Stochastic Adaptive Filtering

This is one of the greater projects of my Master done is 2020, which is at the border of Machine Learning and DSP. The goal was to first understand the theory (stochastic filtering and kernel methods) behind industry standard algorithms and implement a python library together with a testing procedure and write an in-depth analysis of their behavior (11.1k words).

The implemented and analyzed filter algorithms are the forgetting factor recursive least squares filter (ρRLS) and the (kernel) least means squares (KLMS and LMS), all being supervised machine learning algorithms and part of the adaptive finite impulse response (FIR) filters (AF) class. The paper focusses on the search for optimal hyper-parameters and initialization variables depending on the system to be emulated (FIR vs IIR) and the filter usage environment (different noise levels, a radical system change, and next value prediction rather than filtering).

The python library is composed of a main filter object providing most functionalities, from which all filters are derived. The library also provides a dedicated grid search method for hyper-parameter search. Note that the KLMS deviates strongly from its usual form to allow extensive numerical optimization, as described in the paper.

Read or download the paper
The code is here.