Slow Feature Analysis

Setting

Inputs $x(t)\in {\mathcal{R}}^n$, and output signals $y_j(t)=g_j(x(t))$

Objective

Minimize $\mathrm{\Delta }(y_j)={⟨{\dot{y_j}}^2⟩}_t$ under the constraints

1. ${⟨y_j⟩}_t=0$ (zero mean)
2. ${⟨y_j^2⟩}_t=1$ (unit variance)
3. ${⟨y_i{y}_j⟩}_t=0:\mathrm{\forall }i<j$ (decorrelation and order)

Linear case (SFA)

$g(x)=w^{T}f(x)$ where $f(x)={(f_1(x),\dots f_M(x))}^T$ defines a non-linear basis. Then, we can write

Under the Lagrangian formulation, we get the objective function

Taking the gradient and setting it to zero gives the generalized eigenvalue problem:

The cool thing is that this shows that SFA isn't just about the correlation structure in the data but the joint correlation of the data and its dynamics.