Skip to content

Wiener Filter

Context

Given the following system:

+++-

where

  • \(x(n) \in \mathbb{C}\) is the input sample
  • \(\hat{ \mathbf{h} } = \left[ \hat{h}_{-\infty}, ... , \hat{h}_{-1}, \hat{h}_0, \hat{h}_{1}, ..., \hat{h}_{\infty} \right]\) is a linear filter
  • \(y(n) \in \mathbb{C}\) is the output of the unknown system
  • \(v(n) \in \mathbb{C}\) is the interference added to \(y(n)\)
  • \(\hat{y}(n) = \sum_{i=-\infty}^{\infty}(\hat{h_i}^*x(n-i)) \in \mathbb{C}\) is the estimated output of the unknown system
  • \(\left\{\cdot\right\}^*\) is conjugate

Note that only \(x(n)\), \(\hat{y}(n)\), \(d(n)\), and \(e(n)\) are observable, while \(y(n)\), and \(v(n)\) are not directly observable.

We want to design \(\hat{ \mathbf{h} }\) so that its output, \(\hat{y}(n)\), is statistically a good estimate of \(d(n)\).

Problem

Assuming that

  1. \(x(n)\) is wide-sense stationary,
  2. \(d(n)\) is wide-sense stationary,
  3. \(x(n)\) and \(d(n)\) are jointly wide-sense stationary,
  4. We use the mean square error (MSE) as the cost function to evaluate whether \(\hat{y}(n)\) is a good estimate of \(d(n)\),

How do we choose \(\hat{ \mathbf{h} }\) to minimize the MSE between \(\hat{y}(n)\) and \(d(n)\)?

Solution

To minimize the MSE subject to \(\hat{ \mathbf{h} }\), we need to solve the following equation:

\[ \nabla _{{\hat {\mathbf {h} }}^{H}} E \left( \left| e(n) \right|^2 \right) = 0 \]

Or equivalently,

\[ \frac{\partial}{\partial \hat{h}_i^* } E \left( \left| e(n) \right|^2 \right) = 0 \qquad \forall i \in \mathbb{N} \]

The equation above can be written as

\[ \begin{align} \frac{\partial}{\partial \hat{h}_i^* } E \left( \left| e(n) \right|^2 \right) &= \frac{\partial}{\partial \hat{h}_i^* } E \left( e(n)e^*(n) \right) \\ &= E \left( \frac{\partial e(n)}{\partial \hat{h}_i^* } \cdot e^*(n) \right) + E \left( e(n) \cdot \frac{\partial e^*(n)}{\partial \hat{h}_i^* } \right) \\ &= E \left( x(n-i) e^*(n) \right) +E \left( e(n) x^*(n-i) \right) \\ &= E \left( x(n-i)d^*(n) \right) - \sum_{j=-\infty}^{\infty} \hat{h}_j E \left( x(n-i)x^*(n-j) \right) + E \left( d(n)x^*(n-i) \right) - \sum_{j=-\infty}^{\infty} \hat{h}_j^* E \left( x(n-j)x^*(n-i) \right) \\ &= \phi_{xd}(i) - \sum_{j=-\infty}^{\infty} \hat{h}_j \phi_{xx}(i-j) + \left( \phi_{xd}(i) - \sum_{j=-\infty}^{\infty} \hat{h}_j \phi_{xx}(i-j) \right)^* \\ &= 0 \end{align} \]

where

  • \(\phi_{xd}(i)\) is the cross-correlation of \(x(n)\) and \(d(n)\),
  • \(\phi_{xx}(i)\) is the autocorrelation of \(x(n)\)

So the optimal \(\hat{ \mathbf{h} }\) should satisfy the following equation:

\[ \phi_{xd}(i) - \sum_{j=-\infty}^{\infty} \hat{h}_j \phi_{xx}(i-j) = 0 \]

In the case of \(\hat{ \mathbf{h} }\) being an \(p\)-tap FIR filter, the equation can be written in the matrix form:

\[ \underbrace{ \left[\begin{array}{cccc} \phi_{xx}(0) & \phi_{xx}(-1) & \cdots & \phi_{xx}(-p+1) \\ \phi_{xx}(1) & \phi_{xx}(0) & \cdots & \phi_{xx}(-p+2) \\ \vdots & \vdots & \ddots & \vdots \\ \phi_{xx}(p-1) & \phi_{xx}(p-2) & \cdots & \phi_{xx}(0) \end{array}\right] }_{\mathbf{\Phi_{xx}} } \underbrace{ \left[\begin{array}{c} \hat{h}_0 \\ \hat{h}_1 \\ \vdots \\ \hat{h}_{p-1} \end{array}\right] }_{\mathbf{ \hat{h} } } = \underbrace{ \left[\begin{array}{c} \phi_{xd}(0) \\ \phi_{xd}(1) \\ \vdots \\ \phi_{xd}(p-1) \end{array}\right] }_{\mathbf{\Phi_{xd}} } \]

and the optimal \(\mathbf{ \hat{h} }\) would be

\[ \mathbf{ \hat{h} } = \mathbf{\Phi_{xx}^{-1}} \mathbf{\Phi_{xd}} \]

Reference