Introduction to adaptive filtering and its applications.ppt

By
Asst.Prof.Dr.Thamer M.Jamel
Department of Electrical Engineering
University of Technology
Baghdad – Iraq

Introduction
Linear filters :
the filter output is a linear function of the filter input
Design methods:
 The classical approach
frequency-selective filters such as
low pass / band pass / notch filters etc
 Optimal filter design
Mostly based on minimizing the mean-square value
of the error signal
2

Wiener filter
 work of Wiener in 1942 and Kolmogorov in
1939
 it is based on a priori
statistical information
 when such a priori
information is not available,
which is usually the case,
it is not possible to design
a Wiener filter in the first
place.

Adaptive filter
 the signal and/or noise characteristics are often
nonstationary and the statistical parameters vary
with time
 An adaptive filter has an adaptation algorithm, that is
meant to monitor the environment and vary the filter
transfer function accordingly
 based in the actual signals received, attempts to find
the optimum filter design

Adaptive filter
 The basic operation now involves two processes :
1. a filtering process, which produces an output signal
in response to a given input signal.
2. an adaptation process, which aims to adjust the filter
parameters (filter transfer function) to the (possibly
time-varying) environment
Often, the (average) square value of the error signal is
used as the optimization criterion

Adaptive filter
• Because of complexity of the optimizing algorithms
most adaptive filters are digital filters that perform
digital signal processing
 When processing
analog signals,
the adaptive filter
is then preceded
by A/D and D/A
convertors.

Adaptive filter
• The generalization to adaptive IIR filters leads to
stability problems
• It’s common to use
a FIR digital filter
with adjustable
coefficients

Applications of Adaptive Filters:
Identification
 Used to provide a linear model of an unknown
plant
 Applications:
 System identification

Inverse Modeling
 Used to provide an inverse model of an unknown
plant
 Applications:
 Equalization (communications channels)

Prediction
 Used to provide a prediction of the present
value of a random signal
 Applications:
 Linear predictive coding

Interference Cancellation
 Used to cancel unknown interference from a primary
signal
 Applications:
 Echo / Noise cancellation
hands-free carphone, aircraft headphones etc

Example:
Acoustic Echo Cancellation

Introduction to adaptive filtering and its applications.ppt

LMS Algorithm
• Most popular adaptation algorithm is LMS
Define cost function as mean-squared error
• Based on the method of steepest descent
Move towards the minimum on the error surface to get to
minimum
gradient of the error surface estimated at every iteration

LMS Adaptive Algorithm
• Introduced by Widrow & Hoff in 1959
• Simple, no matrices calculation involved in the
adaptation
• In the family of stochastic gradient algorithms
• Approximation of the steepest – descent method
• Based on the MMSE criterion.(Minimum Mean
square Error)
• Adaptive process containing two input signals:
• 1.) Filtering process, producing output signal.
• 2.) Desired signal (Training sequence)
• Adaptive process: recursive adjustment of filter tap
weights

LMS Algorithm Steps
 Filter output
 Estimation error
 Tap-weight adaptation
17
     





1
0
*
M
k
k n
w
k
n
u
n
y
     
n
y
n
d
n
e 

       
n
e
k
n
u
n
w
1
n
w *
k
k 





















 
































signal
error
vector
input
tap
parameter
rate
-
learning
vector
weight
-
tap
of
value
old
vector
weigth
-
tap
of
value
update

Stability of LMS
 The LMS algorithm is convergent in the mean square
if and only if the step-size parameter satisfy
 Here max is the largest eigenvalue of the correlation
matrix of the input data
 More practical test for stability is
 Larger values for step size
 Increases adaptation rate (faster adaptation)
 Increases residual mean-squared error

• Given the following function we need to obtain the vector that would give us
the absolute minimum.
• It is obvious that
give us the minimum.
(This figure is quadratic error function (quadratic bowl) )
STEEPEST DESCENT EXAMPLE
2
2
2
1
2
1 )
,
( C
C
c
c
Y 

,
0
2
1 
 C
C
1
C
2
C
y
Now lets find the solution by the steepest descend method

• We start by assuming (C1 = 5, C2 = 7)
• We select the constant . If it is too big, we miss the minimum. If it is too
small, it would take us a lot of time to het the minimum. I would select =
0.1.
• The gradient vector is:


]
[
2
1
]
[
2
1
]
[
2
1
]
[
2
1
]
1
[
2
1
9
.
0
1
.
0
2
.
0
n
n
n
n
n
C
C
C
C
C
C
y
C
C
C
C



























































2
1
2
1
2
2
C
C
dc
dy
dc
dy
y
• So our iterative equation is:








































567
.
0
405
.
0
:
3
3
.
6
5
.
4
:
2
7
5
:
1
2
1
2
1
2
1
C
C
Iteration
C
C
Iteration
C
C
Iteration




























0
0
lim
013
.
0
01
.
0
:
60
......
]
[
2
1
2
1
n
n
C
C
C
C
Iteration
As we can see, the vector [c1,c2] converges to the value which would yield the
function minimum and the speed of this convergence depends on .

1
C
2
C
y
Initial guess
Minimum

LMS – CONVERGENCE GRAPH
This graph illustrates the LMS algorithm. First we start from
guessing the TAP weights. Then we start going in opposite the
gradient vector, to calculate the next taps, and so on, until we get
the MMSE, meaning the MSE is 0 or a very close value to it.(In
practice we can not get exactly error of 0 because the noise is a
random process, we could only decrease the error below a desired
minimum)
Example for the Unknown Channel of 2nd order:
Desired Combination of taps
Desired Combination of taps

Adaptive Array Antenna
 Adaptive Arrays
Linear Combiner
Interference
SMART ANTENNAS

Applications are many
Digital Communications
(OFDM , MIMO , CDMA, and
RFID)
Channel Equalisation
Adaptive noise cancellation
Adaptive echo cancellation
System identification
Smart antenna systems
Blind system equalisation
And many, many others

Introduction
Wireless communication is the most
interesting field of communication
these days, because it supports mobility
(mobile users). However, many
applications of wireless comm. now
require high-speed communications
(high-data-rates).

What is the ISI
Inter-symbol-interference, takes place when a
given transmitted symbol is distorted by other
transmitted symbols.
Cause of ISI
ISI is imposed due to band-limiting effect of
practical channel, or also due to the multi-path
effects (delay spread).

Definition of the Equalizer:
the equalizer is a digital filter that provides an
approximate inverse of channel frequency
response.
Need of equalization:
is to mitigate the effects of ISI to decrease the
probability of error that occurs without
suppression of ISI, but this reduction of ISI
effects has to be balanced with prevention of
noise power enhancement.

Types of Equalization techniques
Linear Equalization techniques
which are simple to implement, but greatly enhance
noise power because they work by inverting channel
frequency response.
Non-Linear Equalization techniques
which are more complex to implement, but have much
less noise enhancement than linear equalizers.

Equalization Techniques
Fig.3 Classification of equalizers

Linear equalizer with N-taps, and (N-1) delay elements.
Go

Table of various algorithms and their trade-offs:
algorithm Multiplying-
operations
complexity convergence tracking
LMS Low slow poor
MMSE Very high fast good
RLS High fast good
Fast
kalman
Fairly
Low
fast good
RLS-
DFE
High fast good
2 3
N toN
2 1
N 
2
2.5 4.5
N N

20 5
N 
2
1.5 6.5
N N


Adaptive Filter Block Diagram
Adaptive Filter Block Diagram
d(n) Desired
y(n)
e(n)
+
-
x(n)
Filter Input
Adaptive Filter
e(n) Error Output
Filter Output

The LMS Equation
 The Least Mean Squares Algorithm (LMS)
updates each coefficient on a sample-by-sample
basis based on the error e(n).
 This equation minimises the power in the error
e(n).
)
(
)
(
)
( n
x
n
e
n
w
1)
(n
w k
k
k 


 

The Least Mean Squares Algorithm
 The value of µ (mu) is critical.
 If µ is too small, the filter reacts slowly.
 If µ is too large, the filter resolution is poor.
 The selected value of µ is a compromise.

Audio Noise Reduction
 A popular application of acoustic noise reduction is
for headsets for pilots. This uses two microphones.
Block Diagram of a Noise Reduction Headset
d(n) = speech + noise
y(n)
e(n)
+
-
x(n) = noise'
Adaptive Filter
e(n)
Speech Output
Filter Output
(noise)
Far Microphone
Near Microphone

Setting the Step size (mu)
 The rate of
convergence of the
LMS Algorithm is
controlled by the
“Step size (mu)”.
 This is the critical
variable.

Trace of Input to Model
“Input” = Signal + Noise.

Trace of LMS Filter Output
“Output” starts at
zero and grows.

Trace of LMS Filter Error
“Error” contains
the noise.

Typical C6713 DSK Setup
USB to PC to +5V
Headphones Microphone

New Trends in Adaptive Filtering
 Partial Updating Weights.
 Sub-band adaptive filtering.
 Adaptive Kalman filtering.
 Affine Projection Method.
 Time-Space adaptive processing.
 Non-Linear adaptive filtering:-
Neural Networks.
The Volterra Series Algorithm .
Genetic & Fuzzy.
• Blind Adaptive Filtering.

Introduction to adaptive filtering and its applications.ppt

More Related Content

What's hot (20)

Similar to Introduction to adaptive filtering and its applications.ppt (20)

More from debeshidutta2 (10)

Recently uploaded (20)

Introduction to adaptive filtering and its applications.ppt