Stochastic Processes - part 3

At t = 0, flip a coin every T seconds, if a head shows take
a step s to the right and if a tail shows take a step of s to
the left.
We have a discrete-state stochastic process called random
walk.
After nT seconds our position is at:
X(nT) = ks
|{z}
k heads
− (n − k)s
| {z }
(n−k) tails
= ms, m = 2k − n
AKU-EE/Stochastic/HA, 1st Semester, 85-86 – p.1/134

P{X(nT) = ms} =

n
k

0.5k
0.5n−k
, m = 2k − n
If xi is the ith step
E{xi} = s/2 − s/2 = 0, E{x2
i } = s2
/2 + (−s)2
/2 = s2
E{X(nT)} = E{x1 + x2 + · · · + xn} = 0, E{x2
(nT)} = ns2

For large t:
P{X(nT) = ms} =

n
k

0.5k
0.5n−k
∼
exp

−(k−np)2
2npq

√
2πnpq
For p = q = 0.5, this approximation is valid for
−
√
npq 6 k − np 6
√
npq ⇒ −
√
n
2
6 m −
n
2
6
√
n
2

Then the Gaussian approximation converts to:
P{X(nT) = ms} =
exp

−
(
(m+n)
2
−n/2)2
2n/4

p
2πn/4
=
e−m2/2n
pnπ
2
Independent increments: n1 n2 6 n3 n4 ⇒ X(n4T) −
X(n3T) is independent of X(n2T) − X(n1T)

Wiener Process
The limiting form of random walk as T → 0 and n → ∞ is
the Wiener process:
W(t) = lim
T→0
n→∞
X(t)
For random walk we had:
E{x2
(t = nT)} = ns2
=
t
T
s2
For a meaningful Wiener process we must have: s2
= αT,
i.e., s → 0 ⇔
√
T → 0

For random walk:
P{X(nT) = ms} =

n
k

pk
qn−k
=
e−m2/2n
pnπ
2
The Wiener step: w = ms
m
√
n
=
w/s
p
t/T
=
w
√
t
√
T
s
=
w
αt

Then the 1st order PDF for the Wiener process is:
f(w; t) =
1
σ
√
2π
exp

−
w2
2σ2

The variance of a random walk is npq = n/4, and the
std.dev=
√
n/2

The ACF for a Wiener process:
RWW (t1, t2) = E{W(t1)W
∗
(t2)}
Because of independent increment property:
E{[W(t2) − W(t1)]
∗
W(t1)} = 0
E{W
∗
(t2)W(t1) − |W(t1)|2
} = RWW (t1, t2) − αt1 = 0, t1 t2
⇒ RWW (t1, t2) =

αt1, t1 t2
αt2, t1 t2
The Wiener process is a non-stationary process.

Poisson process
P{N(t1, t2) = k} = e−λt (λt)k
k!
, λ =
n
T
For non-overlapping intervals (t1, t2), (t3, t4) then RVs
N(t1, t2) and N(t3, t4) are independent
E{N(t)} = λt
E {N(t1) [N(t2) − N(t1)]} = E{N(t1)}E{N(t2) − N(t1)} =
λt1 · λ(t2 − t1)

E

N2
(t)

= λt + (λt)2
ACF:
RNN (t1, t2) =

λt1 + λ2
t1t2, t1 t2
λt2 + λ2
t1t2, t1 t2
The Poisson process is a non-stationary process.

x are called Poisson points.
N(t)
t
x x x x
Figure 1: Poisson process

Telegraph signal
We form a process X(t), using the Poisson points, such
that X(t) = 1 if the number of points in (0, t) is even and
X(t) = −1 if the number of points in (0, t) is odd.

X(t)
t
-
-
-1
1
Figure 2: Telegraph signal

1st order PDF:
P{X(t) = 1} = e−λt
cosh λt
P{X(t) = −1} = e−λt
sinh λt

2nd order PDF:
P{X(t1) = 1, X(t2) = 1} = e−λt2
cosh λt2e−λτ
cosh λτ
P{X(t1) = 1, X(t2) = −1} = e−λt2
sinh λt2e−λτ
sinh λτ
P{X(t1) = −1, X(t2) = 1} = e−λt2
cosh λt2e−λτ
sinh λτ
P{X(t1) = −1, X(t2) = −1} = e−λt2
sinh λt2e−λτ
cosh λτ
where t2 t1 and τ = t2 − t1

A review: For every stochastic process we have
X
i,j
aia∗
j R(t1, t2) 0, ∀~
a
This is the positive (semi)-definite property.
And vice-verse, given a positive (semi)-definite function
R(t1, t2) we can find a stochastic process with ACF R(t1, t2).

Autocovariance function:
C(t1, t2) = R(t1, t2) − η(t1)η
∗
(t2)
Correlation coefficient:
r(t1, t2) =
C(t1, t2)
p
C(t1, t1)C(t2, t2)
, |r(t1, t2)| 6 1
Cross-correlation function:
RXY (t1, t2) = E{X(t1)Y
∗
(t2)} = R
∗
Y X(t2, t1)

Cross- covariance function:
CXY (t1, t2) = RXY (t1, t2) − ηX(t1)η
∗
Y (t2)
Orthogonal processes:
RXY (t1, t2) = 0, ∀t1, t2
Uncorrelated processes:
CXY (t1, t2) = 0, ∀t1, t2

a-dependent process:
If for a process C(t1, t2) = 0 for |t1 − t2| a correlation
a-dependent process:
If for a process R(t1, t2) = 0 for |t1 − t2| a
White noise process:
If the value of a process are uncorrelated for every
ti, tj, j 6= i ⇒ C(ti, tj) = 0, i 6= j or
C(ti, tj) = q(ti)δ(ti − tj), q(·) 0

A process X(t) is normal if RV {X(t1), · · · , X(tn)} are
jointly normal for every n, and {t1, · · · , tn}. The only
statistics required are:
E{X(t)} = η(t), E{X(ti)X(tj)} = R(ti, tj)
f(X(t1), · · · , X(tn)) =
exp −0.5X̄C−1
X̄T

p
(2π)n∆
where X̄ = ~
X − ~
η(t) and Cij = E{X(ti)X(tj)} − η(ti)η(tj)

A point process is a set of random points ti on the time axis.
The sequence Z1 = t1, Z2 = t2 − t1, · · · , Zn = tn − tn−1 is
called a renewal process

If X(t) is a WS cyclostationary process, then the shifted
process X̄(t) = X(t − θ), θ ∼ Unif(0, T) is WSS.
Mean : η̄ = 1
T
R T
0
η(t) dt
ACF : R̄(τ) = 1
T
R T
0
R(t + τ, t) dt
Proof: Note: Mean ACF are periodic⇒
E{X(t − θ)} = E{E{X(t − θ)}|θ} = E{
due to cyclo.
z }| {
η(t − θ) }
=
1
T
Z T
0
η(t − θ)dθ

E{E{X(t + τ − θ)X(t − θ)}|θ} = E{R(t + τ − θ, t − θ)}
=
1
T
Z T
0
R(t + τ − θ, t − θ)dθ =
1
T
Z T
0
R(t1 + τ, t1)dt1

Differentiator:
L{X(t)} = X′
(t), ηX′ (t) = η′
X(t), RXX′ (t1, t2) =
∂RXX(t1, t2)
∂t2
because
E{X(t1)X′
(t2)} = E{X(t1)
d
dt2
X(t2)} =
∂
∂t2
E{X(t1)X(t2)}
RX′X′ (t1, t2) =
∂
∂t1
RXX(t1, t2) =
∂2
∂t1∂t2
RXX(t1, t2)
An integrator follows the same rule.

Linear constant coefficient differential equation:
n
X
k=0
akY (k)
(t) = X(t), X(k)
(0) = 0, k = 0, 1, · · · , n − 1
E
( n
X
k=0
akY (k)
(t)
)
= E{X(t)}
n
X
k=0
akη
(k)
Y (t) = ηX(t), E

Y (k)
(t)

=
dk
dtk
E{Y (t)}
η
(k)
X (0) = 0, k = 0, 1, · · · , n − 1

X(t1)
n
X
k=0
akY (k)
(t2) = X(t1)X(t2)
E
(
X(t1)
n
X
k=0
akY (k)
(t2)
)
= E{X(t1)X(t2)}
n
X
k=0
akE{X(t1)Y (k)
(t2)} = RXX(t1, t2)
n
X
k=0
ak
∂k
∂tk
2
RXY (t1, t2) = RXX(t1, t2), R
(k|n−1
0 )
XY (t1, 0) = 0

Y (t2)
n
X
k=0
akY (k)
(t1) = X(t1)Y (t2)
n
X
k=0
ak
∂k
∂tk
1
RY Y (t1, t2) = RXY (t1, t2), R
(k|n−1
0 )
Y Y (0, t2) = 0
initial conditions:
X(k)
(0) = 0 ⇒ R
(k|n−1
0 )
XY (t1, 0) = 0, R
(k|n−1
0 )
Y Y (0, t2) = 0

Ergodicity
Time averaging can replace ensemble averaging
If we have access to a large number of samples a single
ensemble suffices to estimate the desired statistics.
X̄ ≈ lim
T→∞
1
T
Z T/2
−T/2
X(t; ξ) dt
This can only be true iff η(t) =constant.

Mean-Ergodic processes
Given a stochastic process X(t) with E{X(t)} = η
ηT =
1
2T
Z T
−T
X(t)dt, ηT = a RV
The process is mean-ergodic iff ηT −→
T →∞
η

E{ηT } =
1
2T
Z T
−T
E{X(t)}dt = η
Var(ηT ) = E{(ηT − η)2
} −→
T →∞
0
If this holds true then
ηT → η

A process X(t) is mean-ergodic iff
1
4T2
Z T
−T
Z T
−T
CXX(t1, t2)dt1dt2 −→
T →∞
0
Proof:
1
4T2
Z T
−T
Z T
−T
CXX(t1, t2)dt1dt2 = Var(ηT )

If X(t) is WSS then CXX(t1, t2) = CXX(t1 − t2) ⇒
if ω1 = 0, ω2 = 0 ⇒
R ∞
−∞
R ∞
−∞
CXX(τ)u(t1 − T)u(t1 + T)u(t2 − T)u(t2 +
T)e−jω1t1−jω2t2
dt1dt2 = SXX(ω) ∗ 2 sin ωT
ω
· 2 sin ωT
ω

This is because u(t1 − T)u(t1 + T)u(t2 − T)u(t2 + T) are
separable kernels.
F−1
2 sin ωT
ω
· 2 sin ωT
ω

= 2T − |τ|

Hence, the Mean-Ergodicity condition for WSS process is:
1
2T
Z 2T
−2T
CXX(τ)

1 −
|τ|
2T

dτ −→
T →∞
0

Correlation-Ergodic processes
A process X(t) is correlation ergodic if
Zλ(t) = X(t)X(t + λ)
is mean-ergodic.
X(t) is correlation-ergodic iff Zλ(t) is mean-ergodic for all
λ.
RT =
1
2T
Z T
−T
X(t)X(t + λ)dt −→
T →∞
RXX(λ)

RZZ(τ) = E{Zλ(t + τ)Zλ(t)}
CZZ(τ) = RZZ(τ) − (E{Z(λ)})2
, E{Z(λ)} = RXX(λ)
Satisfies
1
2T
Z 2T
−2T
CZZ(τ)

1 −
|τ|
2T

dτ −→
T →∞
0

For a deterministic signal X(t), the spectrum is well defined:
If represents its Fourier transform,
X(ω) =
Z +∞
−∞
X(t)e−jωt
dt,
then |X(ω)|2
represents its energy spectrum. This follows
from Parseval’s theorem since the signal energy is given by
Z +∞
−∞
x2
(t)dt = 1
2π
Z +∞
−∞
|X(ω)|2
dω = E.

Thus |X(ω)|2
∆ω represents the signal energy in the band
(ω, ω + ∆ω).
However for stochastic processes, a direct application of
Fourier transform generates a sequence of random vari-
ables for every ω. Moreover, for a stochastic process,
E{|X(t)|2
} represents the ensemble average power (instan-
taneous energy) at the instant t.

To obtain the spectral distribution of power versus
frequency for stochastic processes, it is best to avoid
infinite intervals to begin with, and start with a finite interval
(T, T)
XT (ω) =
Z T
−T
X(t)e−jωt
dt
|XT (ω)|2
2T
=
1
2T

2

The above equation represents the power distribution as-
sociated with that realization based on (T, T). Notice the
above equation represents a random variable for every ω
and its ensemble average gives, the average power distri-
bution based on (T, T).

Thus
PT (ω)=E

|XT (ω)|2
2T

=
1
2T
Z T
−T
Z T
−T
E{X(t1)X∗
(t2)}e−jω(t1−t2)
dt1dt2
=
1
2T
Z T
−T
Z T
−T
RXX(t1, t2)e−jω(t1−t2)
dt1dt2

Thus if X(t) is assumed to be WSS then
RXX(t1, t2) = RXX(t1 − t2), we have
PT (ω) =
1
2T
Z T
−T
Z T
−T
RXX(t1 − t2)e−jω(t1−t2)
dt1dt2.

PT (ω) =
1
2T
Z 2T
−2T
RXX(τ)e−jωτ
(2T − |τ|)dτ
=
Z 2T
−2T
RXX(τ)e−jωτ
(1 −
|τ|
2T
)dτ ≥ 0
Finally letting T → ∞
SXX(ω) = lim
T→∞
PT (ω) =
Z +∞
−∞
RXX(τ)e−jωτ
dτ ≥ 0
RXX(ω)
F.T.
←→ SXX(ω) ≥ 0.

Wiener-Khinchin theorem:
RXX(τ) = 1
2π
Z +∞
−∞
SXX(ω)ejωτ
dω
1
2π
Z +∞
−∞
SXX(ω)dω = RXX(0) = E{|X(t)|2
} = P, the total power.

The nonnegative-definiteness property of the ACF
translates into the “nonnegative” property for power
spectrum, since
n
X
i=1
n
X
j=1
aia∗
j RXX(ti − tj) =
n
X
i=1
n
X
j=1
aia∗
j
1
2π
Z +∞
−∞
SXX(ω)ejω(ti−tj)
dω
= 1
2π
Z +∞
−∞
SXX(ω)

2
dω ≥ 0.

RXX(τ) nonnegative - definite ⇔ SXX(ω) ≥ 0.
If X(t) is a real WSS process, then RXX(τ) = RXX(−τ) so
that
SXX(ω) =
Z +∞
−∞
RXX(τ)e−jωτ
dτ
=
Z +∞
−∞
RXX(τ) cos ωτdτ
= 2
Z ∞
0
RXX(τ) cos ωτdτ = SXX(−ω) ≥ 0

If a WSS process X(t) with autocorrelation function
RXX(τ) is applied to a linear system with impulse response
h(t), then the cross correlation function RXY (τ) and the
output autocorrelation function RY Y (τ) are
RXY (τ) = RXX(τ) ∗ h∗
(−τ), RY Y (τ) = RXX(τ) ∗ h∗
(−τ) ∗ h(τ).
SXY (ω) = F{RXX(ω) ∗ h∗
(−τ)} = SXX(ω)H∗
(ω)

SY Y (ω) = F{RY Y (τ)} = SXY (ω)H(ω)
= SXX(ω)|H(ω)|2
.

The cross spectrum need not be real or nonnegative; How-
ever the output power spectrum is real and nonnegative
and is related to the input spectrum and the system transfer
function as in. The above equations can be used for system
identification as well.

WSS White Noise Process:
If W(t) is a WSS white noise process
RWW (τ) = qδ(τ) ⇒ SWW (ω) = q.
Thus the spectrum of a white noise process is flat, thus jus-
tifying its name. Notice that a white noise process is unre-
alizable since its total power is indeterminate.

If the input to an unknown system is a white noise process,
then the output spectrum is given by
SY Y (ω) = q|H(ω)|2

Note that the output spectrum captures the amplitude of
transfer function of system characteristics entirely, and for
rational systems may be used to determine the pole/zero
locations of the underlying system.

A WSS white noise process W(t) is passed through a low
pass filter (LPF) with bandwidth B/2. The autocorrelation
function of the output process is determined as follows:
SXX(ω) = q|H(ω)|2
=

q, |ω| ≤ B/2
0, |ω| B/2
.

RXX(τ) =
Z B/2
−B/2
SXX(ω)ejωτ
dω = q
Z B/2
−B/2
ejωτ
dω
= qB
sin(Bτ/2)
(Bτ/2)
= qBsinc(Bτ/2)

smoothing:
Y (t) =
1
2T
Z t+T
t−T
X(τ)dτ
represent a “smoothing” operation using a moving window
on the input process X(t). The spectrum of the output Y (t)
in term of that of X(t) is determined as follows:
Y (t) =
Z +∞
−∞
h(t − τ)X(τ)dτ = h(t) ∗ X(t)

where h(t) = (1/2T)[u(t + T) − u(t − T)].
SY Y (ω) = SXX(ω)|H(ω)|2
.
H(ω) =
Z +T
−T
1
2T
e−jωt
dt = sinc(ωT)
SY Y (ω) = SXX(ω)sinc2
(ωT).

Note that the effect of the smoothing operation is to sup-
press the high frequency components in the input and
the equivalent linear system acts as a low-pass filter
(continuous- time moving average) with bandwidth 2π/T in
this case, because the first zero of sinc(·) is at π/T

Discrete-Time Processes
For discrete-time WSS stochastic processes X(nT) with
autocorrelation sequence {rk}

∞
−∞ or formally defining a
continuous time process
X(t) =
X
n
X(nT)δ(t − nT),

we get the corresponding autocorrelation function to be
RXX(τ) =
+∞
X
k=−∞
rkδ(τ − kT).
SXX(ω) =
+∞
X
k=−∞
rke−jωT
≥ 0,
SXX(ω) = SXX(ω + 2π/T)

so that SXX(ω) is a periodic function with period
2B =
2π
T
This gives the inverse relation
rk =
1
2B
Z B
−B
SXX(ω)ejkωT
dω

r0 = E{|X(nT)|2
} =
1
2B
Z B
−B
SXX(ω)dω
represents the total power of the discrete-time process
X(nT).

SXY (ω) = SXX(ω)H∗
(ejω
)
SY Y (ω) = SXX(ω)|H(ejω
)|2
H(ejω
) =
+∞
X
n=−∞
h(nT)e−jωnT

Matched Filter: Let r(t) represent a deterministic signal
s(t) corrupted by noise. Thus
r(t) = s(t) + w(t), 0 t t0
where r(t) represents the observed data, and it is passed
through a receiver with impulse response h(t). The output
y(t) is given by

y(t) = ys(t) + n(t)
ys(t) = s(t) ∗ h(t), n(t) = w(t) ∗ h(t),
and it can be used to make a decision about the presence
or absence of s(t) in r(t).

Towards this, one approach is to require that the receiver
output signal to noise ratio (SNR)0 at time instant t0 be
maximized.
(SNR)0 =
Output signal power at t = t0
Average output noise power
= |ys(t0)|2
E{|n(t)|2}
= |ys(t0)|2
1
2π
R +∞
−∞ Snn(ω)dω
=

1
2π
R +∞
−∞ S(ω)H(ω)ejωt0 dω

2
1
2π
R +∞
−∞ SW W (ω)|H(ω)|2dω

represents the output SNR, and the problem is to maximize
(SNR)0 by optimally choosing the receiver filter H(ω).

Optimum Receiver for White Noise Input:
The simplest input noise model assumes w(t) to be white
noise with spectral density N0
(SNR)0 =

R +∞
−∞
S(ω)H(ω)ejωt0
dω

2
2πN0
R +∞
−∞
|H(ω)|2dω

and a direct application of Cauchy-Schwarz’ inequality
(SNR)0 ≤ 1
2πN0
Z ∞
−∞
|S(ω)|2
dω =
R ∞
0
s2
(t)dt
N0
=
Es
N0
(1)
H(ω) = S∗
(ω)e−jωt0
h(t) = s(t0 − t).
(2)
The optimum choice for t0 = T.

If the receiver is not causal, the optimum causal receiver
can be shown to be
hopt(t) = s(t0 − t)u(t)
and the corresponding maximum (SNR)0 in that case is
given by
(SNR0) = 1
N0
Z t0
0
s2
(t)dt

Optimum Transmit Signal:
In practice, the signal s(t) may be the output of a target
that has been illuminated by a transmit signal f(t) of finite
duration T. In that case
s(t) = f(t) ∗ q(t) =
Z T
0
f(τ)q(t − τ)dτ,
(3)

where q(t) represents the target impulse response. One
interesting question in this context is to determine the opti-
mum transmit signal f(t) with normalized energy that maxi-
mizes the receiver output SNR at t = t0.

We note that for a given s(t), Eq. (2) represents the op-
timum receiver, and (1) gives the corresponding maximum
(SNR)0. To maximize (SNR)0 in (1), we may substitute (3)
into (1).

2
dt
= 1
N0
Z T
0
Z T
0
Z ∞
0
q(t − τ1)q∗
(t − τ2)dt
| {z }
Ω(τ1,τ2)
f(τ2)dτ2f(τ1)dτ1
= 1
N0
Z T
0
Z T
0
Ω(τ1, τ2)f(τ2)dτ2

f(τ1)dτ1 ≤ λmax/N0
Ω(τ1, τ2) =
Z ∞
0
q(t − τ1)q∗
(t − τ2)dt

and λmax is the largest eigenvalue of the integral equation
Z T
0
Ω(τ1, τ2)f(τ2)dτ2 = λmaxf(τ1), 0 τ1 T.
(4)
Z T
0
f2
(t)dt = 1.
(5)

Observe that the kernel Ω(τ1, τ2) captures the target char-
acteristics so as to maximize the output SNR at the obser-
vation instant, and the optimum transmit signal is the so-
lution of the integral equation in (4) subject to the energy
constraint in (5).

If the noise is not white, one approach is to whiten the input
noise first by passing it through a whitening filter, and then
proceed with the whitened output as before

r(t) = s(t) + w(t)
g(t)
sg(t) + n(t)
Whitening filter
w(t) is the colored noise.
n(t) is the white noise.

Notice that the signal part of the whitened output sg(t)
equals
sg(t) = s(t) ∗ g(t)

Whitening Filter:
What is a whitening filter? From the discussion above, the
output spectral density of the whitened noise process
equals unity, since it represents the normalized white noise
by design.
1 = Snn(ω) = SWW (ω)|G(ω)|2
,
|G(ω)|2
=
1
SWW (ω)
.

To be useful in practice, it is desirable to have the whitening
filter to be stable and causal as well. Moreover, at times its
inverse transfer function also needs to be implementable so
that it needs to be stable as well.

where H(s) together with its inverse function 1/H(s) rep-
resent two filters that are both analytic in ℜ(s) 0. Thus
H(s) and its inverse 1/H(s) can be chosen to be stable and
causal. Such a filter is known as the Wiener factor, and
since has all its poles and zeros in the left half plane, it rep-
resents a minimum phase factor.

In the rational case, if X(t) represents a real process, then
SXX(ω) is even and hence (6) is
0 ≤ SXX(ω2
) = S̃XX(−s2
)|s=jω = H(s)H(−s)|s=jω.

Example:
SXX(ω) =
(ω2
+ 1)(ω2
− 2)2
(ω4 + 1)
S̃XX(−s2
) =
(1 − s2
)(2 + s2
)2
1 + s4
.
The left half factors are
H(s) =
(s + 1)(s −
√
2j)(s +
√
2j)

s + 1+j
√
2

s + 1−j
√
2
=
(s + 1)(s2
+ 2)
s2 +
√
2s + 1

H(s) represents the Wiener factor for the spectrum
SXX(ω).
We observe that the poles and zeros (if any) on the jω-axis
appear in even multiples in SXX(ω) and hence half of them
may be paired with H(s) (and the other half with H(s)) to
preserve the factorization condition in (6).
Notice that H(s) is stable, and so is its inverse.

More generally, if H(s) is minimum phase, then ln H(s) is
analytic on the right half plane so that
H(ω) = A(ω)e−jϕ(ω)
(7)
ln H(ω) = ln A(ω) − jϕ(ω) =
Z +∞
0
b(t)e−jωt
dt.
ln A(ω) =
Z t
0
b(t) cos ωtdt
ϕ(ω) =
Z t
0
b(t) sin ωtdt

since cos ωt and sin ωt are Hilbert transform pairs, it follows
that the phase function φ(ω) in (7) is given by the Hilbert
transform of ln A(ω). Thus
ϕ(ω) = H{ln A(ω)}.
Eq. (7) may be used to generate the unknown phase func-
tion of a minimum phase factor from its magnitude.

For discrete-time processes, the factorization conditions
take the form Z π
−π
SXX(ω)dω∞
Z π
−π
ln SXX(ω)dω − ∞.
SXX(ω) = |H(ejω
)|2
H(z) =
∞
X
k=0
h(k)z−k

H(z) is analytic together with its inverse in |z| 1. This
unique minimum phase function represents the Wiener fac-
tor in the discrete-case.

Innovation:
A system Γ(s) is called minimum-phase if it is causal,
stable and its inverse(1/Γ(s)) is also causal and stable.
This means that both Γ(s) and 1/Γ(s) are analytic for
ℜ{s} 0

Whitening:
Given a stationary process X(t), whitening amounts to
finding a minimum-phase system such that its response to
X(t) is an orthonormal process I(t).
I(t) =
Z ∞
−∞
γ(α)X(t − α) dα, RII(τ) = δ(τ)
X(t) =
Z ∞
−∞
ℓ(β)I(t − β) dβ

L(s) =
1
Γ(s)
,

Γ(s), Whitening filter
L(s), Innovation filter
SXX(s) = L(s)L(−s) SII(s)
| {z }
=1
, SXX(ω) = |L(ω)|2
A process X(t) is called regular iff its PSD can be factored
as SXX(s) = L(s)L(−s)

Rational spectra recursion equations:
Y [n] +
N
X
i=1
aiY [n − i] =
M
X
j=0
bjX[n − j]
Y (z)
X(z)
=
PM
j=0 bjz−j
1 +
PN
i=1 aiz−i
=
N(z)
D(z)
= H(z)

Example, continuous process:
SXX(ω) =
3
ω4 − ω2 + 1
|s=jω =
3
(s/j)4 − (s/j)2 + 1
s4
+ s2
+ 1 = 0 ⇒ s2
= −0.5 ± j
√
3
2
=
√
3
s2 + s + 1
| {z }
L(s)
√
3
s2 − s + 1
| {z }
L(−s)
Γ(s) =
1
√
3
(s2
+ s + 1)

Example, discrete process:
SXX(ω) =
5 − 4 cos ωT
10 − 6 cos ωT
⇒ SXX(z) =
5 − 2(z + z−1
)
10 − 3(z + z−1)
SXX(z) =
2
3
(z − 2)(z − 0.5)
(z − 3)(z − 1/3)
⇒
L(z) =
r
2
3

z − 0.5
z − 1/3

,
1
Γ(z)
=
r
2
3

z − 0.5
z − 1/3


Stochastic Processes - part 3

More Related Content

What's hot (20)

Similar to Stochastic Processes - part 3 (20)

Recently uploaded (20)

Stochastic Processes - part 3