Hw4 2016 | Applied Stochastic process MTH 412 IIT Kanpur
0 likes107 views
This document contains 6 homework problems for a stochastic processes class. Problem 1 asks to prove the uniqueness of the stationary distribution for an ergodic Markov chain. Problem 2 gives a transition probability matrix and asks to find its recurrent and transient classes, and the probability and expected time of absorption. Problem 3 asks to find the nth power of a partitioned transition matrix. Problems 4 and 5 ask about the existence of stationary distributions for infinite recurrent and finite chains with transients. Problem 6 asks to prove an inequality relating the nth passage time and the next passage time.
1 of 1
Download to read offline
More Related Content
More from Vivekananda Samiti (20)
Ad
Recently uploaded (20)
Ad
Hw4 2016 | Applied Stochastic process MTH 412 IIT Kanpur
- 1. Home Work 4 MTH 412 Applied Stochastic Process 1. Prove that for an Ergodic Markov chain the stationary distribution is unique. 2. Consider the following transition probability matrix with the state space S = {0, 1, 2, 3} P = 1/6 1/3 1/2 0 1/2 1/2 0 0 1/6 1/3 1/2 0 0 1/6 1/3 1/2 Find the recurrent and transient classes. Find the probability that starting from the transient state i, it is going to get absorbed in the given recurrent state j. Find the expected time to get absorbed in a recurrent class. 3. For a finite absorbing chain suppose the transition probability matrix P is written as follows: P = P1 0 R1 Q Find Pn . 4. Prove or give a counter example that for an infinite recurrent Markov chain stationary probability distribution does not exist. 5. Prove or give a counter example that for a finite Markov chain with transient states stationary probability distribution does not exist. 6. Prove the following: TNn ≤ n < TNn+1. 1