Question

Let \( \{X_n\}_{n \geq 0} \) be a time-homogeneous discrete time Markov chain with either finite or countable state space \( S \). Then which one of the following statements is true?

• A. There is at least one recurrent state
• B. If there is an absorbing state, then there exists at least one stationary distribution
• C. If all the states are positive recurrent, then there exists a unique stationary distribution
• D. If \( \{X_n\}_{n \geq 0} \) is irreducible, \( S = \{1, 2\} \) and \( [\pi_1 \, \pi_2] \) is a stationary distribution, then \( \lim_{n \to \infty} P(X_n = i \mid X_0 = i) = \pi_i \) for \( i = 1, 2 \)

Hint:

For irreducible Markov chains with a stationary distribution, the chain will converge to the stationary distribution as \( n \to \infty \).

Answer 1

The Correct Option is B

- Option (A) is true because a time-homogeneous Markov chain always has at least one recurrent state.
- Option (B) is false because if there is an absorbing state, it does not necessarily guarantee the existence of a stationary distribution.
- Option (C) is true because if all states are positive recurrent, a unique stationary distribution exists.
- Option (D) is true by the fundamental limit theorem of Markov chains, which states that for an irreducible Markov chain with a stationary distribution, \( \lim_{n \to \infty} P(X_n = i \mid X_0 = i) = \pi_i \).
Final Answer: \[ \boxed{\text{(D) If \( \{X_n\}_{n \geq 0} \) is irreducible, \( S = \{1, 2\} \) and \( [\pi_1 \, \pi_2] \) is a stationary distribution, then \( \lim_{n \to \infty} P(X_n = i \mid X_0 = i) = \pi_i \).}} \]