Question

Consider a Markov chain \( \{ X_n : n = 1, 2, \dots \} \) with state space \( S = \{1, 2, 3\} \) and transition probability matrix
\[ P = \begin{pmatrix} 0 & \frac{1}{2} & \frac{1}{2} <br> \frac{1}{3} & 0 & \frac{2}{3} <br> \frac{2}{5} & \frac{3}{5} & 0 \end{pmatrix}. \]
Define
\[ \pi = \left( \frac{18}{67}, \frac{24}{67}, \frac{25}{67} \right). \]
Which of the following options is/are correct?

• A. \( \pi \) is a stationary distribution of \( P \)
• B. \( \pi^T \) is an eigenvector of \( P^T \)
• C. \( \Pr(X_3 = 1 \mid X_1 = 1) = \frac{11}{30} \)
• D. At least one state is transient

Hint:

When dealing with Markov chains, ensure that the stationary distribution satisfies \( \pi P = \pi \), and that eigenvectors of \( P^T \) correspond to the stationary distribution.

Answer 1

The Correct Option is A, B, C

Step 1: Checking if \( \pi \) is a stationary distribution of \( P \).
For \( \pi \) to be a stationary distribution, it must satisfy:
\[ \pi P = \pi. \]
Multiplying \( \pi \) by the transition matrix \( P \), we check if \( \pi P = \pi \).
After computing the matrix product, we find that \( \pi P = \pi \), confirming that \( \pi \) is indeed a stationary distribution.
Step 2: Checking if \( \pi^T \) is an eigenvector of \( P^T \).
Since \( \pi \) is a stationary distribution, it follows that \( \pi^T \) is an eigenvector of \( P^T \) with eigenvalue 1. Thus, option (B) is correct.
Step 3: Checking the probability \( \Pr(X_3 = 1 \mid X_1 = 1) \).
We calculate the conditional probability \( \Pr(X_3 = 1 \mid X_1 = 1) \) by using the transition probabilities and the Chapman-Kolmogorov equation:
\[ \Pr(X_3 = 1 \mid X_1 = 1) = P_{11}^3 + P_{12} P_{21} + P_{13} P_{31}. \]
After computing the sum, we find that \( \Pr(X_3 = 1 \mid X_1 = 1) = \frac{11}{30} \), so option (C) is correct.
Thus, the correct answer is \( \boxed{(A), (B), (C)} \).