Question

A system has three possible states, 0, 1 and 2. Every hour it makes a transition to a different state, which is determined by a coin flip. For example, from state 0, it makes a transition to state 1 or state 2 with probabilities 0.5 and 0.5.
a. Find the transition probability matrix.
b. Find the three-step transition probability matrix.
c. Find the steady-state distribution of the Markov chain.

Answer 1

Answer:

Step-by-step explanation:

From the given information;

the transition probability matrix (TPM) for state 0, 1, 2 is:

$P =\left[\begin{array}{ccc}0&0.5&0.5\\0.5&0&0.5\\0.5&0.5&0\end{array}\right]$

The three-step TPM  is done by a simple matrix multiplication which is computed as follows:

$P^3 = \left[\begin{array}{ccc}0.250&0.3757 &0.375\\0.375&0.250&0.375\\0.375&0.375&0.250\end{array}\right]$

The steady-state distribution of the Markov Chain is  determined by first solving the system πp = π together with the normalizing condition

here;

π₁ + π₂ + π₃ = 1

So;

0.5π₂ + 0.5π₃ = π₁

⇒ π₂ + π₃ = 2π₁

0.5π₁ + 0.5π₃ = π₂

⇒ π₁ + π₃ = 2π₂

0.5π₁ + 0.5π₂ = π₃

⇒ π₁ + π₂ = 2π₃

Thus, π₁ + π₂ + π₃ = 1

This can be re-written as:

1 - π₁ = 2π₁

1 - π₂ = 2π₂

1 - π₃ = 2π₃

∴

1 = 3π₁

π₁ = 1/3

1 = 3π₂

π₂ = 1/3

1 = 3π₃

π₃ = 1/3

Hence, the steady-state probability distribution is π = [1/3, 1/3, 1/3]