Question

1.67. Roll a fair die repeatedly and let Y1; Y2; : : : be the resulting numbers. Let Xn D jfY1; Y2; : : : ; Yngj be the number of values we have seen in the first n rolls for n 1 and set X0 D 0. Xn is a Markov chain. (a) Find its transition probability. (b) Let T D minfn W Xn D 6g be the number of trials we need to see all 6 numbers at least once. Find ET.

Answer 1

1.67. Roll a fair die repeatedly and let $\mathbf{Y_1, Y_2...}$ be the resulting numbers. Let $\mathbf{X_n = |\{Y_1, Y_2,...,Y_n\}|}$ be the number of values we have seen in the first n rolls for n ≥ 1 and set X₀ = 0. Xₙ is a Markov chain. (a) The transition probability is:

$\mathbf{P = \left[\begin{array}{ccccc}\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3} \\ \\\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6} \end{array}}\right]$

E(T) = E(Time until absorption) is expressed as:

$\mathbf{E(T) =\ \left[\begin{array}{c}1 \\ \\ 1 \\ \\ 1 \\ \\ 1 \end{array}}\right] }$

The transition probability is the likelihood of a system transitioning from one phase to another.  If a Markov chain seems to be in state a, the transition probability $\mathbf{p_{ab}}$ , is the likelihood of transitioning to state b at the next time interval.

From the information given:

• first n rolls for n ≥ 1 and set X₀ = 0.

• the transition probability matrix (tpm) can be computed as:

$\mathbf{P = \left[\begin{array}{ccccc}\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3} \\ \\\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6} \end{array}}\right]$

B.

• Let $\mathbf{T = min \{n : X_n = 6\}}$ be the number of trails we need to see all 6 numbers at least once.

• An absorbing Markov chain is a Markov chain in probability theory that any state might become an absorbing state.

Let take state 6 to turn into an absorbing state, then the transition probability matrix can take the form:

$\mathbf{\implies \left[\begin{array}{ccccc}\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3} \\ \\\ 1&0&0&0&0 \end{array}}\right]$

$\mathbf{Q = \left[\begin{array}{cccc}\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\ \dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6} \\ \\ \dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3} \\ \end{array}}\right]$

As such F = (I-Q)⁻¹

$\mathbf{F \implies \left[\begin{array}{cccc}1&0&0&0\\ \\ 0&1&0&0 \\ \\0&0&1&0 \\ \\ 0&0&0&1\end{array}\right] - \left[\begin{array}{cccc}\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6}&\dfrac{1}{6} \\ \\ \dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3}&\dfrac{1}{6} \\ \\ \dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{6}&\dfrac{1}{3} \\ \end{array}}\right] }$

$\mathbf{F \implies \ \left[\begin{array}{cccc}\dfrac{2}{3}&-\dfrac{1}{6}&-\dfrac{1}{6}&-\dfrac{1}{6} \\ \\-\dfrac{1}{6}&\dfrac{2}{3}&-\dfrac{1}{6}&-\dfrac{1}{6} \\ \\ -\dfrac{1}{6}&-\dfrac{1}{6}&\dfrac{2}{3}&-\dfrac{1}{6} \\ \\ -\dfrac{1}{6}&-\dfrac{1}{6}&-\dfrac{1}{6}&\dfrac{2}{3} \\ \end{array}}\right] }$

∴

E(T) = E(Time until absorption) is expressed as:

$\mathbf{E(T) =\ \left[\begin{array}{c}1 \\ \\ 1 \\ \\ 1 \\ \\ 1 \end{array}}\right] }$

Therefore, from the above explanation, we have clearly understood the transition probability and the E(Time until absorption).

Learn more about Markov Chain here:

https://brainly.com/question/7247701?referrer=searchResults