Problem 15.1 Easy Free

Two-State Chain Return Time

Each day is classified good (G) or bad (B). If today is G, tomorrow is G with probability 35\frac{3}{5} (and B otherwise); if today is B, tomorrow is B with probability 710\frac{7}{10} (and G otherwise). You wake up on a bad day. Find the expected number of days until the next bad day.

Work it out, then compare with the answer.

SolutionShow the worked solution

Let TT be the number of days until the next bad day, counting from today (a bad day). Condition on tomorrow.

  • With probability 710\frac{7}{10} tomorrow is bad, and then T=1T=1.
  • With probability 310\frac{3}{10} tomorrow is good. From a good day, each subsequent day is bad with probability 25\frac{2}{5}, independently of the past. So the further wait is Geom ⁣(25)\mathrm{Geom}\!\left(\frac{2}{5}\right), with mean 52\frac{5}{2}. In this case E[T∣tomorrow good]=1+52\mathbb{E}[T \mid \text{tomorrow good}] = 1 + \frac{5}{2}.

Hence

E[T]=710⋅1+310(1+52)=1+34=74.\mathbb{E}[T] = \frac{7}{10}\cdot 1 + \frac{3}{10}\left(1+\frac{5}{2}\right) = 1 + \frac{3}{4} = \frac{7}{4}.
E[T]=74\boxed{\mathbb{E}[T]=\tfrac{7}{4}}

Alternative. The mean return time to a state of an irreducible finite chain is 1/π1/\pi of that state. Balancing the flow between the two states gives

πG⋅25=πB⋅310.\pi_G\cdot\frac{2}{5}=\pi_B\cdot\frac{3}{10}.

So πG:πB=3:4\pi_G:\pi_B = 3:4, and therefore πB=47\pi_B=\frac{4}{7}. The mean return time to BB is then 1πB=74\frac{1}{\pi_B}=\frac{7}{4}.

Generalisation. Write p=P(G→B)p=\mathbb{P}(G\to B) and q=P(B→G)q=\mathbb{P}(B\to G). Then πB=pp+q\pi_B=\frac{p}{p+q}, and the mean return time to BB is 1+qp1+\frac{q}{p}.