Markov matrices
Problem
Each day for a mid-morning snack I either have an apple or a banana. If I have an apple on one day then the probability that I have an apple on the next day is $\frac 7 {10}$, otherwise I have a banana. If I have a banana on one day then the probablity that I have a banana on the next day is $\frac 4 {10}$.
This information could be displayed as a diagram:
Work out the probability that, if I have an apple on Monday, I have:
- An apple on Wednesday
- A banana on Wednesday
Hint: There are two different ways you can go from an apple on Monday to an apple on Wednesday, and similarly for going from an apple on Monday to a banana on Wednesday.
The information could also be displayed as a matrix (sometimes called a Transition matrix).
$${\bf M} = \begin{pmatrix} 0.7 & 0.6 \\ 0.3 & 0.4 \end{pmatrix}$$
You can use the transition matrix to find probabilities. If you start on an apple you can say that your initial state is $\begin{pmatrix} 1 \\ 0\end{pmatrix}$. You can then find the probability of having an apple or banana next time by calculating:
$$\begin{pmatrix} 0.7 & 0.6 \\ 0.3 & 0.4 \end{pmatrix}\begin{pmatrix} 1 \\ 0\end{pmatrix} = \begin{pmatrix} 0.7 \\ 0.3\end{pmatrix}$$
Calculate ${\bf M}^2$. What do you notice about the first column of your matrix?
Calculate ${\bf M}^3$ and ${\bf M}^4$. What do you think will happen to ${\bf M}^n$ as $n \to \infty$?
On average, what proportion of the days will I choose an apple?
You may find these Matrix Power calculators useful:
Now, instead of just apples and bananas I can choose a cake (or clementine if you want to be healthy!). The diagram below shows the new probabilities.
Write down a transition matrix for this situation.
Use your matrix to work out the probabilities that I have an apple, banana or cake on Friday given that I had an apple on Monday.
On average, what proportion of the days will I eat cake?
There are more matrix problems in this feature.
NOTES AND BACKGROUND
A Markov Chain is a model showing a sequence of events where the probability of an event happening depends only on the current state (this is sometimes called the memorylessness property).
Markov chains have many applications, including Quantum Mechanics, Population Dynamics, Data Compression, Stock Price modelling and ranking the relevance of web pages.
Student Solutions
Work out the probability that, if I have an apple on Monday, I have:
- An apple on Wednesday
Tanish from Pate's Grammar School in the UK, Mateusz from Hautlieu School in Jersey and Nishad from Rugby High School in England found this probability correctly. This is Mateusz's work (click on the image to open a larger version):Image
- A banana on Wednesday
Tanish, Matteusz and Nishad also worked out this probability. This is Tanish's work:Image
Calculate ${\bf M}^2$. What do you notice about the first column of your matrix?
Tanish, Mateusz and Nishad calculated ${\bf M}^2$ and all noticed the same thing. This is Tanish's wok:
Ci Hui from Queensland Academy for Science Mathematics and Technology in Australia showed how to use the transition matrix to get the probabilities of eating an apple and a banana on Wednesday - and therefore showed why these can come from ${\bf M}^2:$
Calculate ${\bf M}^3$ and ${\bf M}^4$. What do you think will happen to ${\bf M}^n$ as $n \to \infty$?
Maiwand from LAE Tottenham in the UK, Tanish, Mateusz and Ci Hui had similar answers. This is Maiwand's work:
To decide what happens to $\bf M^n$ as $n \rightarrow \infty$, Ci Hui plotted graphs (using Excel) showing how the four elements of $\bf M^n$ change for different values of $n$ (click on the image to open a larger version)
Nishad used proof by induction to prove what happens to $\bf M^n$ as $n \rightarrow \infty.$ Click to see Nishad's proof.
On average, what proportion of the days will I choose an apple?
Tanish, Mateusz, Nishad and Ci Hui answered this correctly. This is Nishad's work:
Now, instead of just apples and bananas I can choose a cake.
Write down a transition matrix for this situation.
Tanish, Mateusz, Ci Hui and Nishad wrote down the matrix, and Bethany from England explained how to. Click to see Bethany's work:
Most people put the Cake in the third column and third row, so got a slightly different transition matrix, $\begin{pmatrix} 0 & 0.3 & 0.5 \\ 0.2 & 0.4 & 0 \\ 0.8 & 0.3 & 0.5 \end{pmatrix}.$
Use your matrix to work out the probabilities that I have an apple, banana or cake on Friday given that I had an apple on Monday.
Bethany used repeated matrix multiplication to find the probabilities: 
Tanish, Mateusz and Nishad used $\bf M^4.$ Tanish wrote: 
On average, what proportion of the days will I eat cake?
Tanish, Mateusz, Nishad and Ci Hui all agreed about the answer. This is Mateusz's work, which shows how they got there (click on the image to open a larger version): 
Teachers' Resources
This problem introduces the idea of using a matrix to represent the probabilities of moving between different states, and also how these can be used to investigate long term behaviour.
The problem starts with a situation where movement is between two states, and then moves onto introducing a third state.
Students might find these Matrix Power calculators useful:
They will be especially useful when investigating the long term behaviour of the problem. The decimal calculator might be more useful for this case.
The problem Pass the Parcel takes these ideas and applies them to a popular children's party game.
There are more matrix problems in this feature.