Transition diagram

A very intuitive alternative to the description of a Markov chain through a transition matrix is associating an oriented graph (transition diagram) to a Markov chain to which the following two statements apply:

• Vertices are labeled by the S₁, S₂,…, S_n states (or, briefly, from the indices 1, 2, …, n of the states)
• There is a directed edge that connects the S_i vertex to the S_j vertex if and only if the probability of transition from S_i to S_j is positive (a probability that is in turn used as a label of the edge itself)

It is clear that the transition matrix and transition diagram provide the same information regarding the same Markov chain. To understand this duality, we can look at a simple example. Say that we have a Markov chain with three possible states—1, 2, and 3—and the following transition matrix:

The following diagram shows the transition for the preceding Markov chain. In this diagram, there are three possible states—1, 2, and 3—and the directed edge from each state to other states shows the transition probabilities p_ij. When there is no arrow from state i to state j, it means that p_ij=0:

In the previous diagram, we can see that the arrows that come out of a state always sum up exactly at 1, just as the values of every row in the transition matrix must add up exactly to 1, which represents the probability distribution. From the comparison between the transition matrix and transition diagram, it is possible to understand the duality between the two resources. As always, a diagram is much more illustrative.