Markov chains

A Markov chain is a mathematical model of a random phenomenon that evolves over time in such a way that the past influences the future only through the present. The time can be discrete (a whole variable), continuous (a real variable), or, more generally, a totally ordered whole. In this discussion, only discrete chains are considered. Markov chains were introduced in 1906 by Andrei Andreyevich Markov (1856–1922), from whom the name derives.

The example of a one-dimensional random walk seen in the previous section is a Markov chain; the next value in the chain is a unit that is more or less than the current value with the same probability of occurrence, regardless of the way in which the current value was reached.