官术网_书友最值得收藏!

Creating a Markov chain

Let's get started by creating a Markov chain, on which the MDP is developed.

A Markov chain describes a sequence of events that comply with the Markov property. It is defined by a set of possible states, S = {s0, s1, ... , sm}, and a transition matrix, T(s, s'), consisting of the probabilities of state s transitioning to state s'. With the Markov property, the future state of the process, given the present state, is conditionally independent of past states. In other words, the state of the process at t+1 is dependent only on the state at t. Here, we use a process of study and sleep as an example and create a Markov chain based on two states, s0 (study) and s1 (sleep). Let's say we have the following transition matrix:

In the next section, we will compute the transition matrix after k steps, and the probabilities of being in each state given an initial distribution of states, such as [0.7, 0.3], meaning there is a 70% chance that the process starts with study and a 30% chance that it starts with sleep.

主站蜘蛛池模板: 隆尧县| 巨野县| 兴和县| 峨眉山市| 兴海县| 乌拉特后旗| 西乌珠穆沁旗| 松阳县| 库尔勒市| 儋州市| 定陶县| 湛江市| 佛坪县| 平潭县| 文化| 苏尼特右旗| 陈巴尔虎旗| 泾阳县| 济源市| 靖州| 象州县| 郸城县| 凤阳县| 柯坪县| 开阳县| 仙游县| 文水县| 丰都县| 泸州市| 鸡西市| 广德县| 揭阳市| 韶山市| 茂名市| 修水县| 萝北县| 个旧市| 芜湖县| 莱州市| 湘乡市| 岐山县|