官术网_书友最值得收藏!

Creating a Markov chain

Let's get started by creating a Markov chain, on which the MDP is developed.

A Markov chain describes a sequence of events that comply with the Markov property. It is defined by a set of possible states, S = {s0, s1, ... , sm}, and a transition matrix, T(s, s'), consisting of the probabilities of state s transitioning to state s'. With the Markov property, the future state of the process, given the present state, is conditionally independent of past states. In other words, the state of the process at t+1 is dependent only on the state at t. Here, we use a process of study and sleep as an example and create a Markov chain based on two states, s0 (study) and s1 (sleep). Let's say we have the following transition matrix:

In the next section, we will compute the transition matrix after k steps, and the probabilities of being in each state given an initial distribution of states, such as [0.7, 0.3], meaning there is a 70% chance that the process starts with study and a 30% chance that it starts with sleep.

主站蜘蛛池模板: 保靖县| 耒阳市| 徐汇区| 台江县| 平泉县| 上栗县| 武夷山市| 廉江市| 苍梧县| 峡江县| 德化县| 朝阳区| 嘉峪关市| 文山县| 靖西县| 平湖市| 革吉县| 池州市| 德格县| 西乌珠穆沁旗| 瓦房店市| 宜宾县| 陈巴尔虎旗| 台南市| 开江县| 平山县| 沁阳市| 宾川县| 海原县| 潞城市| 麦盖提县| 会同县| 普定县| 贡嘎县| 渝中区| 巴楚县| 河津市| 龙游县| 乐陵市| 祁门县| 家居|