官术网_书友最值得收藏!

Creating an MDP

Developed upon the Markov chain, an MDP involves an agent and a decision-making process. Let's go ahead with developing an MDP and calculating the value function under the optimal policy.

Besides a set of possible states, S = {s0, s1, ... , sm}, an MDP is defined by a set of actions, A = {a0, a1, ... , an}; a transition model, T(s, a, s'); a reward function, R(s); and a discount factor, ??. The transition matrix, T(s, a, s'), contains the probabilities of taking action a from state s then landing in s'. The discount factor, ??, controls the tradeoff between future rewards and immediate ones.

To make our MDP slightly more complicated, we extend the study and sleep process with one more state, s2 play games. Let's say we have two actions, a0 work and a1 slack. The 3 * 2 * 3 transition matrix T(s, a, s') is as follows:

This means, for example, that when taking the a1 slack action from state s0 study, there is a 60% chance that it will become s1 sleep (maybe getting tired ) and a 30% chance that it will become s2 play games (maybe wanting to relax ), and that there is a 10% chance of keeping on studying (maybe a true workaholic ). We define the reward function as [+1, 0, -1] for three states, to compensate for the hard work. Obviously, the optimal policy, in this case, is choosing a0 work for each step (keep on studying – no pain no gain, right?). Also, we choose 0.5 as the discount factor, to begin with. In the next section, we will compute the state-value function (also called the value function, just the value for short, or expected utility) under the optimal policy.

主站蜘蛛池模板: 台中县| 大洼县| 桦甸市| 定南县| 阳原县| 会东县| 图片| 岳池县| 安庆市| 罗平县| 出国| 东丰县| 永福县| 婺源县| 河西区| 民权县| 青海省| 江安县| 新闻| 中阳县| 清丰县| 攀枝花市| 临猗县| 九江县| 平和县| 沭阳县| 灌云县| 大田县| 永州市| 大足县| 湟源县| 曲松县| 阿鲁科尔沁旗| 汝南县| 阿拉善左旗| 育儿| 青河县| 孝昌县| 辛集市| 安康市| 来安县|