官术网_书友最值得收藏!

Learning the Markov decision process 

The Markov property is widely used in RL, and it states that the environment's response at time t+1 depends only on the state and action at time t. In other words, the immediate future only depends on the present and not on the past. This is a useful property that simplifies the math considerably, and is widely used in many fields such as RL and robotics.

Consider a system that transitions from state s0 to s1 by taking an action a0 and receiving a reward r1, and thereafter from s1 to s2 taking action a1, and so on until time t. If the probability of being in a state s' at time t+1 can be represented mathematically as in the following function, then the system is said to follow the Markov property:

Note that the probability of being in state st+1 depends only on st and at and not on the past. An environment that satisfies the following state transition property and reward function as follows is said to be a Markov Decision Process (MDP):

Let's now define the very foundation of RL: the Bellman equation. This equation will help in providing an iterative solution to obtaining value functions.

主站蜘蛛池模板: 上林县| 黔江区| 当涂县| 玉田县| 常宁市| 民丰县| 鄂托克旗| 黎川县| 万安县| 宁南县| 天等县| 保靖县| 涪陵区| 林甸县| 呈贡县| 上林县| 南汇区| 于田县| 南平市| 闽清县| 巍山| 万源市| 寿光市| 阳城县| 吉木乃县| 商丘市| 嘉义县| 凌云县| 瓦房店市| 内黄县| 莎车县| 突泉县| 全州县| 中阳县| 同江市| 溧水县| 仪征市| 大荔县| 兴海县| 莎车县| 息烽县|