- Hands-On Q-Learning with Python
- Nazia Habib
- 385字
- 2021-06-24 15:13:15
MDPs and state-action diagrams
Note that in the Markov chain examples we discussed, there is only one event that can happen in each state to cause the system to move to the next state. There is no list of actions and no decisions to make about what action to take. In a random walk, we flip the same fair coin each time, and each time we flip the coin, we have a new pair of states that we can potentially enter.
An MDP adds to a Markov chain the presence of a decision-making agent that has a choice of what action to take and rewards to receive, and provides feedback to the agent, affecting its behavior. Recall that an MDP doesn't require any knowledge of any previous states to make a decision on what action to take from the current state.
Let's go back to the state diagram that we discussed in the last chapter. Notice that a state diagram for an MDP is necessarily more complex than a diagram for a Markov chain. It needs to represent the available action and rewards, and the different states that the system can be in:

As we discussed, we have three states and two actions in this environment. Either action can be taken from any state, and the probability of each outcome as a result of taking each action is labeled on the diagram. For example, when we are in state S0 and take action a0, we might end up in S2 or back in S0, each with a 50% probability.
By extension, an MDP that only allows one action from each state and has the same reward for each action (that is, effectively, no rewards at all) will simplify to a Markov chain.
The main takeaway from this discussion of MDPs is that when we assume the role of an agent navigating a stochastic environment, we need to be able to learn lessons and make decisions that can be applied consistently amid the occurrence of random events. We don't always know what the individual outcome of an action will be, but we need to be able to make decisions that will maximize our overall outcome at each step of the process. This is what the algorithms that we develop will work toward achieving.
- Hands-On Artificial Intelligence on Amazon Web Services
- Python Artificial Intelligence Projects for Beginners
- 錯覺:AI 如何通過數(shù)據(jù)挖掘誤導(dǎo)我們
- 教父母學(xué)會上網(wǎng)
- Windows XP中文版應(yīng)用基礎(chǔ)
- Hands-On Cybersecurity with Blockchain
- 80x86/Pentium微型計算機原理及應(yīng)用
- 大數(shù)據(jù)技術(shù)與應(yīng)用
- Implementing Oracle API Platform Cloud Service
- Kubernetes for Developers
- Docker on Amazon Web Services
- Web編程基礎(chǔ)
- 企業(yè)級Web開發(fā)實戰(zhàn)
- 計算機辦公應(yīng)用培訓(xùn)教程
- Embedded Linux Development using Yocto Projects(Second Edition)