Summary

My congratulations, you've made another step towards understanding modern, state-of-the-art RL methods! We learned about some very important concepts that are widely used in deep RL: the value of state, the value of actions, and the Bellman equation in various forms. We saw the value iteration method, which is a very important building block in the area of Q-learning. Finally, we got to know how value iteration can improve our FrozenLake solution.

In the next chapter, we'll learn about deep Q-networks, which started the deep RL revolution in 2013, by beating humans on lots of Atari 2600 games.