官术网_书友最值得收藏!

Epsilon – exploration versus exploitation

Your agent's exploration rate epsilon also ranges from zero to one. As the agent explores its environment, it learns that some actions are better to take than others, but what about states and actions that it hasn't seen yet? We don't want it to get stuck on a local maximum, taking the same currently highest-valued actions over and over when there might be better actions it hasn't tried to take yet.

When you set your epsilon value, there will be a probability equal to epsilon that your agent will take a random (exploratory) action, and a probability equal to 1-epsilon that it will take the current highest Q-valued action for its current state. As we step through a full Q-table update example in the SARSA and the cliff-walking problem section, we'll see how the value that we choose for epsilon affects the rate at which the Q-table converges and the agent discovers the optimal solution. 

As the agent gets more and more familiar with its environment, we want it to start sticking to the high-valued actions it's already discovered and do less exploration of the states it hasn't seen. We achieve this by having epsilon decay over time as the agent learns more about its environment and the Q-table converges on its final optimal values.

There are many different ways to decay epsilon, either by using a constant decay factor or basing the decay factor on some other internal variable. Ideally, we want the epsilon decay function to be directly based on the Q-values that we've already discovered. We'll discuss what this means in the next section.

主站蜘蛛池模板: 小金县| 泌阳县| 鄂尔多斯市| 辽中县| 永泰县| 兰溪市| 嵊泗县| 濉溪县| 武清区| 宜兰县| 阳原县| 日土县| 临武县| 绍兴市| 元朗区| 浑源县| 芮城县| 靖江市| 平山县| 理塘县| 南宁市| 大同县| 昌邑市| 呼伦贝尔市| 伊春市| 甘洛县| 民丰县| 鲜城| 渝中区| 高邮市| 安康市| 都兰县| 搜索| 攀枝花市| 祁门县| 盐城市| 卢龙县| 商城县| 夏津县| 桃源县| 砚山县|