官术网_书友最值得收藏!

Epsilon – exploration versus exploitation

Your agent's exploration rate epsilon also ranges from zero to one. As the agent explores its environment, it learns that some actions are better to take than others, but what about states and actions that it hasn't seen yet? We don't want it to get stuck on a local maximum, taking the same currently highest-valued actions over and over when there might be better actions it hasn't tried to take yet.

When you set your epsilon value, there will be a probability equal to epsilon that your agent will take a random (exploratory) action, and a probability equal to 1-epsilon that it will take the current highest Q-valued action for its current state. As we step through a full Q-table update example in the SARSA and the cliff-walking problem section, we'll see how the value that we choose for epsilon affects the rate at which the Q-table converges and the agent discovers the optimal solution. 

As the agent gets more and more familiar with its environment, we want it to start sticking to the high-valued actions it's already discovered and do less exploration of the states it hasn't seen. We achieve this by having epsilon decay over time as the agent learns more about its environment and the Q-table converges on its final optimal values.

There are many different ways to decay epsilon, either by using a constant decay factor or basing the decay factor on some other internal variable. Ideally, we want the epsilon decay function to be directly based on the Q-values that we've already discovered. We'll discuss what this means in the next section.

主站蜘蛛池模板: 宽甸| 浦县| 杨浦区| 洛浦县| 卓资县| 松阳县| 乌拉特中旗| 溆浦县| 北流市| 微博| 华池县| 贺州市| 达拉特旗| 南宫市| 平塘县| 高陵县| 曲松县| 二手房| 威远县| 古田县| 广平县| 磐石市| 淳化县| 汉沽区| 成都市| 邹平县| 紫阳县| 商都县| 洪泽县| 资源县| 舒兰市| 济南市| 桦甸市| 张家港市| 沁阳市| 三穗县| 富锦市| 桂林市| 鄂温| 文安县| 北票市|