官术网_书友最值得收藏!

Epsilon – exploration versus exploitation

Your agent's exploration rate epsilon also ranges from zero to one. As the agent explores its environment, it learns that some actions are better to take than others, but what about states and actions that it hasn't seen yet? We don't want it to get stuck on a local maximum, taking the same currently highest-valued actions over and over when there might be better actions it hasn't tried to take yet.

When you set your epsilon value, there will be a probability equal to epsilon that your agent will take a random (exploratory) action, and a probability equal to 1-epsilon that it will take the current highest Q-valued action for its current state. As we step through a full Q-table update example in the SARSA and the cliff-walking problem section, we'll see how the value that we choose for epsilon affects the rate at which the Q-table converges and the agent discovers the optimal solution. 

As the agent gets more and more familiar with its environment, we want it to start sticking to the high-valued actions it's already discovered and do less exploration of the states it hasn't seen. We achieve this by having epsilon decay over time as the agent learns more about its environment and the Q-table converges on its final optimal values.

There are many different ways to decay epsilon, either by using a constant decay factor or basing the decay factor on some other internal variable. Ideally, we want the epsilon decay function to be directly based on the Q-values that we've already discovered. We'll discuss what this means in the next section.

主站蜘蛛池模板: 仁寿县| 东乡| 鹤山市| 清远市| 庆安县| 大城县| 宜昌市| 延边| 徐汇区| 吕梁市| 福建省| 洪泽县| 青阳县| 友谊县| 闸北区| 高密市| 彰武县| 翁牛特旗| 华蓥市| 隆安县| 嘉义县| 鹿邑县| 江华| 庆元县| 宜君县| 马山县| 梅河口市| 宜昌市| 孟津县| 玉屏| 嫩江县| 霍城县| 乌拉特前旗| 怀安县| 青浦区| 彩票| 四子王旗| 正蓝旗| 方山县| 阿尔山市| 靖边县|