- Hands-On Q-Learning with Python
- Nazia Habib
- 274字
- 2021-06-24 15:13:12
Epsilon – exploration versus exploitation
Your agent's exploration rate epsilon also ranges from zero to one. As the agent explores its environment, it learns that some actions are better to take than others, but what about states and actions that it hasn't seen yet? We don't want it to get stuck on a local maximum, taking the same currently highest-valued actions over and over when there might be better actions it hasn't tried to take yet.
When you set your epsilon value, there will be a probability equal to epsilon that your agent will take a random (exploratory) action, and a probability equal to 1-epsilon that it will take the current highest Q-valued action for its current state. As we step through a full Q-table update example in the SARSA and the cliff-walking problem section, we'll see how the value that we choose for epsilon affects the rate at which the Q-table converges and the agent discovers the optimal solution.
As the agent gets more and more familiar with its environment, we want it to start sticking to the high-valued actions it's already discovered and do less exploration of the states it hasn't seen. We achieve this by having epsilon decay over time as the agent learns more about its environment and the Q-table converges on its final optimal values.
There are many different ways to decay epsilon, either by using a constant decay factor or basing the decay factor on some other internal variable. Ideally, we want the epsilon decay function to be directly based on the Q-values that we've already discovered. We'll discuss what this means in the next section.
- 大學計算機基礎:基礎理論篇
- 空間機器人遙操作系統及控制
- 極簡AI入門:一本書讀懂人工智能思維與應用
- 機器人創新實訓教程
- 21天學通Java Web開發
- CompTIA Linux+ Certification Guide
- 統計學習理論與方法:R語言版
- Machine Learning with Apache Spark Quick Start Guide
- 智能生產線的重構方法
- Salesforce Advanced Administrator Certification Guide
- 嵌入式GUI開發設計
- 從零開始學JavaScript
- Linux系統管理員工具集
- Dreamweaver+Photoshop+Flash+Fireworks網站建設與網頁設計完全實用
- 青少年VEX IQ機器人實訓課程(初級)