Reinforcement Learning with OpenAI Gym - Value Iteration Frozen Lake - Code Heroku
- 1. Pleaseturn off your webcam If you arejoining from a mobile phone besureto click on Join via Device Audio Weare waiting for other participants to join Wewill begin at 4:30 PM IST
- 2. Mihir Thakkar Founderand Instructor [email protected] Reinforcement Learning with OpenAIGym
- 3. SESSION OBJECTIVES • Quick Recap • Bellman’sEquations • Value Iterationin OpenAI Gym
- 4. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
- 5. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning RL Problem
- 6. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Q Function
- 7. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
- 8. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning
- 9. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Quiz Given the following Reward Table,estimatethe value of Q(A3,East)
- 10. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Quiz Given the following Reward Table,estimatethe value of Q(B3,North)
- 11. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Given a Value Function Extract Policy
- 13. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Value Iteration Algorithm
- 14. Reinforcement Learning Challenges • Access to the Environment • Delayed Reward (Temporal Credit RiskAssignment) • High Cost Actions • Distribution of data changes by the choice of actions you take • Efficient state representations? • Good Rewards functions?
- 15. Thanks
- 17. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Markov Decision Process (MDP)
- 18. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Multi Arm Bandit • Unknown Reward Distribution • Deterministic Actions • Objective:FindSequence of actions whichwillmaximizetotal reward
- 19. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Exploration Vs Exploitation To approximatevaluesof actionsAgent must choose actionsthatare non- optimalto start with. Once an agent has approximatedthe values, it can greedily pick the highest value action.
- 20. www.codeheroku.com Introduction toMachine Learning –Reinforcement Learning Iterative Averaging
Editor's Notes
- #3: In general we saw that RL deals with Making Decisions under uncertainty which core to understand intelligence and simulate it RL also deals with sequence of actions
- #4: Often see a huge gap in the therotical approach which is taught in universities and practical implementations. In this entire course if you have noticed we are trying the bridge that gap
- #5: Y= F(X) F(X) What Happens when we do not know the consequence for our immediate actions Contrast with Supervised ML Delayed Rewards / Sparse Signal RL deals with uncertaininty in envrionments / actions /observations
- #15: Good Rewards – Conversational agent, Treatment pathway for patients