Webpage of the Sequential Decision Making class

The slides of the course and notebooks for the practical sessions will be made online before the class.

The course will be illustated by practical sessions in Python, using jupyter notebook. We will notably use the rlberry library developed in the Scool team.

• Lecture 1: From Batch to Sequential Learning.
• Lecture 2: Stochastic Multi-Armed Bandits.
• Practical session 1: Bandit algorithms.
• Lecture 3: Beyond Classical Bandits.
• Practical session 2: Contextual Bandits.
• Lecture 4: Markov Decision Processes. Code.
• Lecture 5: Dynamic Programming.
• Lecture 6: Reinforcement Learning Algorithms.
• Practical session 3: Dynamic Programming and Q-Learning.
• Lecture 7: Reinforcement learning with Function Approximation.
• Practical session 4: Fitted Q Iteration on a Gridworld. Notebook or Google Colab.
• Lecture 8: Beyond Value Based Methods.
• Lecture 9: Monte Carlo Tree Search.
• Lecture 10: Exploration in RL.
• Practical session 5: Exploration on Mountain Car. Notebook or Google Colab.

References

• Bandit Algorithms. Tor Lattimore and Csaba Szepesvari (2019).
• Reinforcement Learning. Richard Sutton and Andrew Barto (2018 edition).
• Reinforcement Learning Algorithms. Csaba Szepesvari (2009).
• Markov Decision Processes. Martin Puterman (1994).
• Nice lecture notes written by several colleagues: Rémi Munos, Alessandro Lazaric and Aurélien Garivier.
• Material from the 1st Reinforcement Learning Summer School.