Course

Introduction to Machine Learning and its various types, Motivation and Introduction to Reinforcement Learning, Multi arm Bandits, Markov Decision Process, Value functions; Dynamic programming: Policy evaluation and improvement, Value iteration and Policy iteration algorithms

Value prediction problems: Temporal difference learning in finite state spaces Algorithms for large state spaces Control: Closed loop interactive learning, online and active learning in bandits, Q learning in finite MDPs, Q learning with function approximation,

On policy approximation of action values: Value Prediction with Function Approximation, Gradient- Descent Methods, Policy approximation: Actor critic methods, Monte Carlo Methods: Monte-carlo prediction, estimation of action values, off policy prediction via importance sampling,

Preamble

Artificial intelligence techniques face challenges in learning from dynamic environment with minimal data. This course deals with various algorithms to learn such an environment. Elements of Reinforcement Learning, Model Based Learning, Temporal Difference Learning and Policy Search are the main focus topics of this course.

Course Objectives

• Good understanding of various types of algorithms for Reinforcement Learning
• Be able to design an RL system

Course Outcomes

Prerequisites

• Machine Learning basics
• Probability Theory basics
• Programming basics

CO-PO Mapping

Evaluation Pattern – 70:30

• Midterm Exam – 20%
• Lab Assignments – 25%
• Project – 25%
• End Semester Exam – 30%

Text Book / References

1. Sutton and Barto, Reinforcement Learning: An Introduction, The MIT Press Cambridge, Massachusetts London, England, 2015
2. Csaba Szepesvari, Algorithms for Reinforcement Learning, Morgan & Claypool, United States, 2010
3. Xiao, Z. (2024). Reinforcement Learning: Theory and Python Implementation. Springer, 2024, ISBN 9789811949326.
4. Li, S. E. Reinforcement Learning for Sequential Decision and Optimal Control. 2024, ISBN 9789811689093.