Course

Unit 1

Introduction- Reinforcement Learning as MDP- Learnable Functions – Deep Reinforcement Learning Algorithms overview-Deep Learning for Reinforcement Learning- Reinforcement Learning and Supervised Learning. Reinforcement Learning Environment Design: States-Actions-Rewards-Transition Function.

Unit 2

REINFORCE: Policy- The Objective Function-The Policy Gradient- Policy Gradient Derivation- Monte Carlo Sampling- REINFORCE Algorithm-Implementing REINFORCE. SARSA: The Q- and V-Functions -Temporal Difference Learning- Intuition for Temporal Difference Learning- Action Selection in SARSA- Exploration and Exploitation- SARSA Algorithm- On-Policy Algorithms- Implementing SARSA. Deep Q-Networks (DQN): Learning the Q-Function in DQN-Action Selection in DQN-The Boltzmann Policy-Experience Replay- DQN Algorithm- Implementing DQN

Unit 3

Advantage Actor-Critic (A2C): The Actor- The Critic- The Advantage Function- Learning the Advantage Function- A2C Algorithm- Implementing A2C. Proximal Policy Optimization (PPO): Surrogate Objective-Proximal Policy Optimization (PPO) – PPO Algorithm- Implementing PPO

Prerequisite(s): Nil Course

Objectives

• To introduce Reinforcement Learning??
• To introduce techniques used for training artificial neural networks?
• To enable design of deep learning models for classification and sequence analysis?

Course Outcomes

• CO1: Able to understand the mathematical basics of reinforcement learning.
• CO2: Able to understand the working of different types of Reinforcement Learning Agents.
• CO3: Able to formulate a problem as a Reinforcement Learning problem
• CO4: Able to implement Reinforcement Learning algorithms

CO – PO Mapping? 

Text Book(s) 

1. Laura Graesser, Wah Loon Keng, Foundations of Deep Reinforcement Learning: Theory and Practice in Python, Addison-Wesley; 2022.
2. Phil Winder, Reinforcement Learning: Industrial Applications of Intelligent Agents (Grayscale Indian Edition), O’Reilly Media; 2020.
3. Richard S. Sutton, Andrew G. Barto, Francis Bach, Reinforcement Learning: An Introduction, 2^nd Edition, MIT Press; 2018.  

Reference Books 

1. Charu C. Aggarwal, Neural Networks and Deep Learning-A Textbook, Springer Cham; 2018.  
2. Nithin Buduma, Nikhil Buduma, Joe Papa, Fundamentals of Deep Learning, 2^nd Edition, O’Reilly Media;  
3. M Gopal, Deep Learning, Pearson Education India; 2022.