Course

Overview: Robotics introduction, historical perspective on AI and Robotics, Uncertainty in Robotics Reinforcement Learning: Basic overview, examples, elements, Tabular Solution Methods – Multi- armed bandits, Finite Markov decision process, Dynamic programming (Policy Evaluation, Policy Iteration, Value Iteration), Monte Carlo Methods, Temporal-Difference Learning (Q-learning, SARSA).

Approximate Solution Methods – On-policy Prediction with Approximation, Value function approximation, Non-linear function approximation, Reinforcement Learning in robotics, Recursive state estimation: Robot Environment Interaction, Bayes filters, Gaussian filters – The Kalman filter, The Extended Kalman Filter, The information filter, The particle filter Robot motion: Velocity Motion Model, Odometry Motion Model, Motion and maps.

Measurement: Beam Models of Range Finders, Likelihood Fields for Range Finders, Correlation- Based Sensor Models, Feature-Based Sensor Models, Overview of POMDP.

Preamble

In recent years, several off-the-shelf robots have become available and some of them have made their way into our homes, offices, and factories. The ability to program robots has therefore become an important skill; e.g., for robotics research as well as in several companies (such as iRobot, ReThink Robotics, Willow Garage, medical robotics, and others). We study the problem of how a robot can learn to perceive its world well enough to act in it, to make reliable plans, and to learn from its own experience. The focus will be on algorithms and machine learning techniques for autonomous operation of robots.

Course Objectives

• To understand the principles of reinforcement learning which is one of the key learning techniques for robots.
• To understand uncertainty handling in robotics through probabilistic approaches.
• To learn how measurements work for robots.

Course Outcomes

Prerequisites

• Data Structures and Algorithms
• Foundation of Data Science
• Linear Algebra and Optimization
• Principles of AI and ML

Evaluation Pattern – 70:30

• Midterm Exam – 20%
• Lab Assignments – 25%
• Project – 25%
• End Semester Exam – 30%

Text Book / References

1. Sebastian Thrun, Wolfram Burgard, Dieter Fox, Probabilistic Robotics, MIT Press 2005
2. Richard S. Sutton, Andrew G. Barto, Reinforcement Learning: An Introduction”, Second edition, MIT Press, 2018
3. Jens Kober, Jan Peters, Learning Motor Skills: From Algorithms to Robot Experiments, Springer, 2014
4. Francis X. Govers, Artificial Intelligence for Robotics, Packt, 2018