Course

Introduction to Stochastic Processes: Overview of stochastic processes – Introduction to conditional expectations. Conditional Expectations: Definition and properties of conditional expectations – Conditional expectations in the context of stochastic processes – Bayesian inference and decision theory.

Discrete Time Martingales: Definition and properties of martingales – Stopping times and optional stopping theorem – Discrete Time Markov Chains: Definition and properties of Markov chains – Classification of states, ergodicity – Stationary distributions and convergence – Markov decision processes (MDPs) and modelling sequential decision problems.

Poisson Process: Interarrival times and memory lessness – Poisson processes in queuing theory and AI applications – Brownian Motion: – Stochastic integration and differential equations – Geometric Brownian motion and its applications in finance and AI. Elements of Ito Stochastic Calculus: Ito integral and Ito’s lemma – Stochastic differential equations and models in continuous time.

Preamble

This course provides an in-depth exploration of stochastic processes with a focus on their applications in artificial intelligence.

Course Objectives

• Gain a foundational understanding of finite Markov chains in discrete time.
• Develop a comprehensive understanding of martingales in discrete time, covering fundamental theoretical concepts and their practical implications.
• Explore the essential principles of stochastic processes in continuous time, focusing on key examples such as the Poisson process and Brownian motion.
• Acquire insights into elementary stochastic calculus applied to Brownian motion, enhancing comprehension of its theoretical underpinnings.

Course Outcomes

Prerequisites

• None.

CO-PO Mapping

Evaluation Pattern – 70:30

• Midterm Exam – 20%
• Quiz – 20%
• Lab Continuous Assessment – 30%
• End Semester Exam / Project – 30%

Text Book / References

1. Lawler, G. F. (2006). Introduction to Stochastic Processes (2nd ed.). Cambridge University Press.

1. Del Moral, P. (2014). Stochastic Processes: From Applications to Theory, Springer.

3. Shreve, S. E. (2004). Stochastic Calculus for Finance II: Continuous-Time Models (1st ed.). Springer.
4. Brzezniak, Z., & Zastawniak, T. (1999). Basic Stochastic Processes (1st ed.). Springer.
5. Taylor, H. M., & Karlin, S. (2010). An Introduction to Stochastic Modeling (4th ed.). Academic Press.