Course

Recap of Deep neural networks, Variational Autoencoders (VAEs), and attention mechanisms. Generative Adversarial Networks (GANs) GAN components, DCGANs using convolutional layers, control your GAN and build conditional GAN. Use GANs for data augmentation and privacy preservation, GANs applications, Pix2Pix and CycleGAN for image translation. Comparison of generative models, FID method to assess GAN fidelity and diversity, Bias in GAN, StyleGAN techniques

Principles of Transformers, Word embeddings, Introduction to Large Language Models – Decoder-only LLMs: A deep dive into GPT, Encoder Only LLMs-BERT, Prompting – different prompting strategies – Instruction tuning – fine tuning – parameter efficient fine-tuning – quantized fine tuning. Small Language models. Training LLMs using reinforcement Learning.

Evaluating LLMs: Benchmarks, evaluation frameworks and popular leaderboards. Applications/Case study of Large Language Models – Text Generation, Translation, and Summarization – Question Answering, Sentiment Analysis, Chatbots, Application of LLMs in healthcare and Code generation, latest advancements in LLMs.

Preamble

This course offers an in-depth exploration of Generative AI, focusing on Generative Adversarial Networks (GANs) and Large Language Models (LLMs). Generative AI represents a cutting-edge field that aims to create models capable of generating realistic data, encompassing images, music, and text. Students will gain an understanding of generative models’ principles and historical impact. The course covers GANs’ architecture, and types like DCGANs and StyleGANs, and applications It also addresses performance challenges and improvement techniques. For LLMs, the course examines models like GPT-4 and BERT, their training on extensive text data, and applications in NLP tasks such as text generation, translation, and summarization. Students will learn to implement and fine-tune LLMs for specific purposes.

Course Objectives

• To understand basic principles of Artificial Intelligence.
• To understand the basic areas of artificial intelligence including problem solving, knowledge representation, reasoning, decision making, planning, perception and action.
• To understand automatic learning methods in artificial intelligence

Course Outcomes

Prerequisites

• Basic knowledge of Deep Learning
• Familiarity with Python programming

CO-PO Mapping

Evaluation Pattern – 70:30

• Midterm Exam – 20%
• Continuous Assessment – Theory – 10%
• Continuous Assessment – Lab – 40%
• End Semester Exam – 30%

Text Book / References

1. Generative AI in Practice:Bernad Marr, Wiley,March 2024
2. Hands-On Large Language Models by Jay Alammar, Maarten Grootendorst, December 2024 Publisher(s): O’Reilly Media.
3. Ian Goodfellow, YoshuaBengio and Aeron Courville,” Deep Learning”, MIT Press,2016
4. ‘Deep Learning for Natural Language Processing: Develop Deep Learning Models for your Natural Language Problems (Ebook)’, Jason Browlee, Machine Learning Mastery, 2017.
5. Getting Started with Google BERT: Build and train state-of-the-art natural language processing models using BERT by Sudharsan Ravichandiran, Packt Publishing Limited January 2021.
6. Comprehensive Overview of LLMs- A survey paper: https://arxiv.org/pdf/2307.06435. Other research papers on LLM
7. ‘Speech & language processing’ , Daniel Jurafsky, James H Martin, preparation [cited 2020 June 1] Available from: https://web.stanford.edu/~jurafsky/slp3/