Unit 1

Neural Networks: Basic concepts of artificial neurons, single and multilayer perceptron, perceptron learning algorithm, activation functions, loss function. Feed-forward Network Functions – Network Training – Backpropagation – Parameter optimization – Hyperparameter Tuning

Unit 2

Regularization for Deep Learning: Dataset Augmentation – Noise Robustness – Early Stopping – Dropout – Sparse Representation – Bagging and Other Ensemble Methods – Semi-Supervised Learning – Multi-Task Learning – Parameter Tying and Parameter Sharing

Unit 3

Convolutional Networks: The Convolution Operation – Motivation – Pooling – Convolution and Pooling as an Infinitely Strong Prior – Variants of the Basic Convolution Function – ConvNet Architectures – Transfer learning

Unit 4

Graph representation learning – Node embedding models – Knowledge graph embedding models – Graph neural networks – Graph neural network architectures – Graph neural networks and knowledge graphs

Course Objectives:

• Develop a comprehensive understanding of neural networks, covering linear and logistic regression, artificial neurons, single and multi-layer perceptrons, activation functions, and feed-forward network functions.
• Explore regularization techniques for deep learning, including dataset augmentation, noise robustness, semi-supervised learning, multi-task learning, early stopping, and ensemble methods.
• Understand convolutional networks, including the convolution operation, pooling, variants of the basic convolution function, and famous convnet architectures like AlexNet, VGG, ResNet, and EfficientNet.

Course Outcomes:
After completing this course, students should be able to
CO1: Implement deep neural networks and Convolutional Neural Networks for solving problems.
CO2: Employ regularization techniques in deep learning to enhance model robustness and generalization.
CO3: Use transfer learning concepts to solve problems.
CO4: Implement Graph Neural Network to learn the structural relationship in data.

CO-PO mapping

1. Ian Good Fellow,YoshuaBengio, and Aaron Courville. Deep Learning, MIT Press, 2016.
2. C. M. Bishop. Pattern Recognition and Machine Learning, Springer, 2006.
3. Nikhil Buduma. Fundamentals of Deep Learning, First Edition, O’REILLY Media, 2017.
4. M. Mohri, A. Rostamizadeh, and A. Talwalkar. Foundations of Machine Learning, MITPress, 2012.
5. Kevin P. Murphy. Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
6. D. Barber. Bayesian Reasoning and Machine Learning, Cambridge University Press, 2012.
7. https://nptel.ac.in/courses/106105152/
8. William L. Hamilton (2020), Graph Representation Learning, Synthesis Lectures on AI and ML, Vol.  14, No. 3.