Unit 1

Machine learning Basics and introduction, Capacity, Overfitting and under fitting, Hyper parameters, Estimator, Bias and Variance, Maximum likelihood estimation, Stochastic Gradient descent

Unit 2

Deep feedforward networks, Learning XOR, Hidden units, Architecture design, Backpropagation

Unit 3

Regularization, L1 and L2 regularization, Noise robustness, Semi supervised learning, Parameter typing and sharing, Sparse representation, Dropout

Unit 4

Optimization, Challenges in neural network optimization, Parameter initialization strategy, Adaptive learning rates, Optimization algorithms

Unit 5

Convolution operator, Pooling, Structured outputs, Efficient convolution algorithms, Unsupervised features, Convolution Neural networks, Recurrent Neural Networks, Encoder-decoder, LSTM and memory architectures, Optimization for long term dependency

CO1: To understand the computing capacity of single layer neural networks, and the need for multi- layer neural networks.
CO2: Learn to tackle the under-fitting, overfitting, and getting into local optimal solutions when learning an artificial neural network.
CO3: Learn about the deep neural networks, CNN to understand how it differ from a deep traditional FFN both in terms of the number of parameters to be learned and in terms of the learning by back- propagation.
CO4: Learn to design and use CNN both as a stand-alone classifier and in transfer learning settings.
CO5: Learn the necessary theory behind different recurrent neural networks and its applications to sequential data analysis.

Machine Learning Lab II to be performed

1. Support Vector Classifier, Support Vector Machine, ROC Curves, SVM with Multiple Classes.
2. Principal Component Analysis and Clustering.
3. Overfitting and Underfitting Bias and Variance.
4. Gradient Descent Algorithm.
5. Backpropagation.
6. Neural Network Optimization 1.
7. Neural Network Optimization 2.
8. Convolution Neural networks.
9. Recurrent Neural Networks.
10. LSTM and memory architectures.

Textbooks

1. Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press. (Chapters 5-10).
2. Simon J.D. Prince, Understanding Deep Learning, MIT Press Francois Chollet , Deep Learning with Python
3. Sebastian Raschka, Machine Learning with PyTorch and Sci-kit Learn