Unit 1

Introduction, Simple Linear regression, Multiple linear regression, Extensions of the linear model, Classification: overview, Logistic regression, Linear discriminant analysis, comparison of classification methods.

Unit 2

Resampling methods: Cross validation and the bootstrap, Linear model selection and Regularization: Subset selection, Shrinkage methods, Dimension reduction methods, Considerations in high dimensions.

Unit 3

Polynomial regression, step functions, basis functions, regression splines, smoothing splines, local regression, generalized additive models for regression and classification problems, Regression trees, Classification trees, comparison of trees and linear models, Bagging, Random Forests, Boosting.

Unit 4

Support Vector Machines: Hyperplane, Maximum Margin Classifier, Support Vector Classifiers, Support Vector Machines, One vs One Classification and One vs All Classification, Relationship to Logistic Regression. Unsupervised Learning: Principal Component Analysis and its applications, K- Means Clustering and Hierarchical Clustering.

Unit 5

Neural Networks: Introduction, Projection Pursuit Regression, Neural Networks, Fitting Neural Networks, Some issues in Training Neural Networks-Starting Values, Overfitting, Scaling of the Inputs, Number of Hidden Units and Layers, Multiple Minima.

Course Objectives
CO1: Understand the domain of machine learning with respect to the regression and classification and its huge potential for providing solutions to real-life problems.
CO2: Have a good understanding of the fundamental issues and challenges in basic machine learning algorithms in terms of data, model selection, and complexity.
CO3: Understand the problem of Curse of Dimensionality and different methods to tackle it.
CO4: Understand the mathematical framework for machine learning (both supervised and un- supervised learning) and methods to tackle under fitting & overfitting.
CO5: Learn the motivation and theory behind learning an artificial neural networks for machine learning applications.
CO6: Be able to design and implement right machine learning algorithm for a given real-world problem.

Course Outcomes
CO1: Have a good understanding of the fundamental issues and challenges of machine learning data, model selection, model complexity, etc.
CO2: Have an understanding of the strengths and weakness of many popular machine-learning approaches.
CO3: To understand the mathematical relationships within and across Machine Learning algorithms and the paradigms of supervised and un-supervised learning
CO4: Be able to design and implement various machine-learning algorithms in a range of real-world applications

Machine Learning Lab I to be performed

1. Introduction to R: Basic Commands, Graphics, Indexing Data, Loading Data.
2. Linear Regression: Libraries, Simple Linear Regression, Multiple Linear Regression.
3. Logistic Regression, Linear Discriminant Analysis, Quadratic Discriminant Analysis.
4. Cross Validation and Bootstrap, Validation set approach, Leave-One-Out Cross Validation
5. K-Fold Cross Validation, Bootstrap.
6. Subset Selection Methods, Best Subset Selection, Forward and Backward Stepwise Selection.
7. Ridge Regression and the Lasso.
8. Principal Components Regression, Partial Least Squares.
9. Non-Linear Modelling, Polynomial Regression and Step Functions, Splines, GAMS.
10. Decision Trees, Fitting Classification Trees and Regression Trees, Bagging and Random

Forests Textbooks:

1. G. James, R. Tibshirani, An Introduction to Statistical Learning: with applications in R, Springer.
2. T. Hastie, R. Tibshirani, Elements of Statistical Learning: Data mining, Inference and Predicton, Springer.

References:

1. Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press.