Course

PROFESSIONAL ELECTIVES

Electives Electives in Data Science

Unit I

Introduction to Spark: Spark Architecture, Spark Jobs and APIs. Resilient Distributed Datasets- Creating RDDs, Transformation, Actions. Dataframes- Python to RDD communications, Creating Dataframes, Dataframe queries. MLlib -Loading and Transforming the data. Implementation of Machine Learning algorithms such as Classification and Clustering using the MLlib.

Unit II

Approaches to Modelling- Importance of Words in Documents – Hash Functions- Indexes – Secondary Storage -The Base of Natural Logarithms – Power Laws – Map Reduce. Finding similar items: Shingling – LSH – Distance Measures. Mining Data Streams: Stream data model – Sampling data – Filtering streams. Link Analysis: Page Rank, Link Spam.

Unit III

Frequent Item Sets: Market Basket Analysis, A-Priori Algorithm – PCY Algorithm. Recommender Systems, Dimensionality Reduction -SVD, Big data Clustering: Clustering in Non-Euclidean Spaces, BFR, CURE. Structured Streaming: Spark Streaming, Application dataflow

Pre-Requisite(s): 23CSEXXX Machine Learning

Course Objectives

• To understand various scalable machine learning algorithms to solve big data problems.
• To understand the SPARK architecture.
• To implement Machine Learning algorithms using PySpark.

Course Outcomes

CO1: Understand how Machine learning algorithms are made scalable to solve big data problems.

CO2: Implement scalable Machine Learning algorithms using PySpark.

CO3: Apply and compare different strategies for big data analytics using various machine learning algorithms.

CO4: Understand Streaming algorithms to analyze voluminous and high dimensional data.

CO-PO Mapping

Evaluation Pattern: 70:30

*CAT – Can be Quizzes, Assignments, and Reports

*CAL – Can be Lab Assessments, Project, and Report

**End Semester can be theory examination/ lab-based examination/ project presentation

Textbook(s)

AnandRajaRaman, Jure Leskovec and J.D. Ullman, “Mining of Massive Data sets”, e-book, Publisher, 2014.

Reference(s)

Viktor Mayer-Schönberger, Kenneth Cukier, “Big Data: A Revolution That Will Transform How We Live, Work, and Think”, Houghton Mifflin Harcourt, 2013.

Bill Chambers, Matei Zaharia, “Spark: The Definitive Guide”, O’Reilly Media Inc,2018, ISBN: 9781491912218. Kevin P. Murphey, “Machine Learning, a Probabilistic Perspective”, The MIT Press Cambridge, Massachusetts, 2012.