Course

Module 1

Module 1: Learning (7.5 hours)

1. The bag of words, Learning algorithms -Naıve Bayes, Discriminative learning, Loss functions and large-margin classification, Logistic regression, Optimization, Additional topics in classification, Feature selection by regularization, Other views of logistic regression.
2. Feed forward neural networks, designing neural networks, Learning neural networks, Convolutional neural networks.
3. Sentiment and opinion analysis, Word sense disambiguation, Design decisions for text classification, evaluating classifiers, Building datasets.
4. Unsupervised learning, Applications of expectation-maximization, Semi supervised learning, Domain adaptation, other approaches to learning with latent variables

Module 2

Module 2: Sequences and Trees (7.5 hours)

1. N-gram language models, Smoothing and discounting, recurrent neural network language models, evaluating language models, Held-out likelihood, Perplexity, Out-of-vocabulary words.
2. Sequence labeling as classification, Sequence labeling as structure prediction, The Viterbi algorithm, Hidden Markov Models, Discriminative sequence labeling with features, neural sequence labeling, unsupervised sequence labeling.
3. Part-of-speech tagging, Morph syntactic Attributes, Named Entity Recognition, Tokenization, Code switching, Dialogue acts.
4. Regular languages, Context-free languages, mildly context-sensitive languages.
5. Deterministic bottom-up parsing, Ambiguity, Weighted Context-Free Grammars, Learning weighted context-free grammars, Grammar refinement, beyond context-free parsing.
6. Dependency grammar, Graph-based dependency parsing, Transition-based dependency parsing, Applications.

Module 3

Module 3: Meaning (7.5 hours)

1. Meaning and denotation, Logical representations of meaning, Semantic parsing and the lambda calculus, learning semantic parsers.
2. Semantic roles, Semantic role labelling, Abstract Meaning Representation.
3. The distributional hypothesis, Design decisions for word representations, Latent semantic analysis, Brown clusters, neural word embed dings, evaluating word embedding, Distributed representations beyond distributional statistics, Distributed representations of multiword units.
4. Forms of referring expressions, Algorithms for coreference resolution, Representations for coreference resolution, Evaluating coreference resolution.
5. Segments, Entities and reference, Relations.

Module 4

Module 4: Applications (7.5 hours)

1. Entities, Relations, Events, Hedges, denials, and hypotheticals, Question answering and machine reading.
2. Machine translation as a task, Statistical machine translation, neural machine translation Decoding, Training towards the evaluation metric.
3. Data-to-text generation, Text-to-text generation, Dialogue.

Course Description

This course aims to provide a comprehensive view of building real-world natural language processing (NLP) applications. The diverse applications of NLP are based on a common set of ideas, drawing on algorithms, linguistics, logic, statistics, and more. The goal of this course is also to provide an overview of these foundations. The course covers the most important topics in the field, keeping a balance between theory and practice. This course focuses on a compact set of methods unified by the concepts of learning and search, which can solve a remarkable number of problems in NLP. This course teaches how these methods work and how they can be applied to a wide range of tasks: document classification, word sense disambiguation, part-of-speech tagging, named entity recognition, parsing, coreference resolution, relation extraction, discourse analysis, language modeling, machine translation.

Course Outcomes& Learning levels

This course aims to provide a comprehensive background in Natural Language Processing. At the end of this course, the students

1. Will be able to understand the wide spectrum of problem statements, tasks, and solution approaches within NLP
2. Will be able to implement and evaluate different NLP applications and apply machine learning and deep learning methods for this process.
3. Evaluate various algorithms and approaches for the given task, dataset, and stage of the NLP product.
4. Understand best practices, opportunities, and the roadmap for NLP from a business and product leader’s perspective.

Evaluation Pattern

*Based on assignments / Tests / Quizzes / Case Studies / Projects / Term paper / Field visit report.

1. Akmajian, A., R. A. Demers, A. K. Farmer, and R. M. Harnish (2010). Linguistics: An introduction to language and communication (Sixth ed.). Cambridge, MA: MIT press.
2. Arora, S. and B. Barak (2009). Computational complexity: a modern approach. Cambridge University Press