Course

Unit 1

Introduction to Information Theory: Modeling of information sources – uncertainty and information – Entropy – information measures for continuous random variable – source coding theorem – Kraft inequality – source coding algorithms: Huffman coding – arithmetic coding – Lempel-Ziv algorithm. Rate distribution function – Entropy rate of stochastic process – Modeling of communication channels – Binary symmetric channel – binary erasure channel – channel capacity – noisy channel coding theorem – Information capacity theorem – Shannon’s limit – bounds on communication.

Unit 2

Linear block codes – structure – matrix description – Hamming codes – Standard array arithmetic of Galois fields – Integer ring – finite fields based on integer ring – polynomial rings – finite fields based on polynomial rings – primitive elements – Structure of finite fields – Cyclic codes – Structure of cyclic codes – encoding and decoding of cyclic codes.

Unit 3

BCH codes – Generator polynomials in terms of minimal polynomial – Decoding of BCH codes – Reed-Solomon codes – Peterson-Gorenstein – Zierler decoder – Introduction to low density parity check codes – Convolutional Codes: Introduction to Convolutional Codes – Basics of Convolutional Code encoding and decoding – Sequential decoding – Viterbi decoding- Introduction to Turbo codes.

Pre Requisite(s): Communication Theory

Prerequisite(s): Nil Course

Course Objectives

• To provide foundation in information theory that gives understanding about quantitative measures of information and allows analyzing and characterizing the fundamental limits of communication systems
• To provide an insight of Galois fields and primitive polynomials
• To provide an introduction to traditional and modern coding theory and analyzing performance of different decoding algorithm

Course Outcomes

CO1: Able to understand the Information theory fundamentals and the fundamental limits of communication system 

CO2: Able to analyze the basic types of codes and understand the source coding algorithms 

CO3: Able to derive the channel capacity of communication channel models 

CO4: Able to understand the method of encoding and decoding technique of linear block code, cyclic code, convolutional codes 

CO5: Able to carry out implementation of different source coding and channel coding algorithms 

CO – PO Mapping

Text Book(s)

1. Ranjan Bose, “Information Theory, Coding and Cryptography”, Tata McGraw-Hill Publishing Company ltd., New Delhi, Second Edition, 2008.
2. Thomas M Cover and Joy A Thomas, “Elements of Information Theory”, Second Edition John Wiley, 2006.

Reference(s)

1. J.Proakis, M. Salehi, “Fundamentals of Communications systems”, Pearson Education, Second Edition, 2005.
2. Shu Lin and Daniel J.Costello, “Error Control Coding –Fundamentals and Applications”, Pearson, Second Edition, 2004.