Jan 5th 2015

Information Theory (Coursera)

This course is an introduction to information theory, which emphasizes fundamental concepts as well as analytical techniques. Specific topics include: Information Measures, The I-Measure, Zero-Error Data Compression, Weak Typicality, Strong Typicality, Discrete Memoryless Channels, etc.

The lectures are based on the first 11 chapters of Prof. Raymond Yeung’s textbook entitled Information Theory and Network Coding (Springer 2008).

- Chapter 1 The Science of Information

- Chapter 2 Information Measures

- Chapter 3 The I-Measure

- Chapter 4 Zero-Error Data Compression

- Chapter 5 Weak Typicality

- Chapter 6 Strong Typicality

- Chapter 7 Discrete Memoryless Channels

- Chapter 8 Rate-Distortion Theory

- Chapter 9 The Blahut-Arimoto Algorithms

- Chapter 10 Differential Entropy

- Chapter 11 Continuous-Valued Channels

At the completion of this course, the student should be able to:

- Demonstrate knowledge and understanding of the fundamentals of information theory.

- Appreciate the notion of fundamental limits in communication systems and more generally all systems.

- Develop deeper understanding of communication systems.

- Apply the concepts of information theory to various disciplines in information science.

Suggested Readings: