Information theory for physicists

Academic year: 
2014-2015
Date: 
27/11/2014 - 14:00

Information theory for physicists

27 November, 4 December, 11 December

2–3 pm, Physics Library, Physics West, University of Birmingham

The course will introduce key ideas from information theory. Information theory explains how the quantify information or uncertainty about a system, and how much you learn from an inference. Information theory has many applications, and is particularly useful in data analysis; we shall also see how it is connected to thermodynamics.

Lecture 1: Probabilities, inference and information content — An introduction to communications theory, including revision of how to use probabilities (in a Bayesian way) to encode our state of knowledge. An explanation of the Shannon information content.

Lecture 2: Entropy and probability distance — Introduction of the Shannon information entropy for both discrete and continuous distributions. How to measure differences between probability distributions.

Lecture 3: Maximising entropy and thermodynamics — How to use constraints to inform your knowledge (or lack thereof) of a system. How to derive the thermodynamic distributions from a state of ignorance.