Bayesian machine learning

Lecturers

• Rémi Bardenet (CNRS, Univ. Lille), https://rbardenet.github.io
• Gabriel Victorino Cardoso (Mines Paris - PSL Univ.), https://gabrielvc.github.io/
• Julyan Arbel (Inria, Univ. Grenoble-Alpes), https://www.julyanarbel.com

Material

• Incomplete and drafty lecture notes are available in the notes folder. Any comment welcome, live or as a raised issue.
• Practicals and exercises will be made available in the corresponding folders. They are to be done on a voluntary basis. Solutions will be provided on demand.
• Slides for Lecture 1 are here.
• Slides for Lecture 2 are here and here.
• Slides for Lecture 3 are here and here.

Objective of the course

By the end of the course, the students should

• have a high-level view of the main approaches to making decisions under uncertainty.
• be able to detect when being Bayesian helps and why.
• be able to design and run a Bayesian ML pipeline for standard supervised or unsupervised learning.
• have a global view of the current limitations of Bayesian approaches and the research landscape.
• be able to understand the abstract of most Bayesian ML papers.

Topics

• Decision theory
• 50 shades of Bayes: Subjective and objective interpretations
• Bayesian supervised and unsupervised learning
• Bayesian computation for ML: Advanced Monte Carlo and variational methods
• Bayesian nonparametrics
• Bayesian methods for deep learning

Prerequisites

• An undergraduate course in probability.
• It is recommended to have followed either "Probabilistic graphical models" or "Computational statistics" during the first semester.

Organization of courses

• 8x3 hours of lectures, the last session being a student seminar.
• All classes and all material will be in English. Students may write their final report in either French or English.

Validation

• Students form groups. Each group reads and reports on a research paper from a list. We strongly encourage a dash of creativity: students should identify a weak point, shortcoming or limitation of the paper, and try to push in that direction. This can mean extending a proof, implementing another feature, investigating different experiments, etc.
• Deliverables are a small report and a short oral presentation in front of the class, in the form of a student seminar, which will take place during the last lecture.

References

• Parmigiani, G. and Inoue, L. 2009. Decision theory: principles and approaches. Wiley.
• Robert, C. 2007. The Bayesian choice. Springer.
• Murphy, K. 2023. Probabilistic Machine Learning: Advanced Topics. MIT Press. pdf available at this link.
• Ghosal, S., & Van der Vaart, A. W. 2017. Fundamentals of nonparametric Bayesian inference. Cambridge University Press.