Due to the COVID situation, the class is going online on Dec. 9th and Dec. 16th.
The class is going online for two weeks on Dec. 9th and Dec. 16th. You will be given videos corresponding to the lectures (about 1 hour per session), which will be followed by an interactive discussion on Zoom from 12:15pm to 12:45pm. The link is available in the same url giving you access to the data challenge. See instructions in the table below for the videos.Course information
Statistical learning is about the construction and study of systems that can automatically learn from data. With the emergence of massive datasets commonly encountered today, the need for powerful machine learning is of acute importance. Examples of successful applications include effective web search, anti-spam software, computer vision, robotics, practical speech recognition, and a deeper understanding of the human genome. This course gives an introduction to this exciting field, with a strong focus on kernels as a versatile tool to represent data, in combination with (un)supervised learning techniques that are agnostic to the type of data that is learned from. The learning techniques that will be covered include regression, classification, clustering and dimension reduction. We will cover both the theoretical underpinnings of kernels, as well as a series of kernels that are important in practical applications. Finally we will touch upon topics of active research, such as large-scale kernel methods and the use of kernel methods to develop theoretical foundations of deep learning models.
Evaluation
- project (data challenge) (50%) + final exam (50%)
Course outline
- Theory of RKHS and kernels
- Supervised learning with kernels
- Unsupervised learning with kernels
- Kernels for structured data
- Kernels for generative models
- Theoretical foundations of deep learning with kernels
Calendar
The slides, available here, will be those of the MVA master program. The video lectures are also available here
| Date | Time | Lecturer | Room | Topic |
|---|---|---|---|---|
| 30/09/2021 | 11:15 - 12:45 | JM | D207 | Positive definite kernels, slides 1-28 |
| 14/10/2021 | 11:15 - 12:45 | JM | D208 | RKHS, slides 29-47 |
| 21/10/2021 | 11:15 - 12:45 | JM | D208 | Kernel tricks, slides 48-65 |
| 28/10/2021 | 11:15 - 12:45 | JM | D123 | Kernel ridge regression, slides 66-97 |
| 18/11/2021 | 11:15 - 12:45 | JM | D211 | learning theory - large margin classifiers, slides 104-132 |
| 25/11/2021 | 11:15 - 12:45 | JM | D211 | Support vector machines, slides 133-165 |
| 02/12/2021 | 11:15 - 12:45 | JM | D211 | Kernel PCA, kernel K-means |
| 09/12/2021 | 12:15 - 12:45 | JM | D211 | spectral clustering, kernel CCA Video of Lecture 10. |
| 16/12/2021 | 12:15 - 12:45 | JM | D211 | Green, Mercer, Bochner kernels Lecture 12 |
| 06/01/2022 | 11:15 - 12:45 | JM | D211 | Kernels for sequences |
| 13/01/2022 | 11:15 - 12:45 | JM | D211 | Large-scale learning with kernels |
| 20/01/2022 | 11:15 - 12:45 | JM | D211 | Deep learning with kernels |
Reading material
Machine Learning and Statistics
- Vapnik, The nature of statistical learning theory. Springer
- Hastie, Tibshirani, Friedman, The elements of statistical learning. (free online)
- J Shawe-Taylor, N Cristianini. Kernel methods for pattern analysis. 2004.