Statistical Learning

Course Description

Our course aims to provide students with a solid theoretical foundation, but also the practical skills needed to use and develop effective machine learning solutions to a wide variety of problems. We illustrate the use of the models throughout the course with methods implemented in Python. The class covers foundations and recent advances of machine learning techniques, including:

  • Basic concepts: Linear regression, nearest neighbour, parametric vs. non-parametric methods, Bayesian classifiers, the curse of dimensionality, model accuracy, bias-variance trade-off
  • High-dimensional regression and shrinkage methods: Ridge, Lasso, Elastic Net, PCA, Bayesian linear regression, regularization parameter selection
  • Linear classifiers in depth: linear regression for classification, linear and quadratic discriminant analysis
  • Extensions: Logistic regression and regularizations, nonlinear optimization, multi-class problems
  • Nonlinear classifiers: Decision trees, random forests, boosting, support vector machines, perceptron
  • Neural networks: Multi-layer perceptrons, convolutional neural networks, deep learning tutorial
  • Unsupervised learning: Gaussian mixture models, k-means
  • Dimensionality reduction: PCA, nonlinear dimensionality reduction
  • Generative models:  Variational inference, (variational) autoencoders, generative adversarial networks
  • Graphical models: Basic graph theory, d-separation, Markov properties, Bayesian networks, probability propagation
  • Causality: Structural causal models, faithfulness and Markov property revisited, interventions, counterfactuals, graph discover
Date & Time:     Lectures: Wednesdays 12:15 - 13:45. Exercises: Wednesdays 14:15-15:45, Fridays 14:15-15:45. Due to the  COVID situation, we will give the lecture remotely via TUM Zoom (password:930160).  For more information, see Moodle (password: StatLearn2021)
Prerequisites: The participants are expected to know linear algebra and multivariate calculus, basic concepts from linear functional analysis as well as basic concepts in probability theory. We ask all participants to be able to run Python (>3.5) within a Jupyter notebook on their computer.  
Language:      English
Exam:             The final exam will cover the whole material of the course. Some problems from the weekly homework may reappear on the exam. Time and place of the exam will be announced later.
Weekly Exercises:Weekly exercise sheets will include theory, analysis and computational projects and will consist of 3-4 exercises. “Homework” will start in the exercise course and should be completed before the next exercise course where we publish the answers.  You have to be registered to the exercise group via Moodle and TUM online. The exercise sessions will be held remotely. Dates and timing will be announced via Moodle.