Course Overview - CS-433

Machine learning and data analysis are becoming increasingly central in many sciences and applications. In this course, fundamental principles and methods of machine learning and artificial intelligence will be introduced, analyzed and practically implemented.

Course Goals

Define the following basic ML concepts and explain the main differences between them: Regression, classification, clustering, dimensionality reduction.
Implement and apply machine learning methods to real-world problems. Rigorously evaluate the performance of ML methods using cross-validation.
Optimize the main trade-offs such as overfitting, and computational cost vs accuracy.
Experience common pitfalls and how to overcome them, also in an interdisciplinary context (ML4Science).
Explain and understand fundamental theory concepts presented for ML methods.

Syllabus

We will cover the following ML methods and concepts:

Basic regression and classification concepts and methods:
Linear models, overfitting, linear regression, Ridge regression, logistic regression, SVMs, and k-NN
Fundamental concepts:
Cost-functions and optimization, cross-validation and bias-variance trade-off, curse of dimensionality, kernel methods
Neural networks:
Basics, representation power, backpropagation, CNNs, transformer models, regularization, data augmentation, dropout, adversarial examples and robustness
Unsupervised and self-supervised learning:
k-means clustering, Gaussian mixture models, the EM algorithm, generative models, large language models, diffusion models, generative adversarial networks.
Representation learning and dimensionality reduction:
PCA, matrix factorizations, word embeddings for natural language processing, recommender systems

Exercise sessions

Weekly every Thursday 14:15 - 16:00, in person in the following rooms. Room assignment is based on the last two digits of your SCIPER number:

00-16: INR219: capacity 79
17-43: INF1: capacity 126
44-54: INF119: capacity 54
55-74: INJ218: capacity 96
75-82: CO123: capacity 40
83-99: INM202: capacity 86

All labs and projects will be in Python. See the first lab to get started. Don’t worry if you have no experience in it yet, but in that case you should take enough time to thoroughly work on the first (and second) lab.

Prerequisites

We will revise some of basic ML concepts in the first and second weeks of the course. However, you are recommended to go through the list of pre-requisites here to make sure your knowledge is up to date.

Vector and Matrix Algebra.: Vector and matrix multiplication, matrix inverse, rank, eigenvalue decomposition. Refer to first-year courses, or the linear algebra handout on the website, or Gilbert Strang’s book for example.
Vector and Matrix Calculus. Important: The definition of derivative with respect to vectors and matrices. For reference, see this blogpost explained.ai/matrix-calculus, or the Matrix Cookbook for example.
Scientific Computing Languages. Python Basics (see tutorial in the first lab).
Probability and Statistics. Conditional and joint distribution, independence, Bayes’ rule, random variable and expectation, law of large numbers.
Gaussian Distribution. Univariate and multivariate, conditional, joint and marginals.
Writing Scientific Documents using Latex (not required but preferred). Many tutorials are available online, and we provide more resources when we come to Project 1.

Resources

Course Webpage

All materials will be made available on our GitHub repository, including annotated lecture notes, code and exercise solutions: github.com/epfml/ML_course

Most materials will also be linked on the course website.

Lecture Notes

PDF notes for each lecture will be available on the website (and GitHub) after the lecture. For revisions in case of errors, see also on GitHub.

Evaluation

Project 1: 10% of final grade. Due Oct 31.
Project 2: 30% of final grade. Due Dec 18.
Final exam: 60% of final grade. Due TBD.

Project 1 (10%)

The goal of this project is to help you prepare for Project 2. In this first project, you will work in a small group of 3 students (2 only in exceptional cases, pending approval). You will implement the most important methods covered in the lectures and labs so far. Additionally, we will provide you with an interesting real-world dataset, and organize our own competition here.

A detailed project description will be posted on the website very soon.

Team assignment: Your choice. We recommend working in interdisciplinary teams, since the projects require many aspects. Use the discussion forum to find team-mates and form a group of 3.

You will also submit your Python code, and a 2-page PDF report.

Final exam (60%)

A very standard final exam.

It will contain questions on what you have learned during the lectures and exercise sessions.

We will give you a sample exam before for you to practice.

You are allowed to bring one cheat sheet (A4 size paper, both sides can be used).

No calculator, No collaborations. No cell phones. No laptops etc.

Contact Us

Please use the discussion forum for any questions and feedback on the course material or exercises, or email the respective assistants and teachers.

Teaching Assistants

Lectures: 2 × 2h per week.
Exercise sessions: Weekly, hands-on coding assignments.
Projects: Two larger group projects.
Exam: Written, closed book, one A4 crib sheet allowed.

References

Recommended (but not mandatory) textbooks:

G. Strang: Linear Algebra and Learning from Data
C. Bishop: Pattern Recognition and Machine Learning
S. Shalev-Shwartz & S. Ben-David: Understanding Machine Learning
G. James, D. Witten, T. Hastie and R. Tibshirani: An Introduction to Statistical Learning
T. Hastie, R. Tibshirani and J. Friedman: Elements of Statistical Learning
K Murphy: Machine Learning: A Probabilistic Perspective
M. Nielsen: Neural Networks and Deep Learning

Credits

Teaching material by Emtiyaz Khan, Rüdiger Urbanke, Martin Jaggi, Nicolas Flammarion, Tatjana Chavdarova. Additional material and code by Tao Lin, Frederik Kunstner, Yannic Kilcher, Aurelien Lucchi, Thijs Vogels, the assistants from the previous years, and others…