Robustness in Machine Learning (RoMaL)

Machine learning algorithms are designed to be used in the real world, where data is often noisy and malicious users may attempt to exploit the machine learning models for their own gain. With that in mind, it is important to design machine learning algorithms that are robust to such noise and malicious agents without sacrificing their performance.

The study of Robustness in Machine Learning involves understanding the various threat models that noisy data or malicious users pose to learning algorithms and designing algorithms that are robust to such threats. The course will have two main sections, on train time robustness and test time robustness. The rest of the course will involve specialised modules on advanced topics in robustness in ML, depending on the choice of the instructors and guest lecturers.

The course will cover the following main topics

• Train time robustness: Data collected from the real world is often noisy. In this section, we will study the analysis and design of learning algorithms as well as basic statistical estimation procedures when the data is noisy. In particular, we will explore a subset of the following topics

  • Types of noise models (in central as well as distributed learning)

  • Learning theoretic models with noisy data (e.g. PAC learning with (different types of) label noise, SQ learning)

  • Statistical estimation with noisy data (e.g. Robust mean estimation and Robust linear regression)

  • Distributed learning with noisy and faulty clients

• Test time robustness: When deployed in the real world, machine learning models can be vulnerable to perturbations in the test data. In this section, we will study various such threat models, the dangers they pose, and how to protect against them. In particular, we will explore a subset of the following topics

  • Adversarial Robustness

  • Robustness to Distribution Shift

  • Designing Certifiably robust learning and prediction algorithms.

We will also study a few advanced topics in Robustness in ML which will depend on the choice of the instructor and possible guest lecturers. These topics will be mentioned on the absalon page of the course.

WARNING: If you have not taken DIKU's Machine Learning A course, please, carefully check the "Recommended Academic Qualifications" box below. Machine Learning courses given at other places do not necessarily prepare you well for this course, because DIKU's machine learning courses have a stronger theoretical component than average machine learning courses offered elsewhere. It is not advised to take the course if you do not meet the academic qualifications.