Machine Learning B (MLB)

Course content

The course is a continuation of Machine Learning A course and provides deeper theoretical foundations of machine learning and a number of advanced theoretically grounded learning techniques. Tentative list of topics includes:

  • Basics in Optimization Theory
    • Basic properties of functions: convexity, Lipschitzness, gradients, subgradients, etc.
    • Constrained optimization and the method of Lagrange multipliers
    • Stochastic Gradient Descent (SGD)
    • Convergence proof for SGD
    • Alternating optimization methods
  • Basics of Information Theory
    • Entropy
    • Relative entropy (the Kullback-Leibler divergence)
    • The method of types
    • kl inequality for concentration of measure
  • Advanced techniques for analysing generalisation power of learning algorithms
    • Vapnik-Chervonenkis (VC) analysis
    • VC analysis of SVMs
    • VC lower bound
    • PAC-Bayesian analysis
    • PAC-Bayesian analysis of majority vote
  • Kernel Methods
    • Kernels and RKHS
    • SVMs
  • Ensemble classifiers and weighted majority vote
    • Boosting
  • Non-linear dimensionality reduction techniques
  • Bayesian inference
    • Difference between Bayesian and frequentist views
    • Gaussian Processes
Education

BSc Programme in Machine Learning and Data Science

Learning outcome

At course completion, the successful student will have:

Knowledge of

  • advanced understanding of the concept of generalisation;
  • advanced tools for analysis of generalisation power of machine learning algorithms;
  • the mathematical foundations of selected advanced machine learning algorithms.

 

Skills in

  • deriving advanced generalisation bounds for expected prediction quality;
  • applying advanced linear and non-linear techniques for classification and regression;
  • implementing selected advanced machine learning algorithms;
  • visualising and evaluating results obtained with machine learning techniques;
  • using software libraries for solving machine learning problems.

 

Competences in

  • recognising and describing possible applications of machine learning;
  • formalising and rigorously analysing machine learning problems;
  • comparing, appraising and selecting machine learning methods for specific tasks;
  • solving real-world data mining and pattern recognition problems by using machine learning techniques.

Weekly lectures, weekly home assignments, exercise classes

Will be published on Absalon.

It is assumed that the students have successfully passed Machine Learning A or a equivalent course.

The course requires strong mathematical skills and background corresponding to what is achieved on the BSc. in Machine Learning and Data Science.

The course is identical to approximately 50% of NDAB20000U Introduktion til Machine Learning (IntroML)
It is not recommended to pass both this course and the Introduktion til Machine Learning (IntroML).

Written
Oral
Individual
Collective
Continuous feedback during the course of the semester
ECTS
7,5 ECTS
Type of assessment
Written examination, 5 days
The exam is a 5-day written take-home assignment (must be solved individually).
Aid
All aids allowed
Marking scale
7-point grading scale
Censorship form
External censorship
Criteria for exam assessment

See Learning Outcome.

Single subject courses (day)

  • Category
  • Hours
  • Lectures
  • 34
  • Preparation
  • 8
  • Theory exercises
  • 57
  • Practical exercises
  • 57
  • Exam Preparation
  • 25
  • Exam
  • 25
  • English
  • 206