Machine Learning B (MLB)
Course content
The course is a continuation of Machine Learning A course and provides deeper theoretical foundations of machine learning and a number of advanced theoretically grounded learning techniques. Tentative list of topics includes:
- Basics in Optimization Theory
- Basic properties of functions: convexity, Lipschitzness, gradients, subgradients, etc.
- Constrained optimization and the method of Lagrange multipliers
- Stochastic Gradient Descent (SGD)
- Convergence proof for SGD
- Alternating optimization methods
- Basics of Information Theory
- Entropy
- Relative entropy (the Kullback-Leibler divergence)
- The method of types
- kl inequality for concentration of measure
- Advanced techniques for analysing generalisation power of
learning algorithms
- Vapnik-Chervonenkis (VC) analysis
- VC analysis of SVMs
- VC lower bound
- PAC-Bayesian analysis
- PAC-Bayesian analysis of majority vote
- Bernstein-type concentration inequalities, with applications to analysis of learning algorithms
- Kernel Methods
- Kernels and RKHS
- SVMs
- Ensemble classifiers and weighted majority vote
- Boosting
- Bayesian inference
- Difference between Bayesian and frequentist views
WARNING: The course assumes that you have taken DIKU's Machine Learning A course. If you have not taken it, please, carefully check the "Recommended Academic Qualifications" box below and the self-preparation assignment at https://sites.google.com/diku.edu/machine-learning-courses/ml-b. Machine Learning courses given at other places do not necessarily prepare you well for this course. It is not advised taking the course if you do not meet the academic qualifications.
Physical & Online: This is a physical course, but we support remote participation.
BSc Programme in Machine Learning and Data Science
MSc Programme in Actuarial Mathematics
MSc Programme in Mathematics-Economics
MSc Programme in Computer Science
MSc Programme in Computer Science (part time)
MSc Programme in Computer Science (with minor subject)
MSc Programme in Statistics
At course completion, the successful student will have:
Knowledge of
- advanced understanding of the concept of generalisation;
- advanced tools for analysis of generalisation power of machine learning algorithms;
- the mathematical foundations of selected advanced machine learning algorithms.
Skills in
- deriving advanced generalisation bounds for expected prediction quality;
- applying advanced linear and non-linear techniques for classification and regression;
- implementing selected advanced machine learning algorithms;
- visualising and evaluating results obtained with machine learning techniques;
- using software libraries for solving machine learning problems.
Competences in
- recognising and describing possible applications of machine learning;
- formalising and rigorously analysing machine learning problems;
- comparing, appraising and selecting machine learning methods for specific tasks;
- solving real-world data mining and pattern recognition problems by using machine learning techniques.
Weekly lectures, weekly home assignments, exercise classes
Will be published on Absalon.
It is assumed that the students have successfully passed Machine
Learning A course. Machine Learning courses given at other places
do not necessarily prepare you well for this course.
Please, check the self-preparation assignment at
https://sites.google.com/diku.edu/machine-learning-courses/ml-b.
The course requires strong mathematical skills and background
corresponding to what is achieved on the BSc. in Machine Learning
and Data Science. In particular:
1. Knowledge of Linear Algebra corresponding to Lineær algebra i
datalogi course (LinAlgDat)
2. Knowledge of Calculus corresponding to Introduktion til
matematik i naturvidenskab (MatintroNat) or Matematisk analyse og
sandsynlighedsteori i datalogi (MASD).
3.Knowledge of Probability Theory corresponding to
Sandsynligheds-regning og statistik (SS), Grundlæggende statistik
og sandsynlighedsregning (GSS) or Matematisk analyse og
sandsynlighedsteori i datalogi (MASD) and Modelling analysis of
data (MAD).
4.Knowledge of Discrete Mathematics corresponding to Diskret
matematik og formelle sprog (DMFS) or Diskret Matematik og
algoritmer (DMA).
5. Knowledge of programming corresponding to Programmering og
problemløsning (PoP) and experience with programming in
Python.
The course is similar to NDAB21008U Machine Learning B (MLB). Students who have previously passed NDAB21008U Machine Learning B (MLB) are not al-lowed to sign up for this course."
- ECTS
- 7,5 ECTS
- Type of assessment
-
Written assignment, 75 hours
- Type of assessment details
- The exam is a written take-home assignment (must be solved
individually).
*Please note: that the planned exam workload is 25 hours. We provide extra days to allow the students to combine the exam with other potential duties, such as other exams or work commitments - Aid
- All aids allowed
- Marking scale
- 7-point grading scale
- Censorship form
- External censorship
Criteria for exam assessment
See Learning Outcome.
Single subject courses (day)
- Category
- Hours
- Lectures
- 34
- Preparation
- 8
- Theory exercises
- 57
- Practical exercises
- 57
- Exam Preparation
- 25
- Exam
- 25
- English
- 206
Kursusinformation
- Language
- English
- Course number
- NDAK22001U
- ECTS
- 7,5 ECTS
- Programme level
- Full Degree Master
- Duration
-
1 block
- Placement
- Block 2
- Schedulegroup
-
B
- Capacity
- No limit.
The number of seats may be reduced in the late registration period - Studyboard
- Study Board of Mathematics and Computer Science
Contracting department
- Department of Computer Science
Contracting faculty
- Faculty of Science
Course Coordinator
- Sadegh Talebi (7-7233786d666d6e45696e33707a336970)
Teacher
Yevgeny Seldin, Christian Igel & Sadegh Talebi
Timetable
Are you BA- or KA-student?
Courseinformation of students