Machine Learning B (MLB)

The course is a continuation of Machine Learning A course and provides deeper theoretical foundations of machine learning and a number of advanced theoretically grounded learning techniques. Tentative list of topics includes:

• Basics in Optimization Theory
  • Basic properties of functions: convexity, Lipschitzness, gradients, subgradients, etc.
  • Constrained optimization and the method of Lagrange multipliers
  • Stochastic Gradient Descent (SGD)
  • Convergence proof for SGD
  • Alternating optimization methods
• Basics of Information Theory
  • Entropy
  • Relative entropy (the Kullback-Leibler divergence)
  • The method of types
  • kl inequality for concentration of measure
• Advanced techniques for analysing generalisation power of learning algorithms
  • Vapnik-Chervonenkis (VC) analysis
  • VC analysis of SVMs
  • VC lower bound
  • PAC-Bayesian analysis
  • PAC-Bayesian analysis of majority vote
  • Bernstein-type concentration inequalities, with applications to analysis of learning algorithms
• Kernel Methods
  • Kernels and RKHS
  • SVMs
• Ensemble classifiers and weighted majority vote
  • Boosting
• Bayesian inference
  • Difference between Bayesian and frequentist views

WARNING: The course assumes that you have taken DIKU's Machine Learning A course. If you have not taken it, please, carefully check the "Recommended Academic Qualifications" box below and the self-preparation assignment at https://sites.google.com/diku.edu/machine-learning-courses/ml-b. Machine Learning courses given at other places do not necessarily prepare you well for this course. It is not advised taking the course if you do not meet the academic qualifications.

Physical & Online: This is a physical course, but we support remote participation.