Machine Learning A (MLA)

The course introduces basic theory and algorithms of machine learning. The course covers the following tentative list of topics:

• Supervised learning setting
  • Classification
  • Regression
• Unsupervised learning setting
  • Clustering
• Concentration of measure inequalities
  • Markov's
  • Chebyshev's
  • Hoeffding's
• Analysis of generalization in classification
  • Validation and cross-validation
  • Generalisation bound for a single hypothesis
  • Generalisation bound for a finite hypothesis class
  • Occam's razor - generalisation bound for a countably infinite hypothesis class
• Algorithms
  • K-Nearest Neighbors
  • Perceptron
  • Logistic Regression
  • Linear Regression
  • Feature transformations and classification/regression in transformed feature spaces
  • Various forms of regularisation
    • Regularization terms
    • Dimensionality reduction
  • Random Forests and Decision Trees
  • Neural Networks and introduction to Deep Learning
  • Principal Component Analysis (PCA)
  • Clustering algorithms: k-means, k-means++
• Assumptions behind the algorithms taught in the course, their implications, and common pitfalls
  • Overfitting
    • Internal overfitting within algorithms due to overly complex hypothesis spaces
    • Extrenal overfitting outside algorithms due to application of an excessive number of algorithms to a dataset
  • The i.i.d. assumption
    • The i.i.d. assumption is behind everything taught in the course
    • Consequences of violation of the i.i.d. assumption
      • Special case: sampling bias
      • Failure of generalisation guarantees
    • Implications of the i.i.d. assumption
      • Biases in the training data propagate into predictions
  • Correlation ≠ Causality
    • The course only studies statistical correlations / dependencies in the data. Causal inference is not covered in the course.

WARNING: The course assumes solid math and programming skills. Please, check the "Recommended Academic Qualifications" box below and the self-assessment assignment. It is not advised taking the course if you do not meet the academic qualifications.