Numerical Optimisation (NO)

Numerical optimisation is a useful computer tool in many disciplines like image processing, computer vision, machine learning, bioinformatics, eScience, scientific computing and computational physics, computer animation and many more.  This course will build up a toolbox of numerical optimisation methods which the student can use when building solutions in his or her future studies.

This course teaches the basic theory of numerical optimisation methods. The focus is on deep understanding, and how the methods covered during the course works. Both on a theoretical level that goes into deriving the math but also on an implementational level focusing on computer science. A special focus of the course lies on empirical evaluation of the different methods and communication of the results in report form. As a result the course is very practical and there will be weekly group-based programming exercises.

During the course, we will start from the simple gradient descent algorithm and introduce more ideas to improve on this simple approach to create algorithms that are fast and reliable. The topics covered during the course are:

• First-order optimality conditions, Karush-Kuhn-Tucker Conditions, Taylors Theorem, Mean Value Theorem.
• Nonlinear Equation Solving: Newtons Method, etc.
• Linear Search Methods: Newton Methods, Quasi-Newton Methods, etc.
• Trust Region Methods
• And many more...