Numerical Optimization (NO)

Course content

Numerical optimization is a useful computer tool in many disciplines like image processing, computer vision, machine learning, bioinformatics, eScience, scientific computing and computational physics, computer animation and many more. A wide range of problems can be solved using numerical optimization like: inverse kinematics in robotics, image segmentation and registration in medical imaging, protein folding in computational biology, stock portfolio optimization, motion planning and many more.

This course will build up a toolbox of numerical optimization methods which the student can use when building solutions in his or her future studies. Therefore this course is an ideal supplement for students coming from many different fields of science.

This course teaches the basic theory of numerical optimization methods. The focus is on deep learning and of how the methods covered during the course works. Both on a theoretical level that goes into deriving the math but also on an implementational level with focus on computer science and good programming practice.

There will be weekly programming exercises where students will implement the algorithms and methods introduced from theory on their own case-study problems like computing the motion of a robot hand or fitting a model to highly non-linear data or similar problems.

The topics covered during the course are:

  • First order optimality conditions, Karush-Kuhn-Tucker Conditions, Taylors Theorem, Mean Value Theorem.
  • Nonlinear Equation Solving: Newtons Method, etc.
  • Linear Search Methods: Newton Methods, Quasi-Newton Methods, etc.
  • Trust Region Methods: Levenberg-Marquardt, Dog leg method, etc.
  • Linear Least squares fitting, Regression Problems, Normal Equations, etc.
  • And many more...

MSc Programme in Computer Science
MSc Programme in Bioinformatics

MSc Programme in Statistics

Learning outcome

Knowledge of

  • Theory of gradient descent method
  • Theory of Newton and Quasi Newton Methods
  • Theory of Thrust Region Methods
  • Theory of quadratic programming problems
  • First order optimality conditions (KKT conditions)


Skills in

  • Applying numerical optimization problems to solve unconstrained and constrained minimization problems and nonlinear root search problems
  • Reformulating one problem type into another form - i.e. reformulating root search into minimization and vice versa
  • Implementing and testing numerical optimization methods


Competences to

  • Evaluate which numerical optimization methods are best suited for solving a given optimization problem
  • Understand the implications of theoretical theorems and being able to analyze real problems on that basis



Mixture of study groups and project group work with hand-ins.

See Absalon when the course is set up.

We do not impose any programming language, however Python will be the programming language of choice and it is expected that students know how to install and use Python, Numpy and Matplotlib by themselves. It is also expected that students know what matrices and vectors are and that students are able to differentiate vector functions.

Theorems like fundamental theorem of calculus, mean value theorem or Taylor's theorem will be touched upon during the course. The inquisitive students may find more in depth knowledge from Chapters 2, 3, 5, 6 and 13 of R. A. Adams, Calculus, 3rd ed. Addison Wesley.

Academic qualifications equivalent to a BSc degree is recommended.

Continuous feedback during the course of the semester
7,5 ECTS
Type of assessment
Continuous assessment
The assessment is based on 5-7 written assignments (must be individually approved).

The student acting as opponent in respect of fellow students’ work is also considered as part of the continuous assessment.

The final grade is based on an overall assessment.
All aids allowed
Marking scale
7-point grading scale
Censorship form
No external censorship
Several internal examiners
Criteria for exam assessment

See Learning Outcome.

Single subject courses (day)

  • Category
  • Hours
  • Preparation
  • 50
  • Project work
  • 84
  • Exercises
  • 72
  • English
  • 206