## 1. Introduction

This page first describes the Levenberg-Marquardt optimization algorithm, then shows how to use its implementation within the **mrpt-base** C++ library. All the source code discussed here, the implementation of the algorithm itself and examples, are available for download within the MRPT packages.

The following notation and algorithm have been extracted from the report [1].

## 2. Algorithm description

The Levenberg-Marquardt (LM) method consists on an iterative least-square minimization of a cost function based on a modification of the Gauss-Newton method. Let’s state the problem formally before defining the algorithm. We will assume that **derivatives of the cost functions are not available in closed form**, so they will be approximated by finite-difference approximation Finite-difference approximation.

Let be the **parameter vector** to be optimized. We want to find the optimal that minimizes the scalar error function :

**Jacobian**of the error functions as the $ m \times n $ matrix:

The Hessian of the error function is the $ n \times n $ matrix of second order derivatives ($n$ being the length of the parameter vector), and it’s approximated by:

If we don’t have closed form expressions for the derivatives needed for the Jacobian, we can estimate them from finite differences using some increments for each individual variable :

Now, denote as for $t=0,1,2,…$ the **sequence of iterative approximations** to the optimal set of parameters . The first intial guess must be provided by the user. Then, each iteration of the LM method performs:

## 3. C++ Implementation

The LM algorithm is implemented in the C++ template class mrpt::math::CLevenbergMarquardtTempl<T> , and there is an example in ` MRPT/samples/optimize-lm`

, which is described next.

The type `mrpt::math::CLevenbergMarquard`

is a shortcut for the template instantiation `CLevenbergMarquardtTempl<double>`

.

The image below represents the result for this example. The displayed equation is the **error function**, , in this case one-dimensional:

See the source code of the example here: https://raw.github.com/MRPT/mrpt/master/samples/optimize-lm/test.cpp

## References

[1] K. Madsen, H.B. Nielsen, O. Tingleff. Methods for non-linear least squares problems, Informatics and Mathematical Modelling, Technical University of Denmark, April 2004.