This page first describes the Levenberg-Marquardt optimization algorithm, then shows how to use its implementation within the mrpt-base C++ library. All the source code discussed here, the implementation of the algorithm itself and examples, are available for download within the MRPT packages.
The following notation and algorithm have been extracted from the report .
2. Algorithm description
The Levenberg-Marquardt (LM) method consists on an iterative least-square minimization of a cost function based on a modification of the Gauss-Newton method. Let’s state the problem formally before defining the algorithm. We will assume that derivatives of the cost functions are not available in closed form, so they will be approximated by finite-difference approximation Finite-difference approximation.
Let be the parameter vector to be optimized. We want to find the optimal that minimizes the scalar error function :
The Hessian of the error function is the $ n \times n $ matrix of second order derivatives ($n$ being the length of the parameter vector), and it’s approximated by:
If we don’t have closed form expressions for the derivatives needed for the Jacobian, we can estimate them from finite differences using some increments for each individual variable :
Now, denote as for $t=0,1,2,…$ the sequence of iterative approximations to the optimal set of parameters . The first intial guess must be provided by the user. Then, each iteration of the LM method performs:
3. C++ Implementation
The LM algorithm is implemented in the C++ template class mrpt::math::CLevenbergMarquardtTempl<T> , and there is an example in
MRPT/samples/optimize-lm, which is described next.
mrpt::math::CLevenbergMarquard is a shortcut for the template instantiation
The image below represents the result for this example. The displayed equation is the error function, , in this case one-dimensional:
See the source code of the example here: https://raw.github.com/MRPT/mrpt/master/samples/optimize-lm/test.cpp
 K. Madsen, H.B. Nielsen, O. Tingleff. Methods for non-linear least squares problems, Informatics and Mathematical Modelling, Technical University of Denmark, April 2004.