has characteristic has parameter relative tolerance hybridity local optimization algorithm maximum iterations Levenberg-Marquardt 2019-01-18 http://identifiers.org/biosimulators/copasi http://identifiers.org/doi/10.1090/qam/10666 http://identifiers.org/doi/10.1137/0111030 AZ Levenberg-Marquardt is a gradient descent method. It is a hybrid between the steepest descent and the Newton methods. Levenberg first suggested an improvement to the Newton method in order to make it more robust, i.e. to overcome the problem of non-convergence. His suggestion was to add a factor to the diagonal elements of the Hessian matrix of second derivatives when not close to the minimum (this can be judged by how positive definite the matrix is). The effect when this factor is large compared to the elements of Hessian is that the method then becomes the steepest descent method. Later Marquardt suggested that the factor should be multiplicative rather than additive and also defined a heuristic to make this factor increase or decrease. The method known as Levenberg-Marquardt is thus an adaptive method that effectively changes between the steepest descent to the Newton method.