Recent advances in numerical methods for nonlinear equations and nonlinear least squares
Ya-Xiang Yuan
Numerical Algebra, Control & Optimization 2011, 1(1): 15-34 doi: 10.3934/naco.2011.1.15
Nonlinear equations and nonlinear least squares problems have many applications in physics, chemistry, engineering, biology, economics, finance and many other fields. In this paper, we will review some recent results on numerical methods for these two special problems, particularly on Levenberg-Marquardt type methods, quasi-Newton type methods, and trust region algorithms. Discussions on variable projection methods and subspace methods are also given. Some theoretical results about local convergence results of the Levenberg-Marquardt type methods without non-singularity assumption are presented. A few model algorithms based on line searches and trust regions are also given.
keywords: local error bound conditions convergence. nonlinear least squares quasi-Newton Levenberg-Marquardt trust region Nonlinear equations variable projection subspace
Analysis of monotone gradient methods
Yuhong Dai Ya-xiang Yuan
Journal of Industrial & Management Optimization 2005, 1(2): 181-192 doi: 10.3934/jimo.2005.1.181
The gradient method is one simple method in nonlinear optimization. In this paper, we give a brief review on monotone gradient methods and study their numerical properties by introducing a new technique of long-term observation. We find that, one monotone gradient algorithm which is proposed by Yuan recently shares with the Barzilai-Borwein (BB) method the property that the gradient components with respect to the eigenvectors of the function Hessian are decreasing together. This might partly explain why this algorithm by Yuan is comparable to the BB method in practice. Some examples are also provided showing that the alternate minimization algorithm and the other algorithm by Yuan may fall into cycles. Some more efficient gradient algorithms are provided. Particularly, one of them is monotone and performs better than the BB method in the quadratic case.
keywords: monotone cycle. nonmonotone gradient method strictly convex quadratics

Year of publication

Related Authors

Related Keywords

[Back to Top]