# American Institute of Mathematical Sciences

2011, 1(1): 15-34. doi: 10.3934/naco.2011.1.15

## Recent advances in numerical methods for nonlinear equations and nonlinear least squares

 1 State Key Laboratory of Scientific/Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing,Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Zhong Guan Cun Donglu 55, Beijing, 100190, China

Received  June 2010 Revised  September 2010 Published  February 2011

Nonlinear equations and nonlinear least squares problems have many applications in physics, chemistry, engineering, biology, economics, finance and many other fields. In this paper, we will review some recent results on numerical methods for these two special problems, particularly on Levenberg-Marquardt type methods, quasi-Newton type methods, and trust region algorithms. Discussions on variable projection methods and subspace methods are also given. Some theoretical results about local convergence results of the Levenberg-Marquardt type methods without non-singularity assumption are presented. A few model algorithms based on line searches and trust regions are also given.
Citation: Ya-Xiang Yuan. Recent advances in numerical methods for nonlinear equations and nonlinear least squares. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 15-34. doi: 10.3934/naco.2011.1.15
##### References:
 [1] J. Barzilai and J. M. Borwein, Two point step size gradient methods,, IMA J. Numer. Anal., 8 (1988), 141.  doi: 10.1093/imanum/8.1.141.  Google Scholar [2] S. Bellavia, C. Cartis, N. I. M. Gould, B. Morini and Ph. L. Toint, Convergence of a regularized Euclidean residual algorithm for nonlinear least-squares,, SIAM J. Numer. Anal., 48 (2010), 1.  doi: 10.1137/080732432.  Google Scholar [3] L. Bergamaschi, I. Moret and G. Zilli, Inexact quasi-Newton methods for sparse systems of nonlinear equations,, Future Generation Computer Systems, 18 (2001), 41.  doi: 10.1016/S0167-739X(00)00074-1.  Google Scholar [4] E. G. Birgin, N. Krejic and J. M. Martinez, Globally convergent inexact quasi-Newton methods for solving nonlinear systems,, Numerical Algorithms, 32 (2003), 249.  doi: 10.1023/A:1024013824524.  Google Scholar [5] A. Bouaricha and J. J. Moré, Impact of Partial Separability on Large-Scale Optimization,, Computational Optimization and Applications, 7 (1997), 27.  doi: 10.1023/A:1008628114432.  Google Scholar [6] M. H. Cheng, "Quasi-Newton Type Methods for Solving Large-Scale Problems,'', Ph.D. thesis, (2010).   Google Scholar [7] M. H. Cheng and Y. H. Dai, Sparse two-sided rank-one updates for nonlinear equations,, Science in China, 53 (2010), 1.  doi: 10.1007/s11425-010-4056-x.  Google Scholar [8] A. R. Conn, N. J. M. Gould and Ph. L. Toint, "Trust-Region Methods,", MPS-SIAM Series on Optimization, (2000).   Google Scholar [9] J. E. Dennis Jr. and J. J. Moré, Quasi-Newton methods, motivation and theory,, SIAM Review, 19 (1977), 46.  doi: 10.1137/1019005.  Google Scholar [10] J. E. Dennis and R. B. Schnabel, "Numerical Methods for Unconstrained Optimization and Nonlinear Equations,", SIAM, (1993).   Google Scholar [11] J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar [12] J. Y. Fan and Y. X. Yuan, Regularized Newton method with correction for monotone nonlinear equations and its application,, Report, (2010).   Google Scholar [13] R. Fletcher, A model algorithm for composite NDO problem,, Math. Program., 17 (1982), 67.   Google Scholar [14] R. Fletcher, "Practical Methods of Optimization,", Second Edition, (1987).   Google Scholar [15] R. Fletcher and C. Xu, Hybrid methods for nonlinear least squares,, IMA J. Numerical Analysis, 7 (1987), 371.  doi: 10.1093/imanum/7.3.371.  Google Scholar [16] G. H. Golub and V. Pereyra, The differentiation of pseudo-inverses and nonlinear least squares whose variables separable,, SIAM J. Numer. Anal, 10 (1973), 413.  doi: 10.1137/0710036.  Google Scholar [17] G. H. Golub and V. Pereyra, Separable nonlinear least squares: the variable projection method and applications,, Inverse Problems, 19 (2003), 1.  doi: 10.1088/0266-5611/19/2/201.  Google Scholar [18] G. H. Golub and C. F. Van Loan, "Matrix Computations,'' 3rd Edition,, Johns Hopkins University Press, (1996).   Google Scholar [19] N. I. M. Gould, D. Orban and Ph. L. Toint, Numerical methods for large-scale nonlinear optimization,, Acta Numerica, (2005), 299.  doi: 10.1017/S0962492904000248.  Google Scholar [20] N. I. M. Gould and Ph. L. Toint, FILTRANE, a Fortran 95 filter-trust-region package for solving nonlinear least-squares and nonlinear feasibility problems,, ACM Transactions on Mathematical Software (TOMS), 33 (2007), 3.  doi: 10.1145/1206040.1206043.  Google Scholar [21] A. Griewank, "Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation,", Frontiers in Applied Mathematics, (2000).   Google Scholar [22] A. Griewank and A. Walther, On constrained optimization by adjoint based quasi-Newton methods,, Optimizaiton Methods and Software, 17 (2002), 869.  doi: 10.1080/1055678021000060829.  Google Scholar [23] L. Kaufman, A variable projection method for solving separable nonlinear least squares problems,, BIT, 15 (1975), 49.  doi: 10.1007/BF01932995.  Google Scholar [24] C.T. Kelley, "Iterative Methods for Linear and Nonlinear Equations,", SIAM, (1995).   Google Scholar [25] C. T. Kelley, "Solving Nonlinear Equations with Newton's Method,", Fundamentals of Algorithms Series, (2003).   Google Scholar [26] C. Kanzow, N. Yamashita and M. Fukushima, Levenberg-Marquardt methods with storng local convergence properties for solving nonlinear equations with convex constraints,, J. Comp. Appl. Math., 173 (2005), 321.   Google Scholar [27] W. La Cruz and M. Raydan, Nonmonotone spectral methods for large-scale nonlinear sysstems,, Optimization Methods and Software, 18 (2003), 583.  doi: 10.1080/10556780310001610493.  Google Scholar [28] W. La Cruz, J. M. Martinez and M. Raydan, Spectral residual method without gradient information for solving large-scale nonlinear systems of equations,, Math. Comp., 75 (2006), 1429.  doi: 10.1090/S0025-5718-06-01840-0.  Google Scholar [29] D. H. Li and M. Fukushima, A globally and superlinearly convergent Gauss-Newton-based BFGS method for symmetric nonlinear equations,, SIAM J. Numer. Anal., 37 (1999), 152.  doi: 10.1137/S0036142998335704.  Google Scholar [30] D. H. Li and M. Fukushima, A derivative-free line search and global convergence of Broyden-like method for nonlinear equations,, Optimizaiton Methods and Software, 13 (2000), 181.  doi: 10.1080/10556780008805782.  Google Scholar [31] X. Liu and Y. X. Yuan, On the separable nonlinear least squares problems,, J. Comput. Math., 26 (2008), 390.   Google Scholar [32] J. M. Martinez, A quasi-Newton method with modification of one column per iteration,, Computing, 33 (1984), 353.   Google Scholar [33] E. Mizutani and J. W. Demmel, On structure-exploiting trust-region regularized nonlinear least squares algorithms for neural-network learning,, Neural Networks, 16 (2003), 745.  doi: 10.1016/S0893-6080(03)00085-6.  Google Scholar [34] J. J. Moré, The Levenberg-Marquardt algorithm: implementation and theory,, in, (1978), 105.   Google Scholar [35] J. J. Moré, Recent developments in algorithms and software for trust region methods,, in, (1983), 258.   Google Scholar [36] J. J. Moré and D. C. Sorensen, Computing a trust region step,, SIAM J. Sci. Statist. Compute., 4 (1983), 553.  doi: 10.1137/0904038.  Google Scholar [37] Yu. Nesterov, Modified Gauss-Newton scheme with worst-case grarantees for global performance,, Optimization Methods and Sofware, 22 (2007), 469.  doi: 10.1080/08927020600643812.  Google Scholar [38] J. Nocedal and S. J. Wright, "Numerical Optimization,", Springer, (1999).  doi: 10.1007/b98874.  Google Scholar [39] J. M. Ortega and W. C. Rheinboldt, "Iterative Solution of Nonlinear Equations in Several Variables,", Academic Press, (1970).   Google Scholar [40] M. J. D. Powell, A new algorithm for unconstrained optimization,, in, (1970), 31.   Google Scholar [41] M. J. D. Powell, A method for minimizing a sum of squares of non-linear functions without calculating derivatives,, The Computer J., 7 (1965), 303.   Google Scholar [42] M. J. D. Powell, A hybrid method for nonlinear equations,, in, (1970), 87.   Google Scholar [43] M. J. D. Powell, A fast algorithm for nonlinearly constrained optimization calculations,, in, (1978), 144.  doi: 10.1007/BFb0067703.  Google Scholar [44] M. J. D. Powell and Y. X. Yuan, Conditions for superlinear convergence in $l_1$ and $l_\infty$ solutions of overdetermined nonlinear equations,, IMA J. Numerical Analysis, 4 (1984), 241.  doi: 10.1093/imanum/4.2.241.  Google Scholar [45] A. Ruhe and P. Å. Wedin, Algorithms for separable nonlinear least squares problems,, SIAM Review, 22 (1980), 318.  doi: 10.1137/1022057.  Google Scholar [46] Y. Saad, "Iterative Methods for Sparse Linear Systems,", SIAM, (2003).  doi: 10.1137/1.9780898718003.  Google Scholar [47] S. Schlenkrich, A. Griewank and A. Walther, On the local convergence of adjoint Broyden methods,, Math. Program., 121 (2010), 221.  doi: 10.1007/s10107-008-0232-y.  Google Scholar [48] K. Schittkowski, Solving nonlinear least squares problems by a general purpose SQP-method,, in, (1988), 295.   Google Scholar [49] K. Schittkowski, "Numerical Data Fitting in Dynamical Systems,", Applied Optimization Vol. 77, (2002).   Google Scholar [50] L. K. Schubert, Modification of a quasi-Newton method for nonlinear equations with a sparse Jacobian,, Math. Comp., 24 (1970), 27.  doi: 10.1090/S0025-5718-1970-0258276-9.  Google Scholar [51] T. Steihaug, The conjugate gradient method and trust regions in large scale optimization,, SIAM J. Numerical Analysis, 20 (1983), 626.  doi: 10.1137/0720042.  Google Scholar [52] J. Stoer and Y. X. Yuan, A subspace study on conjugate gradient algorithms,, Z. Angew. Math. Mech., 75 (1995), 69.  doi: 10.1002/zamm.19950750118.  Google Scholar [53] W. Y. Sun and Y. X. Yuan, "Optimization Theory and Mehtods: Nonlinear Programming,", Springer Series on Optimization and Its Application, (2006).   Google Scholar [54] Ph. L. Toint, On sparse and sysmmetric matrix updating subject to a linear equation,, Mathematics of Computation, 31 (1977), 954.  doi: 10.1090/S0025-5718-1977-0455338-4.  Google Scholar [55] Ph. L. Toint, Towards an efficient sparsity exploiting Newton method for minimization,, in, (1981), 57.   Google Scholar [56] Ph. L. Toint, On large scale nonlinear least squares calculations,, SIAM J. Sci. Stat. Comput., 8 (1987), 416.  doi: 10.1137/0908042.  Google Scholar [57] Z. H. Wang and Y. X. Yuan, A subspace implementation of quasi-Newton trust region methods for unconstrained optimization,, Numerische Mathematik, 104 (2006), 241.  doi: 10.1007/s00211-006-0021-6.  Google Scholar [58] N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method ,, Computing, 15 (2001), 237.   Google Scholar [59] Y. X. Yuan, Trust region algorithms for nonlinear equations,, Information, 1 (1998), 7.   Google Scholar [60] Y. X. Yuan, On the truncated conjugate gradient method,, Math. Program., 87 (2000), 561.  doi: 10.1007/s101070050012.  Google Scholar [61] Y. X. Yuan, Subspace techniques for nonlinear optimization,, in, (2007), 206.   Google Scholar [62] Y. X. Yuan, Subspace methods for large scale nonlinear equations and nonlinear least squares,, Optimization and Engineering, 10 (2009), 207.  doi: 10.1007/s11081-008-9064-0.  Google Scholar [63] H. C. Zhang, A. R. Conn and K. Scheinberg, A derivative-free algorithm for the least-squares minimization,, , ().   Google Scholar [64] H. C. Zhang, A. R. Conn and K. Scheinberg, On the local convergence of a derivative-free algorithm for least-squares minimization,, , ().   Google Scholar

show all references

##### References:
 [1] J. Barzilai and J. M. Borwein, Two point step size gradient methods,, IMA J. Numer. Anal., 8 (1988), 141.  doi: 10.1093/imanum/8.1.141.  Google Scholar [2] S. Bellavia, C. Cartis, N. I. M. Gould, B. Morini and Ph. L. Toint, Convergence of a regularized Euclidean residual algorithm for nonlinear least-squares,, SIAM J. Numer. Anal., 48 (2010), 1.  doi: 10.1137/080732432.  Google Scholar [3] L. Bergamaschi, I. Moret and G. Zilli, Inexact quasi-Newton methods for sparse systems of nonlinear equations,, Future Generation Computer Systems, 18 (2001), 41.  doi: 10.1016/S0167-739X(00)00074-1.  Google Scholar [4] E. G. Birgin, N. Krejic and J. M. Martinez, Globally convergent inexact quasi-Newton methods for solving nonlinear systems,, Numerical Algorithms, 32 (2003), 249.  doi: 10.1023/A:1024013824524.  Google Scholar [5] A. Bouaricha and J. J. Moré, Impact of Partial Separability on Large-Scale Optimization,, Computational Optimization and Applications, 7 (1997), 27.  doi: 10.1023/A:1008628114432.  Google Scholar [6] M. H. Cheng, "Quasi-Newton Type Methods for Solving Large-Scale Problems,'', Ph.D. thesis, (2010).   Google Scholar [7] M. H. Cheng and Y. H. Dai, Sparse two-sided rank-one updates for nonlinear equations,, Science in China, 53 (2010), 1.  doi: 10.1007/s11425-010-4056-x.  Google Scholar [8] A. R. Conn, N. J. M. Gould and Ph. L. Toint, "Trust-Region Methods,", MPS-SIAM Series on Optimization, (2000).   Google Scholar [9] J. E. Dennis Jr. and J. J. Moré, Quasi-Newton methods, motivation and theory,, SIAM Review, 19 (1977), 46.  doi: 10.1137/1019005.  Google Scholar [10] J. E. Dennis and R. B. Schnabel, "Numerical Methods for Unconstrained Optimization and Nonlinear Equations,", SIAM, (1993).   Google Scholar [11] J. Y. Fan and Y. X. Yuan, On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,, Computing, 74 (2005), 23.  doi: 10.1007/s00607-004-0083-1.  Google Scholar [12] J. Y. Fan and Y. X. Yuan, Regularized Newton method with correction for monotone nonlinear equations and its application,, Report, (2010).   Google Scholar [13] R. Fletcher, A model algorithm for composite NDO problem,, Math. Program., 17 (1982), 67.   Google Scholar [14] R. Fletcher, "Practical Methods of Optimization,", Second Edition, (1987).   Google Scholar [15] R. Fletcher and C. Xu, Hybrid methods for nonlinear least squares,, IMA J. Numerical Analysis, 7 (1987), 371.  doi: 10.1093/imanum/7.3.371.  Google Scholar [16] G. H. Golub and V. Pereyra, The differentiation of pseudo-inverses and nonlinear least squares whose variables separable,, SIAM J. Numer. Anal, 10 (1973), 413.  doi: 10.1137/0710036.  Google Scholar [17] G. H. Golub and V. Pereyra, Separable nonlinear least squares: the variable projection method and applications,, Inverse Problems, 19 (2003), 1.  doi: 10.1088/0266-5611/19/2/201.  Google Scholar [18] G. H. Golub and C. F. Van Loan, "Matrix Computations,'' 3rd Edition,, Johns Hopkins University Press, (1996).   Google Scholar [19] N. I. M. Gould, D. Orban and Ph. L. Toint, Numerical methods for large-scale nonlinear optimization,, Acta Numerica, (2005), 299.  doi: 10.1017/S0962492904000248.  Google Scholar [20] N. I. M. Gould and Ph. L. Toint, FILTRANE, a Fortran 95 filter-trust-region package for solving nonlinear least-squares and nonlinear feasibility problems,, ACM Transactions on Mathematical Software (TOMS), 33 (2007), 3.  doi: 10.1145/1206040.1206043.  Google Scholar [21] A. Griewank, "Evaluating Derivatives. Principles and Techniques of Algorithmic Differentiation,", Frontiers in Applied Mathematics, (2000).   Google Scholar [22] A. Griewank and A. Walther, On constrained optimization by adjoint based quasi-Newton methods,, Optimizaiton Methods and Software, 17 (2002), 869.  doi: 10.1080/1055678021000060829.  Google Scholar [23] L. Kaufman, A variable projection method for solving separable nonlinear least squares problems,, BIT, 15 (1975), 49.  doi: 10.1007/BF01932995.  Google Scholar [24] C.T. Kelley, "Iterative Methods for Linear and Nonlinear Equations,", SIAM, (1995).   Google Scholar [25] C. T. Kelley, "Solving Nonlinear Equations with Newton's Method,", Fundamentals of Algorithms Series, (2003).   Google Scholar [26] C. Kanzow, N. Yamashita and M. Fukushima, Levenberg-Marquardt methods with storng local convergence properties for solving nonlinear equations with convex constraints,, J. Comp. Appl. Math., 173 (2005), 321.   Google Scholar [27] W. La Cruz and M. Raydan, Nonmonotone spectral methods for large-scale nonlinear sysstems,, Optimization Methods and Software, 18 (2003), 583.  doi: 10.1080/10556780310001610493.  Google Scholar [28] W. La Cruz, J. M. Martinez and M. Raydan, Spectral residual method without gradient information for solving large-scale nonlinear systems of equations,, Math. Comp., 75 (2006), 1429.  doi: 10.1090/S0025-5718-06-01840-0.  Google Scholar [29] D. H. Li and M. Fukushima, A globally and superlinearly convergent Gauss-Newton-based BFGS method for symmetric nonlinear equations,, SIAM J. Numer. Anal., 37 (1999), 152.  doi: 10.1137/S0036142998335704.  Google Scholar [30] D. H. Li and M. Fukushima, A derivative-free line search and global convergence of Broyden-like method for nonlinear equations,, Optimizaiton Methods and Software, 13 (2000), 181.  doi: 10.1080/10556780008805782.  Google Scholar [31] X. Liu and Y. X. Yuan, On the separable nonlinear least squares problems,, J. Comput. Math., 26 (2008), 390.   Google Scholar [32] J. M. Martinez, A quasi-Newton method with modification of one column per iteration,, Computing, 33 (1984), 353.   Google Scholar [33] E. Mizutani and J. W. Demmel, On structure-exploiting trust-region regularized nonlinear least squares algorithms for neural-network learning,, Neural Networks, 16 (2003), 745.  doi: 10.1016/S0893-6080(03)00085-6.  Google Scholar [34] J. J. Moré, The Levenberg-Marquardt algorithm: implementation and theory,, in, (1978), 105.   Google Scholar [35] J. J. Moré, Recent developments in algorithms and software for trust region methods,, in, (1983), 258.   Google Scholar [36] J. J. Moré and D. C. Sorensen, Computing a trust region step,, SIAM J. Sci. Statist. Compute., 4 (1983), 553.  doi: 10.1137/0904038.  Google Scholar [37] Yu. Nesterov, Modified Gauss-Newton scheme with worst-case grarantees for global performance,, Optimization Methods and Sofware, 22 (2007), 469.  doi: 10.1080/08927020600643812.  Google Scholar [38] J. Nocedal and S. J. Wright, "Numerical Optimization,", Springer, (1999).  doi: 10.1007/b98874.  Google Scholar [39] J. M. Ortega and W. C. Rheinboldt, "Iterative Solution of Nonlinear Equations in Several Variables,", Academic Press, (1970).   Google Scholar [40] M. J. D. Powell, A new algorithm for unconstrained optimization,, in, (1970), 31.   Google Scholar [41] M. J. D. Powell, A method for minimizing a sum of squares of non-linear functions without calculating derivatives,, The Computer J., 7 (1965), 303.   Google Scholar [42] M. J. D. Powell, A hybrid method for nonlinear equations,, in, (1970), 87.   Google Scholar [43] M. J. D. Powell, A fast algorithm for nonlinearly constrained optimization calculations,, in, (1978), 144.  doi: 10.1007/BFb0067703.  Google Scholar [44] M. J. D. Powell and Y. X. Yuan, Conditions for superlinear convergence in $l_1$ and $l_\infty$ solutions of overdetermined nonlinear equations,, IMA J. Numerical Analysis, 4 (1984), 241.  doi: 10.1093/imanum/4.2.241.  Google Scholar [45] A. Ruhe and P. Å. Wedin, Algorithms for separable nonlinear least squares problems,, SIAM Review, 22 (1980), 318.  doi: 10.1137/1022057.  Google Scholar [46] Y. Saad, "Iterative Methods for Sparse Linear Systems,", SIAM, (2003).  doi: 10.1137/1.9780898718003.  Google Scholar [47] S. Schlenkrich, A. Griewank and A. Walther, On the local convergence of adjoint Broyden methods,, Math. Program., 121 (2010), 221.  doi: 10.1007/s10107-008-0232-y.  Google Scholar [48] K. Schittkowski, Solving nonlinear least squares problems by a general purpose SQP-method,, in, (1988), 295.   Google Scholar [49] K. Schittkowski, "Numerical Data Fitting in Dynamical Systems,", Applied Optimization Vol. 77, (2002).   Google Scholar [50] L. K. Schubert, Modification of a quasi-Newton method for nonlinear equations with a sparse Jacobian,, Math. Comp., 24 (1970), 27.  doi: 10.1090/S0025-5718-1970-0258276-9.  Google Scholar [51] T. Steihaug, The conjugate gradient method and trust regions in large scale optimization,, SIAM J. Numerical Analysis, 20 (1983), 626.  doi: 10.1137/0720042.  Google Scholar [52] J. Stoer and Y. X. Yuan, A subspace study on conjugate gradient algorithms,, Z. Angew. Math. Mech., 75 (1995), 69.  doi: 10.1002/zamm.19950750118.  Google Scholar [53] W. Y. Sun and Y. X. Yuan, "Optimization Theory and Mehtods: Nonlinear Programming,", Springer Series on Optimization and Its Application, (2006).   Google Scholar [54] Ph. L. Toint, On sparse and sysmmetric matrix updating subject to a linear equation,, Mathematics of Computation, 31 (1977), 954.  doi: 10.1090/S0025-5718-1977-0455338-4.  Google Scholar [55] Ph. L. Toint, Towards an efficient sparsity exploiting Newton method for minimization,, in, (1981), 57.   Google Scholar [56] Ph. L. Toint, On large scale nonlinear least squares calculations,, SIAM J. Sci. Stat. Comput., 8 (1987), 416.  doi: 10.1137/0908042.  Google Scholar [57] Z. H. Wang and Y. X. Yuan, A subspace implementation of quasi-Newton trust region methods for unconstrained optimization,, Numerische Mathematik, 104 (2006), 241.  doi: 10.1007/s00211-006-0021-6.  Google Scholar [58] N. Yamashita and M. Fukushima, On the rate of convergence of the Levenberg-Marquardt method ,, Computing, 15 (2001), 237.   Google Scholar [59] Y. X. Yuan, Trust region algorithms for nonlinear equations,, Information, 1 (1998), 7.   Google Scholar [60] Y. X. Yuan, On the truncated conjugate gradient method,, Math. Program., 87 (2000), 561.  doi: 10.1007/s101070050012.  Google Scholar [61] Y. X. Yuan, Subspace techniques for nonlinear optimization,, in, (2007), 206.   Google Scholar [62] Y. X. Yuan, Subspace methods for large scale nonlinear equations and nonlinear least squares,, Optimization and Engineering, 10 (2009), 207.  doi: 10.1007/s11081-008-9064-0.  Google Scholar [63] H. C. Zhang, A. R. Conn and K. Scheinberg, A derivative-free algorithm for the least-squares minimization,, , ().   Google Scholar [64] H. C. Zhang, A. R. Conn and K. Scheinberg, On the local convergence of a derivative-free algorithm for least-squares minimization,, , ().   Google Scholar
 [1] Haiyan Wang, Jinyan Fan. Convergence properties of inexact Levenberg-Marquardt method under Hölderian local error bound. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020068 [2] Jinyan Fan, Jianyu Pan. Inexact Levenberg-Marquardt method for nonlinear equations. Discrete & Continuous Dynamical Systems - B, 2004, 4 (4) : 1223-1232. doi: 10.3934/dcdsb.2004.4.1223 [3] Jinyan Fan. On the Levenberg-Marquardt methods for convex constrained nonlinear equations. Journal of Industrial & Management Optimization, 2013, 9 (1) : 227-241. doi: 10.3934/jimo.2013.9.227 [4] Honglan Zhu, Qin Ni, Meilan Zeng. A quasi-Newton trust region method based on a new fractional model. Numerical Algebra, Control & Optimization, 2015, 5 (3) : 237-249. doi: 10.3934/naco.2015.5.237 [5] Jinyan Fan, Jianyu Pan. On the convergence rate of the inexact Levenberg-Marquardt method. Journal of Industrial & Management Optimization, 2011, 7 (1) : 199-210. doi: 10.3934/jimo.2011.7.199 [6] Liyan Qi, Xiantao Xiao, Liwei Zhang. On the global convergence of a parameter-adjusting Levenberg-Marquardt method. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 25-36. doi: 10.3934/naco.2015.5.25 [7] Johann Baumeister, Barbara Kaltenbacher, Antonio Leitão. On Levenberg-Marquardt-Kaczmarz iterative methods for solving systems of nonlinear ill-posed equations. Inverse Problems & Imaging, 2010, 4 (3) : 335-350. doi: 10.3934/ipi.2010.4.335 [8] Zhou Sheng, Gonglin Yuan, Zengru Cui, Xiabin Duan, Xiaoliang Wang. An adaptive trust region algorithm for large-residual nonsmooth least squares problems. Journal of Industrial & Management Optimization, 2018, 14 (2) : 707-718. doi: 10.3934/jimo.2017070 [9] Petr Knobloch. Error estimates for a nonlinear local projection stabilization of transient convection--diffusion--reaction equations. Discrete & Continuous Dynamical Systems - S, 2015, 8 (5) : 901-911. doi: 10.3934/dcdss.2015.8.901 [10] Yanfei Lu, Qingfei Yin, Hongyi Li, Hongli Sun, Yunlei Yang, Muzhou Hou. Solving higher order nonlinear ordinary differential equations with least squares support vector machines. Journal of Industrial & Management Optimization, 2020, 16 (3) : 1481-1502. doi: 10.3934/jimo.2019012 [11] Yuhong Dai, Nobuo Yamashita. Convergence analysis of sparse quasi-Newton updates with positive definite matrix completion for two-dimensional functions. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 61-69. doi: 10.3934/naco.2011.1.61 [12] Hassan Mohammad, Mohammed Yusuf Waziri, Sandra Augusta Santos. A brief survey of methods for solving nonlinear least-squares problems. Numerical Algebra, Control & Optimization, 2019, 9 (1) : 1-13. doi: 10.3934/naco.2019001 [13] Xin Zhang, Jie Wen, Qin Ni. Subspace trust-region algorithm with conic model for unconstrained optimization. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 223-234. doi: 10.3934/naco.2013.3.223 [14] Basim A. Hassan. A new type of quasi-newton updating formulas based on the new quasi-newton equation. Numerical Algebra, Control & Optimization, 2020, 10 (2) : 227-235. doi: 10.3934/naco.2019049 [15] Liping Zhang, Soon-Yi Wu, Shu-Cherng Fang. Convergence and error bound of a D-gap function based Newton-type algorithm for equilibrium problems. Journal of Industrial & Management Optimization, 2010, 6 (2) : 333-346. doi: 10.3934/jimo.2010.6.333 [16] Enrique Fernández-Cara, Arnaud Münch. Numerical null controllability of semi-linear 1-D heat equations: Fixed point, least squares and Newton methods. Mathematical Control & Related Fields, 2012, 2 (3) : 217-246. doi: 10.3934/mcrf.2012.2.217 [17] Jing Zhou, Cheng Lu, Ye Tian, Xiaoying Tang. A socp relaxation based branch-and-bound method for generalized trust-region subproblem. Journal of Industrial & Management Optimization, 2019  doi: 10.3934/jimo.2019104 [18] Jun Takaki, Nobuo Yamashita. A derivative-free trust-region algorithm for unconstrained optimization with controlled error. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 117-145. doi: 10.3934/naco.2011.1.117 [19] Shummin Nakayama, Yasushi Narushima, Hiroshi Yabe. Memoryless quasi-Newton methods based on spectral-scaling Broyden family for unconstrained optimization. Journal of Industrial & Management Optimization, 2019, 15 (4) : 1773-1793. doi: 10.3934/jimo.2018122 [20] Koichiro Naito. Recurrent dimensions of quasi-periodic solutions for nonlinear evolution equations II: Gaps of dimensions and Diophantine conditions. Discrete & Continuous Dynamical Systems - A, 2004, 11 (2&3) : 449-488. doi: 10.3934/dcds.2004.11.449

Impact Factor: