American Institute of Mathematical Sciences

Advanced
  • Previous Article
    A parallel Gauss-Seidel method for convex problems with separable structure
  • NACO Home
  • This Issue
  • Next Article
    Two-stage stochastic variational inequalities for Cournot-Nash equilibrium with risk-averse players under uncertainty
December  2020, 10(4): 537-555. doi: 10.3934/naco.2020050

Survey of derivative-free optimization

Min Xi 1,2,, , Wenyu Sun 1, and  Jun Chen 3,

1. 

School of Mathematical Sciences, Jiangsu Key Laboratory for NSLSCS, Nanjing Normal University, Nanjing 210023, China
2. 

School of Mathematics and Statistics, Guangdong University of Foreign Studies, Guangzhou 510006, China
3. 

School of Information Engineering, Nanjing Xiaozhuang University, Nanjing 211171, China

* Corresponding author: Min Xi

Received  April 2020 Revised  September 2020 Published  September 2020

Fund Project: This research was supported by the National Natural Science Foundation of China under grants 11571178

In this survey paper we present an overview of derivative-free optimization, including basic concepts, theories, derivative-free methods and some applications. To date, there are mainly three classes of derivative-free methods and we concentrate on two of them, they are direct search methods and model-based methods. In this paper, we first focus on unconstrained optimization problems and review some classical direct search methods and model-based methods in turn for these problems. Then, we survey a number of derivative-free approaches for problems with constraints, including an algorithm we proposed for spherical optimization recently.

Keywords: Derivative-free optimization, direct search methods, model-based methods, unconstrained optimization.
Mathematics Subject Classification: Primary: 90C56; Secondary: 90C30.
Citation: Min Xi, Wenyu Sun, Jun Chen. Survey of derivative-free optimization. Numerical Algebra, Control & Optimization, 2020, 10 (4) : 537-555. doi: 10.3934/naco.2020050
References:
[1]

M. A. Abramson and C. Audet, Convergence of mesh adaptive direct search to second-order stationary points, SIAM Journal on Optimization, 17 (2006), 606-609.  doi: 10.1137/050638382.  Google Scholar

[2]

P. Alberto, F. Nogueira, H. Rocha and L. N. Vicente, Pattern search methods for user-provided points: Application to molecular geometry problems, SIAM Journal on Optimization, 14 (2004), 1216-1236.  doi: 10.1137/S1052623400377955.  Google Scholar

[3]

C. Audet, J. E. Dennis and Jr., A pattern search filter method for nonlinear programming without derivatives, SIAM Journal on Optimization, 14 (2004), 980-1010.  doi: 10.1137/S105262340138983X.  Google Scholar

[4]

C. Audet, J. E. Dennis and Jr., Mesh adaptive direct search algorithms for constrained optimization, SIAM Journal on Optimization, 17 (2006), 188-217.  doi: 10.1137/040603371.  Google Scholar

[5]

C. Audet, Convergence results for generalized pattern search algorithms are tight, Optimization and Engineering, 5 (2004), 101-122.  doi: 10.1023/B:OPTE.0000033370.66768.a9.  Google Scholar

[6]

A. S. Bandeira, K. Scheinberg and L. N. Vicente, Convergence of trust-region methods based on probabilistic models, Mathematical Programming, 24 (2014), 1238-1264.  doi: 10.1137/130915984.  Google Scholar

[7]

A. J. Booker, J. E. Dennis, Jr., P. D. Frank, D. W. Moore and D. B. Serafini, Managing surrogate objectives to optimize a helicopter rotor design-Further experiments, in AIAA Paper 1998–4717, 8th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis, MO, 1998. Google Scholar

[8]

M. J. Box, A new method of constrained optimization and a comparison with other methods, Computer Journal, 8 (1965), 42-52.  doi: 10.1093/comjnl/8.1.42.  Google Scholar

[9]

T. D. Choi and C. T. Kelley, Superlinear convergence and implicit filtering, SIAM Journal on Optimization, 10 (2000), 1149-1162.  doi: 10.1137/S1052623499354096.  Google Scholar

[10]

B. Colson and Ph. L. Toint, Exploiting band structure in unconstrained optimization without derivatives, in Trends in Industrial and Applied Mathematics, Vol. 72 Applied Optimization, (eds. A.H. Siddiqi and M. Kocvera), Kluwer, (2002), 131–147. doi: 10.1023/A:1016090421852.  Google Scholar

[11]

B. Colson and Ph. L. Toint, Optimizing partially separable functions without derivatives, Optimization Methods and Software, 20 (2005), 493-508.  doi: 10.1080/10556780500140227.  Google Scholar

[12]

P. Conejo, E. Karas and P. Ledroso, A trust-region derivative-free algorithm for constrained optimization, Optimization Methods and Software, 30 (2015), 1126-1145.  doi: 10.1080/10556788.2015.1026968.  Google Scholar

[13]

A. R. Conn, N. I. M. Gould and Ph. L. Toint, Trust-Region Methods, SIAM, Philadephia, 2000. doi: 10.1137/1.9780898719857.  Google Scholar

[14]

A. R. Conn, K. Scheinberg and Ph. L. Toint, On the convergence of derivative-free methods for unconstrained optimization, in Approximation Theory and Optimization: Tributes to M. J. D. Powell(eds. A. Iserles and M. Buhmann), Cambridge University Press, Cambridge, (1996), 83–108.  Google Scholar

[15]

A. R. Conn, K. Scheinberg and Ph. L.Toint, Recent progress in unconstrained nonlinear optimization without derivatives, Mathematical Programming, 79 (1997), 397-414.  doi: 10.1016/S0025-5610(97)00054-3.  Google Scholar

[16]

A. R. Conn, K. Scheinberg and L. N. Vicente, Geometry of interpolation sets in derivative free optimization, Mathematical Programming, 111 (2008), 141-172.  doi: 10.1007/s10107-006-0073-5.  Google Scholar

[17]

A. R. Conn, K. Scheinberg and L. N. Vicente, Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points, SIAM Journal on Optimization, 20 (2009), 387-415.  doi: 10.1137/060673424.  Google Scholar

[18]

A. R. Conn, K. Scheinberg and L. N. Vicente, Introduction to Derivative-free Optimization, MPS/SIAM Ser. Optim. 8, SIAM, Philadelphia, 2009. doi: 10.1137/1.9780898718768.  Google Scholar

[19]

A. R. Conn and Ph. L. Toint, An algorithm using quadratic interpolation for unconstrained derivative-free optimization, in Nonlinear Optimization and Application (eds. G. Di. Pillo and F. Giannessi), Plenium Publishing, New York, (1996), 27–47. Google Scholar

[20]

J. E. Dennis Jr. and V. J. Torczon, Direct search methods on parallel machines, SIAM Journal on Optimization, 1 (1991), 448-474.  doi: 10.1137/0801027.  Google Scholar

[21]

L. C. W. Dixon, ASCIM-an accelerated constrained simplex technique, Computer-aided Design, 5 (1973), 22-32.   Google Scholar

[22]

R. Duvigneau and M. Visonneau, Hydrodynamic design using a derivative-free method, Structural and Multidisciplinary Optimization, 28 (2004), 195-205.   Google Scholar

[23]

E. Fermi and N. Metropolis, Los Alamos Unclassified Report LS–1492, in Technical Report, Los Alamos National Laboratory, Los Alamos, NM, 1952. Google Scholar

[24]

K. R. Fowler, J. P. Reese, C. E. Kees, J. E. Dennis, Jr., C. T. Kelley, C. T. Miller, C. Audet, A. J. Booker, G. Couyure, R. W. Darwin, M. W. Farthing, D. E. Finkel, J. M. Gablonsky, G. Gray and T. G. Kolda, A comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems, Advances in Water Resources, 31 (2007), 743-757.   Google Scholar

[25]

P. Gilmore and C. T. Kelley, An implicit filtering algorithm for optimization of functions with many local minima, SIAM Journal on Optimization, 5 (1995), 269-285.  doi: 10.1137/0805015.  Google Scholar

[26]

G. N. Grapiglia, J. Yuan and Y. Yuan, A derivative-free trust-region algorithm for composite nonsmooth optimization, Journal of Computational and Applied Mathematics, 35 (2016), 475-499.  doi: 10.1007/s40314-014-0201-4.  Google Scholar

[27]

W. L. Hare and R. A. Poliquin, Prox-regularity and stability of the proximal mapping, Journal of Convex Analysis, 14 (2007), 589-606.   Google Scholar

[28] G. H. Golub and C. F. Van Loan, Matrix Computations, Johns Hopkins University Press, Baltimore, MD, 2013.   Google Scholar
[29]

R. Hooke and T. A. Jeeves, Direct search solution of numerical and statistical problems, Journal of the Association for Computing Machinery, 8 (1961), 212-229.   Google Scholar

[30]

X. J. Huang and D. T. Zhu, An interior affine scaling cubic regularization algorithm for derivative-free optimization subject to bound constraints, Journal of Computational and Applied Mathematics, 321 (2017), 108-127.  doi: 10.1016/j.cam.2017.02.028.  Google Scholar

[31]

D. L. Keefer, Simpat: Self-bounding direct search method for optimization, Industrial and Engineering Chemistry Process Design and Development, 12 (1973), 92-99.   Google Scholar

[32]

C. T. Kelley, Iterative Methods for Optimization, Frontiers Appl. Math. 18, SIAM, Philadelphia, 1999. doi: 10.1137/1.9781611970920.  Google Scholar

[33]

T. G. Kolda and V. J. Torczon, On the convergence of asynchronous parallel pattern search, SIAM Journal on Optimization, 14 (2004), 939-964.  doi: 10.1137/S1052623401398107.  Google Scholar

[34]

J. Larson and S. C. Billups, Stochastic derivative-free optimization using a trust region framework, Computational Optimization and Applications, 64 (2016), 619-645.  doi: 10.1007/s10589-016-9827-z.  Google Scholar

[35]

T. Levina, Y. Levin, J. Mcgill and M. Nediak, Dynamic pricing with online learning and strategic consumers: An application of the aggregating algorithm, Operations Research, 57 (2009), 327-341.   Google Scholar

[36]

R. M. Lewis and V. Torczon, Pattern search algorithms for bound constrained minimization, SIAM Journal on Optimization, 9 (1999), 1082-1099.  doi: 10.1137/S1052623496300507.  Google Scholar

[37]

R. M. Lewis and V. Torczon, Pattern search methods for linearly constrained minimization, SIAM Journal on Optimization, 10 (2000), 917-941.  doi: 10.1137/S1052623497331373.  Google Scholar

[38]

R. M. Lewis and V. Torczon, A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds, SIAM Journal on Optimization, 12 (2002), 1075-1089.  doi: 10.1137/S1052623498339727.  Google Scholar

[39]

R. M. Lewis, V. Torczon and M. W. Trosset, Direct search methods: then and now, Journal of Computational and Applied Mathematics, 124 (2000), 191-207.  doi: 10.1016/S0377-0427(00)00423-4.  Google Scholar

[40]

S. Lucidi and M. Sciandrone, On the global convergence of derivative-free methods for unconstrained minimization, SIAM Journal on Optimization, 13 (2002), 97-116.  doi: 10.1137/S1052623497330392.  Google Scholar

[41]

G. Liuzzi, S. Lucidi and M. Sciandrone, Sequential penalty derivative-free methods for nonlinear constrained optimization, SIAM Journal on Optimization, 20 (2010), 2614-2635.  doi: 10.1137/090750639.  Google Scholar

[42]

A. L. Marsden, Aerodynamic Noise Control by Optimal Shape Design, Ph.D thesis, Stanford University, Stanford, CA, 2004. Google Scholar

[43]

J. H. May, Linearly Constrained Nonlinear Programming: A Solution Method That Does Not Require Analytic Derivatives, Ph.D thesis, Yale University, New Haven, CT, 1974.  Google Scholar

[44]

J. C. Meza and M. L. Martinez, On the use of direct search methods for the molecular conformation problem, Journal of Computational Chemistry, 15 (1994), 627-632.   Google Scholar

[45]

K. I. M. McKinnon, Convergence of the Nelder-Mead simplex method to a nonstationary point, SIAM Journal on Optimization, 9 (1998), 148-158.  doi: 10.1137/S1052623496303482.  Google Scholar

[46]

J. J. Moré and S. M. Wild, Benchmarking derivative-free optimization algorithms, SIAM Journal on Optimization, 20 (2009), 172-191.  doi: 10.1137/080724083.  Google Scholar

[47]

L. Nazareth and P. Tseng, Gilding the lily: A variant of the Nelder-Mead algorithm based on golden-section search, Computational Optimization and Applications, 22 (2002), 133-144.  doi: 10.1023/A:1014842520519.  Google Scholar

[48]

J. A. Nelder and R. Mead, A simplex method for function minimization, Computer Journal, 7 (1965), 308-313.  doi: 10.1093/comjnl/7.4.308.  Google Scholar

[49]

J. Nocedal and S. J. Wright, Numerical Optimization, 2$^nd$ edition, Springer-Verlag, New York, 2006.  Google Scholar

[50]

M. J. D. Powell, A direct search optimization method that models the objective and constraint functions by linear interpolation, in Advances in Optimization and Numerical Analysis(eds. S. Gomez and J-P. Hennart), Kluwer Academic Publishers, Dordrecht, (1994), 52–67.  Google Scholar

[51]

M. J. D. Powell, A direct search optimization method that models the objective by quadratic interpolation, in Presentation at the 5th Stockholm Optimization Days, 1994. Google Scholar

[52]

M. J. D. Powell, UOBYQA: Unconsrained optimization by quadratic approximation, Mathematical Programning, 92 (2002), 555-582.  doi: 10.1007/s101070100290.  Google Scholar

[53]

M. J. D. Powell, The NEWUOA software for unconstrained optimization without derivatives, in Large-Scale Nonlinear Optimization(eds. G. Di Pillo and M. Roma), Springer, New York, 2006. doi: 10.1007/0-387-30065-1_16.  Google Scholar

[54]

M. J. D. Powell, The BOBYQA algorithm for bound constrained optimization without derivatives, Technical Report, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, 2009.  Google Scholar

[55]

C. J. Price, I. D. Coope and D. Byatt, A convergent variant of the Nelder Mead algorithm, Journal of Optimization Theory and Applications, 113 (2002), 5-19.  doi: 10.1023/A:1014849028575.  Google Scholar

[56]

P. R. Sampaio and Ph. L. Toint, A derivative-free trust-funnel method for equality-constrained, Computational Optimization and Applications, 61 (2015), 25-49.  doi: 10.1007/s10589-014-9715-3.  Google Scholar

[57]

M. Schäfer, B. Karasözen, Y. Uluda$\breve{g}$, K. Yapici and Ö. U$\breve{g}$ur, Numerical method for optimizing stirrer configurations, Computers and Chemical Engineering, 30 (2005), 183-190.   Google Scholar

[58]

K. Scheinberg, Manual for Fortran Software Package DFO v2.0, 2003. Google Scholar

[59]

K. Scheinberg and Ph. L. Toint, Self-correcting geometry in model-based algorithms for derivative-free unconstrained optimization, SIAM Journal on Optimization, 20 (2010), 3512-3532.  doi: 10.1137/090748536.  Google Scholar

[60]

W. Spendley, G. R. Hext and F. R. Himsworth, Sequential application for simplex designs in optimisation and evolutionary operation, Technometrics, 4 (1962), 441-461.  doi: 10.2307/1266283.  Google Scholar

[61]

J. Takaki and N. Yamashita, A derivative-free trust-region algorithm for unconstrained optimization with controlled error, Numerical Algebra, Control and Optimization, 1 (2011), 117-145.  doi: 10.3934/naco.2011.1.117.  Google Scholar

[62]

V. Torczon, Multi-directional Search: A Direct Search Algorithm For Parallel Machines, Ph.D thesis, Rice University, Houston, TX, 1989.  Google Scholar

[63]

V. Torczon, On the convergence of multidirectional search algorithms, SIAM Journal on Optimization, 1 (1991), 123-145.  doi: 10.1137/0801010.  Google Scholar

[64]

V. Torczon, On the convergence of pattern search algorithms, SIAM Journal on Optimization, 7 (1997), 1-25.  doi: 10.1137/S1052623493250780.  Google Scholar

[65]

P. Tseng, Fortified-descent simplicial search method: A general approach, SIAM Journal on Optimization, 10 (2000), 269-288.  doi: 10.1137/S1052623495282857.  Google Scholar

[66]

D. Winfield, Function and Functional Optimization by Interpolation in Data Table, Ph.D Thesis, Harvard University, Cambridge, MA, 1969. Google Scholar

[67]

D. Winfield, Function minimization by interpolation in a data table, J. Inst. Math. Appl., 12 (1973), 339-347.   Google Scholar

[68]

J.-H. Yoon and C. A. Shoemaker, Comparison of optimization methods for ground-water bioremediation, Journal of Water Resources Planning and Management, 125 (1999), 54-63.   Google Scholar

show all references

References:
[1]

M. A. Abramson and C. Audet, Convergence of mesh adaptive direct search to second-order stationary points, SIAM Journal on Optimization, 17 (2006), 606-609.  doi: 10.1137/050638382.  Google Scholar

[2]

P. Alberto, F. Nogueira, H. Rocha and L. N. Vicente, Pattern search methods for user-provided points: Application to molecular geometry problems, SIAM Journal on Optimization, 14 (2004), 1216-1236.  doi: 10.1137/S1052623400377955.  Google Scholar

[3]

C. Audet, J. E. Dennis and Jr., A pattern search filter method for nonlinear programming without derivatives, SIAM Journal on Optimization, 14 (2004), 980-1010.  doi: 10.1137/S105262340138983X.  Google Scholar

[4]

C. Audet, J. E. Dennis and Jr., Mesh adaptive direct search algorithms for constrained optimization, SIAM Journal on Optimization, 17 (2006), 188-217.  doi: 10.1137/040603371.  Google Scholar

[5]

C. Audet, Convergence results for generalized pattern search algorithms are tight, Optimization and Engineering, 5 (2004), 101-122.  doi: 10.1023/B:OPTE.0000033370.66768.a9.  Google Scholar

[6]

A. S. Bandeira, K. Scheinberg and L. N. Vicente, Convergence of trust-region methods based on probabilistic models, Mathematical Programming, 24 (2014), 1238-1264.  doi: 10.1137/130915984.  Google Scholar

[7]

A. J. Booker, J. E. Dennis, Jr., P. D. Frank, D. W. Moore and D. B. Serafini, Managing surrogate objectives to optimize a helicopter rotor design-Further experiments, in AIAA Paper 1998–4717, 8th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis, MO, 1998. Google Scholar

[8]

M. J. Box, A new method of constrained optimization and a comparison with other methods, Computer Journal, 8 (1965), 42-52.  doi: 10.1093/comjnl/8.1.42.  Google Scholar

[9]

T. D. Choi and C. T. Kelley, Superlinear convergence and implicit filtering, SIAM Journal on Optimization, 10 (2000), 1149-1162.  doi: 10.1137/S1052623499354096.  Google Scholar

[10]

B. Colson and Ph. L. Toint, Exploiting band structure in unconstrained optimization without derivatives, in Trends in Industrial and Applied Mathematics, Vol. 72 Applied Optimization, (eds. A.H. Siddiqi and M. Kocvera), Kluwer, (2002), 131–147. doi: 10.1023/A:1016090421852.  Google Scholar

[11]

B. Colson and Ph. L. Toint, Optimizing partially separable functions without derivatives, Optimization Methods and Software, 20 (2005), 493-508.  doi: 10.1080/10556780500140227.  Google Scholar

[12]

P. Conejo, E. Karas and P. Ledroso, A trust-region derivative-free algorithm for constrained optimization, Optimization Methods and Software, 30 (2015), 1126-1145.  doi: 10.1080/10556788.2015.1026968.  Google Scholar

[13]

A. R. Conn, N. I. M. Gould and Ph. L. Toint, Trust-Region Methods, SIAM, Philadephia, 2000. doi: 10.1137/1.9780898719857.  Google Scholar

[14]

A. R. Conn, K. Scheinberg and Ph. L. Toint, On the convergence of derivative-free methods for unconstrained optimization, in Approximation Theory and Optimization: Tributes to M. J. D. Powell(eds. A. Iserles and M. Buhmann), Cambridge University Press, Cambridge, (1996), 83–108.  Google Scholar

[15]

A. R. Conn, K. Scheinberg and Ph. L.Toint, Recent progress in unconstrained nonlinear optimization without derivatives, Mathematical Programming, 79 (1997), 397-414.  doi: 10.1016/S0025-5610(97)00054-3.  Google Scholar

[16]

A. R. Conn, K. Scheinberg and L. N. Vicente, Geometry of interpolation sets in derivative free optimization, Mathematical Programming, 111 (2008), 141-172.  doi: 10.1007/s10107-006-0073-5.  Google Scholar

[17]

A. R. Conn, K. Scheinberg and L. N. Vicente, Global convergence of general derivative-free trust-region algorithms to first- and second-order critical points, SIAM Journal on Optimization, 20 (2009), 387-415.  doi: 10.1137/060673424.  Google Scholar

[18]

A. R. Conn, K. Scheinberg and L. N. Vicente, Introduction to Derivative-free Optimization, MPS/SIAM Ser. Optim. 8, SIAM, Philadelphia, 2009. doi: 10.1137/1.9780898718768.  Google Scholar

[19]

A. R. Conn and Ph. L. Toint, An algorithm using quadratic interpolation for unconstrained derivative-free optimization, in Nonlinear Optimization and Application (eds. G. Di. Pillo and F. Giannessi), Plenium Publishing, New York, (1996), 27–47. Google Scholar

[20]

J. E. Dennis Jr. and V. J. Torczon, Direct search methods on parallel machines, SIAM Journal on Optimization, 1 (1991), 448-474.  doi: 10.1137/0801027.  Google Scholar

[21]

L. C. W. Dixon, ASCIM-an accelerated constrained simplex technique, Computer-aided Design, 5 (1973), 22-32.   Google Scholar

[22]

R. Duvigneau and M. Visonneau, Hydrodynamic design using a derivative-free method, Structural and Multidisciplinary Optimization, 28 (2004), 195-205.   Google Scholar

[23]

E. Fermi and N. Metropolis, Los Alamos Unclassified Report LS–1492, in Technical Report, Los Alamos National Laboratory, Los Alamos, NM, 1952. Google Scholar

[24]

K. R. Fowler, J. P. Reese, C. E. Kees, J. E. Dennis, Jr., C. T. Kelley, C. T. Miller, C. Audet, A. J. Booker, G. Couyure, R. W. Darwin, M. W. Farthing, D. E. Finkel, J. M. Gablonsky, G. Gray and T. G. Kolda, A comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems, Advances in Water Resources, 31 (2007), 743-757.   Google Scholar

[25]

P. Gilmore and C. T. Kelley, An implicit filtering algorithm for optimization of functions with many local minima, SIAM Journal on Optimization, 5 (1995), 269-285.  doi: 10.1137/0805015.  Google Scholar

[26]

G. N. Grapiglia, J. Yuan and Y. Yuan, A derivative-free trust-region algorithm for composite nonsmooth optimization, Journal of Computational and Applied Mathematics, 35 (2016), 475-499.  doi: 10.1007/s40314-014-0201-4.  Google Scholar

[27]

W. L. Hare and R. A. Poliquin, Prox-regularity and stability of the proximal mapping, Journal of Convex Analysis, 14 (2007), 589-606.   Google Scholar

[28] G. H. Golub and C. F. Van Loan, Matrix Computations, Johns Hopkins University Press, Baltimore, MD, 2013.   Google Scholar
[29]

R. Hooke and T. A. Jeeves, Direct search solution of numerical and statistical problems, Journal of the Association for Computing Machinery, 8 (1961), 212-229.   Google Scholar

[30]

X. J. Huang and D. T. Zhu, An interior affine scaling cubic regularization algorithm for derivative-free optimization subject to bound constraints, Journal of Computational and Applied Mathematics, 321 (2017), 108-127.  doi: 10.1016/j.cam.2017.02.028.  Google Scholar

[31]

D. L. Keefer, Simpat: Self-bounding direct search method for optimization, Industrial and Engineering Chemistry Process Design and Development, 12 (1973), 92-99.   Google Scholar

[32]

C. T. Kelley, Iterative Methods for Optimization, Frontiers Appl. Math. 18, SIAM, Philadelphia, 1999. doi: 10.1137/1.9781611970920.  Google Scholar

[33]

T. G. Kolda and V. J. Torczon, On the convergence of asynchronous parallel pattern search, SIAM Journal on Optimization, 14 (2004), 939-964.  doi: 10.1137/S1052623401398107.  Google Scholar

[34]

J. Larson and S. C. Billups, Stochastic derivative-free optimization using a trust region framework, Computational Optimization and Applications, 64 (2016), 619-645.  doi: 10.1007/s10589-016-9827-z.  Google Scholar

[35]

T. Levina, Y. Levin, J. Mcgill and M. Nediak, Dynamic pricing with online learning and strategic consumers: An application of the aggregating algorithm, Operations Research, 57 (2009), 327-341.   Google Scholar

[36]

R. M. Lewis and V. Torczon, Pattern search algorithms for bound constrained minimization, SIAM Journal on Optimization, 9 (1999), 1082-1099.  doi: 10.1137/S1052623496300507.  Google Scholar

[37]

R. M. Lewis and V. Torczon, Pattern search methods for linearly constrained minimization, SIAM Journal on Optimization, 10 (2000), 917-941.  doi: 10.1137/S1052623497331373.  Google Scholar

[38]

R. M. Lewis and V. Torczon, A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds, SIAM Journal on Optimization, 12 (2002), 1075-1089.  doi: 10.1137/S1052623498339727.  Google Scholar

[39]

R. M. Lewis, V. Torczon and M. W. Trosset, Direct search methods: then and now, Journal of Computational and Applied Mathematics, 124 (2000), 191-207.  doi: 10.1016/S0377-0427(00)00423-4.  Google Scholar

[40]

S. Lucidi and M. Sciandrone, On the global convergence of derivative-free methods for unconstrained minimization, SIAM Journal on Optimization, 13 (2002), 97-116.  doi: 10.1137/S1052623497330392.  Google Scholar

[41]

G. Liuzzi, S. Lucidi and M. Sciandrone, Sequential penalty derivative-free methods for nonlinear constrained optimization, SIAM Journal on Optimization, 20 (2010), 2614-2635.  doi: 10.1137/090750639.  Google Scholar

[42]

A. L. Marsden, Aerodynamic Noise Control by Optimal Shape Design, Ph.D thesis, Stanford University, Stanford, CA, 2004. Google Scholar

[43]

J. H. May, Linearly Constrained Nonlinear Programming: A Solution Method That Does Not Require Analytic Derivatives, Ph.D thesis, Yale University, New Haven, CT, 1974.  Google Scholar

[44]

J. C. Meza and M. L. Martinez, On the use of direct search methods for the molecular conformation problem, Journal of Computational Chemistry, 15 (1994), 627-632.   Google Scholar

[45]

K. I. M. McKinnon, Convergence of the Nelder-Mead simplex method to a nonstationary point, SIAM Journal on Optimization, 9 (1998), 148-158.  doi: 10.1137/S1052623496303482.  Google Scholar

[46]

J. J. Moré and S. M. Wild, Benchmarking derivative-free optimization algorithms, SIAM Journal on Optimization, 20 (2009), 172-191.  doi: 10.1137/080724083.  Google Scholar

[47]

L. Nazareth and P. Tseng, Gilding the lily: A variant of the Nelder-Mead algorithm based on golden-section search, Computational Optimization and Applications, 22 (2002), 133-144.  doi: 10.1023/A:1014842520519.  Google Scholar

[48]

J. A. Nelder and R. Mead, A simplex method for function minimization, Computer Journal, 7 (1965), 308-313.  doi: 10.1093/comjnl/7.4.308.  Google Scholar

[49]

J. Nocedal and S. J. Wright, Numerical Optimization, 2$^nd$ edition, Springer-Verlag, New York, 2006.  Google Scholar

[50]

M. J. D. Powell, A direct search optimization method that models the objective and constraint functions by linear interpolation, in Advances in Optimization and Numerical Analysis(eds. S. Gomez and J-P. Hennart), Kluwer Academic Publishers, Dordrecht, (1994), 52–67.  Google Scholar

[51]

M. J. D. Powell, A direct search optimization method that models the objective by quadratic interpolation, in Presentation at the 5th Stockholm Optimization Days, 1994. Google Scholar

[52]

M. J. D. Powell, UOBYQA: Unconsrained optimization by quadratic approximation, Mathematical Programning, 92 (2002), 555-582.  doi: 10.1007/s101070100290.  Google Scholar

[53]

M. J. D. Powell, The NEWUOA software for unconstrained optimization without derivatives, in Large-Scale Nonlinear Optimization(eds. G. Di Pillo and M. Roma), Springer, New York, 2006. doi: 10.1007/0-387-30065-1_16.  Google Scholar

[54]

M. J. D. Powell, The BOBYQA algorithm for bound constrained optimization without derivatives, Technical Report, Department of Applied Mathematics and Theoretical Physics, University of Cambridge, 2009.  Google Scholar

[55]

C. J. Price, I. D. Coope and D. Byatt, A convergent variant of the Nelder Mead algorithm, Journal of Optimization Theory and Applications, 113 (2002), 5-19.  doi: 10.1023/A:1014849028575.  Google Scholar

[56]

P. R. Sampaio and Ph. L. Toint, A derivative-free trust-funnel method for equality-constrained, Computational Optimization and Applications, 61 (2015), 25-49.  doi: 10.1007/s10589-014-9715-3.  Google Scholar

[57]

M. Schäfer, B. Karasözen, Y. Uluda$\breve{g}$, K. Yapici and Ö. U$\breve{g}$ur, Numerical method for optimizing stirrer configurations, Computers and Chemical Engineering, 30 (2005), 183-190.   Google Scholar

[58]

K. Scheinberg, Manual for Fortran Software Package DFO v2.0, 2003. Google Scholar

[59]

K. Scheinberg and Ph. L. Toint, Self-correcting geometry in model-based algorithms for derivative-free unconstrained optimization, SIAM Journal on Optimization, 20 (2010), 3512-3532.  doi: 10.1137/090748536.  Google Scholar

[60]

W. Spendley, G. R. Hext and F. R. Himsworth, Sequential application for simplex designs in optimisation and evolutionary operation, Technometrics, 4 (1962), 441-461.  doi: 10.2307/1266283.  Google Scholar

[61]

J. Takaki and N. Yamashita, A derivative-free trust-region algorithm for unconstrained optimization with controlled error, Numerical Algebra, Control and Optimization, 1 (2011), 117-145.  doi: 10.3934/naco.2011.1.117.  Google Scholar

[62]

V. Torczon, Multi-directional Search: A Direct Search Algorithm For Parallel Machines, Ph.D thesis, Rice University, Houston, TX, 1989.  Google Scholar

[63]

V. Torczon, On the convergence of multidirectional search algorithms, SIAM Journal on Optimization, 1 (1991), 123-145.  doi: 10.1137/0801010.  Google Scholar

[64]

V. Torczon, On the convergence of pattern search algorithms, SIAM Journal on Optimization, 7 (1997), 1-25.  doi: 10.1137/S1052623493250780.  Google Scholar

[65]

P. Tseng, Fortified-descent simplicial search method: A general approach, SIAM Journal on Optimization, 10 (2000), 269-288.  doi: 10.1137/S1052623495282857.  Google Scholar

[66]

D. Winfield, Function and Functional Optimization by Interpolation in Data Table, Ph.D Thesis, Harvard University, Cambridge, MA, 1969. Google Scholar

[67]

D. Winfield, Function minimization by interpolation in a data table, J. Inst. Math. Appl., 12 (1973), 339-347.   Google Scholar

[68]

J.-H. Yoon and C. A. Shoemaker, Comparison of optimization methods for ground-water bioremediation, Journal of Water Resources Planning and Management, 125 (1999), 54-63.   Google Scholar

[1]

Jun Takaki, Nobuo Yamashita. A derivative-free trust-region algorithm for unconstrained optimization with controlled error. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 117-145. doi: 10.3934/naco.2011.1.117
[2]

Bülent Karasözen. Survey of trust-region derivative free optimization methods. Journal of Industrial & Management Optimization, 2007, 3 (2) : 321-334. doi: 10.3934/jimo.2007.3.321
[3]

Sarra Delladji, Mohammed Belloufi, Badreddine Sellami. Behavior of the combination of PRP and HZ methods for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (3) : 377-389. doi: 10.3934/naco.2020032
[4]

A. M. Bagirov, Moumita Ghosh, Dean Webb. A derivative-free method for linearly constrained nonsmooth optimization. Journal of Industrial & Management Optimization, 2006, 2 (3) : 319-338. doi: 10.3934/jimo.2006.2.319
[5]

Shummin Nakayama, Yasushi Narushima, Hiroshi Yabe. Memoryless quasi-Newton methods based on spectral-scaling Broyden family for unconstrained optimization. Journal of Industrial & Management Optimization, 2019, 15 (4) : 1773-1793. doi: 10.3934/jimo.2018122
[6]

Zhong Wan, Chaoming Hu, Zhanlu Yang. A spectral PRP conjugate gradient methods for nonconvex optimization problem based on modified line search. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1157-1169. doi: 10.3934/dcdsb.2011.16.1157
[7]

Wataru Nakamura, Yasushi Narushima, Hiroshi Yabe. Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization. Journal of Industrial & Management Optimization, 2013, 9 (3) : 595-619. doi: 10.3934/jimo.2013.9.595
[8]

Liang Zhang, Wenyu Sun, Raimundo J. B. de Sampaio, Jinyun Yuan. A wedge trust region method with self-correcting geometry for derivative-free optimization. Numerical Algebra, Control & Optimization, 2015, 5 (2) : 169-184. doi: 10.3934/naco.2015.5.169
[9]

Alexandr Golodnikov, Stan Uryasev, Grigoriy Zrazhevsky, Yevgeny Macheret, A. Alexandre Trindade. Optimization of composition and processing parameters for alloy development: a statistical model-based approach. Journal of Industrial & Management Optimization, 2007, 3 (3) : 489-501. doi: 10.3934/jimo.2007.3.489
[10]

Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 377-387. doi: 10.3934/naco.2018024
[11]

M. S. Lee, H. G. Harno, B. S. Goh, K. H. Lim. On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 45-61. doi: 10.3934/naco.2020014
[12]

Mauro Pontani. Orbital transfers: optimization methods and recent results. Numerical Algebra, Control & Optimization, 2011, 1 (3) : 435-485. doi: 10.3934/naco.2011.1.435
[13]

Jie Sun. On methods for solving nonlinear semidefinite optimization problems. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 1-14. doi: 10.3934/naco.2011.1.1
[14]

Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115
[15]

Wenxiong Chen, Shijie Qi. Direct methods on fractional equations. Discrete & Continuous Dynamical Systems, 2019, 39 (3) : 1269-1310. doi: 10.3934/dcds.2019055
[16]

Xin Zhang, Jie Wen, Qin Ni. Subspace trust-region algorithm with conic model for unconstrained optimization. Numerical Algebra, Control & Optimization, 2013, 3 (2) : 223-234. doi: 10.3934/naco.2013.3.223
[17]

Yigui Ou, Wenjie Xu. A unified derivative-free projection method model for large-scale nonlinear equations with convex constraints. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021125
[18]

Mohamed Aly Tawhid. Nonsmooth generalized complementarity as unconstrained optimization. Journal of Industrial & Management Optimization, 2010, 6 (2) : 411-423. doi: 10.3934/jimo.2010.6.411
[19]

Guoqiang Wang, Zhongchen Wu, Zhongtuan Zheng, Xinzhong Cai. Complexity analysis of primal-dual interior-point methods for semidefinite optimization based on a parametric kernel function with a trigonometric barrier term. Numerical Algebra, Control & Optimization, 2015, 5 (2) : 101-113. doi: 10.3934/naco.2015.5.101
[20]

Hong Seng Sim, Chuei Yee Chen, Wah June Leong, Jiao Li. Nonmonotone spectral gradient method based on memoryless symmetric rank-one update for large-scale unconstrained optimization. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021143

 Impact Factor: 

Tools

Metrics

  • PDF downloads (366)
  • HTML views (235)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences