# American Institute of Mathematical Sciences

March  2021, 11(1): 45-61. doi: 10.3934/naco.2020014

## On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization

 1 School of Mathematical Sciences, Sunway University, Selangor, Malaysia 2 Department of Aerospace and Software Engineering, Gyeongsang National University, Jinju, Republic of Korea 3 School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, Australia 4 Department of Electrical and Computer Engineering, Curtin University, Sarawak, Malaysia

* Corresponding author: M. S. LEE (Email: moksianglee@gmail.com)

Received  June 2019 Revised  August 2019 Published  February 2020

A bang-bang iteration method equipped with a component-wise line search strategy is introduced to solve unconstrained optimization problems. The main idea of this method is to formulate an unconstrained optimization problem as an optimal control problem to obtain an optimal trajectory. However, the optimal trajectory can only be generated by impulsive control variables and it is a straight line joining a guessed initial point to a minimum point. Thus, a priori bounds are imposed on the control variables in order to obtain a feasible solution. As a result, the optimal trajectory is made up of bang-bang control sub-arcs, which form an iterative model based on the Lyapunov function's theorem. This is to ensure monotonic decrease of the objective function value and convergence to a desirable minimum point. However, a chattering behavior may occur near the solution. To avoid this behavior, the Newton iterations are then applied to the proposed method via a two-phase approach to achieve fast convergence. Numerical experiments show that this new approach is efficient and cost-effective to solve the unconstrained optimization problems.

Citation: M. S. Lee, H. G. Harno, B. S. Goh, K. H. Lim. On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 45-61. doi: 10.3934/naco.2020014
##### References:

show all references

##### References:
A backtracking procedure to obtain the next iterative point $x(k+1)$ from the current point $x(k)$ on an objective function with two variables
The AGD with rectangular search regions approaching the minimum point $x^{*}$ from the initial point $x(0)$
Performance profiles based on the CPU time
Performance profiles based on the number of iterations
The iterations of the AGDRN method on Booth function (40) in Phase-Ⅰ and Phase-Ⅱ
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $3.79\times 10^{3}$ $3.69\times 10^{2}$ $1.80\times 10^{1}$ 1 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.41\times 10^{1}$ 2 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.27\times 10^{1}$ 3 $2.38\times 10^{1}$ $9.17\times 10^{1}$ $1.15\times 10^{1}$ 4 $4.04\times 10^{1}$ $3.62\times 10^{1}$ $3.09\times 10^{0}$ 5 $1.01\times 10^{1}$ $1.48\times 10^{1}$ $2.78\times 10^{0}$ 6 $4.51\times 10^{0}$ $4.29\times 10^{0}$ $7.52\times 10^{-1}$ 7 $2.10\times 10^{0}$ $2.95\times 10^{0}$ $6.76\times 10^{-1}$ 8 $1.60\times 10^{0}$ $2.60\times 10^{0}$ $1.83\times 10^{-1}$ 9 $1.22\times 10^{0}$ $2.28\times 10^{0}$ $1.64\times 10^{-1}$ 10 $9.13\times 10^{-1}$ $1.99\times 10^{0}$ $1.48\times 10^{-1}$ 11 $6.77\times 10^{-1}$ $1.74\times 10^{0}$ $1.33\times 10^{-1}$ 12 $4.96\times 10^{-1}$ $1.52\times 10^{0}$ $1.20\times 10^{-1}$ 13 $3.57\times 10^{-1}$ $1.33\times 10^{0}$ $1.08\times 10^{-1}$ 14 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $9.71\times 10^{-2}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $3.60\times 10^{1}$ 1 $4.54\times 10^{-18}$ $4.34\times 10^{-9}$ $3.60\times 10^{1}$
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $3.79\times 10^{3}$ $3.69\times 10^{2}$ $1.80\times 10^{1}$ 1 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.41\times 10^{1}$ 2 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.27\times 10^{1}$ 3 $2.38\times 10^{1}$ $9.17\times 10^{1}$ $1.15\times 10^{1}$ 4 $4.04\times 10^{1}$ $3.62\times 10^{1}$ $3.09\times 10^{0}$ 5 $1.01\times 10^{1}$ $1.48\times 10^{1}$ $2.78\times 10^{0}$ 6 $4.51\times 10^{0}$ $4.29\times 10^{0}$ $7.52\times 10^{-1}$ 7 $2.10\times 10^{0}$ $2.95\times 10^{0}$ $6.76\times 10^{-1}$ 8 $1.60\times 10^{0}$ $2.60\times 10^{0}$ $1.83\times 10^{-1}$ 9 $1.22\times 10^{0}$ $2.28\times 10^{0}$ $1.64\times 10^{-1}$ 10 $9.13\times 10^{-1}$ $1.99\times 10^{0}$ $1.48\times 10^{-1}$ 11 $6.77\times 10^{-1}$ $1.74\times 10^{0}$ $1.33\times 10^{-1}$ 12 $4.96\times 10^{-1}$ $1.52\times 10^{0}$ $1.20\times 10^{-1}$ 13 $3.57\times 10^{-1}$ $1.33\times 10^{0}$ $1.08\times 10^{-1}$ 14 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $9.71\times 10^{-2}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $3.60\times 10^{1}$ 1 $4.54\times 10^{-18}$ $4.34\times 10^{-9}$ $3.60\times 10^{1}$
The iterations of the AGDRN method on Powell function (41) in Phase-Ⅰ and Phase-Ⅱ
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $2.60\times 10^{2}$ $2.53\times 10^{2}$ $5.00\times 10^{0}$ 1 $6.56\times 10^{0}$ $1.31\times 10^{1}$ $3.54\times 10^{0}$ 2 $1.31\times 10^{0}$ $4.51\times 10^{0}$ $2.61\times 10^{0}$ 3 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $1.06\times 10^{0}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $5.00\times 10^{-1}$ 1 $-5.49\times 10^{-1}$ $6.63\times 10^{-1}$ $1.36\times 10^{1}$ 2 $-5.82\times 10^{-1}$ $5.20\times 10^{-2}$ $1.09\times 10^{1}$ 3 $-5.82\times 10^{-1}$ $7.61\times 10^{-4}$ $1.06\times 10^{1}$ 4 $-5.82\times 10^{-1}$ $1.71\times 10^{-7}$ $1.06\times 10^{1}$
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $2.60\times 10^{2}$ $2.53\times 10^{2}$ $5.00\times 10^{0}$ 1 $6.56\times 10^{0}$ $1.31\times 10^{1}$ $3.54\times 10^{0}$ 2 $1.31\times 10^{0}$ $4.51\times 10^{0}$ $2.61\times 10^{0}$ 3 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $1.06\times 10^{0}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $5.00\times 10^{-1}$ 1 $-5.49\times 10^{-1}$ $6.63\times 10^{-1}$ $1.36\times 10^{1}$ 2 $-5.82\times 10^{-1}$ $5.20\times 10^{-2}$ $1.09\times 10^{1}$ 3 $-5.82\times 10^{-1}$ $7.61\times 10^{-4}$ $1.06\times 10^{1}$ 4 $-5.82\times 10^{-1}$ $1.71\times 10^{-7}$ $1.06\times 10^{1}$
Comparison of Phase-Ⅱ replacement with other methods to minimize the Gulf research and development function
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ k CPU Time (s) AGDS $5.40\times 10^{-5}$ $6.40\times 10^{-9}$ 37 $9.06\times 10^{-1}$ BTR $5.40\times 10^{-5}$ $5.96\times 10^{-8}$ 81 $1.75\times 10^{0}$ AGDRN $3.11\times 10^{-2}$ $6.61\times 10^{-2}$ 500 $2.15\times 10^{1}$ AGD-RS $5.40\times 10^{-5}$ $1.35\times 10^{-7}$ 83 $1.63\times 10^{0}$
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ k CPU Time (s) AGDS $5.40\times 10^{-5}$ $6.40\times 10^{-9}$ 37 $9.06\times 10^{-1}$ BTR $5.40\times 10^{-5}$ $5.96\times 10^{-8}$ 81 $1.75\times 10^{0}$ AGDRN $3.11\times 10^{-2}$ $6.61\times 10^{-2}$ 500 $2.15\times 10^{1}$ AGD-RS $5.40\times 10^{-5}$ $1.35\times 10^{-7}$ 83 $1.63\times 10^{0}$
Comparison of Phase-Ⅱ replacement with other methods to minimize the Kowalik Osborne function
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ $\boldsymbol{k}$ CPU Time (s) AGDS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $6.09\times 10^{-1}$ BTR $3.08\times 10^{-4}$ $5.76\times 10^{-8}$ 16 $5.94\times 10^{-1}$ AGDRN $5.08\times 10^{-4}$ $3.84\times 10^{-3}$ 499 $2.38\times 10^{1}$ AGD-RS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $4.06\times 10^{-1}$
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ $\boldsymbol{k}$ CPU Time (s) AGDS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $6.09\times 10^{-1}$ BTR $3.08\times 10^{-4}$ $5.76\times 10^{-8}$ 16 $5.94\times 10^{-1}$ AGDRN $5.08\times 10^{-4}$ $3.84\times 10^{-3}$ 499 $2.38\times 10^{1}$ AGD-RS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $4.06\times 10^{-1}$
 [1] Helmut Abels, Andreas Marquardt. On a linearized Mullins-Sekerka/Stokes system for two-phase flows. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020467 [2] Cheng He, Changzheng Qu. Global weak solutions for the two-component Novikov equation. Electronic Research Archive, 2020, 28 (4) : 1545-1562. doi: 10.3934/era.2020081 [3] Mohammed Abdulrazaq Kahya, Suhaib Abduljabbar Altamir, Zakariya Yahya Algamal. Improving whale optimization algorithm for feature selection with a time-varying transfer function. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 87-98. doi: 10.3934/naco.2020017 [4] Peng Luo. Comparison theorem for diagonally quadratic BSDEs. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020374 [5] José Madrid, João P. G. Ramos. On optimal autocorrelation inequalities on the real line. Communications on Pure & Applied Analysis, 2021, 20 (1) : 369-388. doi: 10.3934/cpaa.2020271 [6] Yue Feng, Yujie Liu, Ruishu Wang, Shangyou Zhang. A conforming discontinuous Galerkin finite element method on rectangular partitions. Electronic Research Archive, , () : -. doi: 10.3934/era.2020120 [7] Tian Ma, Shouhong Wang. Topological phase transition III: Solar surface eruptions and sunspots. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020350 [8] Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of a Sobolev type impulsive functional evolution system in Banach spaces. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020049 [9] Predrag S. Stanimirović, Branislav Ivanov, Haifeng Ma, Dijana Mosić. A survey of gradient methods for solving nonlinear optimization. Electronic Research Archive, 2020, 28 (4) : 1573-1624. doi: 10.3934/era.2020115 [10] Yifan Chen, Thomas Y. Hou. Function approximation via the subsampled Poincaré inequality. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 169-199. doi: 10.3934/dcds.2020296 [11] Tommi Brander, Joonas Ilmavirta, Petteri Piiroinen, Teemu Tyni. Optimal recovery of a radiating source with multiple frequencies along one line. Inverse Problems & Imaging, 2020, 14 (6) : 967-983. doi: 10.3934/ipi.2020044 [12] Alberto Bressan, Sondre Tesdal Galtung. A 2-dimensional shape optimization problem for tree branches. Networks & Heterogeneous Media, 2020  doi: 10.3934/nhm.2020031 [13] Bahaaeldin Abdalla, Thabet Abdeljawad. Oscillation criteria for kernel function dependent fractional dynamic equations. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020443 [14] Marion Darbas, Jérémy Heleine, Stephanie Lohrengel. Numerical resolution by the quasi-reversibility method of a data completion problem for Maxwell's equations. Inverse Problems & Imaging, 2020, 14 (6) : 1107-1133. doi: 10.3934/ipi.2020056 [15] Soniya Singh, Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of second order impulsive systems with state-dependent delay in Banach spaces. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020103 [16] Chao Wang, Qihuai Liu, Zhiguo Wang. Periodic bouncing solutions for Hill's type sub-linear oscillators with obstacles. Communications on Pure & Applied Analysis, 2021, 20 (1) : 281-300. doi: 10.3934/cpaa.2020266 [17] Nicolas Rougerie. On two properties of the Fisher information. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2020049 [18] Lingfeng Li, Shousheng Luo, Xue-Cheng Tai, Jiang Yang. A new variational approach based on level-set function for convex hull problem with outliers. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020070 [19] Hua Qiu, Zheng-An Yao. The regularized Boussinesq equations with partial dissipations in dimension two. Electronic Research Archive, 2020, 28 (4) : 1375-1393. doi: 10.3934/era.2020073 [20] Huu-Quang Nguyen, Ya-Chi Chu, Ruey-Lin Sheu. On the convexity for the range set of two quadratic functions. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020169

Impact Factor: