# American Institute of Mathematical Sciences

March  2021, 11(1): 45-61. doi: 10.3934/naco.2020014

## On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization

 1 School of Mathematical Sciences, Sunway University, Selangor, Malaysia 2 Department of Aerospace and Software Engineering, Gyeongsang National University, Jinju, Republic of Korea 3 School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, Australia 4 Department of Electrical and Computer Engineering, Curtin University, Sarawak, Malaysia

* Corresponding author: M. S. LEE (Email: moksianglee@gmail.com)

Received  June 2019 Revised  August 2019 Published  February 2020

A bang-bang iteration method equipped with a component-wise line search strategy is introduced to solve unconstrained optimization problems. The main idea of this method is to formulate an unconstrained optimization problem as an optimal control problem to obtain an optimal trajectory. However, the optimal trajectory can only be generated by impulsive control variables and it is a straight line joining a guessed initial point to a minimum point. Thus, a priori bounds are imposed on the control variables in order to obtain a feasible solution. As a result, the optimal trajectory is made up of bang-bang control sub-arcs, which form an iterative model based on the Lyapunov function's theorem. This is to ensure monotonic decrease of the objective function value and convergence to a desirable minimum point. However, a chattering behavior may occur near the solution. To avoid this behavior, the Newton iterations are then applied to the proposed method via a two-phase approach to achieve fast convergence. Numerical experiments show that this new approach is efficient and cost-effective to solve the unconstrained optimization problems.

Citation: M. S. Lee, H. G. Harno, B. S. Goh, K. H. Lim. On the bang-bang control approach via a component-wise line search strategy for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 45-61. doi: 10.3934/naco.2020014
##### References:

show all references

##### References:
A backtracking procedure to obtain the next iterative point $x(k+1)$ from the current point $x(k)$ on an objective function with two variables
The AGD with rectangular search regions approaching the minimum point $x^{*}$ from the initial point $x(0)$
Performance profiles based on the CPU time
Performance profiles based on the number of iterations
The iterations of the AGDRN method on Booth function (40) in Phase-Ⅰ and Phase-Ⅱ
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $3.79\times 10^{3}$ $3.69\times 10^{2}$ $1.80\times 10^{1}$ 1 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.41\times 10^{1}$ 2 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.27\times 10^{1}$ 3 $2.38\times 10^{1}$ $9.17\times 10^{1}$ $1.15\times 10^{1}$ 4 $4.04\times 10^{1}$ $3.62\times 10^{1}$ $3.09\times 10^{0}$ 5 $1.01\times 10^{1}$ $1.48\times 10^{1}$ $2.78\times 10^{0}$ 6 $4.51\times 10^{0}$ $4.29\times 10^{0}$ $7.52\times 10^{-1}$ 7 $2.10\times 10^{0}$ $2.95\times 10^{0}$ $6.76\times 10^{-1}$ 8 $1.60\times 10^{0}$ $2.60\times 10^{0}$ $1.83\times 10^{-1}$ 9 $1.22\times 10^{0}$ $2.28\times 10^{0}$ $1.64\times 10^{-1}$ 10 $9.13\times 10^{-1}$ $1.99\times 10^{0}$ $1.48\times 10^{-1}$ 11 $6.77\times 10^{-1}$ $1.74\times 10^{0}$ $1.33\times 10^{-1}$ 12 $4.96\times 10^{-1}$ $1.52\times 10^{0}$ $1.20\times 10^{-1}$ 13 $3.57\times 10^{-1}$ $1.33\times 10^{0}$ $1.08\times 10^{-1}$ 14 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $9.71\times 10^{-2}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $3.60\times 10^{1}$ 1 $4.54\times 10^{-18}$ $4.34\times 10^{-9}$ $3.60\times 10^{1}$
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $3.79\times 10^{3}$ $3.69\times 10^{2}$ $1.80\times 10^{1}$ 1 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.41\times 10^{1}$ 2 $3.69\times 10^{2}$ $1.15\times 10^{2}$ $1.27\times 10^{1}$ 3 $2.38\times 10^{1}$ $9.17\times 10^{1}$ $1.15\times 10^{1}$ 4 $4.04\times 10^{1}$ $3.62\times 10^{1}$ $3.09\times 10^{0}$ 5 $1.01\times 10^{1}$ $1.48\times 10^{1}$ $2.78\times 10^{0}$ 6 $4.51\times 10^{0}$ $4.29\times 10^{0}$ $7.52\times 10^{-1}$ 7 $2.10\times 10^{0}$ $2.95\times 10^{0}$ $6.76\times 10^{-1}$ 8 $1.60\times 10^{0}$ $2.60\times 10^{0}$ $1.83\times 10^{-1}$ 9 $1.22\times 10^{0}$ $2.28\times 10^{0}$ $1.64\times 10^{-1}$ 10 $9.13\times 10^{-1}$ $1.99\times 10^{0}$ $1.48\times 10^{-1}$ 11 $6.77\times 10^{-1}$ $1.74\times 10^{0}$ $1.33\times 10^{-1}$ 12 $4.96\times 10^{-1}$ $1.52\times 10^{0}$ $1.20\times 10^{-1}$ 13 $3.57\times 10^{-1}$ $1.33\times 10^{0}$ $1.08\times 10^{-1}$ 14 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $9.71\times 10^{-2}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $2.52\times 10^{-1}$ $1.16\times 10^{0}$ $3.60\times 10^{1}$ 1 $4.54\times 10^{-18}$ $4.34\times 10^{-9}$ $3.60\times 10^{1}$
The iterations of the AGDRN method on Powell function (41) in Phase-Ⅰ and Phase-Ⅱ
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $2.60\times 10^{2}$ $2.53\times 10^{2}$ $5.00\times 10^{0}$ 1 $6.56\times 10^{0}$ $1.31\times 10^{1}$ $3.54\times 10^{0}$ 2 $1.31\times 10^{0}$ $4.51\times 10^{0}$ $2.61\times 10^{0}$ 3 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $1.06\times 10^{0}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $5.00\times 10^{-1}$ 1 $-5.49\times 10^{-1}$ $6.63\times 10^{-1}$ $1.36\times 10^{1}$ 2 $-5.82\times 10^{-1}$ $5.20\times 10^{-2}$ $1.09\times 10^{1}$ 3 $-5.82\times 10^{-1}$ $7.61\times 10^{-4}$ $1.06\times 10^{1}$ 4 $-5.82\times 10^{-1}$ $1.71\times 10^{-7}$ $1.06\times 10^{1}$
 Phase-Ⅰ $k$ $f(x)$ $\|g(x)\|$ $\|\Delta x\|$ 0 $2.60\times 10^{2}$ $2.53\times 10^{2}$ $5.00\times 10^{0}$ 1 $6.56\times 10^{0}$ $1.31\times 10^{1}$ $3.54\times 10^{0}$ 2 $1.31\times 10^{0}$ $4.51\times 10^{0}$ $2.61\times 10^{0}$ 3 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $1.06\times 10^{0}$ Phase-Ⅱ $j$ $f(x)$ $\|g(x)\|$ {det H} 0 $-2.46\times 10^{-1}$ $9.70\times 10^{-1}$ $5.00\times 10^{-1}$ 1 $-5.49\times 10^{-1}$ $6.63\times 10^{-1}$ $1.36\times 10^{1}$ 2 $-5.82\times 10^{-1}$ $5.20\times 10^{-2}$ $1.09\times 10^{1}$ 3 $-5.82\times 10^{-1}$ $7.61\times 10^{-4}$ $1.06\times 10^{1}$ 4 $-5.82\times 10^{-1}$ $1.71\times 10^{-7}$ $1.06\times 10^{1}$
Comparison of Phase-Ⅱ replacement with other methods to minimize the Gulf research and development function
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ k CPU Time (s) AGDS $5.40\times 10^{-5}$ $6.40\times 10^{-9}$ 37 $9.06\times 10^{-1}$ BTR $5.40\times 10^{-5}$ $5.96\times 10^{-8}$ 81 $1.75\times 10^{0}$ AGDRN $3.11\times 10^{-2}$ $6.61\times 10^{-2}$ 500 $2.15\times 10^{1}$ AGD-RS $5.40\times 10^{-5}$ $1.35\times 10^{-7}$ 83 $1.63\times 10^{0}$
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ k CPU Time (s) AGDS $5.40\times 10^{-5}$ $6.40\times 10^{-9}$ 37 $9.06\times 10^{-1}$ BTR $5.40\times 10^{-5}$ $5.96\times 10^{-8}$ 81 $1.75\times 10^{0}$ AGDRN $3.11\times 10^{-2}$ $6.61\times 10^{-2}$ 500 $2.15\times 10^{1}$ AGD-RS $5.40\times 10^{-5}$ $1.35\times 10^{-7}$ 83 $1.63\times 10^{0}$
Comparison of Phase-Ⅱ replacement with other methods to minimize the Kowalik Osborne function
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ $\boldsymbol{k}$ CPU Time (s) AGDS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $6.09\times 10^{-1}$ BTR $3.08\times 10^{-4}$ $5.76\times 10^{-8}$ 16 $5.94\times 10^{-1}$ AGDRN $5.08\times 10^{-4}$ $3.84\times 10^{-3}$ 499 $2.38\times 10^{1}$ AGD-RS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $4.06\times 10^{-1}$
 Methods $\boldsymbol{f(x^{*})}$ $\boldsymbol{\|g(x)\|}$ $\boldsymbol{k}$ CPU Time (s) AGDS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $6.09\times 10^{-1}$ BTR $3.08\times 10^{-4}$ $5.76\times 10^{-8}$ 16 $5.94\times 10^{-1}$ AGDRN $5.08\times 10^{-4}$ $3.84\times 10^{-3}$ 499 $2.38\times 10^{1}$ AGD-RS $3.08\times 10^{-4}$ $8.60\times 10^{-8}$ 14 $4.06\times 10^{-1}$
 [1] G. Deugoué, B. Jidjou Moghomye, T. Tachim Medjo. Approximation of a stochastic two-phase flow model by a splitting-up method. Communications on Pure & Applied Analysis, 2021, 20 (3) : 1135-1170. doi: 10.3934/cpaa.2021010 [2] Hirokazu Saito, Xin Zhang. Unique solvability of elliptic problems associated with two-phase incompressible flows in unbounded domains. Discrete & Continuous Dynamical Systems, 2021  doi: 10.3934/dcds.2021051 [3] Mohammed Abdelghany, Amr B. Eltawil, Zakaria Yahia, Kazuhide Nakata. A hybrid variable neighbourhood search and dynamic programming approach for the nurse rostering problem. Journal of Industrial & Management Optimization, 2021, 17 (4) : 2051-2072. doi: 10.3934/jimo.2020058 [4] Sarra Delladji, Mohammed Belloufi, Badreddine Sellami. Behavior of the combination of PRP and HZ methods for unconstrained optimization. Numerical Algebra, Control & Optimization, 2021, 11 (3) : 377-389. doi: 10.3934/naco.2020032 [5] Zheng Chang, Haoxun Chen, Farouk Yalaoui, Bo Dai. Adaptive large neighborhood search Algorithm for route planning of freight buses with pickup and delivery. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1771-1793. doi: 10.3934/jimo.2020045 [6] Yishui Wang, Dongmei Zhang, Peng Zhang, Yong Zhang. Local search algorithm for the squared metric $k$-facility location problem with linear penalties. Journal of Industrial & Management Optimization, 2021, 17 (4) : 2013-2030. doi: 10.3934/jimo.2020056 [7] Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 377-387. doi: 10.3934/naco.2018024 [8] Peng Luo. Comparison theorem for diagonally quadratic BSDEs. Discrete & Continuous Dynamical Systems, 2021, 41 (6) : 2543-2557. doi: 10.3934/dcds.2020374 [9] Jonathan DeWitt. Local Lyapunov spectrum rigidity of nilmanifold automorphisms. Journal of Modern Dynamics, 2021, 17: 65-109. doi: 10.3934/jmd.2021003 [10] Wenmeng Geng, Kai Tao. Lyapunov exponents of discrete quasi-periodic gevrey Schrödinger equations. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 2977-2996. doi: 10.3934/dcdsb.2020216 [11] Christos Sourdis. A Liouville theorem for ancient solutions to a semilinear heat equation and its elliptic counterpart. Electronic Research Archive, , () : -. doi: 10.3934/era.2021016 [12] Jianfeng Lv, Yan Gao, Na Zhao. The viability of switched nonlinear systems with piecewise smooth Lyapunov functions. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1825-1843. doi: 10.3934/jimo.2020048 [13] Bochao Chen, Yixian Gao. Quasi-periodic travelling waves for beam equations with damping on 3-dimensional rectangular tori. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021075 [14] Mohamed Ouzahra. Approximate controllability of the semilinear reaction-diffusion equation governed by a multiplicative control. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021081 [15] K. Ravikumar, Manil T. Mohan, A. Anguraj. Approximate controllability of a non-autonomous evolution equation in Banach spaces. Numerical Algebra, Control & Optimization, 2021, 11 (3) : 461-485. doi: 10.3934/naco.2020038 [16] Sara Munday. On the derivative of the $\alpha$-Farey-Minkowski function. Discrete & Continuous Dynamical Systems, 2014, 34 (2) : 709-732. doi: 10.3934/dcds.2014.34.709 [17] Michael Grinfeld, Amy Novick-Cohen. Some remarks on stability for a phase field model with memory. Discrete & Continuous Dynamical Systems, 2006, 15 (4) : 1089-1117. doi: 10.3934/dcds.2006.15.1089 [18] Isabeau Birindelli, Françoise Demengel, Fabiana Leoni. Boundary asymptotics of the ergodic functions associated with fully nonlinear operators through a Liouville type theorem. Discrete & Continuous Dynamical Systems, 2021, 41 (7) : 3021-3029. doi: 10.3934/dcds.2020395 [19] Ralf Hielscher, Michael Quellmalz. Reconstructing a function on the sphere from its means along vertical slices. Inverse Problems & Imaging, 2016, 10 (3) : 711-739. doi: 10.3934/ipi.2016018 [20] Raimund Bürger, Christophe Chalons, Rafael Ordoñez, Luis Miguel Villada. A multiclass Lighthill-Whitham-Richards traffic model with a discontinuous velocity function. Networks & Heterogeneous Media, 2021, 16 (2) : 187-219. doi: 10.3934/nhm.2021004

Impact Factor: