\`x^2+y_1+z_12^34\`
Advanced Search
Article Contents
Article Contents

An alternating linearization bundle method for a class of nonconvex optimization problem with inexact information

The authors' work are supported by the Natural Science Foundation of China (Grant No. 11801503, 11701061), and Natural Science Foundation of ShanDong (Grant No. ZR201807061177)

Abstract Full Text(HTML) Figure(3) / Table(4) Related Papers Cited by
  • We propose an alternating linearization bundle method for minimizing the sum of a nonconvex function and a convex function. The convex function is assumed to be "simple" in the sense that finding its proximal-like point is relatively easy. The nonconvex function is known through oracles which provide inexact information. The errors in function values and subgradient evaluations might be unknown, but are bounded by universal constants. We examine an alternating linearization bundle method in this setting and obtain reasonable convergence properties. Numerical results show the good performance of the method.

    Mathematics Subject Classification: Primary: 90C25, 90C30; Secondary: 49J52.

    Citation:

    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  The 3D image of the nonconvex function $ f(x) $

    Figure 2.  The 3D image of the convex function $ h_1(x) $

    Figure 3.  The 3D image of the convex function $ h_2(x) $

    Algorithm 1 An alternating linearization bundle algorithm
    step 0 (Initialization) Select a starting point $ y^{0}\in \mathbb{R}^{n} $ and set $ x^{0}=y^{0}=z^{0}. $ A stopping tolerance tol $ \geq 0 $, $ m\in (0,1) $, a proximal parameter $ u_{0}>0 $. Initialize the iteration counter $ \ell=0 $, the serious step counter $ k=k(\ell)=0 $ with $ j_{0}=0 $, the bundle index sets $ L_{0}^{f}:=\{0\} $. Compute $ f^{0}, \tilde{g}^{0}_{f} $$ \in\partial f(x^{0})+B_{\varepsilon_{0}}(0) $ and the bundle information $ (e_{f}^{0,0}, d_{0}^{0},\bigtriangleup_{0}^{0})=(0,0,0) $. Set $ s_{h}^{-1}\in\partial h(y^{0})=\partial h(x^{0}), $ $ \bar{h}_{-1}(\cdot)=h(y^{0})+ $$ \langle s_{h}^{-1},\cdot-y^{0}\rangle $.
    step 1 (Solving the $ f $-subproblem) Find $ z^{\ell+1} $ by solving subproblem (17), and set
    $ \begin{align} \bar{\varphi}_{\ell}(\cdot) = \check{\varphi}_{\ell}(z^{\ell+1})+\langle s_{\varphi}^{\ell}, \cdot-z^{\ell+1}\rangle \ \ {\text{with}} \ \ s_{\varphi}^{\ell} = u_{\ell}(\hat{x}^{k}-z^{\ell+1})-s_{h}^{\ell-1}. \;\;\;\;(26)\end{align}$
    step 2 (Solving the $ h $-subproblem) Find $ y^{\ell+1} $ by solving subproblem (18), and set
    $ \begin{align} \bar{h}_{\ell}(\cdot) = h(y^{\ell+1})+\langle s_{h}^{\ell}, \cdot-y^{\ell+1}\rangle \ \ {\rm{with}}\ \ s_{h}^{\ell} = u_{\ell}(\hat{x}^{k}-y^{\ell+1})-s_{\varphi}^{\ell}.\;\;\;\;(27) \end{align} $
    Compute $ \delta_{\ell} $ and the aggregate subgradient and linearization error of $ F $
    $ \begin{align} s^{\ell}: = u_{\ell}(\hat{x}^{k}-y^{\ell+1}) \ \ {\rm{with}}\ \ E_{\ell} = \hat{F}^{k}-[\bar{\varphi}_{\ell}(\hat{x}^{k})+\bar{h}_{\ell}(\hat{x}^{k})]. \;\;\;\;\;\; (28)\end{align}$
    step 3 (Stopping criterion) Compute $ f^{\ell+1}, h(y^{\ell+1}), s^{\ell}_{h}\in \partial h(y^{\ell+1}) $, and $ \tilde{g}^{\ell+1}_{f} $$ \in \partial f(y^{\ell+1})+B_{\varepsilon_{\ell+1}}(0). $ If $ \delta_{\ell}\leq $tol, then stop. Select a new index $ L_{\ell+1}^{f} $ satisfying
    $ \begin{align} L_{\ell+1}^{f}\supseteq \bigl\{\ell+1, j_{k}\bigr\} \ \ {\rm{and}}\ \ L_{\ell+1}^{f}\supseteq \bigl\{j\in L_{\ell}^{f}:\lambda_{j}^{\ell}>0\bigr\}.\;\;\;\;\;\;\;\;(29) \end{align}$
    step 4 (Descent test) Compute $ F^{\ell+1}=f^{\ell+1}+h(y^{\ell+1}) $. If $ F^{\ell+1}\leq \hat{F}^{k}-m\delta_{\ell} $, declare a descent step, set $ x^{k+1}=y^{\ell+1} $, $ k(\ell+1)=k+1 $. Otherwise, declare a null step, set $ x^{k+1}=\hat{x}^{k} $, $ k(\ell+1)=k. $
    step 5 (Bundle update and loop) For a proximal parameter $ u_{\ell} $, we use a positive constant $ u_{\max} $ to update it. If the iteration $ \ell $ is a serious step, then set $ u_{\ell+1}\leq u_{\max} $. If the iteration $ \ell $ is a null step, then set $ u_{\ell+1}=u_{\ell} $. Select the new index set $ L_{\ell+1}^{f} $, increase $ \ell $ by 1 and go to step 1.
     | Show Table
    DownLoad: CSV

    Table 1.  Comparison between Algorithm 1, $ \texttt{PPBM}$ and $ \texttt{ALBM}$ for Example 5.1

    $\texttt{Problem}$ $\texttt{Alg.}$ $\texttt{n}$ $ F_{_{\texttt{final}}} $ $ \texttt{Ni} $ $ \texttt{Nd} $ $\texttt{NF}$
    $\texttt{CB2}$ Alg. 1 2 3.342427 2 1 2
    $\texttt{PPBM}$ 3.343146 2 1 2
    $\texttt{ALBM}$ 3.343146 2 1 2
    $\texttt{CB3}$ Alg. 1 2 24.477350 7 4 7
    $\texttt{PPBM}$ 24.479795 9 4 9
    $\texttt{ALBM}$ 24.479795 11 8 11
    $\texttt{LQ}$ Alg. 1 2 -0.998473 13 10 13
    $\texttt{PPBM}$ -0.999989 15 12 15
    $\texttt{ALBM}$ -0.999989 18 17 18
    $\texttt{Mifflin1}$ Alg. 1 2 48.153612 4 3 4
    $\texttt{PPBM}$ 48.153612 3 2 3
    $\texttt{ALBM}$ 48.153612 4 2 4
    $\texttt{Rosen-Suzuki}$ Alg. 1 4 39.698418 6 5 6
    $\texttt{PPBM}$ 39.715617 7 6 7
    $\texttt{ALBM}$ 39.715617 9 8 9
    $\texttt{Shor}$ Alg. 1 5 50.250278 4 3 4
    $\texttt{PPBM}$ 50.250278 5 1 5
    $\texttt{ALBM}$ 50.250278 4 1 4
    $\texttt{MAXL}$ Alg. 1 20 0.557216 17 7 17
    $\texttt{PPBM}$ 0.552786 19 6 19
    $\texttt{ALBM}$ 0.552786 22 2 22
     | Show Table
    DownLoad: CSV

    Table 2.  Comparison between Algorithm 1, $ \texttt{PBM}$ and $ \texttt{IPBM}$ for objective function $ F_1 $

    $\texttt{n}$ $\texttt{Algorithm}$ $ F_{_{\texttt{final}}} $ $ \texttt{Nd} $ $\texttt{Ni}$ $\texttt{Time}$
    5 Alg. 1 4.22e-007 28 28 0.5833
    $\texttt{IPBM}$ 3.13e-007 30 30 0.6032
    $\texttt{PBM}$ 2.37e-007 31 31 0.6817
    6 Alg. 1 1.88e-007 27 28 0.5024
    $\texttt{IPBM}$ 2.39e-007 27 30 0.5924
    $\texttt{PBM}$ 1.43e-006 27 33 1.0164
    7 Alg. 1 3.82e-005 33 36 0.5437
    $\texttt{IPBM}$ 4.83e-007 32 37 0.5771
    $\texttt{PBM}$ 3.34e-006 32 34 1.4873
    8 Alg. 1 4.16e-005 38 41 0.8272
    $\texttt{IPBM}$ 4.63e-005 37 43 0.9136
    $\texttt{PBM}$ 4.25e-003 43 48 5.7492
    9 Alg. 1 5.18e-005 33 36 1.1306
    $\texttt{IPBM}$ 6.48e-006 37 39 1.3848
    $\texttt{PBM}$ 2.41e-004 37 44 2.1447
    10 Alg. 1 4.96e-005 38 43 0.9137
    $\texttt{IPBM}$ 4.17e-006 37 46 0.9747
    $\texttt{PBM}$ 5.43e-005 44 57 4.0254
    11 Alg. 1 4.37e-005 48 55 1.0873
    $\texttt{IPBM}$ 2.73e-006 47 56 1.1473
    $\texttt{PBM}$ 4.67e-005 62 69 1.2734
    12 Alg. 1 4.37e-006 48 65 1.2063
    $\texttt{IPBM}$ 5.46e-006 51 67 1.3593
    $\texttt{PBM}$ 4.73e-006 74 80 1.4932
    13 Alg. 1 4.58e-005 48 68 1.1283
    $\texttt{IPBM}$ 3.28e-006 54 72 1.2863
    $\texttt{PBM}$ 8.53e-005 82 93 2.3602
    14 Alg. 1 4.63e-005 56 65 1.4853
    $\texttt{IPBM}$ 3.17e-005 54 68 1.7437
    $\texttt{PBM}$ 3.14e-006 63 72 2.4185
    15 Alg. 1 1.16e-004 46 58 1.0673
    $\texttt{IPBM}$ 2.41e-004 52 64 1.1536
    $\texttt{PBM}$ 3.38e-003 57 72 1.2183
    20 Alg. 1 4.62e-004 81 96 4.4328
    $\texttt{IPBM}$ 3.84e-003 79 95 4.8753
    $\texttt{PBM}$ 0.0198 104 131 10.7273
     | Show Table
    DownLoad: CSV

    Table 3.  Comparison between Algorithm 1, $ \texttt{PBM}$ and $ \texttt{IPBM}$ for objective function $ F_2 $

    $\texttt{n}$ $\texttt{Algorithm}$ $ F_{_{\texttt{final}}} $ $ \texttt{Ki} $ $\texttt{Ni}$ $\texttt{Time}$
    5 Alg. 1 2.18e-004 21 26 0.2063
    $\texttt{IPBM}$ 3.86e-004 19 26 0.2165
    $\texttt{PBM}$ 5.74e-004 23 31 0.6639
    6 Alg. 1 3.65e-003 27 33 0.3452
    $\texttt{IPBM}$ 3.61e-003 24 29 0.3277
    $\texttt{PBM}$ 3.46e-003 22 31 0.3532
    7 Alg. 1 2.84e-003 36 41 1.5834
    $\texttt{IPBM}$ 2.65e-004 43 56 1.9734
    $\texttt{PBM}$ 2.14e-003 56 74 5.8017
    8 Alg. 1 2.88e-004 24 33 0.4110
    $ \texttt{IPBM}$ 3.28e-004 29 36 0.4368
    $\texttt{PBM}$ 1.29e-003 31 40 1.2455
    9 Alg. 1 2.46e-004 29 53 1.1681
    $\texttt{IPBM}$ 5.49e-004 32 54 1.2518
    $\texttt{PBM}$ 2.74e-003 36 45 1.8386
    10 Alg. 1 3.48e-003 19 34 1.0538
    $\texttt{IPBM}$ 2.86e-003 21 36 1.2023
    $\texttt{PBM}$ 1.55e-002 63 88 7.2973
    11 Alg. 1 5.75e-003 36 57 1.3976
    $\texttt{IPBM}$ 8.64e-003 37 55 1.4683
    $\texttt{PBM}$ 1.17e-002 64 81 11.2496
    12 Alg. 1 1.32e-002 29 54 1.3056
    $\texttt{IPBM}$ 1.22e-002 29 57 1.3205
    $\texttt{PBM}$ 1.24e-002 64 79 1.6228
    13 Alg. 1 4.16e-003 35 71 1.5946
    $\texttt{IPBM}$ 3.144-004 38 63 1.6634
    $\texttt{PBM}$ 2.84e-003 67 83 2.1066
    14 Alg. 1 3.53e-003 30 46 1.2391
    $\texttt{IPBM}$ 3.85e-003 32 48 1.3884
    $\texttt{PBM}$ 7.95e-003 58 81 2.4639
    15 Alg. 1 2.43e-002 41 67 0.9864
    $\texttt{IPBM}$ 1.41e-003 49 76 1.0356
    $\texttt{PBM}$ 4.63e-002 66 84 3.7320
    20 Alg. 1 1.53e-004 33 51 1.4601
    $\texttt{IPBM}$ 3.74e-004 35 56 1.6054
    $\texttt{PBM}$ 7.78e-002 87 119 5.8107
     | Show Table
    DownLoad: CSV
  • [1] H. AttouchJ. BolteP. Redont and A. Soubeyran, Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality, Math. Oper. Res., 35 (2010), 438-457.  doi: 10.1287/moor.1100.0449.
    [2] D. P. Bertsekas, Nonlinear Programming, 2nd edition, Athena Scientific Optimization and Computation Series, Athena Scientific, Belmont, 1999.
    [3] J. BolteS. Sabach and M. Teboulle, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program., 146 (2014), 459-494.  doi: 10.1007/s10107-013-0701-9.
    [4] R. Chartrand, Exact reconstruction of sparse signals via nonconvex minimization, IEEE Signal Process. Lett., 14 (2007), 707-710.  doi: 10.1109/lsp.2007.898300.
    [5] A. Danilidis and P. Georgiev, Approximate convexity and submonotonicity, J. Math. Anal. Appl., 291 (2004), 292-301.  doi: 10.1016/j.jmaa.2003.11.004.
    [6] G. Emiel and C. Sagastizábal, Incremental-like bundle methods with application to energy planning, Comput. Optim. Appl., 46 (2010), 305-332.  doi: 10.1007/s10589-009-9288-8.
    [7] C. Ferrier, Bornes Duales de Problémes d' Optimisation Polynomiaux, PhD thesis, Laboratoire Approximation et Optimisation, Université Paul Sabatier, Toulouse, France, 1997.
    [8] D. GoldfarbS. Ma and K. Scheinberg, Fast alternating linearization methods for minimizing the sum of two convex funcions, Math. Program., 141 (2013), 349-382.  doi: 10.1007/s10107-012-0530-2.
    [9] W. Hare and C. Sagastizábal, Computing proximal points of nonconvex functions, Math. Program., 116 (2009), 221-258.  doi: 10.1007/s10107-007-0124-6.
    [10] W. Hare and C. Sagastizábal, A redistributed proximal bundle method for nonconvex optimization, SIAM J. Optim., 20 (2010), 2442-2473.  doi: 10.1137/090754595.
    [11] W. HareC. Sagastizábal and M. Solodov, A proximal bundle method for nonsmooth nonconvex functions with inexact information, Comput. Optim. Appl., 63 (2016), 1-28.  doi: 10.1007/s10589-015-9762-4.
    [12] K. C. Kiwiel, An algorithm for nonsmooth convex minimization with errors, Math. Comp., 45 (1985), 173-180.  doi: 10.2307/2008055.
    [13] K. C. Kiwiel, An alternating linearization bundle method for convex optimization and nonlinear multicommodity flow problems, Math. Program., 130 (2011), 59-84.  doi: 10.1007/s10107-009-0327-0.
    [14] K. C. Kiwiel, A proximal bundle method with approximate subgradient linearizations, SIAM J. Optim., 16 (2006), 1007-1023.  doi: 10.1137/040603929.
    [15] K. C. Kiwiel, A proximal-projection bundle method for Lagrangian relaxation, including semidefinite programming, SIAM J. Optim., 17 (2006), 1015-1034.  doi: 10.1137/050639284.
    [16] K. C. KiwielC. H. Rosa and A. Ruszczynski, Proximal decomposition via alternating linearization, SIAM J. Optim., 9 (1999), 668-689.  doi: 10.1137/S1052623495288064.
    [17] D. LiL. P. Pang and S. Chen, A proximal alternating linearization method for nonconvex optimization problems, Optim. Methods Softw., 29 (2014), 771-785.  doi: 10.1080/10556788.2013.854358.
    [18] D. LiL. P. Pang and Z. Q. Xia, An approximate alternating linearization decomposition method, J. Appl. Math. & Informatics, 28 (2010), 1249-1262. 
    [19] L. Lukšan and J. Vlček, Test problems for nonsmooth unconstrained and linearly constrained optimization, Technical Report No. 798, Institute of Computer Science, Academy of Sciences of the Czech Republic, Prague, 2000.
    [20] J. LvL. P. Pang and F. Y. Meng, A proximal bundle method for constrained nonsmooth nonconvex optimization with inexact information, J. Global Optim., 70 (2018), 517-549.  doi: 10.1007/s10898-017-0565-2.
    [21] J. LvL. P. PangN. Xu and Z. H. Xiao, An infeasible bundle method for nonconvex constrained optimization with application to semi-infinite programming problems, Numer. Algorithms, 80 (2019), 397-427.  doi: 10.1007/s11075-018-0490-6.
    [22] F. Y. MengL. P. PangJ. Lv and J. H. Wang, An approximate bundle method for solving nonsmooth equilibrium problems, J. Global Optim., 68 (2017), 537-562.  doi: 10.1007/s10898-016-0490-9.
    [23] D. Noll, Bundle method for non-convex minimization with inexact subgradients and function values, in Computational and Analytical Mathematics, Springer Proceedings in Mathematics & Statistics, 50, Springer, New York, 2013, 555–592. doi: 10.1007/978-1-4614-7621-4_26.
    [24] W. OliveiraC. Sagastizábal and S. Scheimberg, Inexact bundle methods for two-stage stochastic programming, SIAM J. Optim., 21 (2011), 517-544.  doi: 10.1137/100808289.
    [25] L. P. PangJ. Lv and J. H. Wang, Constrained incremental bundle method with partial inexact oracle for nonsmooth convex semi-infinite programming problems, Comput. Optim. Appl., 64 (2016), 433-465.  doi: 10.1007/s10589-015-9810-0.
    [26] R. T. Rockafellar and R. J. B. Wets, Variational Analysis, Springer, Berlin, 1998. doi: 10.1007/978-3-642-02431-3.
    [27] M. V. Solodov, On approximations with finite precision in bundle methods for nonsmooth optimization, J. Optim. Theory Appl., 119 (2003), 151-165.  doi: 10.1023/B:JOTA.0000005046.70410.02.
    [28] J. E. Springarn, Submonotone subdifferentials of Lipschitz functions, Trans. Amer. Math. Soc, 264 (1981), 77-89.  doi: 10.2307/1998411.
    [29] C. M. TangJ. B. Jian and G. Y. Li, A proximal-projection partial bundle method for convex constrained minimax problems, J. Ind. Manag. Optim., 15 (2019), 757-774.  doi: 10.3934/jimo.2018069.
    [30] C. M. TangJ. M. Lv and J. B. Jian, An alternating linearization bundle method for a class of nonconvex nonsmooth optimization problems, J. Inequal. Appl., 101 (2018), 1-23.  doi: 10.1186/s13660-018-1683-1.
    [31] Y. YangL. P. PangX. F. Ma and J. Shen, Constrained nonconvex nonsmooth optimization via proximal bundle method, J. Optim. Theory Appl., 163 (2014), 900-925.  doi: 10.1007/s10957-014-0523-9.
  • 加载中

Figures(3)

Tables(4)

SHARE

Article Metrics

HTML views(1012) PDF downloads(396) Cited by(0)

Access History

Other Articles By Authors

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return