\`x^2+y_1+z_12^34\`
Advanced Search
Article Contents
Article Contents

Accelerated bregman operator splitting with backtracking

  • * Corresponding author: Xianqi Li

    * Corresponding author: Xianqi Li 
The first author is supported by NSF grant DMS-1319050, NSF/DMS 1719932 and AFOSR/Eglin (FA8651-08-D-0108). The second author was partially supported by UFII fellowship and AFRL Mathematical Modeling and Optimization Institute.
Abstract Full Text(HTML) Figure(8) / Table(4) Related Papers Cited by
  • This paper develops two accelerated Bregman Operator Splitting (BOS) algorithms with backtracking for solving regularized large-scale linear inverse problems, where the regularization term may not be smooth. The first algorithm improves the rate of convergence for BOSVS [5] in terms of the smooth component in the objective function by incorporating Nesterov's multi-step acceleration scheme under the assumption that the feasible set is bounded. The second algorithm is capable of dealing with the case where the feasible set is unbounded. Moreover, it allows more aggressive stepsize than that in the first scheme by properly selecting the penalty parameter and jointly updating the acceleration parameter and stepsize. Both algorithms exhibit better practical performance than BOSVS and AADMM [21], while preserve the same accelerated rate of convergence as that for AADMM. The numerical results on total-variation based image reconstruction problems indicate the effectiveness of the proposed algorithms.

    Mathematics Subject Classification: Primary: 90C25, 90C52; Secondary: 65Y20, 68W40.

    Citation:

    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  The objective function values and relative error vs. CPU time for first instance

    Figure 2.  The objective function values and relative error vs.CPU time for second instance

    Figure 3.  Sensitivity maps of data1

    Figure 4.  Left: True image; Right: Cartesian mask for data 1

    Figure 5.  Comparison of 400 iterations of BOSVS, TVAL3, ALADMML, and ABOSVS-Ⅱ in partially parallel image reconstruction (Top) and their differences with true image (Bottom) using data1. From left to right: BOSVS, TVAL3, ALADMML, and ABOSVS-Ⅱ

    Figure 6.  Sensitivity maps of data2

    Figure 7.  Left: True image; Right: Cartesian mask for data 2

    Figure 8.  Comparison of 400 iterations of BOSVS, TVAL3, ALADMML, and ABOSVS-Ⅱ in partially parallel image reconstruction (Top) and their differences with true image (Bottom) using data2. From left to right: BOSVS, TVAL3, ALADMML, and ABOSVS-Ⅱ.

    Table 1.  Parameter settings for ABOSVS-I and ABOSVS-Ⅱ in (4.1). Note that $\rho$ and $\tau$ are only used in the first instance

    ParametersασηminCβkρτ
    Values102||A||2/101001/k11
     | Show Table
    DownLoad: CSV

    Table 2.  Parameter settings for ABOSVS-Ⅱ in 4.2

    ParametersασηminC
    Values10-10n2||A||2/5100
     | Show Table
    DownLoad: CSV

    Table 3.  Comparison of objective function value, relative error, and CPU time in seconds using data1

    AlgorithmsObjective valueRelative errorCPU
    BOSVS15.468410.05962528.0
    TVAL356.72630.050228.6
    ALADMML15.468100.049966.5
    ABOSVS-Ⅱ15.445350.047949.7
     | Show Table
    DownLoad: CSV

    Table 4.  Comparison of objective function value, relative error, and CPU time in seconds using data2

    AlgorithmsObjective valueRelative errorCPU
    BOSVS5007e+20.06467448.4
    TVAL31704e+30.050681.2
    ALADMML5.006e+20.0499180.0
    ABOSVS-Ⅱ4.970e+20.0479144.7
     | Show Table
    DownLoad: CSV
  • [1] A. Beck and M. Teboulle, Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems, IEEE Transactions on Image Processing, 18 (2009), 2419-2434.  doi: 10.1109/TIP.2009.2028250.
    [2] K. BlockM. Uecker and J. Frahm, Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint, Magnetic Resonance in Medicine, 57 (2007), 1086-1098.  doi: 10.1002/mrm.21236.
    [3] S. BoydN. ParikhE. ChuB. Peleato and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends® in Machine Learning, 3 (2011), 1-122.  doi: 10.1561/2200000016.
    [4] Y. ChenW. HagerF. HuangD. PhanX. Ye and W. Yin, Fast algorithms for image reconstruction with application to partially parallel MR imaging, SIAM Journal on Imaging Sciences, 5 (2012), 90-118.  doi: 10.1137/100792688.
    [5] Y. ChenW. HagerM. Yashtini and X. Ye, Bregman operator splitting with variable stepsize for total variation image reconstruction, Computational Optimization and Applications, 54 (2013), 317-342.  doi: 10.1007/s10589-012-9519-2.
    [6] W. Deng and W. Yin, On the global and linear convergence of the generalized alternating direction method of multipliers, Journal of Scientific Computing, 66 (2012), 1-28.  doi: 10.1007/s10915-015-0048-x.
    [7] D. Donoho, Compressed sensing, IEEE Transactions on Information Theory, 52 (2006), 1289-1306.  doi: 10.1109/TIT.2006.871582.
    [8] D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Computers & Mathematics with Applications, 2 (1976), 17-40.  doi: 10.1016/0898-1221(76)90003-1.
    [9] R. Glowinski and A. Marroco, Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problémes de Dirichlet non linéaires, Revue française d'automatique, informatique, recherche opérationnelle. Analyse numérique, 9 (1975), 41-76. 
    [10] D. GoldfarbS. Ma and K. Scheinberg, Fast multiple-splitting algorithms for convex optimization, SIAM Journal on Optimization, 22 (2012), 533-556.  doi: 10.1137/090780705.
    [11] T. GoldsteinB. O'DonoghueS. Setzer and R. Baraniuk, Fast alternating direction optimization methods, SIAM Journal on Imaging Sciences, 7 (2014), 1588-1623.  doi: 10.1137/120896219.
    [12] T. Goldstein and S. Osher, The split Bregman method for L1-regularized problems, SIAM Journal on Imaging Sciences, 2 (2009), 323-343.  doi: 10.1137/080725891.
    [13] W. HagerC. NgoM. Yashtini and H. Zhang, An alternating direction approximate Newton algorithm for ill-conditioned inverse problems with application to parallel MRI, Name of the Journal, 3 (2015), 139-162.  doi: 10.1007/s40305-015-0078-y.
    [14] W. HagerM. Yashtini and H. Zhang, An $\backslash$mathcalO(1/k) convergence rate for the variable stepsize Bregman operator splitting algorithm, SIAM Journal on Numerical Analysis, 54 (2016), 1535-1556.  doi: 10.1137/15100401X.
    [15] M. Hong and Z. Luo, On the linear convergence of the alternating direction method of multipliers, Mathematical Programming, 162 (2017), 165-199, arXiv:1208.3922.  doi: 10.1007/s10107-016-1034-2.
    [16] C. LiW. YinH. Jiang and Y. Zhang, An efficient augmented Lagrangian method with applications to total variation minimization, Computational Optimization and Applications, 56 (2013), 507-530.  doi: 10.1007/s10589-013-9576-1.
    [17] Q. Liu, J. Luo, S. Wang M. Xiao and M. Ye, An augmented Lagrangian multi-scale dictionary learning algorithm EURASIP Journal on Advances in Signal Processing, 2011 (2011), p58. doi: 10.1186/1687-6180-2011-58.
    [18] M. LustigD. Donoho and J. Pauly, Sparse MRI: The application of compressed sensing for rapid MR imaging, Magnetic Resonance in Medicine, 58 (2007), 1182-1195.  doi: 10.1002/mrm.21391.
    [19] R. Monteiro and B. Svaiter, Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers, SIAM Journal on Optimization, 23 (2013), 475-507.  doi: 10.1137/110849468.
    [20] Y. Nesterov, Smooth minimization of non-smooth functions, Mathematical Programming, 103 (2005), 127-152.  doi: 10.1007/s10107-004-0552-5.
    [21] Y. OuyangY. ChenG. Lan and E. Pasiliao, An accelerated linearized alternating direction method of multipliers, SIAM Journal on Imaging Sciences, 8 (2015), 644-681.  doi: 10.1137/14095697X.
    [22] K. PruessmannM. WeigerP. Börnert and P. Boesiger, Advances in sensitivity encoding with arbitrary k-space trajectories, Magnetic Resonance in Medicine, 46 (2001), 638-651. 
    [23] K. ScheinbergD. Goldfarb and X. Bai, Fast first-order methods for composite convex optimization with backtracking, Foundations of Computational Mathematics, 14 (2014), 389-417.  doi: 10.1007/s10208-014-9189-9.
    [24] L. Shepp and B. Logan, The Fourier reconstruction of a head section, IEEE Transactions on Nuclear Science, 21 (1974), 21-43.  doi: 10.1109/TNS.1974.6499235.
    [25] Y. WangJ. YangW. Yin and Y. Zhang A new alternating minimization algorithm for total variation image reconstruction, A new alternating minimization algorithm for total variation image reconstruction, SIAM Journal on Imaging Sciences, 1 (2008), 248-272.  doi: 10.1137/080724265.
    [26] B. Wohlberg, Efficient Algorithms for Convolutional Sparse Representations, IEEE Transactions on Image Processing, 25 (2016), 301-315.  doi: 10.1109/TIP.2015.2495260.
    [27] J. YangY. Zhang and W. Yin, A fast alternating direction method for TVL1-L2 signal reconstruction from partial Fourier data, IEEE Journal of Selected Topics in Signal Processing, 4 (2010), 288-297. 
    [28] X. YeY. Chen and F. Huang, Computational acceleration for MR image reconstruction in partially parallel imaging, IEEE Transactions on Medical Imaging, 30 (2011), 1055-1063. 
    [29] X. ZhangM. BurgerX. Bresson and S. Osher, Bregmanized nonlocal regularization for deconvolution and sparse reconstruction, SIAM Journal on Imaging Sciences, 3 (2010), 253-276.  doi: 10.1137/090746379.
    [30] X. ZhangM. Burger and S. Osher, A unified primal-dual algorithm framework based on Bregman iteration, Journal of Scientific Computing, 46 (2011), 20-46.  doi: 10.1007/s10915-010-9408-8.
    [31] M. Zhu and T. Chan, An efficient primal-dual hybrid gradient algorithm for total variation image restoration, UCLA CAM Report, (2008), 8-34. 
    [32] M. ZhuS. Wright and T. Chan, Duality-based algorithms for total-variation-regularized image restoration, Computational Optimization and Applications, 47 (2010), 377-400.  doi: 10.1007/s10589-008-9225-2.
  • 加载中

Figures(8)

Tables(4)

SHARE

Article Metrics

HTML views(770) PDF downloads(142) Cited by(0)

Access History

Other Articles By Authors

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return