Optimal estimation of $\ell_1$-regularization prior from a regularized empirical Bayesian risk standpoint

Hui Huang; Eldad Haber; Lior Horesh

doi:10.3934/ipi.2012.6.447

Previous Article
On the inverse doping profile problem
IPI Home
This Issue
Next Article
Probing for inclusions in heat conductive bodies

August 2012, 6(3): 447-464. doi: 10.3934/ipi.2012.6.447

Optimal estimation of $\ell_1$-regularization prior from a regularized empirical Bayesian risk standpoint

Hui Huang ^1, , Eldad Haber ^2, and Lior Horesh ^3,

1.	Shenzhen Key Lab of Visual Computing and Visual Analytics, Shenzhen Institute of Advanced Technology, Shenzhen, Guangdong, 518055, China
2.	Department of Mathematics and Earth and Ocean Science, The University of British Columbia, Vancouver, BC, V6T 1Z2, Canada
3.	Business Analytics and Mathematical Sciences, IBM T J Watson Research Center, Yorktown Heights, NY, 10598, United States

Received March 2011 Revised April 2012 Published September 2012

Abstract
Related Papers

We address the problem of prior matrix estimation for the solution of $\ell_1$-regularized ill-posed inverse problems. From a Bayesian viewpoint, we show that such a matrix can be regarded as an influence matrix in a multivariate $\ell_1$-Laplace density function. Assuming a training set is given, the prior matrix design problem is cast as a maximum likelihood term with an additional sparsity-inducing term. This formulation results in an unconstrained yet nonconvex optimization problem. Memory requirements as well as computation of the nonlinear, nonsmooth sub-gradient equations are prohibitive for large-scale problems. Thus, we introduce an iterative algorithm to design efficient priors for such large problems. We further demonstrate that the solutions of ill-posed inverse problems by incorporation of $\ell_1$-regularization using the learned prior matrix perform generally better than commonly used regularization techniques where the prior matrix is chosen a-priori.

Keywords: empirical Bayesian estimation, nonlinear and nonsmooth optimization., optimal design, $\ell_1$-regularization prior matrix.

Mathematics Subject Classification: Primary: 49N45; Secondary: 62F1.

Citation: Hui Huang, Eldad Haber, Lior Horesh. Optimal estimation of $\ell_1$-regularization prior from a regularized empirical Bayesian risk standpoint. Inverse Problems & Imaging, 2012, 6 (3) : 447-464. doi: 10.3934/ipi.2012.6.447

References:

[1]	H. Akaike, A new look at the statistical model identification, IEEE Trans. Auto. Cont., 19 (1974), 716-723. doi: 10.1109/TAC.1974.1100705. Google Scholar
[2]	A. Aravkin, An $l_1$-Laplace robust kalman smoother, IEEE Trans. Auto. Cont., 56 (2011), 2898-2911. doi: 10.1109/TAC.2011.2141430. Google Scholar
[3]	U. Ascher, E. Haber and H. Huang, On effective methods for implicit piecewise smooth surface recovery, SIAM J. Scient. Comput., 28 (2006), 339-358. doi: 10.1137/040617261. Google Scholar
[4]	U. Ascher, K. van den Doel, H. Huang and B. Svaiter, Gradient descent and fast artificial time integration, M2AN, Math. Model. Numer. Anal., 43 (2009), 689-708. Google Scholar
[5]	J. A. Bilmes, "Factored Sparse Inverse Covariance Matrices," Proc. IEEE Intl. Conf. on Acous., Spee., and Sig. Proc., 2000. Google Scholar
[6]	A. M. Bruckstein, D. L. Donoho and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Review, 51 (2009), 34-81. doi: 10.1137/060657704. Google Scholar
[7]	E. J. Candès and D. L. Donoho, "Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges,", 1999., (). Google Scholar
[8]	E. J. Candès and M. B. Wakin, An introduction to compressive sampling, IEEE In Sig. Proc. Magaz., 25 (2008), 21-30. doi: 10.1109/MSP.2007.914731. Google Scholar
[9]	T. Chan and J. Shen, "Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods," SIAM, 2005. Google Scholar
[10]	X. Chen and W. Zhou., Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization, SIAM J. Imaging Sci., 3 (2010), 765-790. doi: 10.1137/080740167. Google Scholar
[11]	M. Cheney, D.Isaacson and J. C. Newell, Electrical impedance tomography, SIAM Review, 41 (1999), 85-101. doi: 10.1137/S0036144598333613. Google Scholar
[12]	J. Dahl, L. Vandenberghe and V. Roychowdhury, Covariance selection for non-chordal graphs via chordal embedding, Optim. Meth. and Soft., 23 (2008), 501-520. doi: 10.1080/10556780802102693. Google Scholar
[13]	Y. H. Dai and R. Fletcher, Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming, Numer. Math., 100 (2005), 21-47. doi: 10.1007/s00211-004-0569-y. Google Scholar
[14]	A. d'Aspremont, O. Banerjee and L. E. Ghaoui, First-order methods for sparse covariance selection, SIAM J. Matrix Anal. Appl., 30 (2008), 56-66. doi: 10.1137/060670985. Google Scholar
[15]	M. N. Do and M. Vetterli, The contourlet transform: an efficient directional multiresolution image representation, IEEE Trans. on Image Proc., 14 (2005), 2091-2106. doi: 10.1109/TIP.2005.859376. Google Scholar
[16]	W. Dong, L. Zhang, G. Shi and X. Wu, Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization, IEEE Trans. Image Proc., 20 (2011), 1838-1857. doi: 10.1109/TIP.2011.2108306. Google Scholar
[17]	M. Elad, P. Milanfar and R. Rubinstein, Analysis versus synthesis in signal priors, Inver. Problems, 23 (2007), 947-968. doi: 10.1088/0266-5611/23/3/007. Google Scholar
[18]	K. Engan, K. Skretting and J. H. Husφy, Family of iterative ls-based dictionary learning algorithms, ils-dla, for sparse signal representation, Digit. Sig. Proc., 17 (2007), 32-49. doi: 10.1016/j.dsp.2006.02.002. Google Scholar
[19]	M. A. T. Figueiredo, R. D. Nowak and S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. on Sig. Proc., 1 (2007), 586-597. Google Scholar
[20]	M. A. T. Figueiredo, R. D. Nowak and S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. in Sig. Proc., 1 (2007), 586-597. Google Scholar
[21]	J. Friedman, T. Hastie and R. Tibshirani, Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9 (2008), 432-441. doi: 10.1093/biostatistics/kxm045. Google Scholar
[22]	A. Genkin, D. Lewis and D. Madigan, Large-scale bayesian logistic regression for text categorization, Technometrics, 49 (2007), 291-304. doi: 10.1198/004017007000000245. Google Scholar
[23]	M. Girolami, A variational method for learning sparse and overcomplete representations, Neu. Comput., 13 (2001), 2517-2532. doi: 10.1162/089976601753196003. Google Scholar
[24]	G. H. Golub, M. T. Heath and G. Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics, 21 (1979), 215-223. doi: 10.1080/00401706.1979.10489751. Google Scholar
[25]	E. Haber, U. Ascher and D. Oldenburg, Inversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach, Geophysics, 69 (2004), 1216-1228. Google Scholar
[26]	E. Haber, D. Oldenburg and A. Celler, Recovery of dynamic parameters in SPECT, IEEE trans. on Nuc. Sci., 44 (1997), 2425-2430. Google Scholar
[27]	P. C. Hansen, "Rank Deficient and Ill-Posed Problems," SIAM, Philadelphia, 1998. doi: 10.1137/1.9780898719697. Google Scholar
[28]	L. Horesh and E. Haber, Sensitivity computation of the $l_1$ minimization problem and its application to dictionary design of ill-posed problems, Inver. Problems, 25 (2009), 20pp. doi: 10.1088/0266-5611/25/9/095009. Google Scholar
[29]	N. Johnson and S. Kotz, "Continuous Multivariate Distributions," Wiley, New York, 1972. Google Scholar
[30]	S. M. Kakade, S. Shalev-Shwartz and A. Tewari, "Regularization Techniques for Learning with Matrices," J. Mach. Learning Research, accepted, 2012. Google Scholar
[31]	S. Kotz, T. J. Kozubowski and K. Podgórski, "Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance," Birkhäuser, Boston, 2001. Google Scholar
[32]	K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T. W. Lee and T. J. Sejnowski, Dictionary learning algorithms for sparse representation, Neu. Comput., 15 (2003), 349-396. doi: 10.1162/089976603762552951. Google Scholar
[33]	G. Kutyniok, D. Labate, W. Q. Lim and G. Weiss, Sparse multidimensional representation using shearlets, in "Proc. of SPIE," 5914, (2005), 254-262. doi: 10.1117/12.613494. Google Scholar
[34]	M. S. Lewicki and T. J. Sejnowski, Learning overcomplete representations, Neu. Comput., 12 (2000), 337-365. doi: 10.1162/089976600300015826. Google Scholar
[35]	A. S. Lewis and M. L. Overton, "Nonsmooth Optimization Via BFGS," SIAM J. Optim., in press, 2011. Google Scholar
[36]	Z. Lu, Smooth optimization approach for sparse covariance selection, SIAM J. Optim., 19 (2009), 807-1827. doi: 10.1137/070695915. Google Scholar
[37]	J. Mairal, F. Bach, J. Ponce, G. Sapiro and A. Zisserman, Supervised dictionary learning, 21 (2009), 1033-1040. Google Scholar
[38]	S. Mallat, "A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way," Academic Press, 2008. Google Scholar
[39]	D. Mihovilovic and R. N. Bracewell, Adaptive chirplet representation of signals on time-frequency plane, Electronics Letters, 27 (1991), 1159-1161. doi: 10.1049/el:19910723. Google Scholar
[40]	J. F. Murray and K. Kreutz-Delgado, Learning sparse overcomplete codes for images, J. VLSI Sig. Proc., 45 (2006), 97-110. doi: 10.1007/s11265-006-9774-5. Google Scholar
[41]	C. Papageorgiou and T. Poggio, "A Trainable Object Detection System: Car Detection in Static Images," Technical Report 1673, mitai, (CBCL Memo 180), October 1999. Google Scholar
[42]	A. Pázman, "Foundations of Optimum Experimental Design," Springer-Verlag, New York, 1986. Google Scholar
[43]	E. Le Pennec and S. Mallat, Sparse geometric image representations with bandelets, IEEE Trans. on Image Proc., 14 (2005), 423-438. doi: 10.1109/TIP.2005.843753. Google Scholar
[44]	G. Peyré and J. Fadili, Learning analysis sparsity priors, in "Proc. of Sampta'11," 2011. Google Scholar
[45]	R. Rubinstein, M. Zibulevsky and M. Elad, Double sparsity: learning sparse dictionaries for sparse signal approximation, IEEE Trans. on Sig. Proc., 58 (2010), 1553-1564. Google Scholar
[46]	G. Sapiro, "Geometric Partial Differential Equations and Image Analysis," Cambridge, 2001. Google Scholar
[47]	O. Scherzer, Scale-space methods and regularization for denoising and inverse problems, Advances in Image and Elect. Phys., 128 (2003), 445-530. Google Scholar
[48]	G. Schwarz, Estimating the dimension of a model, The Annals of Stat., 6 (1978), 461-464. doi: 10.1214/aos/1176344136. Google Scholar
[49]	D. F. Shanno and K. Phua, Matrix conditioning and nonlinear optimization, Math. Program., 14 (1978), 149-160. doi: 10.1007/BF01588962. Google Scholar
[50]	V. Sindhwani and A. Ghoting, "Large-Scale Distributed Non-Negative Sparse Coding and Sparse Dictionary Learning," SIGKDD, 2012. Google Scholar
[51]	M. Sugiyama and H. Ogawa, Optimal design of regularization term and regularization parameter by subspace information criterion, Neu. Netw., 15 (2002), 349-361. doi: 10.1016/S0893-6080(02)00022-9. Google Scholar
[52]	R. Tibshirani, Regression shrinkage and selection via the lasso, J. Royal. Statist. Soc B., 58 (1996), 267-288. Google Scholar
[53]	A. N. Tychonoff, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl., 4 (1963), 1035-1038. Google Scholar
[54]	E. van den Berg and M. P. Friedlander, Probing the pareto frontier for basis pursuit solutions, SIAM J. Scient. Comput., 31 (2008), 890-912. doi: 10.1137/080714488. Google Scholar
[55]	V. Vapnik, "Statistical Learning Theory," Wiley, 1998. Google Scholar
[56]	C. R. Vogel, "Computational Methods for Inverse Problem," SIAM, Philadelphia, 2002. doi: 10.1137/1.9780898717570. Google Scholar
[57]	H. Wang, B. Li and C. Leng, Shrinkage tuning parameter selection with a diverging number of parameters, J. R. Statist. Soc. B, 71 (2009), 671-683. doi: 10.1111/j.1467-9868.2008.00693.x. Google Scholar
[58]	D. P. Wipf and B. D. Rao, Sparse bayesian learning for basis selection, IEEE Trans. on Sig. Proc., 52 (2004), 2153-2164. Google Scholar
[59]	Z. J. Xiang, H. Xu and P. J. Ramadge, Learning sparse representations of high dimensional data on large scale dictionaries, 24 (2011), 900-908. Google Scholar
[60]	X. Chen and C. Zhang, Smoothing projected gradient method and its application to stochastic linear complementarity problems, SIAM J. Optim., 20, (2009), 627-649. doi: 10.1137/070702187. Google Scholar

show all references

References:

[1]	H. Akaike, A new look at the statistical model identification, IEEE Trans. Auto. Cont., 19 (1974), 716-723. doi: 10.1109/TAC.1974.1100705. Google Scholar
[2]	A. Aravkin, An $l_1$-Laplace robust kalman smoother, IEEE Trans. Auto. Cont., 56 (2011), 2898-2911. doi: 10.1109/TAC.2011.2141430. Google Scholar
[3]	U. Ascher, E. Haber and H. Huang, On effective methods for implicit piecewise smooth surface recovery, SIAM J. Scient. Comput., 28 (2006), 339-358. doi: 10.1137/040617261. Google Scholar
[4]	U. Ascher, K. van den Doel, H. Huang and B. Svaiter, Gradient descent and fast artificial time integration, M2AN, Math. Model. Numer. Anal., 43 (2009), 689-708. Google Scholar
[5]	J. A. Bilmes, "Factored Sparse Inverse Covariance Matrices," Proc. IEEE Intl. Conf. on Acous., Spee., and Sig. Proc., 2000. Google Scholar
[6]	A. M. Bruckstein, D. L. Donoho and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Review, 51 (2009), 34-81. doi: 10.1137/060657704. Google Scholar
[7]	E. J. Candès and D. L. Donoho, "Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges,", 1999., (). Google Scholar
[8]	E. J. Candès and M. B. Wakin, An introduction to compressive sampling, IEEE In Sig. Proc. Magaz., 25 (2008), 21-30. doi: 10.1109/MSP.2007.914731. Google Scholar
[9]	T. Chan and J. Shen, "Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods," SIAM, 2005. Google Scholar
[10]	X. Chen and W. Zhou., Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization, SIAM J. Imaging Sci., 3 (2010), 765-790. doi: 10.1137/080740167. Google Scholar
[11]	M. Cheney, D.Isaacson and J. C. Newell, Electrical impedance tomography, SIAM Review, 41 (1999), 85-101. doi: 10.1137/S0036144598333613. Google Scholar
[12]	J. Dahl, L. Vandenberghe and V. Roychowdhury, Covariance selection for non-chordal graphs via chordal embedding, Optim. Meth. and Soft., 23 (2008), 501-520. doi: 10.1080/10556780802102693. Google Scholar
[13]	Y. H. Dai and R. Fletcher, Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming, Numer. Math., 100 (2005), 21-47. doi: 10.1007/s00211-004-0569-y. Google Scholar
[14]	A. d'Aspremont, O. Banerjee and L. E. Ghaoui, First-order methods for sparse covariance selection, SIAM J. Matrix Anal. Appl., 30 (2008), 56-66. doi: 10.1137/060670985. Google Scholar
[15]	M. N. Do and M. Vetterli, The contourlet transform: an efficient directional multiresolution image representation, IEEE Trans. on Image Proc., 14 (2005), 2091-2106. doi: 10.1109/TIP.2005.859376. Google Scholar
[16]	W. Dong, L. Zhang, G. Shi and X. Wu, Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization, IEEE Trans. Image Proc., 20 (2011), 1838-1857. doi: 10.1109/TIP.2011.2108306. Google Scholar
[17]	M. Elad, P. Milanfar and R. Rubinstein, Analysis versus synthesis in signal priors, Inver. Problems, 23 (2007), 947-968. doi: 10.1088/0266-5611/23/3/007. Google Scholar
[18]	K. Engan, K. Skretting and J. H. Husφy, Family of iterative ls-based dictionary learning algorithms, ils-dla, for sparse signal representation, Digit. Sig. Proc., 17 (2007), 32-49. doi: 10.1016/j.dsp.2006.02.002. Google Scholar
[19]	M. A. T. Figueiredo, R. D. Nowak and S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. on Sig. Proc., 1 (2007), 586-597. Google Scholar
[20]	M. A. T. Figueiredo, R. D. Nowak and S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. in Sig. Proc., 1 (2007), 586-597. Google Scholar
[21]	J. Friedman, T. Hastie and R. Tibshirani, Sparse inverse covariance estimation with the graphical lasso, Biostatistics, 9 (2008), 432-441. doi: 10.1093/biostatistics/kxm045. Google Scholar
[22]	A. Genkin, D. Lewis and D. Madigan, Large-scale bayesian logistic regression for text categorization, Technometrics, 49 (2007), 291-304. doi: 10.1198/004017007000000245. Google Scholar
[23]	M. Girolami, A variational method for learning sparse and overcomplete representations, Neu. Comput., 13 (2001), 2517-2532. doi: 10.1162/089976601753196003. Google Scholar
[24]	G. H. Golub, M. T. Heath and G. Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics, 21 (1979), 215-223. doi: 10.1080/00401706.1979.10489751. Google Scholar
[25]	E. Haber, U. Ascher and D. Oldenburg, Inversion of 3D electromagnetic data in frequency and time domain using an inexact all-at-once approach, Geophysics, 69 (2004), 1216-1228. Google Scholar
[26]	E. Haber, D. Oldenburg and A. Celler, Recovery of dynamic parameters in SPECT, IEEE trans. on Nuc. Sci., 44 (1997), 2425-2430. Google Scholar
[27]	P. C. Hansen, "Rank Deficient and Ill-Posed Problems," SIAM, Philadelphia, 1998. doi: 10.1137/1.9780898719697. Google Scholar
[28]	L. Horesh and E. Haber, Sensitivity computation of the $l_1$ minimization problem and its application to dictionary design of ill-posed problems, Inver. Problems, 25 (2009), 20pp. doi: 10.1088/0266-5611/25/9/095009. Google Scholar
[29]	N. Johnson and S. Kotz, "Continuous Multivariate Distributions," Wiley, New York, 1972. Google Scholar
[30]	S. M. Kakade, S. Shalev-Shwartz and A. Tewari, "Regularization Techniques for Learning with Matrices," J. Mach. Learning Research, accepted, 2012. Google Scholar
[31]	S. Kotz, T. J. Kozubowski and K. Podgórski, "Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance," Birkhäuser, Boston, 2001. Google Scholar
[32]	K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T. W. Lee and T. J. Sejnowski, Dictionary learning algorithms for sparse representation, Neu. Comput., 15 (2003), 349-396. doi: 10.1162/089976603762552951. Google Scholar
[33]	G. Kutyniok, D. Labate, W. Q. Lim and G. Weiss, Sparse multidimensional representation using shearlets, in "Proc. of SPIE," 5914, (2005), 254-262. doi: 10.1117/12.613494. Google Scholar
[34]	M. S. Lewicki and T. J. Sejnowski, Learning overcomplete representations, Neu. Comput., 12 (2000), 337-365. doi: 10.1162/089976600300015826. Google Scholar
[35]	A. S. Lewis and M. L. Overton, "Nonsmooth Optimization Via BFGS," SIAM J. Optim., in press, 2011. Google Scholar
[36]	Z. Lu, Smooth optimization approach for sparse covariance selection, SIAM J. Optim., 19 (2009), 807-1827. doi: 10.1137/070695915. Google Scholar
[37]	J. Mairal, F. Bach, J. Ponce, G. Sapiro and A. Zisserman, Supervised dictionary learning, 21 (2009), 1033-1040. Google Scholar
[38]	S. Mallat, "A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way," Academic Press, 2008. Google Scholar
[39]	D. Mihovilovic and R. N. Bracewell, Adaptive chirplet representation of signals on time-frequency plane, Electronics Letters, 27 (1991), 1159-1161. doi: 10.1049/el:19910723. Google Scholar
[40]	J. F. Murray and K. Kreutz-Delgado, Learning sparse overcomplete codes for images, J. VLSI Sig. Proc., 45 (2006), 97-110. doi: 10.1007/s11265-006-9774-5. Google Scholar
[41]	C. Papageorgiou and T. Poggio, "A Trainable Object Detection System: Car Detection in Static Images," Technical Report 1673, mitai, (CBCL Memo 180), October 1999. Google Scholar
[42]	A. Pázman, "Foundations of Optimum Experimental Design," Springer-Verlag, New York, 1986. Google Scholar
[43]	E. Le Pennec and S. Mallat, Sparse geometric image representations with bandelets, IEEE Trans. on Image Proc., 14 (2005), 423-438. doi: 10.1109/TIP.2005.843753. Google Scholar
[44]	G. Peyré and J. Fadili, Learning analysis sparsity priors, in "Proc. of Sampta'11," 2011. Google Scholar
[45]	R. Rubinstein, M. Zibulevsky and M. Elad, Double sparsity: learning sparse dictionaries for sparse signal approximation, IEEE Trans. on Sig. Proc., 58 (2010), 1553-1564. Google Scholar
[46]	G. Sapiro, "Geometric Partial Differential Equations and Image Analysis," Cambridge, 2001. Google Scholar
[47]	O. Scherzer, Scale-space methods and regularization for denoising and inverse problems, Advances in Image and Elect. Phys., 128 (2003), 445-530. Google Scholar
[48]	G. Schwarz, Estimating the dimension of a model, The Annals of Stat., 6 (1978), 461-464. doi: 10.1214/aos/1176344136. Google Scholar
[49]	D. F. Shanno and K. Phua, Matrix conditioning and nonlinear optimization, Math. Program., 14 (1978), 149-160. doi: 10.1007/BF01588962. Google Scholar
[50]	V. Sindhwani and A. Ghoting, "Large-Scale Distributed Non-Negative Sparse Coding and Sparse Dictionary Learning," SIGKDD, 2012. Google Scholar
[51]	M. Sugiyama and H. Ogawa, Optimal design of regularization term and regularization parameter by subspace information criterion, Neu. Netw., 15 (2002), 349-361. doi: 10.1016/S0893-6080(02)00022-9. Google Scholar
[52]	R. Tibshirani, Regression shrinkage and selection via the lasso, J. Royal. Statist. Soc B., 58 (1996), 267-288. Google Scholar
[53]	A. N. Tychonoff, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl., 4 (1963), 1035-1038. Google Scholar
[54]	E. van den Berg and M. P. Friedlander, Probing the pareto frontier for basis pursuit solutions, SIAM J. Scient. Comput., 31 (2008), 890-912. doi: 10.1137/080714488. Google Scholar
[55]	V. Vapnik, "Statistical Learning Theory," Wiley, 1998. Google Scholar
[56]	C. R. Vogel, "Computational Methods for Inverse Problem," SIAM, Philadelphia, 2002. doi: 10.1137/1.9780898717570. Google Scholar
[57]	H. Wang, B. Li and C. Leng, Shrinkage tuning parameter selection with a diverging number of parameters, J. R. Statist. Soc. B, 71 (2009), 671-683. doi: 10.1111/j.1467-9868.2008.00693.x. Google Scholar
[58]	D. P. Wipf and B. D. Rao, Sparse bayesian learning for basis selection, IEEE Trans. on Sig. Proc., 52 (2004), 2153-2164. Google Scholar
[59]	Z. J. Xiang, H. Xu and P. J. Ramadge, Learning sparse representations of high dimensional data on large scale dictionaries, 24 (2011), 900-908. Google Scholar
[60]	X. Chen and C. Zhang, Smoothing projected gradient method and its application to stochastic linear complementarity problems, SIAM J. Optim., 20, (2009), 627-649. doi: 10.1137/070702187. Google Scholar

[1]	Huining Qiu, Xiaoming Chen, Wanquan Liu, Guanglu Zhou, Yiju Wang, Jianhuang Lai. A fast $\ell_1$-solver and its applications to robust face recognition. Journal of Industrial & Management Optimization, 2012, 8 (1) : 163-178. doi: 10.3934/jimo.2012.8.163
[2]	Pengbo Geng, Wengu Chen. Unconstrained $ \ell_1 $-$ \ell_2 $ minimization for sparse recovery via mutual coherence. Mathematical Foundations of Computing, 2020, 3 (2) : 65-79. doi: 10.3934/mfc.2020006
[3]	Shaojun Lan, Yinghui Tang, Miaomiao Yu. System capacity optimization design and optimal threshold $N^{}$ for a $GEO/G/1$ discrete-time queue with single server vacation and under the control of Min($N, V$)-policy. Journal of Industrial & Management Optimization*, 2016, 12 (4) : 1435-1464. doi: 10.3934/jimo.2016.12.1435
[4]	H. T. Banks, D. Rubio, N. Saintier, M. I. Troparevsky. Optimal design for parameter estimation in EEG problems in a 3D multilayered domain. Mathematical Biosciences & Engineering, 2015, 12 (4) : 739-760. doi: 10.3934/mbe.2015.12.739
[5]	Bin Li, Kok Lay Teo, Cheng-Chew Lim, Guang Ren Duan. An optimal PID controller design for nonlinear constrained optimal control problems. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1101-1117. doi: 10.3934/dcdsb.2011.16.1101
[6]	El-Sayed M.E. Mostafa. A nonlinear conjugate gradient method for a special class of matrix optimization problems. Journal of Industrial & Management Optimization, 2014, 10 (3) : 883-903. doi: 10.3934/jimo.2014.10.883
[7]	Qia Li, Na Zhang. Capped $\ell_p$ approximations for the composite $\ell_0$ regularization problem. Inverse Problems & Imaging, 2018, 12 (5) : 1219-1243. doi: 10.3934/ipi.2018051
[8]	Jinying Ma, Honglei Xu. Empirical analysis and optimization of capital structure adjustment. Journal of Industrial & Management Optimization, 2020, 16 (3) : 1037-1047. doi: 10.3934/jimo.2018191
[9]	Xueting Cui, Xiaoling Sun, Dan Sha. An empirical study on discrete optimization models for portfolio selection. Journal of Industrial & Management Optimization, 2009, 5 (1) : 33-46. doi: 10.3934/jimo.2009.5.33
[10]	Mohamed Aly Tawhid. Nonsmooth generalized complementarity as unconstrained optimization. Journal of Industrial & Management Optimization, 2010, 6 (2) : 411-423. doi: 10.3934/jimo.2010.6.411
[11]	Didi Lv, Qingping Zhou, Jae Kyu Choi, Jinglai Li, Xiaoqun Zhang. Nonlocal TV-Gaussian prior for Bayesian inverse problems with applications to limited CT reconstruction. Inverse Problems & Imaging, 2020, 14 (1) : 117-132. doi: 10.3934/ipi.2019066
[12]	Johnathan M. Bardsley. A theoretical framework for the regularization of Poisson likelihood estimation problems. Inverse Problems & Imaging, 2010, 4 (1) : 11-17. doi: 10.3934/ipi.2010.4.11
[13]	Jiangqi Wu, Linjie Wen, Jinglai Li. Resampled ensemble Kalman inversion for Bayesian parameter estimation with sequential data. Discrete & Continuous Dynamical Systems - S, 2021 doi: 10.3934/dcdss.2021045
[14]	Junxiong Jia, Jigen Peng, Jinghuai Gao. Posterior contraction for empirical bayesian approach to inverse problems under non-diagonal assumption. Inverse Problems & Imaging, 2021, 15 (2) : 201-228. doi: 10.3934/ipi.2020061
[15]	Wu Chanti, Qiu Youzhen. A nonlinear empirical analysis on influence factor of circulation efficiency. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 929-940. doi: 10.3934/dcdss.2019062
[16]	Giancarlo Bigi. Componentwise versus global approaches to nonsmooth multiobjective optimization. Journal of Industrial & Management Optimization, 2005, 1 (1) : 21-32. doi: 10.3934/jimo.2005.1.21
[17]	Nobuko Sagara, Masao Fukushima. trust region method for nonsmooth convex optimization. Journal of Industrial & Management Optimization, 2005, 1 (2) : 171-180. doi: 10.3934/jimo.2005.1.171
[18]	Burak Ordin, Adil Bagirov, Ehsan Mohebi. An incremental nonsmooth optimization algorithm for clustering using $ L_1 $ and $ L_\infty $ norms. Journal of Industrial & Management Optimization, 2020, 16 (6) : 2757-2779. doi: 10.3934/jimo.2019079
[19]	K. T. Arasu, Manil T. Mohan. Optimization problems with orthogonal matrix constraints. Numerical Algebra, Control & Optimization, 2018, 8 (4) : 413-440. doi: 10.3934/naco.2018026
[20]	Victor Churchill, Rick Archibald, Anne Gelb. Edge-adaptive $ \ell_2 $ regularization image reconstruction from non-uniform Fourier data. Inverse Problems & Imaging, 2019, 13 (5) : 931-958. doi: 10.3934/ipi.2019042

2020 Impact Factor: 1.639

American Institute of Mathematical Sciences

Optimal estimation of $\ell_1$-regularization prior from a regularized empirical Bayesian risk standpoint

References:

References:

Tools

Metrics

Other articles
by authors

Tools

Metrics

Other articles
by authors

American Institute of Mathematical Sciences

Optimal estimation of $\ell_1$-regularization prior from a regularized empirical Bayesian risk standpoint

References:

References:

Tools

Metrics

Other articlesby authors

Tools

Metrics

Other articlesby authors

Other articles
by authors

Other articles
by authors