Article Contents
Article Contents

# Convergence of a randomized Douglas-Rachford method for linear system

The research of the second author is partially supported by the NSFC grants 11871279, 11571178

• In this article, we propose a randomized Douglas-Rachford(DR) method for linear system. This algorithm is based on the cyclic DR method. We consider a linear system as a feasible problem of finding intersection of hyperplanes. In each iteration, the next iteration point is determined by a random DR operator. We prove the convergence of the iteration points based on expectation. And the variance of the iteration points declines to zero. The numerical experiment shows that the proposed algorithm performs better than the cyclic DR method.

Mathematics Subject Classification: 65F10, 65F20, 65K05, 90C25, 90C15, 15A06.

 Citation:

• Figure 1.  Comparison of number of projections/reflections between randomized Kaczmarz(RK), cyclic DR(CDR) and randomized DR(RDR) when the size of $A$ and $b$ is $m = 1000$ and $n = 100$

Table 1.  Number of Projections/Reflections(NPR) for $n = 50$

 $m$ $1000$ $2000$ $3000$ $4000$ $5000$ NPR CDR 1264 1404 1282 1488 1390 RK 725.3 701.1 696.1 671.3 683.8 RDR 705.2 683.8 696 669.6 698.8

Table 2.  Number of Projections/Reflections(NPR) for $n = 100$

 $m$ $1000$ $2000$ $3000$ $4000$ $5000$ NPR CDR 2620 2604 2692 2770 2770 RK 1508.4 1419 1425.5 1422.5 1395.1 RDR 1523.2 1425.8 1409.6 1421.4 1381.8

Table 3.  Number of Projections/Reflections(NPR) for $n = 150$

 $m$ $1000$ $2000$ $3000$ $4000$ $5000$ NPR CDR 4272 4286 4104 4096 3854 RK 2466.9 2194.4 2175.4 2092.1 2152 RDR 2574.2 2227.8 2160.8 2133 2113.8
•  [1] Z.-Z. Bai and W.-T. Wu, On convergence rate of the randomized kaczmarz method, Linear Algebra and Its Applications, 553 (2018), 252-269.  doi: 10.1016/j.laa.2018.05.009. [2] Z.-Z. Bai and W.-T. Wu, On greedy randomized kaczmarz method for solving large sparse linear systems, SIAM Journal on Scientific Computing, 40 (2018), A592–A606. doi: 10.1137/17M1137747. [3] H. H. Bauschke, Projection algorithms: results and open problems, in Studies in Computational Mathematics, Elsevier, 8 (2001), 11–22. doi: 10.1016/S1570-579X(01)80003-3. [4] H. H. Bauschke and J. M. Borwein, On the convergence of von neumann's alternating projection algorithm for two sets, Set-Valued Analysis, 1 (1993), 185-212.  doi: 10.1007/BF01027691. [5] H. H. Bauschke and J. M. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Review, 38 (1996), 367-426.  doi: 10.1137/S0036144593251710. [6] H. H. Bauschke, J. M. Borwein and A. S. Lewis, The method of cyclic projections for closed convex sets in hilbert space, Contemporary Mathematics, 204 (1997), 1-38.  doi: 10.1090/conm/204/02620. [7] H. H. Bauschke, P. L. Combettes et al., Convex Analysis and Monotone Operator Theory in Hilbert Spaces, vol. 408, Springer, 2011. doi: 10.1007/978-1-4419-9467-7. [8] J. M. Borwein and M. K. Tam, A cyclic douglas–rachford iteration scheme, Journal of Optimization Theory and Applications, 160 (2014), 1-29.  doi: 10.1007/s10957-013-0381-x. [9] J. P. Boyle and R. L. Dykstra, A method for finding projections onto the intersection of convex sets in hilbert spaces, in Advances in order restricted statistical inference, Springer, (1986), 28–47. doi: 10.1007/978-1-4613-9940-7_3. [10] C. Brezinski, Projection methods for linear systems, Journal of Computational and Applied Mathematics, 77 (1997), 35-51.  doi: 10.1016/S0377-0427(96)00121-5. [11] J. Douglas and H. H. Rachford, On the numerical solution of heat conduction problems in two and three space variables, Transactions of the American mathematical Society, 82 (1956), 421-439.  doi: 10.2307/1993056. [12] R. L. Dykstra, An algorithm for restricted least squares regression, Journal of the American Statistical Association, 78 (1983), 837-842. [13] T. Elfving, Block-iterative methods for consistent and inconsistent linear equations, Numerische Mathematik, 35 (1980), 1-12.  doi: 10.1007/BF01396365. [14] M. Griebel and P. Oswald, Greedy and randomized versions of the multiplicative schwarz method, Linear Algebra and Its Applications, 437 (2012), 1596-1610.  doi: 10.1016/j.laa.2012.04.052. [15] S. Karczmarz, Angenäherte auflösung von systemen linearer gleichungen, Bulletin International de l'Académie Polonaise des Sciences et des Lettres. Classe des Sciences Mathématiques et Naturelles. Série A, Sciences Mathématiques, 35 (1937), 355-357. [16] D. Needell, R. Zhao and A. Zouzias, Randomized block kaczmarz method with projection for solving least squares, Linear Algebra and its Applications, 484 (2015), 322-343.  doi: 10.1016/j.laa.2015.06.027. [17] J. Nutini, B. Sepehry, I. Laradji, M. Schmidt, H. Koepke and A. Virani, Convergence rates for greedy kaczmarz algorithms, and faster randomized kaczmarz rules using the orthogonality graph, in Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, AUAI Press, (2016), 547–556. [18] T. Strohmer and R. Vershynin, A randomized kaczmarz algorithm with exponential convergence, Journal of Fourier Analysis and Applications, 15 (2008), 262. doi: 10.1007/s00041-008-9030-4. [19] J. Von Neumann, The geometry of orthogonal spaces, Functional operators-vol. II, Annals of Mathematics Studies, 22 (1950), This is a reprint of mimeographed lecture notes, first distributed in 1933.

Figures(1)

Tables(3)