December  2017, 11(6): 997-1025. doi: 10.3934/ipi.2017046

Analysis of a variational model for motion compensated inpainting

1. 

Istituto per le Applicazioni del Calcolo, CNR, Via dei Taurini 19,00185 Roma, Italy

2. 

Dipartimento di Matematica e Informatica, Universitá della Calabria, Via Pietro Bucci, Arcavacata di Rende, 87036 Cosenza, Italy

Received  October 2016 Revised  June 2017 Published  September 2017

We study a variational problem for simultaneous video inpainting and motion estimation. We consider a functional proposed by Lauze and Nielsen [25] and we study, by means of the relaxation method of the Calculus of Variations, a slightly modified version of this functional. The domain of the relaxed functional is constituted of functions of bounded variation and we compute a representation formula of the relaxed functional. The representation formula shows the role of discontinuities of the various functions involved in the variational model. The present study clarifies the variational properties of the functional proposed in [25] for motion compensated video inpainting.

Citation: Riccardo March, Giuseppe Riey. Analysis of a variational model for motion compensated inpainting. Inverse Problems & Imaging, 2017, 11 (6) : 997-1025. doi: 10.3934/ipi.2017046
References:
[1]

L. Ambrosio, Variational problems in SBV and image segmentation, Acta Appl. Math., 17 (1989), 1-40.  doi: 10.1007/BF00052492.  Google Scholar

[2] L. AmbrosioN. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, New York, 2000.   Google Scholar
[3]

L. Ambrosio and D. Pallara, Integral representation of relaxed functionals on $BV(\mathbb{R}^n, \mathbb{R}^k)$ and polyhedral approximation, Indiana Univ. Math. J., 42 (1993), 295-321.  doi: 10.1512/iumj.1993.42.42015.  Google Scholar

[4]

G. AubertR. Deriche and P. Kornprobst, Computing optimal flow via variational techniques, SIAM J. App. Math., 60 (2000), 156-182.  doi: 10.1137/S0036139998340170.  Google Scholar

[5]

G. Aubert and P. Kornprobst, A mathematical study of the relaxed optical flow problem in the space $BV(Ω)$, SIAM J. Math. Anal., 30 (1999), 1282-1308.  doi: 10.1137/S003614109834123X.  Google Scholar

[6] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd Edition, Springer, New York, 2006.   Google Scholar
[7]

M. Bertalmio, A. Bertozzi and G. Sapiro, Navier-Stokes, fluid-dynamics and image and video inpainting, in Proceedings of Computer Vision and Pattern Recognition, 2001,355-362. doi: 10.1109/CVPR.2001.990497.  Google Scholar

[8]

K. BrediesK. Kunish and T. Pock, Total generalized variation, SIAM J. Imaging Sciences, 3 (2010), 492-526.  doi: 10.1137/090769521.  Google Scholar

[9]

K. BrediesK. Kunish and T. Valkonen, Properties of $L^1-TGV^2$: The one-dimensional case, J. Math. Anal. Appl., 398 (2013), 438-454.  doi: 10.1016/j.jmaa.2012.08.053.  Google Scholar

[10]

T. Brox, A. Bruhn, N. Papenberg and J. Weickert, High accuracy optical flow estimation based on a theory for warping, Proceedings of the 8th European Conference on Computer Vision (eds. T. Pajdla and J. Matas), Springer, 3024 (2004), 25-36. doi: 10.1007/978-3-540-24673-2_3.  Google Scholar

[11]

A. Buffa and C. Ortner, Compact embeddings of broken Sobolev spaces and applications, IMA J. Numer. Anal., 29 (2009), 827-855.  doi: 10.1093/imanum/drn038.  Google Scholar

[12]

G. Buttazzo, Semicontinuity, Relaxation and Integral Representation in the Calculus of Variations, Pitman Research Notes in Mathematics Series, Vol. 207, Longman Scientific & Technical, UK, 1989.  Google Scholar

[13]

J. P. Cocquerez, L. Chanas and J. Blanc-Talon, Simultaneous inpainting and motion estimation of highly degraded video-sequences, in Scandinavian Conference on Image Analysis, LNCS, Springer-Verlag, 2749 (2003), 523-530. Google Scholar

[14]

S. ContiJ. Ginster and M. Rumpf, A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM: Mathematical Modelling and Numerical Analysis, 49 (2015), 1463-1487.  doi: 10.1051/m2an/2015036.  Google Scholar

[15]

A. Corbo Esposito and R. De Arcangelis, Comparison results for some types of relaxation of variational integral functionals, Ann. Mat. Pura Appl., 164 (1993), 155-193.  doi: 10.1007/BF01759320.  Google Scholar

[16]

F. Demengel, Fonctions á hessien borné, Annales de l'Institut Fourier, 34 (1984), 155-190.   Google Scholar

[17]

F. Demengel and R. Temam, Convex functions of a measure and applications, Indiana University Mathematics Journal, 33 (1984), 673-709.  doi: 10.1512/iumj.1984.33.33036.  Google Scholar

[18]

M. GiaquintaG. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅰ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 143-156.   Google Scholar

[19]

M. GiaquintaG. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅱ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 157-172.   Google Scholar

[20] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, Boston, 1984.  doi: 10.1007/978-1-4684-9486-0.  Google Scholar
[21]

C. Goffman and J. Serrin, Sublinear functions of measures and variationals integrals, Duke Math. J., 31 (1964), 159-178.   Google Scholar

[22]

B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203.   Google Scholar

[23]

S. H. KellerF. Lauze and M. Nielsen, Deintarlacing using variational methods, IEEE Trans. Image Proc., 17 (2008), 2015-2028.  doi: 10.1109/TIP.2008.2003394.  Google Scholar

[24]

S. H. KellerF. Lauze and M. Nielsen, Video super-resolution using simultaneous motion and intensity calculations, IEEE Trans. Image Proc., 20 (2011), 1870-1884.  doi: 10.1109/TIP.2011.2106793.  Google Scholar

[25]

F. Lauze and M. Nielsen, A Variational algorithm for motion compensated inpainting, in British Machine Vision Conference (eds. S. Barman, A. Hoppe and T. Ellis editors), BMVA, 2 (2004), 777-787. Google Scholar

[26]

Y. G. Reshetnyak, Weak convergence of completely additive vector functions on a set, (Russian) Sibirski Mat. Zh., 9 (1968), 1386-1394; translation in Siberian Math. J., 9 (1968), 1039-1045.  Google Scholar

[27]

S. UrasF. GirosiA. Verri and V. Torre, A computational approach to motion perception, Biol. Cybern., 60 (1988), 79-87.   Google Scholar

[28] W. P. Ziemer, Weakly differentiable functions, Springer-Verlag, New York, 1989.  doi: 10.1007/978-1-4612-1015-3.  Google Scholar

show all references

References:
[1]

L. Ambrosio, Variational problems in SBV and image segmentation, Acta Appl. Math., 17 (1989), 1-40.  doi: 10.1007/BF00052492.  Google Scholar

[2] L. AmbrosioN. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, New York, 2000.   Google Scholar
[3]

L. Ambrosio and D. Pallara, Integral representation of relaxed functionals on $BV(\mathbb{R}^n, \mathbb{R}^k)$ and polyhedral approximation, Indiana Univ. Math. J., 42 (1993), 295-321.  doi: 10.1512/iumj.1993.42.42015.  Google Scholar

[4]

G. AubertR. Deriche and P. Kornprobst, Computing optimal flow via variational techniques, SIAM J. App. Math., 60 (2000), 156-182.  doi: 10.1137/S0036139998340170.  Google Scholar

[5]

G. Aubert and P. Kornprobst, A mathematical study of the relaxed optical flow problem in the space $BV(Ω)$, SIAM J. Math. Anal., 30 (1999), 1282-1308.  doi: 10.1137/S003614109834123X.  Google Scholar

[6] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd Edition, Springer, New York, 2006.   Google Scholar
[7]

M. Bertalmio, A. Bertozzi and G. Sapiro, Navier-Stokes, fluid-dynamics and image and video inpainting, in Proceedings of Computer Vision and Pattern Recognition, 2001,355-362. doi: 10.1109/CVPR.2001.990497.  Google Scholar

[8]

K. BrediesK. Kunish and T. Pock, Total generalized variation, SIAM J. Imaging Sciences, 3 (2010), 492-526.  doi: 10.1137/090769521.  Google Scholar

[9]

K. BrediesK. Kunish and T. Valkonen, Properties of $L^1-TGV^2$: The one-dimensional case, J. Math. Anal. Appl., 398 (2013), 438-454.  doi: 10.1016/j.jmaa.2012.08.053.  Google Scholar

[10]

T. Brox, A. Bruhn, N. Papenberg and J. Weickert, High accuracy optical flow estimation based on a theory for warping, Proceedings of the 8th European Conference on Computer Vision (eds. T. Pajdla and J. Matas), Springer, 3024 (2004), 25-36. doi: 10.1007/978-3-540-24673-2_3.  Google Scholar

[11]

A. Buffa and C. Ortner, Compact embeddings of broken Sobolev spaces and applications, IMA J. Numer. Anal., 29 (2009), 827-855.  doi: 10.1093/imanum/drn038.  Google Scholar

[12]

G. Buttazzo, Semicontinuity, Relaxation and Integral Representation in the Calculus of Variations, Pitman Research Notes in Mathematics Series, Vol. 207, Longman Scientific & Technical, UK, 1989.  Google Scholar

[13]

J. P. Cocquerez, L. Chanas and J. Blanc-Talon, Simultaneous inpainting and motion estimation of highly degraded video-sequences, in Scandinavian Conference on Image Analysis, LNCS, Springer-Verlag, 2749 (2003), 523-530. Google Scholar

[14]

S. ContiJ. Ginster and M. Rumpf, A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM: Mathematical Modelling and Numerical Analysis, 49 (2015), 1463-1487.  doi: 10.1051/m2an/2015036.  Google Scholar

[15]

A. Corbo Esposito and R. De Arcangelis, Comparison results for some types of relaxation of variational integral functionals, Ann. Mat. Pura Appl., 164 (1993), 155-193.  doi: 10.1007/BF01759320.  Google Scholar

[16]

F. Demengel, Fonctions á hessien borné, Annales de l'Institut Fourier, 34 (1984), 155-190.   Google Scholar

[17]

F. Demengel and R. Temam, Convex functions of a measure and applications, Indiana University Mathematics Journal, 33 (1984), 673-709.  doi: 10.1512/iumj.1984.33.33036.  Google Scholar

[18]

M. GiaquintaG. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅰ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 143-156.   Google Scholar

[19]

M. GiaquintaG. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅱ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 157-172.   Google Scholar

[20] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, Boston, 1984.  doi: 10.1007/978-1-4684-9486-0.  Google Scholar
[21]

C. Goffman and J. Serrin, Sublinear functions of measures and variationals integrals, Duke Math. J., 31 (1964), 159-178.   Google Scholar

[22]

B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203.   Google Scholar

[23]

S. H. KellerF. Lauze and M. Nielsen, Deintarlacing using variational methods, IEEE Trans. Image Proc., 17 (2008), 2015-2028.  doi: 10.1109/TIP.2008.2003394.  Google Scholar

[24]

S. H. KellerF. Lauze and M. Nielsen, Video super-resolution using simultaneous motion and intensity calculations, IEEE Trans. Image Proc., 20 (2011), 1870-1884.  doi: 10.1109/TIP.2011.2106793.  Google Scholar

[25]

F. Lauze and M. Nielsen, A Variational algorithm for motion compensated inpainting, in British Machine Vision Conference (eds. S. Barman, A. Hoppe and T. Ellis editors), BMVA, 2 (2004), 777-787. Google Scholar

[26]

Y. G. Reshetnyak, Weak convergence of completely additive vector functions on a set, (Russian) Sibirski Mat. Zh., 9 (1968), 1386-1394; translation in Siberian Math. J., 9 (1968), 1039-1045.  Google Scholar

[27]

S. UrasF. GirosiA. Verri and V. Torre, A computational approach to motion perception, Biol. Cybern., 60 (1988), 79-87.   Google Scholar

[28] W. P. Ziemer, Weakly differentiable functions, Springer-Verlag, New York, 1989.  doi: 10.1007/978-1-4612-1015-3.  Google Scholar
[1]

Xuemei Chen, Julia Dobrosotskaya. Inpainting via sparse recovery with directional constraints. Mathematical Foundations of Computing, 2020, 3 (4) : 229-247. doi: 10.3934/mfc.2020025

[2]

Huu-Quang Nguyen, Ya-Chi Chu, Ruey-Lin Sheu. On the convexity for the range set of two quadratic functions. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020169

[3]

Aihua Fan, Jörg Schmeling, Weixiao Shen. $ L^\infty $-estimation of generalized Thue-Morse trigonometric polynomials and ergodic maximization. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 297-327. doi: 10.3934/dcds.2020363

[4]

Thabet Abdeljawad, Mohammad Esmael Samei. Applying quantum calculus for the existence of solution of $ q $-integro-differential equations with three criteria. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020440

[5]

Yu Zhou, Xinfeng Dong, Yongzhuang Wei, Fengrong Zhang. A note on the Signal-to-noise ratio of $ (n, m) $-functions. Advances in Mathematics of Communications, 2020  doi: 10.3934/amc.2020117

[6]

Sumit Arora, Manil T. Mohan, Jaydev Dabas. Approximate controllability of a Sobolev type impulsive functional evolution system in Banach spaces. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020049

[7]

Djamel Aaid, Amel Noui, Özen Özer. Piecewise quadratic bounding functions for finding real roots of polynomials. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 63-73. doi: 10.3934/naco.2020015

[8]

Bo Chen, Youde Wang. Global weak solutions for Landau-Lifshitz flows and heat flows associated to micromagnetic energy functional. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020268

[9]

Bernard Bonnard, Jérémy Rouot. Geometric optimal techniques to control the muscular force response to functional electrical stimulation using a non-isometric force-fatigue model. Journal of Geometric Mechanics, 2020  doi: 10.3934/jgm.2020032

2019 Impact Factor: 1.373

Metrics

  • PDF downloads (48)
  • HTML views (142)
  • Cited by (3)

Other articles
by authors

[Back to Top]