Advanced Search
Article Contents
Article Contents

Optical flow on evolving sphere-like surfaces

  • * Corresponding author

    * Corresponding author 
Abstract Full Text(HTML) Figure(12) / Table(2) Related Papers Cited by
  • In this work we consider optical flow on evolving Riemannian 2-manifolds which can be parametrised from the 2-sphere. Our main motivation is to estimate cell motion in time-lapse volumetric microscopy images depicting fluorescently labelled cells of a live zebrafish embryo. We exploit the fact that the recorded cells float on the surface of the embryo and allow for the extraction of an image sequence together with a sphere-like surface. We solve the resulting variational problem by means of a Galerkin method based on vector spherical harmonics and present numerical results computed from the aforementioned microscopy data.

    Mathematics Subject Classification: 35A15, 68U10, 92C55, 33C55, 92C37, 92C17, 53A05.


    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  Frames 140 (left) and 141 (right) of the volumetric zebrafish microscopy images recorded during early embryogenesis. The sequence contains a total number of 151 frames recorded at time intervals of $120 \, \mathrm{s}$. Fluorescence response is indicated by blue colour and is proportional to the observed intensity. The embryonic axis of the animal forms around the clearly visible dent. All dimensions are in micrometer ($\mu$m).

    Figure 2.  Depicted are frames no. 140 (left) and 141 (right) of the processed zebrafish microscopy sequence. Top and bottom row differ by a rotation of 180 degrees around the $x^{3}$-axis. All dimensions are in micrometer ($\mu$m).

    Figure 11.  From left to right, top to bottom: a) colour-coded surface velocity $\mathbf{\hat{V}}$, b) signed norm $\text{sign}(\partial_{t} \tilde{\rho}) \left\| {{\bf{\hat V}}} \right\|$, c) optical flow $\mathbf{\hat{v}}$ for $\alpha = 1$, and d) total motion $\mathbf{\hat{M}} = \mathbf{\hat{V}} + \mathbf{\hat{v}}$. Values are computed for the interval between frames 140 and 141. All surfaces are depicted in a top view.

    Figure 12.  Top view of the optical flow field computed on a static spherical geometry. The same values of α as in Fig. 10 were used.

    Figure 3.  Schematic illustration of a cut through the surfaces $\mathcal{S}^2$ and $\mathcal{M}_{t}$ intersecting the origin. In addition, we show a radial line along which the extension $\bar{f}(t, \cdot)$ is constant. The surface normals are shown in grey.

    Figure 5.  Commutative diagram relating spaces $\Omega$, $\mathcal{S}^{2}$, and $\mathcal{M}_{t}$, and tangent vector fields. We highlight that $\mathbf{y}_{p}$ is the coordinate representation, see Sec. 2.1, of a particular tangential vector spherical harmonic $\mathbf{\tilde{y}}_{p}$ and $\mathbf{\hat{y}}_{p}$ is its uniquely identified tangent vector field on $\mathcal{M}_{t}$.

    Figure 4.  Illustration of trajectories through the evolving surface. Their corresponding velocities are shown in grey.

    Figure 6.  Illustration of a triangular face (filled gray) intersecting the sphere $\mathcal{S}^{2}$ at the vertices (hollow circles). The six nodal points consist of the vertices of the triangle together the edge midpoints (filled black dots). The approximated sphere-like surface is shown by the hatched gray area. A radial line passing through the vertex $v_{i}$ is shown. The hollow circle indicates the intersection with $\mathcal{S}^{2}$ at which $\bar{f}(v_{i})$ in (49) is taken. $\bar{f}$ itself, as described in Sec. 5.2, is assigned by taking the maximum image intensity along the drawn radial line between the two cross marks.

    Figure 7.  Frames no. 140 (left) and 141 (right) of the processed image sequence in a top view. The embryo's body axis is oriented from bottom left to top right.

    Figure 9.  Tangent vector field minimising $\mathcal{E}_{\alpha}$. Depicted is the colour-coded optical flow field computed between frames 140 and 141 for different values of $\alpha$. The bottom row differs from the top view by a rotation of 180 degrees around the $x^{3}$-axis. From left to right: a) $\alpha = 10^{-2}$, b) $\alpha = 10^{-1}$, c) $\alpha = 1$, and d) $\alpha = 10$.

    Figure 8.  Function $\tilde{\rho}_{h}$ obtained by minimising $\mathcal{F}_{\beta}$ for frames 140 (left column) and 141 (right column). Colour corresponds to the radius ($\mu$m) of the fitted surface. The top row depicts $\mathcal{S}_{h}^{2}$ in a top view.

    Figure 10.  Top view of the optical flow field computed for different values of $\alpha$. From left to right, top to bottom: a) $\alpha = 10^{-2}$, b) $\alpha = 10^{-1}$, c) $\alpha = 1$, and d) $\alpha = 10$.

    Table 1.  Summary of notation used throughout the paper.

    $\Omega$ coordinate domain
    $I$ time interval
    $\mathcal{S}^2$ 2-sphere
    $\mathcal{M}$ family of sphere-like surfaces $\mathcal{M}_{t}$
    $T_{y}\mathcal{M}_{t}$ tangent plane at $y \in \mathcal{M}_{t}$
    $\mathbf{\tilde{N}}, \mathbf{\hat{N}}$ outward unit normals to $\mathcal{S}^2$ and $\mathcal{M}$
    $\bf{x}$ , $\bf{y}$ parametrisations of $\mathcal{S}^2$ and $\mathcal{M}$
    $D\bf{x}$ , $D\bf{y}$ gradient matrix of $\bf{x}$ and $\bf{y}$
    $\{ \partial_{1} \bf{x}, \partial_{2} \bf{x} \}$ basis for $T\mathcal{S}^2$
    $\{ \partial_{1} \bf{y}, \partial_{2} \bf{y} \}$ basis for $T\mathcal{M}$
    $\{ \mathbf{\hat{e}}_{1}, \mathbf{\hat{e}}_{2} \}$ orthonormal basis for $T\mathcal{M}_{t}$
    $\mathbf{\hat{V}}$ surface velocity of $\mathcal{M}$
    $\tilde{\phi}, D\tilde{\phi}$ smooth map from $\mathcal{S}^{2}$ to $\mathcal{M}$ and its differential
    $\tilde{f}$ , $\hat{f}$ , $f$ scalar function on $\mathcal{S}^2$ , $\mathcal{M}$ , and their coordinate version
    $\nabla_{\mathcal{S}^2} \tilde{f}$ , $\nabla_{\mathcal{M}} \hat{f}$ surface gradient on $\mathcal{S}^2$ and $\mathcal{M}_{t}$
    $\mathbf{\tilde{v}}$ , $\mathbf{\hat{v}}$ , $\mathbf{v}$ tangent vector fields on $\mathcal{S}^2$ , $\mathcal{M}$ , and their coordinate version
    $\nabla_{\mathbf{\hat{u}}} \mathbf{\hat{v}}$ covariant derivative of $\mathbf{\hat{v}}$ along direction $\mathbf{\hat{u}}$ on $\mathcal{M}_{t}$
    $\bar{f}$ , $\mathbf{\bar{v}}$ radially constant extensions of $\hat{f}$ and $\mathbf{\hat{v}}$ to $\mathbb{R}^{3} \setminus \{ 0 \}$
    $\tilde{Y}_{n,j}$ scalar spherical harmonic of degree $n$ and order $j$
    $\mathbf{\tilde{y}}_{n,j}^{(i)}$ vector spherical harmonic of degree $n$ , order $j$ , and type $i$
    $\mathbf{\hat{y}}_{n,j}^{(i)}$ pushforward of $\mathbf{\tilde{y}}_{n,j}^{(i)}$ via the differential $D\tilde{\phi}$
     | Show Table
    DownLoad: CSV

    Table 2.  Radii $R$ of the colour disks used for colour-coded visualisation of tangent vector fields.

    $R$ $8.92$ $4.62$ $2.90$ $2.19$ $12.07$ $2.90$ $12.02$ $9.23$ $4.87$ $2.92$ $2.10$
     | Show Table
    DownLoad: CSV
  • [1] F. AmatW. LemonD. P. MossingK. McDoleY. WanK. BransonE. W. Myers and P. J. Keller, Fast, accurate reconstruction of cell lineages from large-scale fluorescence microscopy data, Nat. Meth., 11 (2014), 951-958.  doi: 10.1038/nmeth.3036.
    [2] F. AmatE. W. Myers and P. J. Keller, Fast and robust optical flow for time-lapse microscopy using super-voxels, Bioinformatics, 29 (2013), 373-380.  doi: 10.1093/bioinformatics/bts706.
    [3] K. Atkinson and W. Han, Spherical Harmonics and Approximations on the Unit Sphere: An Introduction volume 2044 of Lecture Notes in Mathematics, Springer, Heidelberg, 2012. doi: 10.1007/978-3-642-25983-8.
    [4] G. AubertR. Deriche and P. Kornprobst, Computing optical flow via variational techniques, SIAM J. Appl. Math., 60 (2000), 156-182.  doi: 10.1137/S0036139998340170.
    [5] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing, volume 147 of Applied Mathematical Sciences, Springer, New York, 2 edition, 2006.
    [6] S. BakerD. ScharsteinJ. P. LewisS. RothM. J. Black and R. Szeliski, A database and evaluation methodology for optical flow, Int. J. Comput. Vision, 92 (2011), 1-31.  doi: 10.1109/ICCV.2007.4408903.
    [7] M. BauerM. Grasmair and C. Kirisits, Optical flow on moving manifolds, SIAM J. Imaging Sciences, 8 (2015), 484-512.  doi: 10.1137/140965235.
    [8] M. Botsch, L. Kobbelt, M. Pauly, P. Alliez and B. Lévy, Polygon Mesh Processing, A K Peters, 2010. doi: 10.1201/b10688.
    [9] M. P. do Carmo, Differential Geometry of Curves and Surfaces, Prentice-Hall, 1976.
    [10] M. P. do Carmo, Riemannian Geometry, Birkhäuser, 1992.
    [11] L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions, Studies in Advanced Mathematics. CRC Press, Boca Raton, FL, 1992.
    [12] W. Freeden and M. Schreiner, Spherical functions of mathematical geosciences. A scalar, vectorial, and tensorial setup, Berlin: Springer, 2009.
    [13] D. Gilbarg and N. Trudinger, Elliptic Partial Differential Equations of Second Order, Classics in Mathematics. Springer Verlag, Berlin, 2001.
    [14] E. Hebey, Sobolev Spaces on Riemannian Manifolds, volume 1635 of Lecture Notes in Mathematics, SV, Berlin, 1996. doi: 10.1007/BFb0092907.
    [15] E. Hebey, Nonlinear Analysis on Manifolds: Sobolev Spaces and Inequalities, Courant Lecture Notes in Mathematics. New York University, Courant Institute of Mathematical Sciences, New York; American Mathematical Society, Providence, RI, 1999.
    [16] K. Hesse, I. H. Sloan and R. S. Womersley, Numerical integration on the sphere, In W. Freeden, M. Z. Nashed, and T. Sonar, editors, Handbook of Geomathematics, pages 1187-1219. Springer, 2010.
    [17] M. W. Hirsch, Differential Topology, volume 33 of Graduate Texts in Mathematics, Springer-Verlag, New York, 1994.
    [18] B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203.  doi: 10.1016/0004-3702(81)90024-2.
    [19] A. Imiya, H. Sugaya, A. Torii and Y. Mochizuki, Variational analysis of spherical images, In A. Gagalowicz and W. Philips, editors, Computer Analysis of Images and Patterns, volume 3691 of Lecture Notes in Computer Science, pages 104-111. Springer Berlin, Heidelberg, 2005. doi: 10.1007/11556121_14.
    [20] P. J. Keller, Imaging morphogenesis: Technological advances and biological insights Science, 340 (2013), 1234168. doi: 10.1126/science.1234168.
    [21] C. B. KimmelW. W. BallardS. R. KimmelB. Ullmann and T. F. Schilling, Stages of embryonic development of the zebrafish, Devel. Dyn., 203 (1995), 253-310.  doi: 10.1002/aja.1002030302.
    [22] C. Kirisits, L. F. Lang and O. Scherzer, Optical flow on evolving surfaces with an application to the analysis of 4D microscopy data, In A. Kuijper, K. Bredies, T. Pock, and H. Bischof, editors, SSVM'13: Proceedings of the fourth International Conference on Scale Space and Variational Methods in Computer Vision, volume 7893 of Lecture Notes in Computer Science, pages 246-257, Berlin, Heidelberg, 2013. Springer-Verlag. doi: 10.1007/978-3-642-38267-3_21.
    [23] C. KirisitsL. F. Lang and O. Scherzer, Decomposition of optical flow on the sphere, GEM. Int. J. Geomath., 5 (2014), 117-141.  doi: 10.1007/s13137-013-0055-8.
    [24] C. KirisitsL. F. Lang and O. Scherzer, Optical flow on evolving surfaces with space and time regularisation, J. Math. Imaging Vision, 52 (2015), 55-70.  doi: 10.1007/s10851-014-0513-4.
    [25] J. M. Lee, Riemannian Manifolds volume 176 of Graduate Texts in Mathematics, Springer-Verlag, New York, 1997. doi: 10.1007/b98852.
    [26] J. M. Lee, Introduction to Smooth Manifolds, volume 218 of Graduate Texts in Mathematics, Springer, New York, 2 edition, 2013.
    [27] J. Lefévre and S. Baillet, Optical flow and advection on 2-Riemannian manifolds: A common framework, IEEE Trans. Pattern Anal. Mach. Intell., 30 (2008), 1081-1092. 
    [28] S. G. Megason and S. E. Fraser, Digitizing life at the level of the cell: High-performance laser-scanning microscopy and image analysis for in toto imaging of development, Mech. Dev., 120 (2003), 1407-1420.  doi: 10.1016/j.mod.2003.07.005.
    [29] C. MelaniM. CampanaB. LombardotB. RizziF. VeronesiC. ZanellaP. BourgineK. MikulaN. Peyriéras and A. Sarti, Cells tracking in a live zebrafish embryo, In Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS 2007), (2007), 1631-1634.  doi: 10.1109/IEMBS.2007.4352619.
    [30] V. Michel, Lectures on Constructive Approximation. Fourier, Spline, and Wavelet Methods on the Real Line, the Sphere, and The Ball, New York, NY: Birkhäuser, 2013. doi: 10.1007/978-0-8176-8403-7.
    [31] T. MizoguchiH. VerkadeJ. K. HeathA. Kuroiwa and Y. Kikuchi, Sdf1/Cxcr4 signaling controls the dorsal migration of endodermal cells during zebrafish gastrulation, Development, 135 (2008), 2521-2529.  doi: 10.1242/dev.020107.
    [32] M. A. Penna and K. A. Dines, A simple method for fitting sphere-like surfaces, IEEE Trans. Pattern Anal. Mach. Intell., 29 (2007), 1673-1678.  doi: 10.1109/TPAMI.2007.1114.
    [33] P. Quelhas, A. M. Mendonça and A. Campilho, Optical flow based arabidopsis thaliana root meristem cell division detection, In A. Campilho and M. Kamel, editors, Image Analysis and Recognition, volume 6112 of Lecture Notes in Computer Science, pages 217-226. Springer Berlin Heidelberg, 2010. doi: 10.1007/978-3-642-13775-4_22.
    [34] B. Schmid, G. Shah, N. Scherf, M. Weber, K. Thierbach, C. Campos Pérez, I. Roeder, P. Aanstad and J. Huisken, High-speed panoramic light-sheet microscopy reveals global endodermal cell dynamics Nat. Commun. , 4 (2013), p2207. doi: 10.1038/ncomms3207.
    [35] Ch. Schnörr, Determining optical flow for irregular domains by minimizing quadratic functionals of a certain class, Int. J. Comput. Vision, 6 (1991), 25-38. 
    [36] T. Schuster and J. Weickert, On the application of projection methods for computing optical flow fields, Inverse Probl. Imaging, 1 (2007), 673-690.  doi: 10.3934/ipi.2007.1.673.
    [37] A. Torii, A. Imiya, H. Sugaya and Y. Mochizuki, Optical Flow Computation for Compound Eyes: Variational Analysis of Omni-Directional Views, In M. De Gregorio, V. Di Maio, M. Frucci, and C. Musio, editors, Brain, Vision, and Artificial Intelligence, volume 3704 of Lecture Notes in Computer Science, pages 527-536. Springer Berlin, Heidelberg, 2005. doi: 10.1007/11565123_51.
    [38] H. Triebel, Theory of Function Spaces. II, volume 84 of Monographs in Mathematics, Birkhäuser Verlag, Basel, 1992. doi: 10.1007/978-3-0346-0419-2.
    [39] R. M. Warga and C. Nüsslein-Volhard, Origin and development of the zebrafish endoderm, Development, 126 (1999), 827-838. 
    [40] J. Weickert, A. Bruhn, T. Brox and N. Papenberg, A survey on variational optic flow methods for small displacements, In O. Scherzer, editor, Mathematical Models for Registration and Applications to Medical Imaging, volume 10 of Mathematics in Industry, pages 103-136. Springer, Berlin Heidelberg, 2006. doi: 10.1007/978-3-540-34767-5_5.
    [41] J. Weickert and Ch. Schnörr, A theoretical framework for convex regularizers in PDE-based computation of image motion, Int. J. Comput. Vision, 45 (2001), 245-264. 
    [42] J. Weickert and Ch. Schnörr, Variational optic flow computation with a spatio-temporal smoothness constraint, J. Math. Imaging Vision, 14 (2001), 245-255. 
  • 加载中




Article Metrics

HTML views(1547) PDF downloads(53) Cited by(0)

Access History

Other Articles By Authors



    DownLoad:  Full-Size Img  PowerPoint