2016, 6(3): 263-296. doi: 10.3934/naco.2016011

Robust and flexible landmarks detection for uncontrolled frontal faces in the wild

1. 

Department of Computing, Curtin University, Perth, Western Australia

Received  June 2015 Revised  August 2016 Published  September 2016

In this paper, we propose a robust facial landmarking scheme for frontal faces which can be applied on both controlled and uncontrolled environment. This scheme is based on improvement/extension of the tree-structured facial landmarking scheme proposed by Zhu and Ramanan. The whole system is divided into two main parts: face detection and face landmarking. In the face detection part, we proposed a Tree-structured Filter Model (TFM) combined with Viola and Jones face detector to significantly reduce the false positives while maintaining high accuracy. For the facial landmarking step, we improve the accuracy and the amount of the facial landmarks by readjusting the face structure to provide better geometrical information. Furthermore, we expand the face models into Multi-Resolution (MR) models with the adaptive landmark approach via landmark reduction to train the face models to be able to detect facial landmarks on face images with resolutions as low as 30x30 pixels. Our experiments show that our proposed approaches can improve the accuracy of facial landmark detection on both controlled and uncontrolled environment. Furthermore, they also show that our MR models are more robust on detecting facial components (eyebrows, eyes, nose, and mouth) on very small faces.
Citation: A. Liang, C. Wang, W. Liu, L. Li. Robust and flexible landmarks detection for uncontrolled frontal faces in the wild. Numerical Algebra, Control & Optimization, 2016, 6 (3) : 263-296. doi: 10.3934/naco.2016011
References:
[1]

T. L. Berg, A. C. Berg, J. Edwards and D. A. Forsyth, Who's in the picture?, in NIPS, (2004).   Google Scholar

[2]

O. Çeliktutan, S. Ulukaya and B. Sankur, A comparative study of face landmarking techniques,, EURASIP Journal on Image and Video Processing, 2013 (2013).   Google Scholar

[3]

P. Conilione and D. Wang, Fuzzy approach for semantic face image retrieval,, The Computer Journal, 55 (2012), 1130.   Google Scholar

[4]

T. F. Cootes, C. J. Taylor, D. H. Cooper and J. Graham, Active shape models-their training and application,, Computer vision and image understanding, 61 (1995), 38.   Google Scholar

[5]

N. Dalal and B. Triggs, Histograms of oriented gradients for human detection,, in Computer Vision and Pattern Recognition, (2005), 886.   Google Scholar

[6]

L. Ding and A. M. Martinez, Features versus context: An approach for precise and detailed detection and delineation of faces and facial features,, IEEE Trans. Pattern Anal. Mach. Intell., 32 (2010), 2022.   Google Scholar

[7]

P. F. Felzenszwalb, R. B. Girshick, D. McAllester and D. Ramanan, Object detection with discriminatively trained part-based models,, Pattern Analysis and Machine Intelligence, 32 (2010), 1627.   Google Scholar

[8]

P. F. Felzenszwalb and D. P. Huttenlocher, Pictorial structures for object recognition,, International Journal of Computer Vision, 61 (2005), 55.   Google Scholar

[9]

G. D. Forney Jr, The viterbi algorithm,, Proceedings of the IEEE, 61 (1973), 268.   Google Scholar

[10]

Y. Freund and R. E. Schapire, A desicion-theoretic generalization of on-line learning and an application to boosting,, in Computational learning theory, (1995), 23.   Google Scholar

[11]

J. H. Friedman, Multivariate adaptive regression splines,, The annals of statistics, 19 (1991), 1.  doi: 10.1214/aos/1176347963.  Google Scholar

[12]

R. Gross, I. Matthews, J. Cohn, T. Kanade and S. Baker, Multi-pie,, Image Vision Computing, 28 (2010), 807.   Google Scholar

[13]

V. Jain and E. Learned-Miller, FDDB: A Benchmark for Face Detection in Unconstrained Settings,, Technical Report UM-CS-2010-009, (2010), 2010.   Google Scholar

[14]

A. Kasinski, A. Florek and A. Schmidt, The PUT face database,, Image Processing and Communications, 13 (2008), 59.   Google Scholar

[15]

J. C. Klontz and A. K. Jain, A case study of automated face recognition: The boston marathon bombings suspects,, IEEE Computer, 46 (2013), 91.   Google Scholar

[16]

M. Koestinger, P. Wohlhart, P. M. Roth and H. Bischof, Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization,, in First IEEE International Workshop on Benchmarking Facial Image Analysis Technologies, (2011).   Google Scholar

[17]

V. Le, J. Brandt, Z. Lin, L. Bourdev and T. S. Huang, Interactive facial feature localization,, in Proceedings of the 12th European conference on Computer Vision - Volume Part III, (2012), 679.   Google Scholar

[18]

A. Liang, W. Liu, L. Li, M. R. Farid and V. Le, Accurate facial landmarks detection for frontal faces with extended tree-structured models,, in Pattern Recognition (ICPR), (2014), 538.   Google Scholar

[19]

A. Liang, C. Wang, W. Liu and L. Li, A novel landmark detector system for multi resolution frontal faces,, in Digital lmage Computing: Techniques and Applications (DlCTA), (2014), 1.   Google Scholar

[20]

D. G. Lowe, Distinctive image features from scale-invariant keypoints,, International journal of computer vision, 60 (2004), 91.   Google Scholar

[21]

A. M. Martínez and R. Benavente, The AR Face Database,, Technical Report 24, (1998).   Google Scholar

[22]

S. Milborrow, Stasm User Manual,, , (2013).   Google Scholar

[23]

S. Milborrow and F. Nicolls, Active Shape Models with SIFT Descriptors and MARS,, VISAPP, ().   Google Scholar

[24]

S. Milborrow and F. Nicolls, Locating facial features with an extended active shape model,, in Computer Vision-ECCV 2008, (2008), 504.   Google Scholar

[25]

A. Torralba, K. P. Murphy and W. T. Freeman, Sharing visual features for multiclass and multiview object detection,, Pattern Analysis and Machine Intelligence, 29 (2007), 854.   Google Scholar

[26]

P. Viola and M. J. Jones, Robust real-time face detection,, International journal of computer vision, 57 (2004), 137.   Google Scholar

[27]

X. Xiong and F. De la Torre, Supervised descent method and its applications to face alignment,, in Computer Vision and Pattern Recognition (CVPR), (2013), 532.   Google Scholar

[28]

W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld, Face recognition: A literature survey,, ACM Computing Surveys (CSUR), 35 (2003), 399.   Google Scholar

[29]

X. Zhu and D. Ramanan, Face detection, pose estimation and landmark localization in the wild,, , ().   Google Scholar

[30]

X. Zhu and D. Ramanan, Face detection, pose estimation, and landmark localization in the wild,, in Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2012), 2879.   Google Scholar

show all references

References:
[1]

T. L. Berg, A. C. Berg, J. Edwards and D. A. Forsyth, Who's in the picture?, in NIPS, (2004).   Google Scholar

[2]

O. Çeliktutan, S. Ulukaya and B. Sankur, A comparative study of face landmarking techniques,, EURASIP Journal on Image and Video Processing, 2013 (2013).   Google Scholar

[3]

P. Conilione and D. Wang, Fuzzy approach for semantic face image retrieval,, The Computer Journal, 55 (2012), 1130.   Google Scholar

[4]

T. F. Cootes, C. J. Taylor, D. H. Cooper and J. Graham, Active shape models-their training and application,, Computer vision and image understanding, 61 (1995), 38.   Google Scholar

[5]

N. Dalal and B. Triggs, Histograms of oriented gradients for human detection,, in Computer Vision and Pattern Recognition, (2005), 886.   Google Scholar

[6]

L. Ding and A. M. Martinez, Features versus context: An approach for precise and detailed detection and delineation of faces and facial features,, IEEE Trans. Pattern Anal. Mach. Intell., 32 (2010), 2022.   Google Scholar

[7]

P. F. Felzenszwalb, R. B. Girshick, D. McAllester and D. Ramanan, Object detection with discriminatively trained part-based models,, Pattern Analysis and Machine Intelligence, 32 (2010), 1627.   Google Scholar

[8]

P. F. Felzenszwalb and D. P. Huttenlocher, Pictorial structures for object recognition,, International Journal of Computer Vision, 61 (2005), 55.   Google Scholar

[9]

G. D. Forney Jr, The viterbi algorithm,, Proceedings of the IEEE, 61 (1973), 268.   Google Scholar

[10]

Y. Freund and R. E. Schapire, A desicion-theoretic generalization of on-line learning and an application to boosting,, in Computational learning theory, (1995), 23.   Google Scholar

[11]

J. H. Friedman, Multivariate adaptive regression splines,, The annals of statistics, 19 (1991), 1.  doi: 10.1214/aos/1176347963.  Google Scholar

[12]

R. Gross, I. Matthews, J. Cohn, T. Kanade and S. Baker, Multi-pie,, Image Vision Computing, 28 (2010), 807.   Google Scholar

[13]

V. Jain and E. Learned-Miller, FDDB: A Benchmark for Face Detection in Unconstrained Settings,, Technical Report UM-CS-2010-009, (2010), 2010.   Google Scholar

[14]

A. Kasinski, A. Florek and A. Schmidt, The PUT face database,, Image Processing and Communications, 13 (2008), 59.   Google Scholar

[15]

J. C. Klontz and A. K. Jain, A case study of automated face recognition: The boston marathon bombings suspects,, IEEE Computer, 46 (2013), 91.   Google Scholar

[16]

M. Koestinger, P. Wohlhart, P. M. Roth and H. Bischof, Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization,, in First IEEE International Workshop on Benchmarking Facial Image Analysis Technologies, (2011).   Google Scholar

[17]

V. Le, J. Brandt, Z. Lin, L. Bourdev and T. S. Huang, Interactive facial feature localization,, in Proceedings of the 12th European conference on Computer Vision - Volume Part III, (2012), 679.   Google Scholar

[18]

A. Liang, W. Liu, L. Li, M. R. Farid and V. Le, Accurate facial landmarks detection for frontal faces with extended tree-structured models,, in Pattern Recognition (ICPR), (2014), 538.   Google Scholar

[19]

A. Liang, C. Wang, W. Liu and L. Li, A novel landmark detector system for multi resolution frontal faces,, in Digital lmage Computing: Techniques and Applications (DlCTA), (2014), 1.   Google Scholar

[20]

D. G. Lowe, Distinctive image features from scale-invariant keypoints,, International journal of computer vision, 60 (2004), 91.   Google Scholar

[21]

A. M. Martínez and R. Benavente, The AR Face Database,, Technical Report 24, (1998).   Google Scholar

[22]

S. Milborrow, Stasm User Manual,, , (2013).   Google Scholar

[23]

S. Milborrow and F. Nicolls, Active Shape Models with SIFT Descriptors and MARS,, VISAPP, ().   Google Scholar

[24]

S. Milborrow and F. Nicolls, Locating facial features with an extended active shape model,, in Computer Vision-ECCV 2008, (2008), 504.   Google Scholar

[25]

A. Torralba, K. P. Murphy and W. T. Freeman, Sharing visual features for multiclass and multiview object detection,, Pattern Analysis and Machine Intelligence, 29 (2007), 854.   Google Scholar

[26]

P. Viola and M. J. Jones, Robust real-time face detection,, International journal of computer vision, 57 (2004), 137.   Google Scholar

[27]

X. Xiong and F. De la Torre, Supervised descent method and its applications to face alignment,, in Computer Vision and Pattern Recognition (CVPR), (2013), 532.   Google Scholar

[28]

W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld, Face recognition: A literature survey,, ACM Computing Surveys (CSUR), 35 (2003), 399.   Google Scholar

[29]

X. Zhu and D. Ramanan, Face detection, pose estimation and landmark localization in the wild,, , ().   Google Scholar

[30]

X. Zhu and D. Ramanan, Face detection, pose estimation, and landmark localization in the wild,, in Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2012), 2879.   Google Scholar

[1]

Xiyou Cheng, Zhitao Zhang. Structure of positive solutions to a class of Schrödinger systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020461

[2]

Reza Lotfi, Zahra Yadegari, Seyed Hossein Hosseini, Amir Hossein Khameneh, Erfan Babaee Tirkolaee, Gerhard-Wilhelm Weber. A robust time-cost-quality-energy-environment trade-off with resource-constrained in project management: A case study for a bridge construction project. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020158

[3]

Xin-Guang Yang, Lu Li, Xingjie Yan, Ling Ding. The structure and stability of pullback attractors for 3D Brinkman-Forchheimer equation with delay. Electronic Research Archive, 2020, 28 (4) : 1395-1418. doi: 10.3934/era.2020074

 Impact Factor: 

Metrics

  • PDF downloads (182)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]