Deep learning-based multiple moving vehicle detection and tracking using a nonlinear fourth-order reaction-diffusion based multi-scale video object analysis

Tudor Barbu

doi:10.3934/dcdss.2022083

Article Contents

2023, Volume 16, Issue 1: 16-32. Doi: 10.3934/dcdss.2022083

This issue Previous Article Qualitative and quantitative analysis for a nonlocal and nonlinear reaction-diffusion problem with in-homogeneous Neumann boundary conditions Next Article Qualitative and quantitative analysis of a nonlinear second-order anisotropic reaction-diffusion model of an epidemic infection spread

Deep learning-based multiple moving vehicle detection and tracking using a nonlinear fourth-order reaction-diffusion based multi-scale video object analysis

Tudor Barbu

Institute of Computer Science of the Romanian Academy -– Iasi Branch, Bld. Carol I, no. 8, Iaşi, Romania

Received: February 2022

Revised: March 2022

Early access: March 2022

Published: January 2023

Abstract / Introduction Full Text(HTML) Figure(3) / Table(2) Related Papers Cited by

Abstract

A novel automatic vehicle detection and tracking framework is proposed in this research work. First, the moving vehicles are detected in the frames of the video sequence by combining some deep learning and Gaussian Mixture Model-based object detection techniques. Then, the correspondence between the video objects detected in successive frames is determined using a multi-scale analysis of those objects. A scale-space representation is created by applying the numerical approximation algorithm that solves a nonlinear fourth-order reaction-diffusion based model whose mathematical validity is rigorously investigated here. A color image feature extraction is then performed at each scale using SURF and HOG-based features and the feature vectors determined at multiple scales are next concatenated into a final descriptor. A novel instance matching-based vehicle tracking technique using the distances between these feature vectors is then proposed. The results of the performed detection and tracking simulations are finally discussed.

Keywords:

Multiple vehicle detection and tracking,
deep learning,
Gaussian Mixture Models,
convolutional neural networks,
nonlinear fourth-order reaction-diffusion,
well-posed PDE model,
multi-scale analysis,
SURF descriptor,
HOG features,
video instance matching.

Mathematics Subject Classification: Primary: 35Axx, 68Txx; Secondary: 35A01, 35A02, 49K40, 62H30, 68T07, 68U10, 94A08, 62M40, 65D18.

Citation:

Full Text(HTML)

Figure 1. Example of moving vehicle detection process

Download: Full-size image PowerPoint slide

Figure 2. Architecture of the vehicle detection and tracking framework

Download: Full-size image PowerPoint slide

Figure 3. Moving vehicle tracking example

Download: Full-size image PowerPoint slide

Table 1. Moving vehicle detection method comparison

Detection technique	Precision	Recall
The proposed approach	0.8423	0.8257
Aggregated Channels Network (ACF)	0.6741	0.6824
Gaussian Mixture Models (GMM)	0.6843	0.6722
Faster R-CNN	0.8192	0.8192
R-CNN	0.6581	0.6625
YOLO-v2	0.7632	0.7811
Frame differencing (FD)	0.5826	0.5972
HOG + SVM	0.6149	0.6349

| Show Table

DownLoad: CSV

Table 2. Multiple vehicle tracking method comparison

Tracking approach	Precision	Recall
The proposed technique	0.8401	0.8216
GMM + Mean-shift tracking	0.7148	0.7332
GMM + Kalman filter	0.7643	0.7527
EB + IoU tracker	0.8576	0.8615
SIFT tracking	0.6233	0.6181
SIFT + Kalman filtering	0.7486	0.7391
Frame differencing (FD)+object matching	0.5801	0.5879
HOG + SVM + feature matching	0.6102	0.6213

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	M. Atefian and H. Mahdavi-Nasab, A robust mean-shift tracking using GMM background subtraction, Journal of Basic and Applied Scientific Research, 3 (2013), 596-607.
[2]	A. Banharnsakun and S. Tanathong, Object detection based on template matching through use of best-so-far ABC, Computational Intelligence and Neuroscience, 2014 (2014), 919406. doi: 10.1155/2014/919406.
[3]	T. Barbu, Comparing various voice recognition techniques, 2009 Proceedings of the 5th Conference on Speech Technology and Human-Computer Dialogue, (2009), 1-6. doi: 10.1109/SPED.2009.5156172.
[4]	T. Barbu, An automatic unsupervised pattern recognition approach, Proceedings of the Romanian Academy, Series A: Mathematics, Physics, Technical Sciences, Information Science, 7 (2006), 73–78.
[5]	T. Barbu, Multiple object detection and tracking in sonar movies using an improved temporal differencing approach and texture analysis, Politehn. Univ. Bucharest Sci. Bull. Ser. A Appl. Math. Phys., 74 (2012), 27-40.
[6]	T. Barbu, An automatic face detection system for RGB images, International Journal of Computers, Communications & Control, 6 (2011), 21-32. doi: 10.15837/ijccc.2011.1.2197.
[7]	T. Barbu, Pedestrian detection and tracking using temporal differencing and HOG features, Computers & Electrical Engineering, 40 (2014), 1072-1079. doi: 10.1016/j.compeleceng.2013.12.004.
[8]	T. Barbu, Novel approach for moving human detection and tracking in static camera video sequences, Proc. Rom. Acad. Ser. A Math. Phys. Tech. Sci. Inf. Sci., 13 (2012), 269-277.
[9]	T. Barbu, Novel Diffusion-Based Models for Image Restoration and Interpolation, Book Series: Signals and Communication Technology, Springer International Publishing, 2019. doi: 10.1007/978-3-319-93006-0.
[10]	T. Barbu, Robust contour tracking model using a variational level-set algorithm, Numer. Funct. Anal. Optim., 35 (2014), 263-274. doi: 10.1080/01630563.2013.850436.
[11]	T. Barbu, SVM-based human cell detection technique using histograms of oriented gradients, Mathematical Methods for Information Science & Economics: Proceedings of the 3rd International Conference for the Applied Mathematics and Informatics, AMATHI '12, Montreux, Switzerland, Dec. 29–31, (2012), 156–160.
[12]	T. Barbu, A. Miranville and C. Moroşanu, A qualitative analysis and numerical simulations of a nonlinear second-order anisotropic diffusion problem with Non-homogeneous Cauchy-Neumann boundary conditions, Appl. Math. Comput., 350 (2019), 170-180. doi: 10.1016/j.amc.2019.01.004.
[13]	T. Barbu and C. Moroşanu, Image restoration using a nonlinear second-order parabolic PDE-based scheme, Analele Stiintifice ale Universitatii Ovidius Constanta, Seria Matematica, 25 (2017), 33–48. doi: 10.1515/auom-2017-0003.
[14]	T. Barbu and C. Moroşanu, Compound PDE-based additive denoising solution combining an improved anisotropic diffusion model to a 2D Gaussian filter kernel, East Asian J. Appl. Math., 9 (2019), 1-12. doi: 10.4208/eajam.270318.260518.
[15]	H. Bay, T. Tuytelaars and L. V. Gool, Surf: Speeded up robust features, In European Conference on Computer Vision, (2006), Springer, Berlin, Heidelberg, 404–417. doi: 10.1007/11744023_32.
[16]	M. Betke, E. Haritaoglu and L. S. Davis, Real-time multiple vehicle detection and tracking from a moving vehicle, Machine Vision and Applications, 12 (2000), 69-83. doi: 10.1007/s001380050126.
[17]	E. Bochinski, V. Eiselein and T. Sikora, High-speed tracking-by-detection without using image information, In 2017 14th IEEE International conference on advanced video and signal based surveillance (AVSS), (2017), 1–6. doi: 10.1109/AVSS.2017.8078516.
[18]	T. Burghardt and J. Calic, Real-time face detection and tracking of animals, In 2006 8th Seminar on Neural Network Applications in Electrical Engineering, (2006), 27–32. doi: 10.1109/NEUREL.2006.341167.
[19]	T. F. Chan and L. A. Vese, Active contours without edges, IEEE Transactions on Image Processing, 10 (2001), 266-277. doi: 10.1109/83.902291.
[20]	W. Chantara, J.-H. Mun, D.-W. Shin and Y.-S. Ho, Object tracking using adaptive template matching, IEIE Transactions on Smart Processing and Computing, 4 (2015), 1-9. doi: 10.5573/IEIESPC.2015.4.1.001.
[21]	R. Chavan and S. R. Gengaje, Multiple object detection using GMM technique and tracking using Kalman filter, Int. J. Comput. Appl., 172 (2017), 20-25. doi: 10.5120/ijca2017915102.
[22]	X. Chen, C. Xi and J. Cao, Research on moving object detection based on improved mixture Gaussian model, Optik, 126 (2015), 2256-2259. doi: 10.1016/j.ijleo.2015.05.122.
[23]	G. Ciaparrone, F. L. Sánchez, S. Tabik, L. Troiano, R. Tagliaferri and F. Herrera, Deep learning in video multi-object tracking: A survey, Neurocomputing, 381 (2020), 61-88. doi: 10.1016/j.neucom.2019.11.023.
[24]	N. Dalal and B. Triggs, Histograms of oriented gradients for human detection, 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), 1 (2005), 886-893. doi: 10.1109/CVPR.2005.177.
[25]	J. Du, Understanding of object detection based on CNN family and YOLO, Journal of Physics: Conference Series, 1004 (2018), 012029. doi: 10.1088/1742-6596/1004/1/012029.
[26]	J. Farooq, Object detection and identification using SURF and BoW model, 2016 International Conference on Computing, Electronic and Electrical Engineering (ICE Cube), (2016), 318-323. doi: 10.1109/ICECUBE.2016.7495245.
[27]	P. F. Felzenszwalb, R. B. Girshick, D. McAllester and D. Ramanan, Object detection with discriminatively trained part-based models, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (2010), 1627-1645. doi: 10.1109/TPAMI.2009.167.
[28]	F. Ghorban, J. Marín, Y. Su, A. Colombo and A. Kummert, Aggregated channels network for real-time pedestrian detection, Tenth International Conference on Machine Vision (ICMV 2017), 10696 (2018), 106960I. doi: 10.1117/12.2309864.
[29]	M. Han, A. Sethi, W. Hua and Y. Gong, A detection-based multiple object tracking method, 2004 International Conference on Image Processing, ICIP'04, 5 (2004), 3065-3068.
[30]	C. C. Hsu and G. T. Dai, Multiple object tracking using particle swarm optimization, International Journal of Electronics and Communication Engineering, 6 (2012), 744-747.
[31]	P. Johnson, Finite Difference for PDEs, School of Mathematics, University of Manchester, Semester I, 2008.
[32]	K. Kale, S. Pawar and P. Dhulekar, Moving object tracking using optical flow and motion vector estimation, In 2015 4th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO) (trends and future directions), (2015), 1–6. doi: 10.1109/ICRITO.2015.7359323.
[33]	C. Küblbeck and A. Ernst, Face detection and tracking in video sequences using the modifiedcensus transformation, Image and Vision Computing, 24 (2006), 564-572.
[34]	D.-H. Lee, CNN-based single object detection and tracking in videos and its application to drone detection, Multimedia Tools and Applications, 80 (2021), 34237-34248. doi: 10.1007/s11042-020-09924-0.
[35]	X. Li and X. Guo, A HOG feature and SVM based method for forward vehicle detection with single camera, 2013 5th International Conference on Intelligent Human-Machine Systems and Cybernetics, 1 (2013), 263-266. doi: 10.1109/IHMSC.2013.69.
[36]	J. L. Lions, Quelques Methods de Resolution des Problémes aux Limites Nonlinéares, Dunod. Gauthier-Villars, Paris, 1969.
[37]	L. Liu, W. Ouyang, X. Wang, P. Fieguth, J. Chen, X. Liu and M. Pietikäinen, Deep learning for generic object detection: A survey, International Journal of Computer Vision, 128 (2020), 261-318. doi: 10.1007/s11263-019-01247-4.
[38]	K. Mantripragada, F. C. Trigo, F. P. Martins and A. de Toledo Fleury, Vehicle tracking using feature matching and Kalman filtering, In Proc. International Congress of Mechanical Engineering, (2013), 361–370.
[39]	A. Nadeem, A. Jalal and K. Kim, Human actions tracking and recognition based on body parts detection via Artificial neural network, In 2020 3rd International conference on advancements in computational sciences (ICACS), (2020), 1–6. doi: 10.1109/ICACS47775.2020.9055951.
[40]	T. Nguyen, E.-A. Park, J. Han, D.-C. Park and S.-Y. Min, Object detection using scale invariant feature transform, In Genetic and Evolutionary Computing, Springer, Cham, 238 (2014), 65–72. doi: 10.1007/978-3-319-01796-9_7.
[41]	H. A. Patel and D. G. Thakore, Moving object tracking using Kalman filter, International Journal of Computer Science and Mobile Computing, 2 (2013), 326-332.
[42]	F. Porikli and A. Yilmaz, Object detection and tracking, Video Analytics for Business Intelligence, Springer, (2012), 3–41. doi: 10.1007/978-3-642-28598-1_1.
[43]	S. Ren, K. He, R. Girshick and J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks, EEE Transactions on Pattern Analysis and Machine Intelligence, 39 (2017), 1137-1149. doi: 10.1109/TPAMI.2016.2577031.
[44]	K. B. Saran and G. Sreelekha, Traffic video surveillance: Vehicle detection and classification, 2015 International Conference on Control Communication & Computing India (ICCC), (2015), 516-521. doi: 10.1109/ICCC.2015.7432948.
[45]	S. S. Sengar and S. Mukhopadhyay, A novel method for moving object detection based on block based frame differencing,, 2016 3rd International Conference on Recent Advances in Information Technology (RAIT), IEEE, (2016), 467-472. doi: 10.1109/RAIT.2016.7507946.
[46]	V. Srikrishnan, T. Nagaraj and S. Chaudhuri, Fragment based tracking for scale and orientation adaptation, In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, (2008), 328–335. IEEE. doi: 10.1109/ICVGIP.2008.19.
[47]	P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 2001 (2001), I-I. doi: 10.1109/CVPR.2001.990517.
[48]	L. Wang, Y. Lu, H. Wang, Y. Zheng, H. Ye and X. Xue, Evolving boxes for fast vehicle detection, 2017 IEEE International Conference on Multimedia and Expo (ICME), (2017), 1135-1140. doi: 10.1109/ICME.2017.8019461.
[49]	L. Wen, D. Du, Z. Cai, Z. Lei, M.-C. Chang, H. Qi, ... S. Lyu, UA-DETRAC: A new benchmark and protocol for multi-object detection and tracking, Computer Vision and Image Understanding, 193 (2020), 102907. doi: 10.1016/j.cviu.2020.102907.
[50]	A. Wu, S. Zhao, C. Deng and W. Liu, Generalized and discriminative few-shot object detection via SVD-dictionary enhancement, Advances in Neural Information Processing Systems, 34 (2021).
[51]	X. Wu, D. Sahoo and S. C. Hoi, Recent advances in deep learning for object detection, Neurocomputing, 396 (2020), 39-64.
[52]	R. Xiao, L. Zhu and H. J. Zhang, Boosting chain learning for object detection, Proceedings of Ninth IEEE International Conference on Computer Vision, (2003), 709-715.
[53]	C. Yang, R. Duraiswami and L. Davis, Efficient mean-shift tracking via a new similarity measure, In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 1 (2005), 176–183.
[54]	Y. Yuan, H. Yang, Y. Fang and W. Lin, Visual object tracking by structure complexity coefficients, IEEE Transactions on Multimedia, 17 (2015), 1125-1136. doi: 10.1109/TMM.2015.2440996.
[55]	M. Zhang and V. Ciesielski, Using back propagation algorithm and genetic algorithm to train and refine neural networks for object detection, International Conference on Database and Expert Systems Applications, Springer, Berlin, Heidelberg, 1677 (1999), 626–635. doi: 10.1007/3-540-48309-8_58.
[56]	J. Zhou and J. Hoang, Real time robust human detection and tracking system., In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)-Workshops, (2005), 149–149.