A Novel Approach for Real Time Multi-Scene Violent Activities Recognition with Modified ResNet50 and LSTM

A Novel Approach for Real Time Multi-Scene Violent Activities Recognition with Modified ResNet50 and LSTM
  IJETT-book-cover           
  
© 2022 by IJETT Journal
Volume-70 Issue-8
Year of Publication : 2022
Authors : Devang Jani, Anand Mankodia
DOI : 10.14445/22315381/IJETT-V70I8P231

How to Cite?

Devang Jani, Anand Mankodia, "A Novel Approach for Real Time Multi-Scene Violent Activities Recognition with Modified ResNet50 and LSTM," International Journal of Engineering Trends and Technology, vol. 70, no. 8, pp. 292-309, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I8P231

Abstract
Day by day, the demand for autonomous video surveillance systems has been escalating due to inefficient manual inspection power of identifying anomalies in recorded videos by human beings. Currently, most video surveillance systems use manual video inspection after detecting suspicious activities or trying to use manual inspection of videos once the complaint regarding anomalous, violent or suspicious activities is filed at a particular area or location. Implementation of the real-time scanning of the video stream from the multiple cameras at the central level and a single camera at the edge level is a very big challenge due to the requirement of GPU, computational hardware as well a large amount of computation power with different provocations for the mutual type of human abnormal activities behaviors. The proposed methods represented in this paper provide a novel idea about real-time recognition of nine different mutual violent actions and Normal nonviolent actions using modified deep learning models, namely ResNet50 in association with LSTM. The proposed method provides Nine different diversified violent activities, specifically Attacking, Fighting, hitting with an object, Kicking, Punching, Pushing, Shooting with a Gun, Slapping, stabbing with a knife and one Nonviolent activity that is Normal class. A total of ten violent & nonviolent classes with an accuracy of 87.60% were developed and tested using TensorFlow, Keras and Supercomputing facilities. The Proposed Method is unaccustomed to multi-class violent activity recognition in a real-time environment. Real-time violent activities recognition is a summons as the violent recognition algorithms available to date can provide the decision regarding the events that occurs in the video, whether violent or nonviolent. It can work in short-length recorded videos in a non-real-time environment, a post-effect type of processing that cannot prevent future violent activities as the proposed methods can. The design of the Multi-class Violent recognition model is an arduous and much more time-consuming task due to the performance of each class affecting the overall accuracy and efficiency of the model. It is also annoying and tiresome because it requires continuous time of some days without interruption for the model's training

Keywords
Resnet50, LSTM, Violent and Nonviolent Classes, Surveillance System, Real-Time.

Reference
[1] D. Jani and A. Mankodia, "A Review on State of the Art Abnormal Activity Recognition Approaches," International Journal of Emerging Trends in Engineering Research, vol. 9, no. 3, pp. 182–188, 2021. Doi: 10.30534/ijeter/2021/05932021.
[2] D. Jani and A. Mankodia, "ITEE Journal Deep Learning Based Multi Class Human Violent Action Recognition," 2021. [Online]. Available: http://www.iteejournal.org/v10no5oct21_pdf2.pdf
[3] D. Jani and A. Mankodia, "Comprehensive Analysis of Object Detection and Tracking Methodologies from Surveillance Videos," 2021. doi: 10.1109/ICCMC51019.2021.9418373.
[4] Hardik Bhatt, Anand Mankodia,” A Comprehensive review on Content Based Image Retrieval System: Features & Challenges,” In: K. Kotecha et al. Eds., Data Science and Intelligent Applications, Lecture Notes on Data Engineering and Communications Technologies, vol. 52, pp. 63-74, 2020. https://doi.org/10.1007/978-981- 15-4474-3_7
[5] A. P. Mankodia, S. K. Shah, "Design of User Interface for Shot Boundary Detection using Classical Methods", proceeding of International Journal of Innovative Research in Electrical, Electronics, Instrumentation and Control Engineering, vol. 1, no. 7, pp. 287-293, 2013.
[6] A. P. Mankodia, S. K. Shah, "Analysis of Histogram Based Shot Segmentation Techniques for Video Summarization", proceeding of International Journal of Electronics Communications and Electrical Engineering, vol. 4, no. 4, pp. 2-11, 2014.
[7] Ş. Aktı, G. A. Tataroğlu, and H. K. Ekenel, "Vision-based Fight Detection from Surveillance Cameras," 2020. Doi: 10.1109/IPTA.2019.8936070.
[8] F. U. M. Ullah, A. Ullah, K. Muhammad, I. U. Haq, and S. W. Baik, "Violence Detection using Spatiotemporal Features with 3D Convolutional Neural Network," Sensors Switzerland, vol. 19, no. 11, 2019. Doi: 10.3390/s19112472.
[9] P. Vashistha, J. P. Singh, and M. A. Khan, "A Comparative Analysis of Different Violence Detection Algorithms from Videos," in Lecture Notes in Networks and Systems, Springer. vol. 94, pp. 577–589, 2020. Doi: 10.1007/978-981-15-0694-9_54.
[10] M. Sharma and R. Baghel, "Video Surveillance for Violence Detection Using Deep Learning," in Lecture Notes on Data Engineering and Communications Technologies, Springer Science and Business Media Deutschland GmbH, vol. 37, pp. 411–420, 2020. Doi: 10.1007/978-981-15-0978-0_40.
[11] S. R. Dinesh Jackson et al., "Real Time Violence Detection Framework for Football Stadium Comprising of Big Data Analysis and Deep Learning Through Bidirectional LSTM," Computer Networks, vol. 151, pp. 191–200, 2019. Doi: 10.1016/j.comnet.2019.01.028.
[12] B. Pradeepa, A. Viji, and J. J. Anthesious, "A Multi-Temporal Framework for Human Violent Event Analysis in Video Surveillance using CNN."
[13] Z. Islam, M. Rukonuzzaman, R. Ahmed, Md. H. Kabir, and M. Farazi, "Efficient Two-Stream Network for Violence Detection Using Separable Convolutional LSTM," 2021. Doi: 10.1109/IJCNN52387.2021.9534280.
[14] “IEEE Instrumentation and Measurement Society and Institute of Electrical and Electronics Engineers,” IST 2019 : 2019 IEEE International Conference on Imaging Systems & Techniques, Abu Dhabi, UAE, 2019 .
[15] A. Pandey, T. Samarth, and S. Raghuram, "Using Images for Real-Time Violence Detection in the Edge," in Lecture Notes in Electrical Engineering, vol. 656, pp. 677–685, 2020. Doi: 10.1007/978-981-15-3992-3_58.
[16] Deniz, Oscar & Serrano Gracia, Ismael & Bueno, Gloria & Kim, Tae-Tyun, “Fast Violence Detection in Video,” VISAPP 2014 - Proceedings of the 9th International Conference on Computer Vision Theory and Applications, vol. 2, 2014.
[17] Ribeiro, Pedro &Audigier, Romaric & Pham, Quoc-Cuong, “RIMOC, A Feature to Discriminate Unstructured Motions: Application to Violence Detection for Video-Surveillance,” Computer Vision and Image Understanding, vol. 144, 2015. 10.1016/j.cviu.2015.11.001.
[18] V. Machaca Arceda, K. Fernández Fabián and J. C. Gutíerrez, "Real Time Violence Detection in Video," International Conference on Pattern Recognition Systems (ICPRS-16), pp. 1-7, 2016. Doi: 10.1049/ic.2016.0030.
[19] Qian, Huimin& Mao, Yaobin& Xiang, Wenbo& Wang, Zhiquan, “Recognition of Human Activities using SVM Multi-Class Classifier,” Pattern Recognition Letters, vol. 31, pp. 100-111, 2010. 10.1016/j.patrec.2009.09.019.
[20] Al-Naji A, Chahl J, “Detection of Cardiopulmonary Activity and Related Abnormal Events Using Microsoft Kinect Sensor,” Sensors Basel, vol. 18, no. 3, pp. 920, 2018. Doi: 10.3390/s18030920.
[21] Wang, Siqi& Zeng, Yijie& Liu, Qiang& Zhu, Chengzhang& Zhu, En& Yin, Jianping, “Detecting Abnormality without Knowing Normality: A Two-stage Approach for Unsupervised Video Abnormal Event Detection,” pp. 636-644, 2018. 10.1145/3240508.3240615.
[22] H. Alvari, S. Sarkar and P. Shakarian, "Detection of Violent Extremists in Social Media," 2nd International Conference on Data Intelligence and Security (ICDIS), pp. 43-47, 2019. Doi: 10.1109/ICDIS.2019.00014.
[23] Mu, Guankun& Cao, Haibing &Jin, Qin, “Violent Scene Detection Using Convolutional Neural Networks and Deep Audio Features,” vol. 663, pp. 451-463, 2016. 10.1007/978-981-10-3005-5_37.
[24] Gargi Sharma, and Gourav Shrivastava. "Crop Disease Prediction using Deep Learning Techniques - A Review" SSRG International Journal of Computer Science and Engineering, vol. 9, no. 4, Apr. 2022, pp. 23- 28. Crossref, https://doi.org/10.14445/23488387/IJCSE-V9I4P104
[25] M.A. Muthiah, E. Logashamugam, N.M. Nandhitha "Performance evaluation of Googlenet, Squeezenet, and Resnet50 in the classification of Herbal images" International Journal of Engineering Trends and Technology 69.3(2021):229-232.
[26] Zulaiha Parveen A, Mr.T.Senthil Kumar "The Deep Learning Methodology For Improved Breast Cancer Diagnosis In MRI" International Journal of Computer and Organization Trends 11.3 (2021): 11-14.
[27] Zhijie Yang, Hongbin Huang "Garbage Classification Based On Deep Residual Weakly Supervised Learning Model" International Journal of Recent Engineering Science 7.3(2020):47-51.