IJBTT

Emotion Understanding from Facial Expressions using Stacked Generative Adversarial Network (GAN) and Deep Convolution Neural Network (DCNN)

Emotion Understanding from Facial Expressions using Stacked Generative Adversarial Network (GAN) and Deep Convolution Neural Network (DCNN)
	© 2022 by IJETT Journal
	Volume-70 Issue-10
	Year of Publication : 2022
	Authors : T. Kujani, V. Dhilip Kumar
	DOI : 10.14445/22315381/IJETT-V70I10P212

How to Cite?

T. Kujani, V. Dhilip Kumar, "Emotion Understanding from Facial Expressions using Stacked Generative Adversarial Network (GAN) and Deep Convolution Neural Network (DCNN)," International Journal of Engineering Trends and Technology, vol. 70, no. 10, pp. 98-110, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I10P212

Abstract
Facial expressions play a vital role in nonverbal communication in understanding the behavior of human beings. Face recognition applications are applied in several fields, including authentication through biometrics, security-enhancing applications, controlling automobiles, finding mental health, in a personal interview to understand the personality, and detecting driver drowsiness. Despite several advancements, face recognition remains challenging due to variances in the effects of the images, such as illumination variation, poses, occlusions, and facial expressions. Although many techniques, such as iris and fingerprint scanning, yield prominent accuracy, face recognition is an admirable technique that is applied for many real-time recognitions and is a human-centric identification method. In this paper, facial emotion recognition with Deep Convolution Neural Network (DCNN) is used for evaluating the face expressions considering the advantage of extending the training dataset using Generative Adversarial Networks (GAN) and traditional augmentation methods. This work has explored face behavior detection using emotion identification in real-time video surveillance using combined GAN and Deep CNN. The one significant part of this work is the custom-developed deep convolution layers with suitable optimizers. Using the proposed system, the basic human expressions which play a major role in behavior understanding can be classified effectively, irrespective of gender, facial orientation, race, and age, using GAN and DCNN. The FER2013, CK+, and Custom datasets were used in experiments, and the obtained performance was compared to that of cutting-edge techniques.

Keywords
Behavior, Classification, Convolution neural network, Expression, Generative adversarial network.

Reference
[1] Radford, Alec, Luke Metz, and Soumith Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks," arXiv preprint arXiv:1511.06434, 2015.
[2] R. Kosti, J.M. Álvarez, A. Recasens and A. Lapedriza, "Context based Emotion Recognition using Emotic Dataset", IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2019.
[3] Dharanya V, Raj A.N.J. and Gopi V.P, “Facial Expression Recognition through Person-Wise Regeneration of Expressions using Auxiliary Classifier Generative Adversarial Network (AC-GAN) Based Model,” Journal of Visual Communication and Image Representation, vol. 77, pp. 103110, 2021.
[4] K. Schindler, L. Van Gool, and B. de Gelder, “Recognizing Emotions Expressed by Body Pose: A Biologically Inspired Neural Model,” Neural Networks, vol. 21, no. 9, pp. 1238–1246, 2008.
[5] Wang, Junhuan. "Improved Facial Expression Recognition Method Based on GAN," Scientific Programming, vol. 2021, 2021.
[6] T. Nishime, S. Endo, K. Yamada, N. Toma, and Y. Akamine, “Feature Acquisition from Facial Expression Image using Convolutional Neural Networks,” Journal of Robotics, Networking and Artificial Life, vol. 3, no. 1, pp. 9-12, 2016
[7] Chen, An, Hang Xing, and Feiyu Wang, "A Facial Expression Recognition Method using Deep Convolutional Neural Networks Based On Edge Computing," IEEE Access, vol. 8, pp. 49741-49751, 2020.
[8] M. Alhussein, “Automatic Facial Emotion Recognition using Weber Local Descriptor for E-Healthcare System,” Cluster Computing, vol. 19, no. 1, pp. 99–108, 2016. http://dx. doi.org/10.1007/s10586-016-0535-3
[9] H. Yang, Z. Zhang, L. Yin, “Identity-Adaptive Facial Expression Recognition through Expression Regeneration using Conditional Generative Adversarial Networks,” in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition FG 2018, Xi’an, pp. 294–301, 2018. http://dx.doi.org/10.1109/FG.2018. 00050.
[10] Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,” in Advances in Neural Information Processing Systems, pp. 2672–2680, 2014.
[11] G. Ali, A. Ali, F. Ali et al., “Artificial Neural Network-Based Ensemble Approach for Multicultural Facial Expressions Analysis,” IEEE Access, vol. 8, no. 1, pp. 134950–134963, 2020.
[12] Zhu L, Chen Y, Ghamisi P, & Benediktsson J. A, “Generative Adversarial Networks for Hyperspectral Image Classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 9, pp. 5046-5063, 2018.
[13] Lucey, Patrick, et al., “The Extended Cohn-Kanade Dataset (Ck+): A Complete Dataset for Action Unit and Emotion-Specified Expression,” Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE Computer Society Conference on IEEE, 2010.
[14] Nikhil Kumar Singh, Gokul Rajan V, "Facial Emotion Recognition in Python," SSRG International Journal of Computer Science and Engineering, vol. 7, no. 6, pp. 20-23, 2020. Crossref, https://doi.org/10.14445/23488387/IJCSE-V7I6P106
[15] Liu, Ping, Shizhong Han, Zibo Meng, and Yan Tong. ”Facial Expression Recognition via a Boosted Deep Belief Network,” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1805-1812, 2014.
[16] Z. Zhang, M. Lyons, M. Schuster, and S. Akamatsu. “Comparison between Geometry-Based and Gabor-Wavelets-Based Facial Expression Recognition using Multi-Layer Perceptron,” In FG, pp. 454–459, 1998.
[17] G. Zhao and M. Pietiainen, “Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 915–928, 2007.
[18] Toisoul A, Kossaifi J, Bulat A, Tzimiropoulos G, & Pantic M, “Estimation of Continuous Valence and Arousal Levels from Faces in Naturalistic Conditions,” Nature Machine Intelligence, vol. 3, no. 1, pp. 42-50, 2021.
[19] Caramihale, Traian, Dan Popescu, and Loretta Ichim, "Emotion Classification using a Tensorflow Generative Adversarial Network Implementation," Symmetry, vol. 10, no. 9, pp. 414, 2018.
[20] Kahou SE, Bouthillier X, Lamblin P, et al., “EmoNets: Multimodal Deep Learning Approaches for Emotion Recognition in Video,” Journal on Multimodal User Interfaces, vol. 10, pp. 99-111, 2016.
[21] Sajjad, Muhammad, et al., "Human Behavior Understanding in Big Multimedia Data using CNN-Based Facial Expression Recognition," Mobile Networks and Applications, vol. 25, no. 4, pp. 1611-1621, 2020.
[22] Quinn, Minh-An, Grant Sivesind, and Guilherme Reis. "Real-Time Emotion Recognition from Facial Expressions," Standford University, 2017.
[23] Ozdemir, Mehmet Akif, et al., "Real Time Emotion Recognition from Facial Expressions using CNN Architecture," 2019 Medical Technologies Congress tiptekno, IEEE, 2019.
[24] Bhatti Y. K, Jamil A, Nida N, Yousaf M. H, Viriri S, & Velastin S. A, “Facial Expression Recognition of Instructor using Deep Features and Extreme Learning Machine,” Computational Intelligence and Neuroscience, 2021.
[25] P. Deivendran, P. Suresh Babu, G. Malathi, K. Anbazhagan, R. Senthil Kumar, "Emotion Recognition for Challenged People Facial Appearance in Social using Neural Network," International Journal of Engineering Trends and Technology, vol. 70, no. 6, pp. 272- 278, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I6P228
[26] Carrier P. L, Courville A, Goodfellow I. J, Mirza M, & Bengio Y, “FER-2013 Face Database,” University of Montreal, 2013.