3D CNN-Residual Neural Network Based Multimodal Medical Image Classification

3D CNN-Residual Neural Network Based Multimodal Medical Image Classification
  IJETT-book-cover           
  
© 2022 by IJETT Journal
Volume-70 Issue-10
Year of Publication : 2022
Authors : B. Suryakanth, S A. Hari Prasad
DOI : 10.14445/22315381/IJETT-V70I10P236

How to Cite?

B. Suryakanth, S A. Hari Prasad, "3D CNN-Residual Neural Network Based Multimodal Medical Image Classification," International Journal of Engineering Trends and Technology, vol. 70, no. 10, pp. 371-380, 2022. Crossref, https://doi.org/10.14445/22315381/IJETT-V70I10P236

Abstract
Multimodal medical imaging has become incredibly common in biomedical imaging. From multimodality clinical visual information, meaningful information has been derived using clinical image classification. Computed tomography (CT) and Magnetic resonance imaging (MRI) are some imaging approaches. Different imaging technologies provide different imaging information for the same part. Traditional ways of illness classification are effective, but in today's environment, 3D images are used to identify diseases. Compared to 1D and 2D images, 3D images have a very clear vision. The suggested approach uses 3D Residual Convolutional Neural Network (CNN ResNet) for the 3D image classification. Various methods are available for classifying the disease, like cluster, KNN, and ANN. Traditional techniques are not trained to classify 3D images, so an advanced approach is introduced in the proposed method to predict the 3D images. Initially, the multimodal 2D medical image data is taken. This 2D input image is turned into 3D image data because 3D images give more information than 2D image data. After employing guided filtering to integrate the 3D CT and MRI data, the resultant image is filtered for further processing. The fused image is then augmented. Finally, this fused image is fed to 3DCNN ResNet for classification purposes. The 3DCNN ResNet classifies the image data and produces the output as five different stages of the disease. The proposed method achieves 98% of accuracy. Thus, the designed model effectively predicted the disease's stage.

Keywords
Designed modal, Fused image, Guided filtering, Multimodal medical image.

Reference
[1] K. Dutta, “Densely Connected Recurrent Residual (Dense R2UNet) Convolutional Neural Network for Segmentation of Lung CT Images,” arXiv preprint arXiv: 2102.00663, 2021. https://doi.org/10.48550/arXiv.2102.00663.
[2] I. Kitanovski, G. Strezoski, I. Dimitrovski, G. Madjarov, and S. Loskovska, “Multimodal Medical Image Retrieval System,” Multimedia Tools and Applications, vol. 76, no. 2, pp. 2955-2978, 2017. https://doi.org/10.1007/s11042-016-3261-1.
[3] X. Cheng, L. Zhang, and Y. Zheng, “Deep Similarity Learning for Multimodal Medical Images,” Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, vol. 6, no. 3, pp. 248-252, 2018. https://doi.org/10.1080/21681163.2015.1135299.
[4] A. Qayyum, S. M. Anwar, M. Awais and M. Majid, “Medical Image Retrieval using Deep Convolutional Neural Network,” Neurocomputing, vol. 266, pp. 8-20, 2017. https://doi.org/10.1016/j.neucom.2017.05.025.
[5] H. Hermessi, O. Mourali, and E. Zagrouba, “Multimodal Medical Image Fusion Review,” Theoretical Background and Recent Advances Signal Processing, pp. 108036, 2021. https://doi.org/10.1016/j.sigpro.2021.108036.
[6] T. Tirupal, B. C. Mohan and S. S. Kumar, “Multimodal Medical Image Fusion Techniques–A Review,” Current Signal Transduction Therapy, vol. 16, no. 2, pp. 142-163, 2021.https://doi.org/10.2174/1574362415666200226103116.
[7] S. Farjana Farvin, S. Krishna Mohan, "A Comparative Study on Lung Cancer Detection using Deep Learning Algorithms," SSRG International Journal of Computer Science and Engineering, vol. 9, no. 5, pp. 1-4, 2022. Crossref, https://doi.org/10.14445/23488387/IJCSE-V9I5P101.
[8] Z. Guo, X. Li, H. Huang, N. Guo, and Q. Li, “Medical Image Segmentation Based on Multimodal Convolutional Neural Network: Study on Image Fusion Schemes,” In 2018 IEEE 15th International Symposium on Biomedical Imaging, IEEE, pp. 903-907, 2018. https://doi: 10.1109/ISBI.2018.8363717.
[9] S. Singh and R. S. Anand, “Multimodal Medical Image Fusion using Hybrid Layer Decomposition with CNN-Based Feature Mapping and Structural Clustering,” IEEE Transactions on Instrumentation and Measurement, vol. 69, no. 6, pp. 3855-3865, 2019. https:// doi: 10.1109/TIM.2019.2933341.
[10] M. Manchanda and R. Sharma, “A Novel Method of Multimodal Medical Image Fusion using Fuzzy Transform,” Journal of Visual Communication and Image Representation, vol. 40, pp. 197-217, 2016. https://doi.org/10.1016/j.jvcir.2016.06.021.
[11] Lakshman Karthik Ramkumar, Sudharsana Premchand, Gokul Karthi Vijayakumar, "Sign Language Recognition using Depth Data and CNN," SSRG International Journal of Computer Science and Engineering, vol. 6, no. 1, pp. 9-14, 2019. Crossref, https://doi.org/10.14445/23488387/IJCSE-V6I1P102.
[12] F. Deeba, F. A. Dharejo, and M. Zawish, Y. Zhou, et al., “Multimodal-Boost: Multimodal Medical Image Super-Resolution using MultiAttention Network with Wavelet Transform,” arXiv preprint arXiv: 2110.11684, 2021. https://doi.org/10.48550/arXiv.2110.11684.
[13] Z. Tang, K. Chen, M. Pan, M. Wang, and Z. Song, “An Augmentation Strategy for Medical Image Processing Based on Statistical Shape Model and 3D Thin Plate Spline for Deep Learning,” IEEE Access, vol. 7, pp. 133111-133121, 2019. https://doi: 10.1109/ACCESS.2019.2941154.
[14] C. Yu, R. Han, M. Song, C. Liu, and C. I. Chang, “A Simplified 2D-3D CNN Architecture for Hyperspectral Image Classification Based on Spatial–Spectral Fusion,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp. 2485- 2501, 2020. https://doi: 10.1109/JSTARS.2020.2983224.
[15] H. R. Roth, H. Oda, X. Zhou, N. Shimizu, Y. Yang, Y. Hayashi, M. Oda, M. Fujiwara, K. Misawa, and K. Mori, “An Application of Cascaded 3D Fully Convolutional Networks for Medical Image Segmentation,” Computerized Medical Imaging and Graphics, vol .66, pp. 90-99, 2018. https://doi.org/10.1016/j.compmedimag.2018.03.001.
[16] Ms.S.Supraja, Dr.P.Ranjith Kumar, "An Intelligent Traffic Signal Detection System Using Deep Learning," SSRG International Journal of VLSI & Signal Processing, vol. 8, no. 1, pp. 5-9, 2021. Crossref, https://doi.org/10.14445/23942584/IJVSP-V8I1P102.
[17] M. J. Horry, S. Chakraborty, M. Paul, A. Ulhaq, B. Pradhan, M. Saha and N. Shukla, “COVID-19 Detection through Transfer Learning using Multimodal Imaging Data,” IEEE Access, vol. 8, pp. 149808-149824, 2020. https://doi: 10.1109/ACCESS.2020.3016780.
[18] M. Ahmadi, A. Sharifi, M. J. Fard, and N. Soleimani, “Detection of Brain Lesion Location in MRI Images using Convolutional Neural Network and Robust PCA,” International Journal of Neuroscience, pp.1-12, 2021. https://doi.org/10.1080/00207454.2021.1883602.
[19] B. Rajalingam and R. Priya, “Multimodal Medical Image Fusion Based on Deep Learning Neural Network for Clinical Treatment Analysis,” International Journal of ChemTech Research, vol. 11, no. 6, pp. 160-176, 2018. http://dx.doi.org/10.20902/IJCTR.2018.110621.
[20] J. Zhang, Y. Xie, Q. Wu and Y. Xia, “Medical Image Classification using Synergic Deep Learning,” Medical Image Analysis, vol. 54, pp. 10-19, 2019.
[21] X. Chai, F. Shao, Q. Jiang, and Y. S. Ho, “Roundness-Preserving Warping for Aesthetic Enhancement-Based Stereoscopic Image Editing,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 4, pp. 1463-1477, 2020. https://doi: 10.1109/TCSVT.2020.3010181.
[22] B. Rajalingam, F. Al-Turjman, R. Santhoshkumar, and M. Rajesh, “Intelligent Multimodal Medical Image Fusion with Deep Guided Filtering,” Multimedia Systems, pp.1-15, 2020. https://doi.org/10.1007/s00530-020-00706-0.
[23] V. V. Rani, G. Vasavi and K. K. Kumar, “A Detailed Review on Image Augmentation and Segmentation of Brain MRI Images using Deep Learning,” Paideuma Journal, 2018. https://doi.org/10.1016/j.jocs.2018.12.003.
[24] J. Zhao, L. Hu, Y. Dong, L. Huang, S. Weng, and D. Zhang, “A Combination Method of Stacked Autoencoder and 3D Deep Residual Network for Hyperspectral Image Classification,” International Journal of Applied Earth Observation and Geoinformation, vol. 102, pp. 102459, 2021. https://doi.org/10.1016/j.jag.2021.102459.
[25] Alzheimers Disease 5 Class Dataset ADNI, [Online]. Available: https://www.kaggle.com/madhucharan/alzheimersdisease5classdatasetadni.