Assisting Visually Impaired People through Real-time Depth Estimation using Stereo Vision

 

Assisting Visually Impaired People through Real-time Depth Estimation using Stereo Vision
  IJETT-book-cover           
  
© 2023 by IJETT Journal
Volume-71 Issue-11
Year of Publication : 2023
Author : Moncef Aharchi, M’hamed Ait Kbir
DOI : 10.14445/22315381/IJETT-V71I11P225

How to Cite?

Moncef Aharchi, M’hamed Ait Kbir, "Assisting Visually Impaired People through Real-time Depth Estimation using Stereo Vision," International Journal of Engineering Trends and Technology, vol. 71, no. 11, pp. 236-246, 2023. Crossref, https://doi.org/10.14445/22315381/IJETT-V71I11P225

Abstract
Visually impaired individuals face significant challenges in their daily lives, particularly during mobility. The loss of vision greatly reduces their ability to detect obstacles, increasing the risks of falls and collisions. However, by employing stereoscopic vision techniques that simulate human binocular vision, it is possible to restore depth perception and help the visually impaired avoid obstacles more effectively. This article presents an in-depth study of the steps involved in creating a depth map using two cameras as part of a system designed to assist the visually impaired during their mobility. The technique utilized by this system relies on the use of stereoscopic vision to provide valuable assistance in detecting obstacles to visually impaired individuals. By analyzing the captured images from the two cameras, the system constructs a depth map that accurately represents the spatial information of the environment. This depth map is a crucial tool in assisting visually impaired individuals in safely navigating their surroundings. The system can detect and alert users to potential obstacles in real time, enhancing their mobility and reducing the risks they face. This article work emphasizes the importance of developing systems that utilize stereoscopic vision to create depth maps for assisting the visually impaired during mobility. By providing valuable assistance in obstacle detection, these innovations have the potential to improve visually impaired people's daily safety and autonomy greatly.

Keywords
Camera calibration, Depth map, Rectification, Stereo matching, Visual impairment.

References
[1] Blindness and Vision Impairment, World Health Organization, 2023. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment
[2] Arjun Pardasani et al., “Smart Assistive Navigation Devices for Visually Impaired People,” 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS), pp. 725-729, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[3] B. Vamsi Krishna, and K. Aparna, “IoT-Based Indoor Navigation Wearable System for Blind People,” Artificial Intelligence and Evolutionary Computations in Engineering Systems, pp. 413-421, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Rahul Dhod et al., “Low Cost GPS and GSM Based Navigational Aid for Visually Impaired People,” Wireless Personal Communications, vol. 92, pp. 1575-1589, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[5] G. Balakrishnan, and G. Sainarayanan, “Stereo Image Processing Procedure for Vision Rehabilitation,” Applied Artificial Intelligence, vol. 22, no. 6, pp. 501-522, 2008.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Larry Matthies et al., “Computer Vision on Mars,” International Journal of Computer Vision, vol. 75, pp. 67-92, 2007.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Faleh Alqahtani et al., “Detection and Tracking of Faces in 3D Using a Stereo Camera Arrangements,” International Journal of Machine Learning and Computing, vol. 9, no. 1, pp. 35-43, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Abdulkadir Akin, “Real-Time High-Resolution Multiple-Camera Depth Map Estimation Hardware and its Applications,” Thesis, Infoscience EPFL Scientific Publications, pp. 1-221, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Cécile Riou et al., “Calibration and Disparity Maps for a Depth Camera Based on a Four-Lens Device,” Journal of Electronic Imaging, vol. 24, no. 6, pp. 1-11, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Taih´u Pire et al., “Stereo Vision Obstacle Avoidance Using Depth and Elevation Maps,” Laboratorio de Robotica y Sistemas Embebidos, pp. 1-3, 2012.
[Google Scholar] [Publisher Link]
[11] Cristina Cattaneo, G. Mainetti, and R. Sala, “The Importance of Camera Calibration and Distortion Correction to Obtain Measurements with Video Surveillance Systems,” Journal of Physics: Conference Series, vol. 658, pp. 1-8, 2015.
[CrossRef] [Google Scholar] [Publisher Link]
[12] V. Dwivedi, A. Kothari, and K. Raviya, “Depth and Disparity Extraction Structure for Multi View Images Video Frame-A Review,” European Journal of Academic Essays, vol. 1, no. 10, pp. 29-35, 2014.
[Google Scholar] [Publisher Link]
[13] Chao Yang, Fugen Zhou, and Xiangzhi Bai, “3D Reconstruction through Measure Based Image Selection,” 2013 Ninth International Conference on Computational Intelligence and Security, pp. 377-381, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[14] Ajay Kumar Mishra et al., “3D Surveillance System Using Multiple Cameras,” Videometrics IX, vol. 6491, pp. 11-18, 2007.
[CrossRef] [Google Scholar] [Publisher Link]
[15] Vincent Nozick, “Camera Array Image Rectification and Calibration for Stereoscopic and Autostereoscopic Displays,” Annals of Telecommunications, vol. 68, pp. 581-596, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Andrey V. Kudryavtsev, Sounkalo Dembélé, and Nadine Piat, “Stereo-Image Rectification for Dense 3D Reconstruction in Scanning Electron Microscope,” 2017 International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS), pp. 1-6, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[17] Charles Loop, and Zhengyou Zhang, “Computing Rectifying Homographies for Stereo Vision,” 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), pp. 125-131, 1999.
[CrossRef] [Google Scholar] [Publisher Link]
[18] Andrea Fusiel lo, “Tutorial on Rectification of Stereo Images,” Dipartimento di Matematica e Informatica Università di Udine, pp. 1-12, 1999.
[Google Scholar] [Publisher Link]
[19] Ruichao Xiao et al., “DSR: Direct Self-Rectification for Uncalibrated Dual-Lens Cameras,” 2018 International Conference on 3D Vision (3DV), pp. 561-569, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[20] David Lowe, “Object Recognition from Local Scale-Invariant Features,” Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1150-157, 1999.
[CrossRef] [Google Scholar] [Publisher Link]
[21] Vineetha Vijayan, and K.P. Pushpalatha, “FLANN based Matching with SIFT Descriptors for Drowsy Features Extraction,” 2019 Fifth International Conference on Image Information Processing (ICIIP), pp. 600-605, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[22] David G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, vol. 60, pp. 91- 110, 2004.
[CrossRef] [Google Scholar] [Publisher Link]
[23] Tayeb Basta, “Is the Fundamental Matrix Really Independent of the Scene Structure?,” International Journal of Signal Processing, Image Processing and Pattern Recognition, vol. 7, no. 5, pp. 149-168, 2014.
[CrossRef] [Google Scholar] [Publisher Link]
[24] Justinas Miseikis et al., “Automatic Calibration of a Robot Manipulator and Multi 3D Camera System,” 2016 IEEE/SICE International Symposium on System Integration (SII), pp. 735-741, 2016.
[CrossRef] [Google Scholar] [Publisher Link]
[25] Jiang Hong-zhi et al., “Phase-Based Stereo Matching Using Epipolar Line Rectification,” Guangxue Jingmi Gongcheng, vol. 19, no. 10, pp. 2520-2525, 2011.
[CrossRef] [Google Scholar]
[26] Fran ̧cois Darmon, and Pascal Monasse, “The Polar Epipolar Rectification,” Image Processing on Line, vol. 11, pp. 56-75, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[27] Richard Hartley, and Rajiv Gupta, “Computing Matched-Epipolar Projections,” Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 549-555, 1993.
[CrossRef] [Google Scholar] [Publisher Link]
[28] Rahul Raguram, Jan-Michael Frahm, and Marc Pollefeys, “A Comparative Analysis of RANSAC Techniques Leading to Adaptive Real-Time Random Sample Consensus,” European Conference on Computer Vision, pp. 500-513, 2008.
[CrossRef] [Google Scholar] [Publisher Link]
[29] Richard I. Hartley, “Theory and Practice of Projective Rectification,” International Journal of Computer Vision, vol. 35, pp. 115-127, 1999.
[CrossRef] [Google Scholar] [Publisher Link]
[30] Monika Gupta, and Sapna Malik, “Dual Dynamic Programming with Quantization for Disparity Map,” SSRG International Journal of Computer Science and Engineering, vol. 4, no. 3, pp. 5-9, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[31] Heiko Hirschmüller, “Stereo Processing by Semiglobal Matching and Mutual Information,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328-341, 2008.
[CrossRef] [Google Scholar] [Publisher Link]
[32] CV::ximgproc::DisparityWLSFilter Class Reference, Open Source Computer Vision, 2023. [Online]. Available:
https://docs.opencv.org/3.4/d9/d51/classcv_1_1ximgproc_1_1DisparityWLSFilter.html
[33] Serdar Solak, and Emine Doğru Bolat, “A New Hybrid Stereovision-Based Distance-Estimation Approach for Mobile Robot Platforms,” Computers and Electrical Engineering, vol. 67, pp. 672-689, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[34] Fredy Mart´ınez, Edwar Jacinto, and Fernando Mart´ınez, “Obstacle Detection for Autonomous Systems Using Stereoscopic Images and Bacterial Behaviour,” International Journal of Electrical and Computer Engineering, vol. 10, no. 2, pp. 2164-2172, 2020.
[CrossRef] [Google Scholar] [Publisher Link]
[35] Emre Dandil, and Kerim Kürşat Çevik, “Computer Vision based Distance Measurement System Using Stereo Camera View,” 2019 3 rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), pp. 1-4, 2019.
[CrossRef] [Google Scholar] [Publisher Link]