This paper brings forth state-of-the-art blind assistance, and the system here provides a combination of an OpenCV-based DNN module based on YOLOv3’s algorithm; the system would support real-time object detection, with associated auditory notifications to enable visually impaired individuals to understand their autonomy couched with safety about improved fast recognition of objects allied with auditory responses. The live video captures from a webcam were done and further processed by the YOLOv3 algorithm. Optimize to work in real time for accurate object identification. Relayed objects are then passed on to the user through voice alerts generated using GTTS technology, thus passing on all the relevant information in an audio format. Another intuitive aspect of the system is that it is adaptive. It may alter the voice outputs so that they would be in the language of preference to the user. It is easier and more accessible to use it. In addition, the precision of the system in the recognition of large categories of objects makes it very versatile and, therefore, very valuable in making life quality enhancement for visually impaired people.

YOLOv3 (you only look once), DNN (deep neural network), GTTS (google text-to-speech), OpenCV, visually impaired

🔒 This is a subscription article

Full text is available to subscribers and institutional members. Please choose an option below to access it.

1. Jabnoun H, Benzarti F, Amiri H. Object detection and identification for blind people in video scene. In: 2015 15th International Conference on Intelligent Systems Design and Applications (ISDA). Marrakech, Morocco: IEEE; 2015 Dec 14. p. 363–7.
2. Auvray M, Hanneton S, O’Regan JK. Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘The Voice’. Perception. 2007 Mar;36(3):416–30.
3. Jabnoun H, Benzarti F, Amiri H. Visual substitution system for blind people based on SIFT description. In: 2014 6th International Conference of Soft Computing and Pattern Recognition (SoCPaR). Tunis, Tunisia: IEEE; 2014 Aug 11; p. 300–5.
4. Bilgaiyan S, James S, Bhonsle SS, Shahdeo S. Android based signboard detection using image and voice alert system. In: 2016 IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT). Bangalore, India: IEEE; 2016 May 20. p. 1488–91.
5. Cazan A, Vârbanescu R, Popescu D. Algorithms and techniques for image to sound conversion for helping the visually impaired people: Application proposal. In: 2007 14th International Workshop on Systems, Signals and Image Processing and 6th EURASIP Conference on Speech and Image Processing, Multimedia Communications and Services. Maribor, Slovenia: IEEE. 2007 Jun. p. 471–4.
6. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016 Jun 26–Jul 1; Las Vegas, NV, USA. p. 779–88.
7. Kim DH, Lee S, Jeon J, Song BC. Real-time purchase behavior recognition system based on deep learning-based object detection and tracking for an unmanned product cabinet. Expert Syst Appl. 2020 Apr;143:113063.
8. Dandekar Y, Shinde K, Gangan J, Firdausi S, Bharne S. Weed plant detection from agricultural field images using YOLOv3 algorithm. In: 2022 6th International Conference on Computing, Communication, Control and Automation (ICCUBEA). Pune, India: IEEE; 2022 Aug 26. p. 1–4.
9. Haggui O, Bayd H, Magnier B, Aberkane A. Human detection in moving fisheye camera using an improved YOLOv3 framework. In: 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP). Tampere, Finland: IEEE; 2021 Oct 6. p. 1–6.
10. Kumar S, Kumar C. Drone/UAVs surveillance feeds video object recognition using transfer learning. In: 2022 International Conference on Futuristic Technologies (INCOFT). Tiruchirappalli, India: IEEE; 2022 Nov 25. p. 1–4.
11. Nguyen TT. Real-time object detection & grasping for a robotic arm [dissertation]. San Francisco (CA): San Francisco State University; 2018.
12. Devi MR, Kumar VV, Sivakumar P. A review of image classification and object detection on machine learning and deep learning techniques. In: 2021 5th International Conference on Electronics, Communication and Aerospace Technology (ICECA). Coimbatore, India: IEEE; 2021 Dec 2; p. 1–8.
13. Jain D, Tripathi GS, Verma A. Modeling and design of an autonomous amphibious vehicle with obstacle avoidance for surveillance applications. In: International Conference on Intelligent Computing for Sustainable Development; 2023 Aug 25; Chandigarh, India. Cham: Springer Nature Switzerland; 2023. p. 246–57.
14. Bakar MH. Fusion sparse and shaping reward function in soft actor-critic deep reinforcement learning for mobile robot navigation [master’s thesis]. Batu Pahat (Malaysia): Universiti Tun Hussein Onn Malaysia; 2022.
15. Baruah S, Dutta S, Chetia A, Das MP, Choudhury S, Saikia N, et al. Helmet and number plate detection using different YOLO algorithms. In: Sustainable Development, Innovation and Green Technology (ICASDIGT-2024); 2025. p. 473.

This paper brings forth state-of-the-art blind assistance, and the system here provides a combination of an OpenCV-based DNN module based on YOLOv3’s algorithm; the system would support real-time object detection, with associated auditory notifications to enable visually impaired individuals to understand their autonomy couched with safety about improved fast recognition of objects allied with auditory responses. The live video captures from a webcam were done and further processed by the YOLOv3 algorithm. Optimize to work in real time for accurate object identification. Relayed objects are then passed on to the user through voice alerts generated using GTTS technology, thus passing on all the relevant information in an audio format. Another intuitive aspect of the system is that it is adaptive. It may alter the voice outputs so that they would be in the language of preference to the user. It is easier and more accessible to use it. In addition, the precision of the system in the recognition of large categories of objects makes it very versatile and, therefore, very valuable in making life quality enhancement for visually impaired people.

YOLOv3 (you only look once), DNN (deep neural network), GTTS (google text-to-speech), OpenCV, visually impaired

🔒 This is a subscription article

Full text is available to subscribers and institutional members. Please choose an option below to access it.

1. Jabnoun H, Benzarti F, Amiri H. Object detection and identification for blind people in video scene. In: 2015 15th International Conference on Intelligent Systems Design and Applications (ISDA). Marrakech, Morocco: IEEE; 2015 Dec 14. p. 363–7.
2. Auvray M, Hanneton S, O’Regan JK. Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘The Voice’. Perception. 2007 Mar;36(3):416–30.
3. Jabnoun H, Benzarti F, Amiri H. Visual substitution system for blind people based on SIFT description. In: 2014 6th International Conference of Soft Computing and Pattern Recognition (SoCPaR). Tunis, Tunisia: IEEE; 2014 Aug 11; p. 300–5.
4. Bilgaiyan S, James S, Bhonsle SS, Shahdeo S. Android based signboard detection using image and voice alert system. In: 2016 IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT). Bangalore, India: IEEE; 2016 May 20. p. 1488–91.
5. Cazan A, Vârbanescu R, Popescu D. Algorithms and techniques for image to sound conversion for helping the visually impaired people: Application proposal. In: 2007 14th International Workshop on Systems, Signals and Image Processing and 6th EURASIP Conference on Speech and Image Processing, Multimedia Communications and Services. Maribor, Slovenia: IEEE. 2007 Jun. p. 471–4.
6. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016 Jun 26–Jul 1; Las Vegas, NV, USA. p. 779–88.
7. Kim DH, Lee S, Jeon J, Song BC. Real-time purchase behavior recognition system based on deep learning-based object detection and tracking for an unmanned product cabinet. Expert Syst Appl. 2020 Apr;143:113063.
8. Dandekar Y, Shinde K, Gangan J, Firdausi S, Bharne S. Weed plant detection from agricultural field images using YOLOv3 algorithm. In: 2022 6th International Conference on Computing, Communication, Control and Automation (ICCUBEA). Pune, India: IEEE; 2022 Aug 26. p. 1–4.
9. Haggui O, Bayd H, Magnier B, Aberkane A. Human detection in moving fisheye camera using an improved YOLOv3 framework. In: 2021 IEEE 23rd International Workshop on Multimedia Signal Processing (MMSP). Tampere, Finland: IEEE; 2021 Oct 6. p. 1–6.
10. Kumar S, Kumar C. Drone/UAVs surveillance feeds video object recognition using transfer learning. In: 2022 International Conference on Futuristic Technologies (INCOFT). Tiruchirappalli, India: IEEE; 2022 Nov 25. p. 1–4.
11. Nguyen TT. Real-time object detection & grasping for a robotic arm [dissertation]. San Francisco (CA): San Francisco State University; 2018.
12. Devi MR, Kumar VV, Sivakumar P. A review of image classification and object detection on machine learning and deep learning techniques. In: 2021 5th International Conference on Electronics, Communication and Aerospace Technology (ICECA). Coimbatore, India: IEEE; 2021 Dec 2; p. 1–8.
13. Jain D, Tripathi GS, Verma A. Modeling and design of an autonomous amphibious vehicle with obstacle avoidance for surveillance applications. In: International Conference on Intelligent Computing for Sustainable Development; 2023 Aug 25; Chandigarh, India. Cham: Springer Nature Switzerland; 2023. p. 246–57.
14. Bakar MH. Fusion sparse and shaping reward function in soft actor-critic deep reinforcement learning for mobile robot navigation [master’s thesis]. Batu Pahat (Malaysia): Universiti Tun Hussein Onn Malaysia; 2022.
15. Baruah S, Dutta S, Chetia A, Das MP, Choudhury S, Saikia N, et al. Helmet and number plate detection using different YOLO algorithms. In: Sustainable Development, Innovation and Green Technology (ICASDIGT-2024); 2025. p. 473.