Machine Learning and IoT-Based Messaging Device for Blind, Deaf, and Dumb People

Abstract

Recent advances in science and technology have improved the standard of living of human beings and made the life comfortable. But the life of people with visual, speech, and hearing impairments is still uncomfortable in terms of communication. They do not have a proper/simple messaging device that suits their standard of living. The devices that are currently available for them to use have certain limitations in aspects like the cost, simplicity, and limited to vicinal communication. According to reports from World Health Organization, there are about 300 million people who are deaf, 1million who are dumb and 285 million who are blind. This paper describes the device prototype that was designed through which people with visual, speech, and hearing impairments can send messages easily based on their hand gestures. An appropriate machine learning model is used for this task of decoding the meaning of hand gesture that the deaf or dumb person is using. The model was trained on the prepared dataset which consists of the hand gesture images and the corresponding labels. The message decoded by this machine learning model from the hand gestures is sent as mail to the designated person using Amazon web services (AWS), simple email service (SES). This device is mostly suitable for deaf and dumb people compared to blind people in terms of ease to use it.