Researchers at Humanitarian Technology (HuT) Labs of Amrita Vishwa Vidyapeetham, ranked the 7th Best University in India in the 2023 National Institution Ranking Framework, have developed India’s first device that enables people with speech impairment to communicate with the world by using simple eye signs.

The device, called Netravaad, consists of a camera, display, speaker, controller and a rechargeable battery that lasts for six hours on a single charge. The camera detects the eye sign of the user, which is converted into alphabet, word or a sentence by a customized AI algorithm called Sharani. The detected word and sentences are displayed on the screen. In addition, the voice through speakers helps the external world understand the patient’s feelings, and the patient also gets a feel of speaking to others. Netravaad will advance in its development and become available in regional languages such as Malayalam and Hindi. Additionally, the user interface will be highly interactive, ensuring a comfortable experience for users.

Said Prof. Rajesh Kannan Megalingam, Director, Humanitarian Technology (HuT) Labs, Associate Professor, ECE Dept. Head, Amrita Center for Advanced Robotics, “For communication, we created a new language using eye signs, called Netravaani. It allows users to utilize all 26 English alphabets to create any word and sentence. It also supports the numbers 0 to 9, with which the users can create any number. Thought the device is currently set in English, Netravaad is also undergoing regional language trials in Malayalam and Hindi. More languages are planned to be added in the future.We intend to make a highly interactive user interface to increase the comfort level of patient as its future phase.”

Netravaad is portable and lightweight device that can be adjusted to any position for the patient’s comfort. It can be used while sitting in a chair, lying on the bed or even while travelling. The Netravaad Unit houses a display monitor, camera, speaker, a smash box and a rechargeable 24V battery. The complete unit is housed in a box and positioned on a swivel platform that can be adjusted to face in any direction as needed. The camera detects the user’s eye movements and converts them into words or sentences, which are displayed on the screen and verbalized through the speaker.

Prof. Kannan further added, “Netravaad supports two different modes, English 1 and English 2. In the first mode, patients can select the ten most used words for day-to-day activities, allowing for quick communication with caretakers or family members for any immediate need. The English 2 mode has word, sentence, and number sub-modes. In the word or sentence sub-mode, users can form any word or sentence, respectively, with eye signs. An advanced prediction algorithm also allows them to select pre-defined words or sentences. In the number sub-mode, the users can communicate numbers. The patient can choose any of the mode or sub-mode using the eye sign. The device is user-friendly. Patients are giving training on the device according to their health condition, after which they can easily communicate using Netravaad.”

The patients are trained before they can use Netravaad. The duration of the training varies depending on the patient’s health condition. In addition to training, all the instructions for using Netravaad are displayed on the screen and voice instructions are given using the speaker. After completion of the training, the patient can easily follow the instructions given by the device and communicate with the external world through the voice of Netravaad.

Eye sign language is a process to estimate and track the line of sight of a person, or simply, where a person is looking. It has mainly five categories of eye signs including left, right, top, close and center.

Amrita Vishwa Vidyapeetham

http://www.amrita.edu

Amrita Vishwa Vidyapeetham is a multi-campus, multidisciplinary Institution of Eminence accredited by NAAC. It is also ranked the 7th Best University in India in the 2023 National Institution Ranking Framework (NIRF). Amrita continues to be the No. 1 Private University in India as per THE 2020 world rankings as well. The university has been awarded two UNESCO Chairs namely the UNESCO Chair for Experiential Learning for Sustainable Innovation & Development and the UNESCO Chair on Gender Equality & Women Empowerment. The university headquartered at Coimbatore is spread across six campuses in three states — Kerala, Tamil Nadu, and Karnataka. Amrita partners with academic, industry and governmental institutions across the world to accomplish human-centered, translational, and groundbreaking research. To date, Amrita has more than 500 collaborations from more than 200 signed Memorandums of Understanding (MoUs). Some of Amrita’s partners include Harvard University, Columbia University, King’s College London, KTH – Royal Institute of Technology, VU Amsterdam, the British Geological Society, University of Oxford, Italian National Research Council, Deakin University, and the University of Tokyo. Sri Mata Amritanandamayi Devi (Amma), a world-renowned humanitarian leader is the founder, Chancellor, and guiding light of Amrita Vishwa Vidyapeetham. Amma’s words, thoughts, and vision for education and research have shaped the university’s mission and vision.