See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/304802688

Smart Glasses for the Visually Impaired People

Conference Paper in Lecture Notes in Computer Science · July 2016

DOI: 10.1007/978-3-319-41267-2_82

CITATIONS READS

21 45,258

2 authors:

Maghfirah Ali Tong Boon Tang

International Islamic University Malaysia Universiti Teknologi PETRONAS, Bandar Seri Iskandar, Malaysia
1 PUBLICATION 21 CITATIONS 212 PUBLICATIONS 3,826 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Tong Boon Tang on 20 October 2017.

The user has requested enhancement of the downloaded file.

 Smart Glasses for the Visually Impaired People

Esra Ali Hassan and Tong Boon Tang(&)

Department of Electrical and Electronic Engineering,

Universiti Teknologi PETRONAS, 32610 Bandar Seri Iskandar, Perak, Malaysia
[email protected], [email protected]

Abstract. People with visual impairment face various problems in their daily
life as the modern assistive devices are often not meeting the consumer
requirements in term of price and level of assistance. This paper presents a new
design of assistive smart glasses for visually impaired students. The objective is
to assist in multiple daily tasks using the advantage of wearable design format.
As a proof of concept, this paper only presents one example application, i.e. text
recognition technology that can help reading from hardcopy materials. The
building cost is kept low by using single board computer raspberry pi 2 as the
heart of processing and the raspberry pi 2 camera for image capturing. Exper-
iment results demonstrate that the prototype is working as intended.

Keywords: Visually impaired ! Text recognition ! OCR ! Raspberry pi 2

1 Introduction

The number of visually impaired people is growing over the past decades. As reported
by the world health organization (WHO), about 285 million people worldwide are
estimated to be visually impaired [1]. However, until now many schools and jobs
cannot accommodate them mainly due to lack of assistive technologies and economic
barriers [2]. As a result, 90 % of them still live in low level of income [1]. Even when
the new aids or technologies become available, they are either too expensive ($3000
and above), or affordable ($200) but with single or limited task functions only [3].
Among all assistive devices, wearable devices are found to be the most useful
because they are hand free or require minimum use of hands [4]. The most popular type
is head mounted device. Their main advantage is that the device points naturally at the
viewing direction, thus eliminates the need of additional direction instructions, unlike
other devices [5]. This paper presents a new design of smart glasses that can provide
assistance in multiple tasks while maintaining at a low building cost. The design uses
the new raspberry pi 2 single board computer, a camera, and an earpiece to convey
information to the user. Due to page limit, we only demonstrate reading task only. The
experiment results and how additional tasks may be added are discussed.

© Springer International Publishing Switzerland 2016

K. Miesenberger et al. (Eds.): ICCHP 2016, Part II, LNCS 9759, pp. 579–582, 2016.
DOI: 10.1007/978-3-319-41267-2_82
[email protected]
580 E. Ali Hassan and T.B. Tang

2 Related Works

The most popular reading device is Braille reader which can read and/or write using an
arrangement of dots to form different letters [4]. Another device is the audio book
which read books or newspaper saved in audio format by certain suppliers [6]. Screen
reader and e-book readers read digital content from computer screen, and convert the
text to an audio format using a text-to-speech synthesizer [6].
Eyewear devices are the most recent technology. OrCam is a commercially released
glasses that use an embedded computer with a gesture (a point of finger) recognition
system to perform different tasks includes reading and convey them to the user in an
audio format [7]. Esight is another eyewear technology for people with low vision. It
captures and processes live scenes, and displays them back on a specialized screen in
front of the user’s eyes [8] (Table 1).

Table 1. Comparison between proposed design and available assistive devices

Device Functionality & No. of Price Remarks
tasks
Braille Reading and writing (two $1000– Only support tactile materials
readers tasks) $3000
[4]
Audio Reading (one task) $25 per Only for certain available books
books month
[6]
Screen Reading digital format (one $150– Only for digital content
reader task) $1000
[6]
OrCam [7] Multitask - includes reading $2,500 Non affordable price
Esight [8] Re-display the live scenes $15,000 Non affordable price and only for
for visually impaired to people with low vision, but not
see (multitask) total blindness
Proposed Proposed as reading. Has $100– Limited by performance and
design the capacity to be $150 accuracy of hardware
multitask

3 Proposed Smart Glasses Design and Implementation

Smart eyewear design depends mainly on the processing unit, which is the raspberry pi
2, in this case. The main hardware is a Linux based ARM processor that accepts a
micro SD card and thus allows us to increase the number of task functions as we wish.
A raspberry pi camera was used for image acquisition. It was connected to the rasp-
berry pi using a flex cable, and was fixed on the top middle of the glasses for optimal
image capturing. The raspberry pi has an audio port which connects to earpiece. The
raspberry pi GPIO port was configured to receive input from push button switches. To

[email protected]
 Smart Glasses for the Visually Impaired People 581

identify the text easier, the reading material is placed within a customly-designed frame
with red borders.
The general principle of operation for such glasses is by giving instructions via
switches and listening to the output through an earpiece. Similarly in this case, the user
starts the task mode by a push of the button. For text recognition mode, the glasses will
first confirm if the text area is correctly positioned and readable. Otherwise, it will ask
the user to change the orientation of the material. After confirmation, the view is
processed in real time to get the image sent to an optical character recognition
(OCR) software for text extraction and subsequently forwarded to a text-to-speech
synthesizer. The text is then read through the audio output port.
The image processing adopted in this work were implemented by using Simulink
(Mathworks, Natick, MA). In the reading mode, the main challenge is the image
quality, text position and orientation in the image. Therefore, the first step is to detect
the red borders and the frame orientation. To simplify subsequent image processing, we
propose an indicator to inform user if the image is skewed significantly or part of the
frame is cropped. Once the text area is localized and cropped, image is enhanced by
noise filtering, contrast enhancement (histogram matching technique) and morpho-
logical operations. Tesseract OCR engine [9] is used in the last step to extract the text
before converting into audio output.

4 Results

Several sample texts were prepared and tested. Figure 1 shows an example reading text
and the experiment results with the proposed smart glasses. Admittedly the text is
relatively simple, but it proves the basic concept of our design. Future works include

Fig. 1. Text recognition: (a) original image, (b) image after enhancement, (c) Tesseract OCR
result of reading material. (Color figure online)

[email protected]
 582 E. Ali Hassan and T.B. Tang

implementing additional image processing and more robust text recognition technique
to compensate for the low quality images from the raspberry pi camera.

5 Conclusion and Future Work

This paper presents a new concept of smart glasses designed for visually impaired
people using low cost single board computer raspberry pi 2 and its camera. For the
demonstration purpose, the glasses are designed to perform text recognition. The
system capability however can be easily extended to multiple tasks by adding more
models to the core program, albeit restricted by the size of the raspberry pi SD card.
Each model represents a specific task or mode. The user can have the desired task run
independently from the other tasks. The system design, working mechanism and
principles were discussed along with some experiment results. This new concept is
expected to improve the visually impaired students’ lives despite their economic sit-
uations. Immediate future work includes assessing the user-friendliness and optimizing
the power management of the computing unit.

References
1. WHO|Visual impairment and blindness. WHO, 7 April 1948. http://www.who.int/
mediacentre/factsheets/fs282/en/. Accessed Oct 2015
2. Unisco. Modern Stage of SNE Development: Implementation of Inclusive Education. In: Icts
in Education for People with Special Needs, Moscow, Kedrova: Institute For Information
Technologies in Education UNESCO, pp. 12–14 (2006)
3. Low vision assistance. EnableMart (1957). https://www.enablemart.com/vision/low-vision-
assistance. Accessed Oct 2015
4. Velázquez, R.: Wearable assistive devices for the blind. In: Lay-Ekuakille, A.,
Mukhopadhyay, S.C. (eds.) Wearable and Autonomous Systems. LNEE, vol. 75, pp. 331–
349. Springer, Heidelberg (2010)
5. Jafri, R., Ali, S.A.: Exploring the potential of eyewear-based wearable display devices for use
by the visually impaired. In: International Conference on User Science and Engineering, Shah
Alam, 2–5 September 2014
6. The Macular Degeneration Foundation, Low Vision Aids & Technology, Sydney, Australia:
The Macular Degeneration Foundation, July 2012
7. OrCam, OrCam. http://www.orcam.com. Accessed Dec 2015
8. Esight. http://esighteyewear.com/
9. Smith, R.: An overview of the Tesseract OCR engine. In: ICDAR 2007, pp. 629–633 (2007).
doi:10.1109/ICDAR.2007.56

[email protected]

View publication stats