INTRODUCTION

Communication is of paramount importance in today’s data driven society. Shopping at a

supermarket, asking for directions, event planning, or even national security depend on

reliable communication. To the average person, everyday routines such as dentist

appointments or having dinner with a friend would be difficult without the ability of

speaking in order to convey ideas.

Sign language is the only communication tool used by deaf people to communicate to each

other. However, normal people do not understand sign language, and this will create a large

communication barrier between deaf people and normal people. In addition, the sign

language is also not easy to learn due to its natural differences in sentence structure and

grammar. Therefore, there is a need to develop a system which can help in translating the

sign language into text and voice to ensure the effective communication can be easily take

place in this community.

Speech and gestures are expressions, which are mostly used in communication between

human beings Getting the data is the first step. The second step is that recognizing the sign

or gesture once it has been captured is much more challenging, especially in a continuous

stream. In fact currently, this is the focus of the research. The objective of this paper is to

design a simple embedded system based communicating device for deaf and dumb people.

In our day to day life most of the task we carryout involves speaking and hearing. The deaf

and dumb or paralyzed people have difficulty in communicating with others who cannot

understand sign language and miss-interpreters. In this paper, we designed a simple

embedded system based device for solving this problem. So, that we come on conclusion to

make a simple prototype by taking some of those gesture and convert it into audio and visual

form so that they can understand by everyone. For that we are making use of Arduino Nano

Board as Atmega 328 Controller board to interface all of the sensors and actuators.
LITERATURE SURVEY

According to Kumar, Gurjar and Singh , the glove has four flex sensors each sits on each

finger. The microcontroller consistently checks the bowing of flex sensor. At the point when

the signal of the letters make particular word based on the sequence appeared in the LCD.

The glove includes a few contact sensors, which help in recognizing couple of comparable

motions like of "U" and "V". The precision of each flex sensor is constrained past a specific

point. Smaller hands will bring about a bigger level of twist.

Lin and Villalba demonstrated calculation to make an interpretation of gesture based

communication into communicated in English. Each individual's hand is a novel size and

shape, and we intended to make a gadget that could give dependable interpretations paying

little mind to those distinctions. Our gadget utilizes five Spectra Symbol Flex-Sensors that

we use to evaluate how much each finger is bowed, and the MPU-6050 (a three-hub

accelerometer and whirligig) can recognize the introduction and rotational development of

the hand. These sensors are perused, arrived at the midpoint of, and orchestrated into

bundles utilizing an ATmega1284p microcontroller.

According to the journal this framework depicts talk capable hand glove framework which

goes for interpretation of gesture based communication to dissect content info and voice.

This framework comprises of a discussion capable glove that can be worn by a hard of

hearing/moronic individual to encourage the correspondence progressively with other

individuals. The framework deciphers the hand finger movement to relating letters utilizing

Contact switch sensors and an Arduino Board. Our primary objective is to distinguish 26

letters in order and show message on the LCD. Once the content is gotten on the LCD then
content to discourse change operation is completed lastly a voice yield is acquired. Further,

the content pick up can likewise be seen on a LCD or any convenient hand held gadget.

Electronic Speaking System for Speech Impaired People: Speak Up, In this paper Sign

Language is the only way of communication for speech impaired people. But general people

can't understand the sign language so it becomes difficult for a speech impaired person to

communicate with them. In this project an electronic speaking system was developed to ease

the communication process of speech impaired people. A glove was developed which

consists of five flex sensors. When a gesture is made with the glove, the change in resistance

of flex sensors fed into the Arduino Nano and specific prerecorded audio command for that

gesture is played from SD card through speaker and the text command for that gesture is

displayed on the LCD. There are four gestures that are designed for user input so that user

can play his/her chosen audio commands using those gestures. This device not only helps a

speech impaired person to communicate with a normal person via audio commands but also

helps him/her to communicate with a hearing impaired person by displaying the text

commands on the LCD.

Development of Device for Gesture To Speech Conversion For The Mute Community, The

mute community around the globe has a hard time communicating with the rest of the

world’s population. This communication gap is there because a dumb person uses sign

language which is not comprehensible by a normal person. This project mainly focuses on

removing the barrier of communication between the mute community and the people not

familiar with the concept of sign language so that the messages that a dumb person is trying

to relay is understandable to a person with no knowledge of sign language. The design of the
device is based on embedded systems. Flex sensors and microcontroller are the key

components

SIGN LANGUAGE TO SPEECH CONVERSION, in this paper Human beings interact with

each other to convey their ideas, thoughts, and experiences to the people around them. But

this is not the case for deaf-mute people. Sign language paves the way for deaf-mute people

to communicate. Through sign language, communication is possible for a deaf-mute person

without the means of acoustic sounds. The aim behind this work is to develop a system for

recognizing the sign language, which provides communication between people with speech

impairment and normal people, thereby reducing the communication gap between them.

Compared to other gestures (arm, face, head and body), hand gesture plays an important

role, as it expresses the user’s views in less time. In the current work flex sensor-based

gesture recognition module is developed to recognize English alphabets and few words and

a Text-to-Speech synthesizer based on HMM is built to convert the corresponding text.

An Arduino based Gesture Control System for Human-Computer Interface, In this paper

—The basic goal of Human Computer Interaction System is to improve the interaction

between users and computers by making the computer more receptive to user needs. Human

Computer Interaction with a personal computer is not just limited to keyboard and mouse

interaction today. Interaction between humans comes from different sensory modes like

gesture, speech, facial and body expressions. The paper presents a literature survey

conducted which provides an insight into the different methods that can be adopted and

implemented to achieve hand gesture recognition. It also helps in understanding the

advantages and disadvantages associated with the various techniques. Further, in this paper,

we have proposed a cost-effective gesture recognition system that translates the detected

gestures into actions. The system has two interfaces, the distance sensor interface and the
accelerometer interface. The distance sensor interface translates the gestures for actions such

as shuffling between different applications, volume control, scrolling, keyboard short-cuts

and so on; while the accelerometer interface realizes the typing part of the system.

OBJECTIVES

1. To design the handy and wearable system

2. To design the hand gesture based voice playback system

DESIGN METHODOLOGY

Materials The initial gloves were a woolen one. We will try to choose the fabric or material

as it is flexible and it fixes with the hand tightly. We attached rubber pads and plastic tubes

over the gloves to hold the touch sensor buttons. The touch sensor was used to detect the

touch with the ground pin when it touches the finger. The value of the sensor is then

monitored and as per the value the message will send to android app via bluetooth..

BLOCK DIAGRAM

These signal data we program in arduino to play specific messages like - I am thirsty, I am

hungry, I want to go washroom, I am in trouble all these messages send to mobile app via

bluetooth and google assist use to play the message in voice

HARDWARE USE

Arduino Nano ATmega328,Bluetooh , touch buttons sensor and Battery, LCD

Software Used : Arduino IDE compiler , Android APp

Arduino Nano

Arduino Nano circuit board with Arduino IDE is capable of reading analog or digital
input signals from different sensors, activating the motor, turning LED on/off and do
many other such activities. All functionalities are performed by sending a set of
instructions to the ATtmega328 main microcontroller, on the board via Arduino IDE.
The Arduino board also includes Power USB, Power (Barrel Jack), voltage
regulator, crystal oscillator, voltage pins (3.3v,5v,gnd,vin), A0 to A5 analog pins, icsp
pin, power led indicator, tx and rx leds, 14 digital input/output pins, Aref, and
Arduino reset
The Arduino Uno is a microcontroller board , based on the ATmega328.The Uno
board functioning is different from all other boards in that it does not use the FTDI
USB to serial driver chip. Instead, the Atmega328 is programmed as a USB to serial
converter. The ATmega328 is a low power CMOS 8 bit microcontroller based on the
AVR enhanced RISC architecture structure

The Arduino project started in 2003 as a program for students at the Interaction
Design Institute Ivrea in Ivrea, Italy,[2] aiming to provide a low-cost and easy way for
novices and professionals to create devices that interact with their environment
using sensors and actuators. Common examples of such devices intended for
beginner hobbyists include simple robots, thermostats and motion detectors.
The Arduino project was started at the Interaction Design Institute Ivrea (IDII) in
Ivrea, Italy. At that time, the students used a BASIC Stamp microcontroller at a cost
of $50, a considerable expense for many students. In 2003 Hernando Barragán
created the development platform Wiring as a Master's thesis project at IDII, under
the supervision of Massimo Banzi and Casey Reas. Casey Reas is known for
co-creating, with Ben Fry, the Processing development platform. The project goal
was to create simple, low cost tools for creating digital projects by non-engineers.
The Wiring platform consisted of a printed circuit board (PCB) with an ATmega168
microcontroller, an IDE based on Processing and library functions to easily program
the microcontroller.[4] In 2003, Massimo Banzi, with David Mellis, another IDII
student, and David Cuartielles, added support for the cheaper ATmega8
microcontroller to Wiring. But instead of continuing the work on Wiring, they forked
the project and renamed it Arduino.

The initial Arduino core team consisted of Massimo Banzi, David Cuartielles, Tom
Igoe, Gianluca Martino, and David Mellis, but Barragán was not invited to
participate.

Following the completion of the Wiring platform, lighter and less expensive versions
were distributed in the open-source community.It was estimated in mid-2011 that
over 300,000 official Arduinos had been commercially produced, and in 2013 that
700,000 official boards were in users' hands.
In October 2016, Federico Musto, Arduino's former CEO, secured a 50% ownership
of the company. In April 2017, Wired reported that Musto had "fabricated his
academic record.... On his company's website, personal LinkedIn accounts, and
even on Italian business documents, Musto was until recently listed as holding a
PhD from the Massachusetts Institute of Technology. In some cases, his biography
also claimed an MBA from New York University." Wired reported that neither
University had any record of Musto's attendance, and Musto later admitted in an
interview with Wired that he had never earned those degrees.

Around that same time, Massimo Banzi announced that the Arduino Foundation
would be "a new beginning for Arduino." But a year later, the Foundation still hasn't
been established, and the state of the project remains unclear.

The controversy surrounding Musto continued when, in July 2017, he reportedly

pulled many Open source licenses, schematics, and code from the Arduino website,
prompting scrutiny and outcry.

In October 2017, Arduino announced its partnership with ARM Holdings (ARM). The
announcement said, in part, "ARM recognized independence as a core value of
Arduino ... without any lock-in with the ARM architecture.” Arduino intends to
continue to work with all technology vendors and architectures

FLEX SENSOR

For understanding the use of FLEX SENSOR consider:

Case1: Where you want to check whether the surface of a device or thing is leveled or not.

Say you want a device to check whether a window or door is open or not. At that time a Flex

sensor could be used. The sensor could be fixed at door edge and when the door opens the
Flex sensor gets flexed. With the sensor being flexed its parameters changes which could be

designed to provide an alert.

Case2: Where you want to measure the FLEX or BENT or ANGLE change of any

instrument or device. The FLEX SENSOR internal resistance changes almost linearly with

its flex angle. So by sticking the sensor to the instrument we can have the flex angle in

electrical parameter of resistance.

HC-05 Bluetooth Module is an easy to use Bluetooth SPP (Serial Port Protocol)

module, designed for transparent wireless serial connection setup. Its

communication is via serial communication which makes an easy way to interface

with controller or PC. HC-05 Bluetooth module provides switching mode between

master and slave mode which means it able to use neither receiving nor

transmitting data. Specification:

Model: HC-05

Input Voltage: DC 5V

Communication Method: Serial Communication

Master and slave mode can be switched

The interfacing between Arduino UNO and PC via HC-05 Bluetooth Module. In this

example, the communication mode is used. *Note: For AT mode (use to change the

default setting or etc), please refer to this link Modify The HC-05 Bluetooth Module

Defaults Using AT Commands. Diagram below shows the hardware connection

between HC-05 Bluetooth Module and Arduino UNO. Besides Arduino, it may
interface with any microcontroller such as PIC and etc. VCC Arduino 5V GND

Arduino GND TXD Arduino Pin RX RXD Arduino Pin TX KEY Connect to the air

for communication mode

HC-05 module is an easy to use Bluetooth SPP (Serial Port Protocol) module,

designed for transparent wireless serial connection setup. Serial port Bluetooth

module is fully qualified Bluetooth V2.0+EDR (Enhanced Data Rate) 3Mbps

Modulation with complete 2.4GHz radio transceiver and baseband. It uses CSR

Bluecore 04-External single chip Bluetooth system with CMOS technology and with

AFH(Adaptive Frequency Hopping Feature). It has the footprint as small as

12.7mmx27mm. Hope it will simplify your overall design/development cycle.

LCD

LCD (Liquid Crystal Display) screen is an electronic display module and finds a wide

range of applications. A 16x2 LCD display is a very basic module that has 2

controllers with 16 Pins which is very commonly used in various devices and circuits.

These modules are preferred over seven segments and other multi-segment LEDs

as they are economical; easily programmable; have no limitation of displaying special

& even custom characters (unlike in seven segments), animations. The status of the

system is displayed using LCD.

A liquid-crystal display (LCD) is a flat-panel display or other electronically modulated

optical device that uses the light-modulating properties of liquid crystals. Liquid

crystals do not emit light directly, instead using a backlight or reflector to produce

images in color or monochrome.[1] LCDs are available to display arbitrary images (as

in a general-purpose computer display) or fixed images with low information content,

which can be displayed or hidden, such as preset words, digits, and seven-segment

displays, as in a digital clock. They use the same basic technology, except that
arbitrary images are made up of a large number of small pixels, while other displays

have larger elements. LCDs can either be normally on (positive) or off (negative),

depending on the polarizer arrangement. For example, a character positive LCD with

a backlight will have black lettering on a background that is the color of the backlight,

and a character negative LCD will have a black background with the letters being of

the same color as the backlight. Optical filters are added to white on blue LCDs to

give them their characteristic appearance.

A 16x2 LCD means it can display 16 characters per line and there are 2 such lines.

In this LCD each character is displayed in 5x7 pixel matrix. This LCD has two

registers, namely, Command and Data.

The command register stores the command instructions given to the LCD. A

command is an instruction given to LCD to do a predefined task like initializing it,

clearing its screen, setting the cursor position, controlling display etc. The data

register stores the data to be displayed on the LCD. The data is the ASCII value of

the character to be displayed on the LCD. Click to learn more about internal structure

of a LCD.
Circuit is designed on EasyEDA online tool.

EasyEDA Provides:

● Simple, Easier, Friendly, and Powerful general drawing capabilities

● Working Anywhere, Anytime, Any Devices
● Real-time Team Cooperation
● Sharing Online
● Thousands of open source projects
● Integrated PCB fabrication and Components purchase chain
● API provide
● Script support
● Schematic Capture
○ LTSpice-based Simulation
○ Spice models and subcircuits create
○ WaveForm viewer and data export(CSV)
○ Netlist export(Spice, Protel/Altium Designer, Pads, FreePCB)
○ Document export(PDF, PNG, SVG)
○ EasyEDA source file export(json)
○ Altium Designer format export
○ BOM export
○ Mutil-sheet schematics
○ Schematic module
○ Theme setting
○ Document recovery
● PCB Layout
○ Design Rules Checking(DRC)
○ Mutil-Layer, 6 copper layer supported
○ Document export(PDF, PNG, SVG)
○ EasyEDA source file export(json)
○ Altium Designer format export
○ BOM export
○ DXF export
○ Photo view
○ 3D View
○ Generate the fabrication file(Gerber)
○ Export Pick and Place file
○ Auto Router
○ PCB module
○ Document recovery
● Import
○ Altium/ProtelDXP ASCII Schematic/PCB
○ Eagle Schematic/PCB/Libraries
○ KiCAD Schematic/PCB/Libraries
○ DXF
● Libraries
○ More than 700,000 public Libraries(Symbol and Footprint)
○ Libraries management
○ Symbol/Subpart create and edit
○ Spice symbol/model create and edit
○ Libraries management
○ Footprint create and edit
easyEDA Editor has a clearly and friendly user interface. You can use its every function

very easily when you become familiar with EasyEDA.

Filter

Before using the Filter, you need to select what module you need in the left navigation

panel, and then you can find projects,files, parts and footprints quickly and easily just by

typing a few letters of the title. For example, if you want to find all files containing

“NE555” in the title, just type “555”, it is non-case-sensitive.

The Filter could only find projects, files and part titles and names. It does not support the

Descriptions and Content fields.

Click the X to clear the filter.

Navigation Panel
The Navigation panel is very important for EasyEDA: The part that you can find all your

projects, files, parts and footprints.

● Project

Here, You can find all of your projects that are private or shared with the public, or fork

from someone else’s.

Except System IC, these options have a content menu. For example, if you drop down to My

Projects and right click an item, you will get a tree menu like :

● EELib

EElib means EasyEDA Libraries, It provides lots of components completed with simulation
models, many of which have been developed for EasyEDA to make your simulation

experience easier.

● Design Manager

Design Manager, you can check each component and net easily, and it will provide

DRC(Design rule check) to help your design better.

● Libraries

Contains schematic symbols and PCB footprints for many available components and

projects and your own libs and modules will show up here.

● LCSC

If you want to buy components to finish your PCBA, you should try the LCSC module,

LCSC.com and EasyEDA are the same company.

EasyEDA partners with China’s largest electronic components online store by customers and

ordering quantity launch https://lcsc.com.

LCSC means Love Components? Save Cost! We suggest to our users to use LCSC parts to

design. Why?

1. Small Quantity & Global Shipping.

2. More Than 25,000 Kinds of Components.

3. All components are genuine.

4. It is easy to order co after design.

5. You can save 40% cost at least.

6. You can use our components’ symbols and package directly in EasyEDA editor.

● JLCPCB

JLCPCB.com, LCSC.com and EasyEDA are the same company group. https://jlcpcb.com

More than 200,000 customers worldwide trust JLC, 8000 + online orders per day,JLCPCB

(Shenzhen JIALICHUANG Electronic Technology Development Co.,Ltd.), is the largest

PCB prototype enterprise in China and a high-tech manufacturer specializing in quick PCB

prototype and small-batch production.Affordable, series quality boards fully manufactured

in China. Fully e-tested. Transparent pricing.

ToolBar

EasyEDA think that, by using the icon is better than the text menu for the toolbar.

EasyEDA’s toolbar can be reconfigured via Common Buttons Setting…

Click on a button to select it. Then you can toggle button visibility by clicking on

Show/Hide or by clicking on the tick space to the left of the button icon. You can change the

button position using Move up and Move Down.

Many of the buttons have been assigned hotkeys, so you can use those to replace the button

actions.

Preview Dialog

The Preview dialog will help you choose components and packages and can help you to

identify schematics and PCB layouts.

You can close or open this dialog via:

View > Preview Window .

● The Preview Dialog has a resizing handle in the bottom right corner.

● The Preview Dialog can’t be closed but double clicking on the top banner will roll up

the panel or you can click the top right corner — . Double clicking top banner again toggles

it back to the selected size.

● Clicking on the little pencil edit tool opens the item in the preview for editing.

Clicking on the location place tool in the top right corner of the preview dialog places the

item onto the canvas. If you try to place PCB footprint into a schematic it will not provide

any action and message.

Wiring Tools

Wiring Tools are document type sensitive: different document types have different tools.
Drawing Tools

To keep EasyEDA’s UI clean and sharp, the Wiring and Drawing tools palettes can be

resized horizontally, rolled up or hidden so if you want to focus on drawing, you can roll up

or hide the others to make more space and reduce the clutter.
Circuit Diagram
PCB Layout
WORKING

THe system is designed as per circuit diagram. When the sensor touches it send the values 0

on the respective pin,. When the value is 0 then one message string will sned to android app

via bluetooth. Bluetooth HC05 is initially connected to the android app. On the respective

touch the message is predefined and so it will send the data to app via bluetooth

The android app is connected with google text to speech and as per the text send to the app

the voice will active and we can hear the words we send

Similarly from the other app if any message types then it will display on LCD
The compilation process

The arduino code is actually just plain old c without all the header part (the includes and all).

when you press the 'compile' button, the IDE saves the current file as arduino.c in the

'lib/build' directory then it calls a makefile contained in the 'lib' directory.

This makefile copies arduino.c as prog.c into 'lib/tmp' adding 'wiringlite.inc' as the

beginning of it. this operation makes the arduino/wiring code into a proper c file (called

prog.c).

After this, it copies all the files in the 'core' directory into 'lib/tmp'. these files are the

implementation of the various arduino/wiring commands adding to these files adds

commands to the language

The core files are supported by pascal stang's procyon avr-lib that is contained in the

'lib/avrlib' directory

At this point the code contained in lib/tmp is ready to be compiled with the c compiler

contained in 'tools'. If the make operation is succesfull then you'll have prog.hex ready to be

downloaded into the processor.

NOTE:the next release will see each architecture (avr/pic/8051) to treated as a 'plug-in' to

the IDE so that the user can just select from a menu the microcontroller board to use and the

IDE will pick the right compilation sequence.

A program for Arduino hardware may be written in any programming language with

compilers that produce binary machine code for the target processor. Atmel provides a

development environment for their 8-bit AVR and 32-bit ARM Cortex-M based

microcontrollers: AVR Studio (older) and Atmel Studio (newer).[

IDE

The Arduino integrated development environment (IDE) is a cross-platform application (for

Windows, macOS, Linux) that is written in the programming language Java. It originated

from the IDE for the languages Processing and Wiring. It includes a code editor with

features such as text cutting and pasting, searching and replacing text, automatic indenting,

brace matching, and syntax highlighting, and provides simple one-click mechanisms to

compile and upload programs to an Arduino board. It also contains a message area, a text

console, a toolbar with buttons for common functions and a hierarchy of operation menus.

The source code for the IDE is released under the GNU General Public License, version 2.

The Arduino IDE supports the languages C and C++ using special rules of code structuring.

The Arduino IDE supplies a software library from the Wiring project, which provides many

common input and output procedures. User-written code only requires two basic functions,

for starting the sketch and the main program loop, that are compiled and linked with a

program stub main() into an executable cyclic executive program with the GNU toolchain,

also included with the IDE distribution. The Arduino IDE employs the program avrdude to
convert the executable code into a text file in hexadecimal encoding that is loaded into the

Arduino board by a loader program in the board's firmware.

Sketch

A sketch is a program written with the Arduino IDE. Sketches are saved on the development

computer as text files with the file extension .ino. Arduino Software (IDE) pre-1.0 saved

sketches with the extension .pde.

A minimal Arduino C/C++ program consist of only two functions:

setup(): This function is called once when a sketch starts after power-up or reset. It is used to

initialize variables, input and output pin modes, and other libraries needed in the sketch.

loop(): After setup() function exits (ends), the loop() function is executed repeatedly in the

main program. It controls the board until the board is powered off or is reset.
PROJECT PIC
CONCLUSION & FUTURE SCOPE

Deaf-mute people face difficulty in communicating with normal people who do not

understand sign language. To overcome this barrier this work proposes a sensor-based

system for deaf-mute people using glove technology. It requires fewer components such as

flex sensor, Arduino and accelerometer and hence its cost is low compared to vision based

gesture recognition systems. In this system the deaf-mute people wear the gloves (with the

resistors and sensors attached to it) to perform hand gestures. First the system will convert

the gesture to the corresponding text and then the speech is synthesized for the

corresponding text by using the text-to speech app. The system consumes very low power

and it is portable. The sensor glove design along with the tactile sensor helps in reducing the

ambiguity in gestures and shows improved accuracy. This project can further be developed

to convert words, phrases and simple sentences by concatenating the alphabets.

REFERENCES

[1] Kiran R, “Digital Vocalizer System for Speech and Hearing Impaired”, International Journal of

Advanced Research in Computer and Communication Engineering Vol. 4, Issue 5, May 2015

[2] Abhijeet Sondhi, Paresh Kasa, “Interacting Device for Deaf & Dumb using Atmega 328

Processor” International Journal Of Engineering And Computer Science ISSN: 2319-7242 Volume 4

Issue 9 Sep 2015

[3] Nisha Kawale , Pradnya Kaspate , “Implementation Paper on Sign Language Using Flex Sensor”,

International Journal of Innovative Research in Computer and Communication Engineering 2018

[4] Komal Vede, Priyanka Vanjare, “Sign to Speech Converter Gloves for Deaf and Dumb People”,

International Journal of Innovative Research in Computer and Communication Engineering 2018

[6] Muhammad Ali Mazidi and Janice GillispieMazidi, “The 8051 Microcontroller and Embedded

Systems”,

[7] Pearson Education. “Wireless accident information using GPS AND GSM” September 15,

2015,

[8] Research Journal of Applied Sciences, Engineering and Technology, Maxwell Scientific

Organization, 2015.