1

Under Supervision of: Presented By:Mr. Mayank Tripathi Vibek Kumar MauryaLecturer Roll. No. 1216630049 Electronics Department EL – Final YearDr. A.I.T.H. Kanpur EL- Final Year

Sign Language to Multi Language Speech Converter

Dr. Ambedkar Institute of Technology for Handicapped Kanpur U.P -208024

2

"Disability will affect the lives of everyone at some point in their life, it is time society changed to acknowledge this"  Disabled World

3

Introduction

The word “Disabled" is defined as having a physical or mental disability : unable to perform one or more natural activities (such as walking or seeing) because of illness, injury, etc.

Figure. 1

4

Facts Over a billion people, around 15% of the world's population, have some form of disability.

Between 110 million and 190 million adults worldwide have significant difficulties in functioning.

Physical disability is defined as when a person's physical functioning, mobility, dexterity, or stamina is limited.

Rates of disability are increasing due to population aging and increases in chronic health conditions - among other causes.

5

Assistive Technology

Assistive Technology (AT) includes a range of technologies, which enable people to build on their abilities and participate as fully as possible at home, school, work and in their community.

AT is used to describe both the products and

the services for people with special needs. Figure. 2

6

Literature Search Arokia Raj et al in [1]. This paper explained the nature and difficulties

associated with building text processing components of TTS systems in Indian languages. They discussed the relevancy of font identification and font-to-Akshara conversion and proposed a TF-IDF based approach for font-identification and a novel approach of conversion from font-to-Akshara using the shapes of the glyphs and the assimilation rules was explained.

Kaveri Kamble et al in [2] Proposed a System Translation of Text to Speech Conversion for Hindi Language. A database has been created from the various domain words and syllables. The given text is analyzed and syllabified based on the syllable segmentation rules.

7

Literature Search Hay Mar Htun, Theingi Zin et al in [5] Proposed a system Text

To Speech Conversion Using Different Speech Synthesis. Develop a TTS model for numbers and words in MATLAB and discuss the speech generating technology concatenative synthesis and is operations and types Unit Selection Synthesis, Diphone Synthesis and Domain Synthesis.

Francesc Al´ıas, Joan Claudi Socor´o et al in [6] discuss a multi-domain text-to-speech (MD-TTS) synthesis strategy for generating speech among different domains and so increasing the flexibility of high quality TTS systems

8

Literature Search Mithileysh Sathiyanarayanan et al in [7] proposed an

unmanned ground vehicle (UGV) capable of being controlled using hand gestures.

R.Sandanalakshmi, P.Abinaya viji et al in[8] proposed that speech to text converter consist of two stages front end analysis and pattern recognisation.

Moniruzzaman bhuiyam and Rich Picking et al in[13] Gesture controlled user interface and identify trends in technology ,application and usability.

9

Inferences Drawn Out of LiteratureReview

A database has been created from the various domain words and Syllables. The given text is analyzed and syllabified based on segmentation rules. The and desired speech is produced by Concatenative speech Synthesis approach, and a system Develop For visually impaired person.

Text to speech system for Hindi using English language is able to speak a loud Hindi word which is typed in English.

Gesture interface is an important technology that support the non-verbal communication language to verbal communication.

10

Scope of the Proposed system To Help the Visually Impaired, Vocally Disabled Aged Persons Patient admitted in Hospital

Figure. 3

11

Proposed System

Technology used For Proposed Project

Gesture Recognition Technology Text To Speech Technology

A Device that convert sign language in multiple speech language.

12

Block Diagrams Proposed System

Micro- ControllerAtmega 16

Flex Sensor

Compara-tor Encoder RF Trans-

mitter

Figure. 4 Block Diagram Of Transmitter

13

Block Diagrams Proposed System

Receiver Decoder

Display

TTS processor

Micro- ControllerAtmega 16 Audio

amplifier

Speaker

Figure. 5 Block Diagram of Receiver

14

Gesture Recognition Technology

Gesture Recognition is the process of understanding and interpreting meaningful movements of the hands, arms, face, or sometimes head.

Gesture Recognition is to recognize specific human Gestures and process them to control device or convey some meaning full data.

15

Text To speech Technology Text To Speech (TTS) synthesis is to convert input text into

intelligible and natural sound speech. Text Analysis Phonatic Analysis Prosodic Analysis

Block Diagram of Text to Speech

Text analysis Phonetic conversion

Prosodic phrasing

Speech database

Unit selection synthesis

Concatenate speech

Input text

Speech

16Figure. 6 Block Diagram of Text to Speech

17

Flow chart for Speech to Text Start

Input number

Compare and extract number

Compare and concatenate number

Output speech

End

Lexicon of digit

Recorded sounds

Figure. 7 Block Diagram of Text to Speech

18

Components list

Flex sensors Microcontroller Comparator RF Encoder/Decoder Transmitter/Receiver Text To Speech Processor Display Speaker Audio Amplifier

19

Flex Sensor

Figure. 8 Flex Sensor

Angle Displacement Measurement Height: 0.43mm (0.017") Flat Resistance: 10K Ohms Resistance Tolerance: ±30% Bend Resistance Range: 60K to 110K Ohms

Key Features

20

Micro-Controller

Figure 9 Micro-Controller Atmega 16

8-bit Microcontroller with 16K Bytes In-System Programmable Flash Memory

131 Powerful Instructions Up to 16 MIPS Throughput at 16

MHz

Key Features

21

Encoder

Figure . 10 Encoder

22

Decoder

Figure. 11 Decoder

23

RF Transmitter and Receiver

Figure. 12 RF Transmitter and Receiver

Range in open space(Standard Conditions) : 100 Meters

RX Receiver Frequency : 433 MHz RX Operating Voltage : 5V TX Frequency Range : 433.92 MHz TX Supply Voltage : 3V ~ 6V

Key Features

24

Text To Speech Processor

Figure . 13 Text To Speech Processor

The TTS256 is an 8-bit microprocessor programmed with 600 letter-to-sound rules for the automatic, real-time translation of English text to allophone addresses.

25

Speaker

Figure. 14 Speaker

Small SizePower rating: 0.5WImpedance: 8 ohm

Key Features

26

Liquid Crystal Display (16x2 LCD)

Figure. 15 Display

27

Circuit Diagram of Transmitter

28

Circuit Diagram of Receiver

29

Software's• Proteus 8.1• Win AVR• Atmel Studio 6• MATLAB

30

Conclusions

Assistive Technology service that directly assists a seekers with a disability in the selection, acquisition, or use of an assistive technology device. Our device Help the Visually Impaired, Vocally and Aged Persons at every steps of thier lifes. Device is proposed for multi language communication system like Hindi, English or any other language with sign language.

Future Potential

31

Advanced Speech

Touch

Gesture

Feeling/sence

Figure. 16

32

Estimated Cost for The Proposed Work

The estimated cost for the proposed work is in between 8000 – 10000 Rs.

33

PERT CHART

34

References

1) Arokia Raj, Tanuja Sarkar, Satish Chandra Pammi, Santhosh Yuvaraj, Mohit Bansal, Kishore Prahallad, ‘Text Processing for Text-to-Speech Systems in Indian Languages’ .

2) Kaveri Kamble and Ramesh Kagalkar, ‘Translation of Text to Speech Conversion for Hindi Language’ .

3) Gargi Rajadhyaksha, Siddharth Mody and Sneha Venkateswar, ‘Text to Speech Converter’ .

4) Jaka Sodnik and Sašo Tomažičdescribes, ‘Spatial Speaker: 3D Java Text-to-Speech Converter’ .

5) Hay Mar Htun, Theingi Zin and Hla Myo Tun, ‘Text To Speech Conversion Using Different Speech Synthesis’ .

35

References

8) Nabil Eid, ‘Innovation and technology for person with disabilities’ .

9) Marcelo M. Wanderley, ‘Gesture control of music’ .10) Padmavathi. S, Saipreethy M.S. and Valliammai. V , ‘Indian sign

language character recognition using neural networks’ .11) Noor Adnan ibraheem, RafiqulZaman Khan, ‘Survey on various

Gesture Recognition technologies and techniques’ .12) Sangam P. Borkar and Prof. S.P. Patil, ‘Text to speech system for

konkani (goan) language’ .13) Moniruzzaman bhuiyam and Rich Picking,‘gesture controlled

user interface and identify trends in technology ,application and usability. recognition process was complex

36

Thank you