Post on 22-Nov-2014
description
transcript
Sign Language
Recognition
Neural NetworkUsing
PRESENTED BYBikash Chandra Karmokar (0707019)
&Md. Kibria Siddiquee (0707024)
SUPERVISED BY:Dr. Kazi Md. Rokibul AlamAssociate professor,CSE,KUET,khulna,Bangladesh.
An intelligent approach to recognize sign language for deaf and dumb people of the world
Dedication
First of all we would like to remember the deaf and dumb people of the world for whom we tried to develop a Sign language Recognizer (SLR).
Outline
• Sign language• SLR & its necessity• Helping process of SLR• Working procedure of SLR• Block Diagram of SLR• BP training time & graph• Recognition accuracy • Limitations• Future plan• Papers
What is Sign Language ??
Communicating language used primarily by deaf people.
Uses different medium such as hands, face, or eyes rather than vocal tract or ears for communication purpose.
Communication using sign language
What is SLR ??
Sign language recognizer (SLR) is a tool for recognizing sign language of deaf and dumb people of the world.
Why we need SLR ??
Problems:
• About 2 million people are deaf in our world• They are deprived from various social
activities• They are under-estimated to our society• Communication problem
Continued..
Proposed Solution: SLR
SLR can be a desirable interpreter which can help both the community general and deaf.
How SLR help ?? An Example.....
Suppose a deaf customer went to a shop. She is trying to express her demands to the shopkeeper using sign language but the shopkeeper can not understand her demands.
??
shopkeeper Deaf customer
Continued..
SLR brings the solution for this problem>>
• SLR capture signs shown by deaf man• Convert the signs to text• This text is shown to shopkeeper
Now the shopkeeper can understand the deaf man’s demands
Sign to text conversion using SLR
Sign Converted text
Continued..
Text to sign conversion
When shopkeeper replied to the deaf customer SLR • Convert text to sign• This sign is shown to the deaf customer
Continued..
Now the deaf man can understand the shopkeeper’s speeches
Text to sign conversion using SLR
Continued..
Shopkeeper speech/text Sign
Text to Sign Conversion
Process
Separate each letters
Showing sign
Collecting Text• Text from the writing
place are collected
Continued..
Separate each letter
Showing sign
Collecting Text• From the sentences
each letter are separated and put into an array.
Continued..
Separate each letters
Showing sign
Collecting Text• For each letter a
predefined sign image are shown.
Sign to Text Conversion
How SLR works ??
Normalization
Sign recognition
Sign to text conversion
Image processing & sign detection
Continued..
Image processing & sign detection
Normalization
Sign recognition
Sign to text conversion
• Image capture
• Skin color detection
Continued..
Image processing & sign detection
Normalization
Sign recognition
Sign to text conversion
• Hand gesture detection
• Sign detection
Continued..
Image processing & sign detection
Normalization
Sign recognition
Sign to text conversion
• Reducing image size
200x200 30x33
Continued..
Image processing & sign detection
Normalization
Sign recognition
Sign to text conversion
• Backpropagation implementation
Continued..
Image processing & sign detection
Normalization
Sign recognition
Sign to text conversion
• Converting sign language to Bengali or English text
কv
Block diagram of the SLR
BP Training
Figure: Training error versus number of iteration
Training time for BP
Input size of pixelTraining
Time (min)
30*33 1.545*48 2.860*63 3.7
We have used 50 signs as training input where each sign has 5 samples that make 50 x 5 = 250 samples.
Recognition Accuracy
No. of inputAvg.
Accuracy (%)
10 74
20 65
30 60
Limitations
• Due to brightness and contrast sometimes webcam can hardly detect the expected skin color.
• Because of the similarity of tracking environment background color and skin color the SLR gets unexpected pixels.
• Due to almost similar pattern its become hard to take decision.
Continued..
Continued..
Future Plan
• Real time word recognition of ASL & BSL• Implementing neural network Ensembles • Implementing Genetic algorithm for sign recognition
Required Tools
• Visual studio 2008• XML• Avro Keyboard installed• Aforge .Net• Open CV• Webcam
References
http://www.lifeprint.com/ http://engineeringproject2011.webs.com/ www.c-sharpcorner.com www.codeproject.com http://en.wikipedia.org www.aforgenet.com
Published papers
1. Bikash Chandra Karmokar, Kazi Md. RokibulAlam, Md. KibriaSiddiquee, “An intelligent approach to recognize touchless written Bengali characters”, International Conference on Informatics, Electronics & Vision (ICIEV), ISSN: 2226-2105, 2012, Dhaka, Bangladesh
2. Kazi Md. RokibulAlam, Bikash Chandra Karmokar, Md. KibriaSiddiquee, “A comparison of constructive and pruning algorithms to design neural networks”, Indian Journal of Computer Science and Engineering (IJCSE), ISSN : 0976-5166 Vol. 2 No. 3 Jun-Jul 2011