Social Network Framework for Deaf and Blind People

based on Cloud Computing

Social Network Framework for Deaf and Blind People

based on Cloud Computing

Tarek Gaber Dept. of Computer Science,

Faculty of Computers and Informatics,

Suez Canal University

Member of the Scientific Research Group in Egypt

http://www.egyptscience.net

Fedcsis2013- Krakow- Poland. September 08 – September 11, 2013

2

AgendaAgenda

• Introduction

• Problem definition

• Aim

• Proposed framework• Blind mode

• Deaf mode

• SoNetDBlue mode

• Problem challenges•Arabic Sign Language

•Security Issues

• Conclusion and future works

Fedcsis2013- Krakow- Poland. September 08 – September 11, 2013

3

Around 285 million people

around the world are visually

impaired

Around 360 million of the

worldwide population have

disabling hearing loss

Lack of awareness of the blind/deaf

needs could lead to discrimination

in a single community.

IntroductionIntroduction

4

About 6% of the total Egyptian

population are visually impaired

Nearly 3 million people in Egypt

suffer from hearing impairment,

Few IT projects attempting to

help disabled people to develop

their social skills and to integrate

in the community:

ViZWiZ and Disablitinet.com

Introduction Cont.Introduction Cont.

5

Problem Definition Problem Definition

According to our knowledge, no

proposed system for connecting

blind and deaf people within direct

Arabic language-based

conversations.

Providing these people with a way

allowing them to seamlessly

communicate with each other and

with normal people is of a great the

value to the entire community

6

Can Modern Technologies Help?Can Modern Technologies Help?

7

Social Networks for Blind PeopleSocial Networks for Blind People

Similar to famous and regular social networks.

Constructed to facilitate the interaction and communication among the subscribers.

Special features including audio and media contents.

Examples:

• http://www.disabilinet.com/

• http://theblinduniverse.com/• http://

www.disabledcommunity.net/

Disadvantages: Isolation and sociological problems.

8

Existed Mobile App. for Blind UsersExisted Mobile App. for Blind Users

Mobile device is a portable eye that describes and recognizes objects or people who are naturally invisible for them.

VizWiz: Crowd sourcing (e.g. employees) and Mobile Q&A.

A number of employees (i.e. crowdsourcing) answer the audio questions related to a photo taken by their mobiles (i.e. Mobile Q&A).

VizWiz Social: Free, Social (friends) and automated object recognition.

9

Weakness of Current Mobile app.Weakness of Current Mobile app.

VizWiz: • It is not a free service

• Does not support any automated process for recognizing the captured photos.

VizWiz Social:• Picture obscurity and the inaccurate results

received

• Delay to receive a result, up to a full day

• Privacy issues when sharing their data and photos online

10

Other Solutions Other Solutions

Other work integrate special glasses with mobile devices through Bluetooth technology to allow blind users to identify objects and navigate directions

This integration enhances the accuracy of the obtained results.

Weaknesses

• Only focused on the navigation problem

11

Weaknesses in Existed SolutionsWeaknesses in Existed Solutions

• Overlooking Arabic language

• Communicating deaf with blind has not addressed in any solutions

• Interactions emotions and feelings of both the blinds and their peers have not been considered

12

Design and develop an

IT system to support a

seamless

communication among

blind, deaf and normal

people in Egypt that

should help them to gain

better social skills for a

more successful life.

AimAim

13

Our SolutionOur Solution

Clouding Computing • To speed up the process of object recognition.• To obtain an accurate results within the matching

process.

Mobile Device• Android and iOS based devices • Come with integrated speech recognition and

text-to-speech engines

Integrated Camera • Time-of-Light Camera: a new technology used in

real-time three-dimensional imaging• To be worn by blind users (an eye-level

placement) that will help them to easily capture context-relevant pictures

14

Proposed FrameworkProposed Framework

Cloud Object / Arabic

Text Recognition

Emotion Detection

Sign language Audio

Audio Sign language

frames

Blind mode

Deaf mode

Seamless communication

Image / Video / Audio Posts

Notifications, GPS coordinatesPhone numbers

Time of Flight Camera

Images /Audio

Images /Audio

15

Blind Mode

Blind Mode Blind Mode

Time-of-flight camera

(glasses)

Thin Client(Mobile)

Cloud Main Service

Cloud-Emotion Detection Sign Language --> Text

Frames (Pictures)

Filtered Frames

face detected

Sign language detected

Identify Emotion

Message in TextText Audio

Blind / non Blind

User

16

Deaf ModeDeaf Mode

Thin Client(Mobile)

Cloud Main Service

Word - Sign Language

Audio

Word / seq

Filter communicator speech

break into word segements

sign language

Word - Sign Language

Word / seqsign language

..

word synchronization

Sign langauge

Animate avatar

Deaf User

17

SoNetBDBlue WebsiteSoNetBDBlue Website

Disabled Non-disabled

identify objectIdentify Arabic

Text

Post Text

Post Audio

Attach Picture / Video

<<extend>>

<<extend>>

<<extend>>

Find / Call nearest Friend

• SoNetDBlue is an architecture to connect deaf and blind people with their cycle of friends (with or without disabilities) anywhere and at any time.

• Use case • Post text/audio

messages• Find/call nearest

friends• Identify objects/Arabic

text

18

Dealing with the Arabic language

American Sign Language has accomplished an accuracy of 98% while the best result of the ArSL is 90.55 %

ArSL does not have a documentation system which could be utilized while building a translation corpus.

It is not hand/finger spelling of the Arabic alphabet.

Problem ChallengesProblem Challenges

19

Security issues Privacy issues

Our system aims to help a blind user to get a specific location using GPS which threats the user’s privacy

Captured videos would disclose lots of information about the places visited

Authentication Password for sighted people is not convenient way to

protect accessing mobile Blind needs more convenient method to authenticate

themselves to their mobile

Problem ChallengesProblem Challenges

20

• A Mobile-Cloud framework is presented to help people with visual impairments to gain better social skills that are important for healthy and successful life.

• The framework is structured to bridge the communication gap between the blind and deaf persons

• The paper showed how several technologies and methodologies including social networks which can be integrated to recognize all possible obstacles and persons to assist the blind people to feeling and visualizing the surroundings.

ConclusionConclusion

21

Thanks and Acknowledgement

Thanks and Acknowledgement

http://www.egyptscience.net

Authors Mahmoud ElGayyar, Hany ElYamany, Tarek Gaber,

and Aboul Ella Hassanien

Fedcsis2013- Krakow- Poland. September 08 – September 11, 2013