Exploiting Context-awareness and Social Interaction to Provide Help in Large-scale Environments

transcript

Exploiting Context-awareness and Social Interaction to Provide Help in Large–scale Environments

25 April 2012

Nasim Mahmud

Advisor: Professor Dr. Karin Coninx

Co-advisor: Professor Dr. Kris Luyten

Exploiting Context-awareness and Social Interactionto Provide Help in Large–scale Environments

Why Help?

• Someone is unable to do what he wants to do

• Someone needs some information

• Someone needs guidance

Is this for…?

On my way to San Sebastian, Spain

Need More and More Precise Information

In San Sebastian, Spain

A person is browsing a map

Other people joined the search

Need more reliable information

A menu along with a dictionary

Need more interactive information

Motivation

• People need fine-grained or interactiveinformation

• People need reliable information

• Problems in asking someone for help:

– Who is willing or eligible to provide help

– People are often hesitant to ask strangers

– Finding someone in the vicinity

• How to find a suitable person who can help?

• How to exchange contextual information?

• How to select relevant contextual information and potential groups of help providers?

• How can persons with special need benefit from context-awareness and social computing?

• How can social and context-awareness improve data dissemination?

Context

What am I doing?

Who am I with?

What is possible?

Where am I?

What time is it?

How is the weather?

How is the … ?

What am I doing?

Who am I with?

What is possible?

Where am I?

What time is it?

How is the weather?

How is the … ?

Social-context

Is available…

Social-context

Is available…

Is willing to…

Social-context

Is available…

Is willing to…

Is knowledgeable…

Ubiquitous Help System (UHS)

• People seek

• Precise and fine-grained information

• Often from other person(s)

• From reliable source

– It utilizes

• External context (time, location)

• Internal context (willingness, ability)

• Social Network (FOAF)

Profiles and Preferences

Extract from my foaf profileExtract from other users’ foaf profile

Application logic

My preference Other users’ preferences

Constraints Constraints

Contextual variables

How Does it Work?

…has a question

Profile and preference matched

…has a question

Profile matched

How Does it Work?

…has a question

Profile matched

How Does it Work?

…has a question

Profile matched

…has a question

Profile matched

How Does it Work?

…has a question

Profile matched

…has a question

Profile matched

Client Structure

Query Structure

Distributed Search

SPARQL search

UHS Java Client

How to Exchange Information

• How to exchange contextualinformation

• How to exchange rich media

What am I doing?

Who am I with?

What is possible?

Where am I?

What time is it?

How is the weather?

How is the … ?

Do you like this toy?

Who Can Help with the Question?

• A friend

• A family member

• A colleague

• A familiar person

Related Work

• Search by using social networksFacebook, Facebook questions, Quora, Twitter etc.

• Mobile social Q&A

• Photo-based question and answer }Mobile Social Search

Photo-based Question Answer

Tom Yeh et al.(MM 2008)Community based

VizWiz

Jeffrey P. Bigham et al. (UIST 2010)Crowdsourcing based

Limitations of Existing Solutions

• Limited context-awareness

• Lacking social awareness

• Utilizes community and crowdsourcing

– Not suitable for a range of personal questions

– Not suitable where in-situ help is required

– Not interactive enough

Ubiquitous Help System-Next (UHS-Next)

UHS-Next System

A mobile system that provides

– Context-aware communication

– Media rich communication

– Usage of users’ personal social network

• Take a picture

• Ask a question

• Select a group

• Select contextual information

• Preview the question

• Send

• Take a picture

• Ask a question

• Select a group

• Send

• Take a picture

• Ask a question

• Select a group

• Send

• Take a picture

• Ask a question

• Select a group

• Send

• Take a picture

• Ask a question

• Select a group

• Send

• Take a picture

• Ask a question

• Select a group

• Send

User Test

User Test 1: Finding Help

Where are you?

What does it mean?

Results of User Test 1

• UHS-Next is simple to use

• Voice interaction for ‘spoken audio question’ is needed

• Inspiring result

User Test 2: Spontaneous Social Interaction

• Free use of UHS-Next in real life by

– Two users

– One actor

• For two days

– In office environment

– In daily life situations

Results of User Test 2

• Other use than seeking help

– Spontaneous social interaction

– Sharing cognitive load

– Sharing daily life experiences (Fun moment, “Whose office is this?”)

• Easily embedded in daily life

– Useful

– Easy to use

Remaining Difficulties

• Selecting right context

• Selecting right group of users

To solve these, we propose a mixed-initiative approach

Mixed-initiative Context Filteringand Group Selection Approach

• Our approach selects and prioritizes the contextual data for a question, based on the question content

• Helps to select a group of potential help providers

Mixed-initiative Approach

• Human internal context is subtle to measure by the available technologies

• A fully automated system requires to know all the variable about human-activity and external context

• To reflect that the user’s requirements are satisfied and make sure that the user is in control

Context Selection

• A broad range (e.g., urgency, time, location, weather conditions)

• Which contextual information is important? (e.g., time critical, quality critical)

• How to capture that information? (e.g., urgency, location, reliability)

• How to convey that information? (e.g., I am here (where ‘here’ is unknown to the user))

Ubiquitous Help System-Selection (UHS-Selection)

Asking question in a natural way

From Voice Question and from Sensors

Ubiquitous Help System for Context and Group Selection (UHS-Selection)

Main screen

Voice to Text conversion, user in the control

Voice to Text

Parsing the Text

Language Processing

• Utilize the WordNet dictionary

– A social network of words

– Synonyms, meaning and relevance

• Utilize Named Entity Recognition (NER)

– Structure data in XML

– Customized for the purpose

The UHS-Selection system sets priority to location

Result

Group Selection

• Based on the context priority list (output from the context selection algorithm)

• Current context (e.g., location, heading)

• Current task

• Next task

Group Selection: Visualization

Limitations and Workaround

• Need to know more information about the persons who can provide help (e.g., location)

• Social translucence provides the balance (Erickson et al. (2000))

We have applied the framework in particular application domain, for Persons with Dementia

And in the dynamic social network

Simulated vehicular network

Sharing awareness information in specific context of use – Persons with Dementia

– In the early stage of dementia, they can live their lives as usual, they can go:

– Shopping,

– Bird watching,

– Jogging,

– … …

– When dementia syndrome progresses, they need more attention, and targeted help/ more social and navigational help

Dementia

• Dementia is a term for a syndrome related to the loss of cognitive functions

• An acquired decline in memory and thinking (cognition) due to brain disease that results in significant impairment of personal, social or occupational function

General Needs

A person with dementia needs more independence in terms of :

– Spatial

– Temporal and

– Social

awareness

As the Dementia Syndrome Progresses

• It becomes an important cause of dependencies

• …Persons with dementia, are increasingly dependent on their social environment (likely to be less autonomous)

• In most of the cases, in the early form of dementia the caregiver is a family member(Schulz et al. 2010)

Scenario

(COMuICSer tool . Haesen, M. et al. 2009)

Ubiquitous Help System for Persons with Dementia (UHSd)

Relation Between the Models Used to Develop the System : Part 1/3

Dialog model

Application model

Relation Between the Models Used to Develop the System : Part 1/3

Dialog model

Application model

Relation Between the Models Used to Develop the System Part 2/3

Dialog model

Application model

Dialog model

Application model

Dialog model

Application model

System Overview of UHSd

One Example

To-do (Baker/Buy bread )

– Time (From 10:00 to 11:00)

– Location (Grote Markt Baker)

– Associated contact (Jane, Ilsa, Mark )

Resulting System (UHSd)-Navigation Panel

Resulting System(UHSd)-Communication Panel

Summary of Ubiquitous Help System for Persons with Dementia (UHSd)

• UHSd provides memory aid in terms of– Spatial

– Temporal

– Social awareness

• Provides context-aware support– Ensures (partly) gaining users autonomy

– Ensures feeling of connectedness

Lessons Learned

• We presented an interactive system and observed that applications for people with dementia can be created by explicitly taking context into account in the design process

• Three types of context variables involved in the communication (Space, Time and Social Context)

Geo-Social Interaction for Context-aware Help in Large-scale Public Spaces

• We present

– an approach, how to utilize social and spatio-temporal context to improve information dissemination

– Geo-social relevance with a ‘Dynamic view approach’

– Evaluated using a simulation with real life car data

People who are `on-the-move' often do not have an opportunity to spend long time looking for what they need

Motivation

Social-components

• Person in the network

• Person with matched profile

• Person with matched preferences, help type, urgency

Geo-components

• User’s location

• Distance between users (Help seeker and Help provider)

• Direction of movement

Finding Balance (in Geo-social Components)

Friends

• Friendship is ‘asymmetric’ relation (like Twitter)

• Dynamically updating list

Help Type Matching (Asymmetric)

Validation ( by Simulation in KULeuven)

• Using realistic dataset for cars

• In area of 250 km by 260 km

• Logged simulation data for 24 hours

Socializing Cars Vehicular Network

Conclusion

Improved relevance back propagation technique for routing messages in the network shows better results for each evaluated parameter

Conclusion

Lessons Learned from the Dynamic Social Network

• Social networking capabilities and spatio-temporal context information significantly improves purposeful interaction between individuals

• It improves in terms of both the efficiency of the network data dissemination and the quality of the delivered information

Conclusion

Contributions

• Contributions are situated in– Context-aware computing

– Social computing

• Several approaches and algorithms to support `aware interaction’

• We have developed number of context-aware social computing systems

• We have evaluated the systems

• We have studied dynamic social network systems

Context-aware Social Computing Systems

‘Aware’ Interaction

Future Research Directions

• An evaluation framework for context-aware and social computing system

• Emergency response

• Assistive Technology

Thank youhttp://research.edm.uhasselt.be/~nmahmud/

Exploiting Context-awareness and Social Interaction to Provide Help in Large-scale Environments

Documents