ISBO VIN,Virtual Individual Networks

Workshop 24.10.2007IBBT Gent

2

Content

General overviewThe project in one sentenceOverview of all research activitiesCommon user scenarioOther application domainsTechnical demo set-up

3

General overview

Virtual Individual Networks (VIN)Interdisciplinary Strategic Basic Research (ISBR), Long-term interdisciplinary pre-competitive research100% funded by IBBT.

Project duration: 01.09.2005 - 31. 08.2008

4

Multidisciplinary team

Different research groups are involved: CUO - K.U.LeuvenSMIT - VUB MICT - UGentICRI - K.U.LeuvenCOSIC - K.U.LeuvenUGent - IBCNEDM - UHasseltETRO - VUBNES - IMEC

5

Project summary

Research into the use of computer-mediated audiovisual communication, in order to enrich the participation within open and closed communities.

6

Research overviewCOMPUTER-MEDIATED

AUDIOVISUAL COMMUNICATION

3D viewpointVideo compression

PARTICIPATION ENRICHMENT

OPEN AND CLOSED COMMUNITIES

VIRTUAL INDIVIDUAL NETWORK

Key managementAutomatic data capturingSecure network overlays

Content regulationSociability guidelines

Online/offline communities

Social networksSocial translucence

Social capitalPrivacy

AV emotionanalysis

AV encryption

7

Approach and collaboration within VIN

Individual research goals to allow basic research in different domains

UserLegalTechnological

One common user scenario in which all research groups participate:

to assure the interaction between different research goals,to attain common demonstrators and research results that can be further investigated in GBO projects.

8

Common user scenario 1/4

Erica, a passionate dancer of salsa and tango, is not only a member of a dance-school, but also active in a number of online communities on Latin American dances.

Tonight Erica’s daughter is feeling ill and therefore she decides not to go to the dancing lesson tonight but to stay home in case her daughter needs her. She puts her daughter early to bed and contacts her dancing teacher in the dancing school via interactive television. The dancing teacher receives an indication on his screen that the person that is calling is Erica and that she’s feeling sad.

Erica tells her teacher that she won’t be able to make it to the lesson and that she will try to attend the class virtually by means of interactive television.

Viewpoint interpolation between different camera-inputs makes this conversation with her teacher pleasant since they can keep eye contact.

Content regulation Online / offline communities

AV emotion analysis

Sociability through IDTV

3D viewpointVideo compression

PARTICIPATION ENRICHMENT

OPEN AND CLOSED

NETWORKS

COMPUTER-MEDIATED

AUDIOVISUAL COMMUNICATION

9

She also informed her best friend with her mobile device that she will not be at the dance lesson and asks her if they could meet afterwards in the online community to exchange experiences.

The dancing teacher agrees and tells Erica to configure her webcams. Erica tells the system that only her appearance may be filmed and that the system should block out the background.

This means that her classmates will see her dance and her teacher will be able to give feedback on her dance moves, but they will not see the private environment in which Erica is practising.

On the spur of the moment, Erica decides to allow her husband John, who is working late, full accessto her camera-feed.

Object tracking

Social networks, social capital and social

translucence

Security network overlays

PARTICIPATION ENRICHMENT

OPEN AND CLOSED

NETWORKS

COMPUTER-MEDIATED

AUDIOVISUAL COMMUNICATION

Common user scenario 2/4

Online/offline communitiesOnline/offline sociability

10

During class time, Erica changes viewpoints several times in order to experience the dancing lesson and choreography in the best way possible. Erica’s teacher who can see only Erica and has no authority to see the rest of the picture (i.e. the room in which Erica is dancing) congratulates Erica on her progress.

John’s meeting is finished and takes the bus back home. The network automatically detects that John is on the move.

While he commutes between office and home he decides to see how his wife is doing in the dancing lesson. While he is on the move, a ‘universal communication layer’ provides automatic network connection detection for his PDA.

Riding the bus, John watches his wife dancing in their messy living room and makes a mental note that they really should start looking for a cleaning woman.

Automatic data capturing

UI usability & sociability

Personal networks

Distributed source coding Interface adaptation

PARTICIPATION ENRICHMENT

OPEN AND CLOSED

NETWORKS

COMPUTER-MEDIATED

AUDIOVISUAL COMMUNICATION

Common user scenario 3/4

11

When John arrives home, he tells Erica that he really liked one of the songs she was dancing to. So, after the lesson Erica asks her teacher if she could have this particular piece of music. Erica’s teacher, who believes that she has the right to share this material because it serves educational purposes, replies that she will put the data on the social software service or the private peer-to-peer network used by the dancing school for educational or at the online community of which she’s a member.

The teacher also invites Erica to check her blogon which she will post a few tips and tricks regarding the dance that they have been practising.

Secure networks

Content regulation and legal status of virtual individuals

Offline/online communities Online/offline sociability

PARTICIPATION ENRICHMENT

OPEN AND CLOSED

NETWORKS

COMPUTER-MEDIATED

AUDIOVISUAL COMMUNICATION

Common user scenario 4/4

12

Other application domains

Improve safety and security monitoring in freely accessible locations like train and metro stations

Video Coding, Secure Networking and Encryptionsafely send surveillance images over a network to the central monitoring location (not necessarily close to the monitored building).

Key Managementtemporary direct access to images can be given to the police or fire department in the case of an emergency.

3D viewpoint interpolationgives access to a virtual “flying camera” that can be navigated freely around the scene, and allows a better assessment of the given situation by the security personnel

Microphone Array Technology automatic spatial localization of sounds that may indicate a potentially dangerous situation. pick up sounds that come from these locations for further (automatic) analysis and evaluation.

Audiovisual Emotion Analysis automatic assessment of the emotional state of a single person (is (s)he angry or afraid) or even a group of persons (e.g. a happy group of partygoers, or an angry group of hooligans). Based on this assessment, the system can alert the security personnel to potentially dangerous situations.

13

Other application domains

Improve efficiency and communication at workA construction supervisor on the road would like to confer with the architects at the office

Video Coding, Secure Networking and EncryptionSafely send confidential data over a network to the central offices

Key ManagementOnly the construction supervisor and the architects at the office can get access to the images

3D viewpoint interpolationAllows the architects to have a excellent view of the situation in the field, looking at the location from different anglesThe supervisor and the architects can look each other directly in the eyes, improving communication

SociabilityOffering optimized interaction functionalities like voice communication and virtualdrawing boards that enhance communication between the supervisor and the architects

14

http://www.oxygen.lcs.mit.edu/H21.html

Stereo cameras

3D display

Teacher@ PC

Husband @ Mobile

(On the move)Bluetooth

WLAN

- Eye gazing (viewpoint interpolation)- Display/Interface adaptation- Virtual Private Ad Hoc Network- DSC on mobile (complexity issue)- Transcoding from DSC to MPEG-4- Region of Interest (ROI) tracking- Security

Husband sees Erica dancing in living room, Teacher sees Erica dancing without background

Erica@ Home

ROI of Erica

Eye gazing solved

Technical set-up