ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t1

Frank Allan Hansen, fah@daimi.au.dk

Integrating the Web and the World:Contextual Trails on the Move

Torben B. Pedersen

Frank Allan Hansen

Niels Olof Bouvin

Bent G. Christensen

Kaj Grønbæk

Jevgenij Gagach

Department of Computer Science,

University of Aarhus, Denmark

Department of Computer Science,

Aalborg University, Denmark

Euman Ltd., Denmark

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t2

Frank Allan Hansen, fah@daimi.au.dk

Overview

• Motivation– Hypermedia in the physical world– Utilizing context information

• HyCon: Contextual Hypermedia– HyCon Architecture and Data Model– The HyConExplorer Prototype:

• Browsing and Searching• Producing content: Annotations, Links, and Trails

• The SVG-based Client Framework• Conclusion

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t3

fah@daimi.au.dk

Motivation

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t4

Frank Allan Hansen, fah@daimi.au.dk

Motivation

• Techniques for augmenting the physical world with digital structures

• Linking objects in physical and digital space– Supporting access to digital information related

to the user’s context– Supporting production of information and

documentation in the user’s context

• Beyond the PC:– Supporting interaction through heterogeniuos

mobile devices

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t5

Frank Allan Hansen, fah@daimi.au.dk

Open Hypermedia Structures

• How can Open Hypermedia structures be used to Link objects in physical and digital space?– OH structures are not embedded in the linked resources!– Externally defined, out-of-line links– Write access to the resources is not required– Support multiple sets of structures on the same corpus

of documents or objects• Hypermedia structures:

– Links• Multi-headed, bi-directional, typed, generic

– Annotations (user created comments and footnotes)– Guided tours and information trails– ...

• If you can identify an object you can link it!

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t6

Frank Allan Hansen, fah@daimi.au.dk

Defining Context

• DEY (2001), Understanding and using context:

“Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.”

“A system is context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task.”

• Utilizing the part of the surroundings (the context) which is relevant to the user’s task

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t7

Frank Allan Hansen, fah@daimi.au.dk

Defining Context (2)

• Context-aware application support:– Presentation of information and services– Automatic execution of services– Tagging of context to information to support later retrieval

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t8

Frank Allan Hansen, fah@daimi.au.dk

Hypermedia in the Physical World

• Location based browsing and search relative to the users context (time, place, activity,...)– Geo-tagged information– Unstructured Web information (GBS/Google)

• Digital graffiti – Location based

annotations:The user as producer of information (Photo, video, speak, text notes)

– Leaving trails of information ”Social computing”

Digital graffiti: augmenting the environment without spoiling it

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t9

fah@daimi.au.dk

HyCon: Contextual Hypermedia

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t10

Frank Allan Hansen, fah@daimi.au.dk

The HyCon Platform

Trail ServiceAnnotation Service...

eBag/MediaTrayHyconExplorer...

Data Data Data

Storage Layer

Server Layer

Components

Components

Terminal Layer

Sensor Layer

Data layer

Search

Server communication

Bluetooth communication

Weather

GPS abstraction ...

Profile Location Annotation Link

Service Interfaces

Sensor Interfaces

Infra structure

MySQL

HyperContextServer Components

Servlets,Web services

Symbian applications,Tablet applications,Web browsers

Sensor abstraction

Data layer

Sensor equipment

Bluetooth ID

Weather

RFID

Sound recorder

CameraGPS unit IR

...

Service Interfaces

Local Sensors Remote Sensors

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t11

Frank Allan Hansen, fah@daimi.au.dk

The Sensor Layer

• Local sensors• Remote sensors

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t12

Frank Allan Hansen, fah@daimi.au.dk

The HyCon Data Model

AbstractObject Property ValueId

AttributesObject

Context Location Annotation

Body Link

Arc Locator

Href Geotag

1

N

M

1 NM1

N

M

1 1

2

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t13

Frank Allan Hansen, fah@daimi.au.dk

The HyConExplorer Prototype

• The HyConExplorer Prototype– Implemented in Java, SOAP, and

SVG– Runs on commercially available

tablet PCs connected to the network through WLAN or GRPS connections on cell phones

– The tablet has a mounted camera for capturing low resolution images, video, and audio in the application.

– The user’s physical location is registered through a Bluetooth enabled GPS unit.

Tablet PC

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t14

Frank Allan Hansen, fah@daimi.au.dk

Context Based Browsing

• Navigating resources by browsing• Context based browsing

– Browsing the digital context…– by changing parameters in the physical context

• Direct Physical Navigation– Browsing with your feet– Physically walking or driving from one location to another

affects the time and location parameters– User “bump into” information– Used by tourist guide systems, navigation systems, etc.

• Indirect Representational Navigation– It’s not always desirable nor feasible to have to physically

move around to browse information.– Allows users to investigate information structures in the

system even though they are not actually in the physical context associated with the structures.

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t15

Frank Allan Hansen, fah@daimi.au.dk

Context Based Search (GBS)

• Search for information associated with the user’s context

• Geo Based Search: Web searches are augmented with context information or information derived from sensor data– Searching for shops or restaurants nearby which are

open now– Requires information about the user’s (and shops)

location and the time of day• The benefit of adding criteria based on context

information is of course highly dependent on the search engine used and how the context data can be formulated as an appropriate criteria…

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t16

Frank Allan Hansen, fah@daimi.au.dk

Context Based Search (GBS)

• The user’s position can be acquired from GPS – Search engines are not indexing pages by GPS coordinates

(geo-tagging is not very popular)– Derive GPS coordinates to postal addresses!

• Businesses often provide their postal addresses on their Web sites

• GBS: user supplied search terms are augmented with computed postal addresses

=> The matching Web pages presumably covers the topic of interest and are associated to the given geographical area

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t17

Frank Allan Hansen, fah@daimi.au.dk

Context Based Search (GBS)

NimbusGPS(x,y)

?

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t18

Frank Allan Hansen, fah@daimi.au.dk

Context Based Search (GBS)

1. Context informationand search terms

2. Computedsearch terms

3. Search results4. Hypermediastructures

HyCon server

Externalsearchengine

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t19

Frank Allan Hansen, fah@daimi.au.dk

Context Based Search (GBS)

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t20

Frank Allan Hansen, fah@daimi.au.dk

Context Based Annotations

• Going beyond the classical context-aware browsing and navigation systems

• Support users as active information producers– documenting or commenting on objects or situations

• Combining context sensors with mobile devices equipped with built-in cameras and microphones– Supports creation of video, audio, photo, and text

annotations and automatic tagging with context information

– Take photo in context– Taking notes about a site– Using browsing and searching the information can later

be retrieved by the user, friends, or colleagues

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t21

Frank Allan Hansen, fah@daimi.au.dk

Context Based Trails

• Creating structure!• Organizing information as you go• Leaving (user created) guided tours through

collections of “points of interest” – No requirement for pre-defined tourist guides

• “Social computing”– Combined with the annotation facilities trails

may serve as personalized impressions of a given area and even allow users to share their impressions through discussions linked to points of interests along the trail.

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t22

Frank Allan Hansen, fah@daimi.au.dk

Context Based Trails

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t23

fah@daimi.au.dk

The SVG-based Client Framework

ISIS Katrinebjerg

i n t e r a c t i v e s p a c e s . n e t24

fah@daimi.au.dk

Conclusion