Haystack, MIT

End-User Application Development for the Semantic

Web

Karun Bakshi

kbakshi@mit.edu

Computer Science and Artificial Intelligence Laboratory (CSAIL), MIT

Haystack, MIT

Overview

• Tools for performing information based tasks are lacking.

• We need to fix these problems if we are to fully leverage all the information at our disposal.

• I propose an approach to fix these problems and I think the Semantic Web can help.

• In the process we demonstrate– An application of the Semantic Web– Applications for the Semantic Web

Haystack, MIT

Agenda

• Introduction• Related Work• Approach• Demonstration• Implementation• Discussion• Evaluation• Conclusion

Haystack, MIT

Introduction

Haystack, MIT

Information Based Tasks

• Information Based Tasks– Require multiple sets of information and functionality

– Can reuse and visualize the same information and functionality from other tasks

• Use Cases– Military Commander

– Doctor

– Software Project Manager

Haystack, MIT

A Day in the Life of a Software Project Manager

• Receive E-mail Bug Report from Customer• Check Task Assignments• Log Bug in Bug Tracking Software (automatic e-

mail notification)• Schedule meeting via e-mail and update calendar• Update Project Schedule and Task Assignments• Update Project Budget

Haystack, MIT

How Do We Accomplish Such Tasks Today?

• Users Must– Dig deep and wide to find relevant information and

functionality– Mentally collate information– Re-enter Information Data Synchronization? – Lather, Rinse, Repeat

• A solution that allows aggregating relevant resources would– Increase efficiency– Be mission critical

Haystack, MIT

Status Quo: A Potential Solution?

• Build an uber-Application or build a task-focused application. But…– How do I reuse the calendar or e-mail for

another purpose?– What if I need to track additional information?– What if I do a certain new task more often?– How does another user tackle the same

problem?– How do we handle feature creep?

Haystack, MIT

Source of Problem• Tasks are Fluid. Information and functionality

depends on:– User

• Expertise• Preferences• Task Definition

– Task• Evolve over time• New unanticipated tasks appear• New information is available that can be incorporated

• Applications are static!

Haystack, MIT

Problem Definition

• How do we provide users with an interface supportive of their tasks while avoiding the problems engendered by application shortcomings?

• How Important is this Problem?– WWW more information

– WWW more complex tasks

– WWW lay users

– What about ad hoc multi-domain applications?

Haystack, MIT

Related Work

Haystack, MIT

UI Real EstateManagement

Task Focused Applications

Workspace Builder Tools

Configure Presentation

Configure Content

Configure Operations

Configure Task Workspace

Build Task Workspace

Related Work

WinCuts

MyYahoo!

SNAP`

Data ManagementInterface

Task ManagementInterface

Con

figu

rati

onC

reat

ion

Haystack, MIT

MyYahoo! Allows Selecting

Content Layout Relevant Operations

Limited to News Domain

User can configure settings, rather than create them

Haystack, MIT

SNAP Together Visualization

Linked Content Arbitrary Content Multiple Visualizations No UI Programming

Knowledge of schema and SQL, or use fixed queries Fixed visualizations Primarily for data exploration

Haystack, MIT

WinCuts Allows collaboration Peripheral awareness Supports multiple domains

Cannot be persisted Have to go to source window to change content Pixel based content, not semantic Shareable, but not transferable

Haystack, MIT

Important Ideas

• Window Management• Persistent Habitat• Persistent Queries

– Dynamic Content

– Reusable Content

• User Creatable/Editable• Co-Location of Relevant Operations• Synchronized/Linked content

Haystack, MIT

Important Ideas (contd.)

• Little knowledge of underlying schema• Powerful query ability• Tasks require peripheral awareness of other

happenings• No Programming• Multiple visualizations• Low overhead reconfigurability• Applicable to multiple domains

Haystack, MIT

What’s Missing?

• General Purpose Interface– All above properties– Domain Interoperable (support multiple

domains ad hoc and simultaneously)– Transferable– Semantic Interface

Haystack, MIT

Approach

Haystack, MIT

Our Approach• Combine desirable characteristics in a single system• Information Management Interfaces Should Match

Tasks. • Users Know Tasks Best• Therefore Give User Control Over Building Them

– What are the application building primitives that users must control to match the fluid nature of tasks?

• What we are NOT trying to achieve:– The best implementation for particular techniques,

feature sets, UI widgets, etc.

Haystack, MIT

Our Approach (contd.)

• Unify the Data Model• Let Users Build and Populate a Task

Workspace – Formalize + Structure Existing Ideas/Features

• Content• Presentation• Manipulation

– Give users fragments of functionality to select & aggregate for unique tasks

Haystack, MIT

Semantic Web – Motivation

• Information on the World Wide Web is primarily intended for human consumption with complex semantics distributed across text, images, video, sound

• There is no way for machines to use and leverage such information easily and effectively

• Thus, very little on the WWW can be automated, and use of all the information on the WWW is constrained to the limited bandwidth of human processing

Haystack, MIT

Semantic Web – What is it?

• An initiative to embed semantics into content on the WWW to make it amenable to machine processing.

• Semantics are embedded via metadata annotations• Software (agent) understands the annotations

because they follow a particular schema/ontology• Different schemas can be used to describe the

same entity• But with metadata annotations, we begin to have a

substrate for automation

Haystack, MIT

Semantic Web – The Data Model

• Semi-structured (unenforced and/or partial schema validation) data models are very flexible – Captures multiple domains– Models the web

• Linking content from multiple domains in a growing (multi-schema) web

• No single schema to enforce

Haystack, MIT

Resource Description Framework (RDF)

• A specification/technology that captures and defines such a semi-structured data model for the Semantic Web

• Can be used to capture content from multiple domains

• Provides a common interchange format (XML based) that allows applications to communicate

Haystack, MIT

RDF Example

<URI> “Karun”

<URI>

“Max”

“Blue”

<URI>

“John”<URI>

“Blue”

Karun has a blue cat named Max, and Karun’s friend John has a blue car.

name

color

name

name

hasCat

hasCar

color

friend

URI – Universal Resource Identifier

Literal – String data

Haystack, MIT

RDF Example

Karun has a blue cat named Max, and Karun’s friend John has a blue car.

<URI> “Karun”

<URI>

“Max”

“FF”

<URI>

“John”<URI>

<URI>

name

color

name

name

owns

owns

color

friend

<URI>

type

“Cat”

name

<URI>

type

“Car”name

“00”

“00”

red

green

blue

Haystack, MIT

How Does the Semantic Web Fit In?

• A Problem?– More information– Faster generation of information– Highly granular information

• An Opportunity?– RDF models multiple domains– RDF allows granular access and annotation

Haystack, MIT

What’s Left?

• Tools that let users create a task workspace by manipulating fragments of– Content Queries (Channels) Channel Manager– Presentation Views & Layout View Designer and

Workspace Builder– Manipulation Operations Workspace Builder

• Haystack provides – such fragments at the developer level– other supportive capabilities

• Build tools to expose them to the user

Haystack, MIT

Haystack

• An general purpose information management platform

Services/Agent Framework

User Interface Framework

RDF Store

Haystack, MIT

Browsing Paradigm

Haystack Views

Recursive Rendering

Haystack, MIT

Haystack User Interaction

• Direct Manipulation– Drag & Drop– Context Menus

• Operations– UI Continuation– Currying

• Demonstration

Haystack, MIT

Demonstration

Haystack, MIT

Tool Set

• Workspace Designer

• View Designer

• Channel Manager

Haystack, MIT

Haystack, MIT

Implementation

Haystack, MIT

ImplementationWorkspaces

BuilderChannel Manager

Views Designer

Haystack

ContentX

(selecting)

X

(defining)

PresentationX

(layout)

X

(user views)

X

(developer views)

Manipulation

X

(Select Curried Operations,Dynamic

Binding)

X

(developer operations, drag and

drop, context menus)

Haystack, MIT

Implementation (contd.)

• Each Application Building Primitive has– An ontology– A set of views

• Each tool generates RDF entities according to an ontology

• User navigates to each entity and modifies it using its view(s)

• Each entity can have several views, e.g. Design View vs. Usage View

Haystack, MIT

Implementation (contd.)

• Store views, operation, content description and workspace as metadata along with application data in single format

Haystack, MIT

Workspace DesignerDesign Decisions

? Space Allocation to Winlets? Modality in Workspace Interaction? Uniform Treatment of Views? Dynamic Binding and Currying

Haystack, MIT

View Designer Pattern

• Developer Created Designer

• Exposes domain specific capabilities to create/customize views

• Named view that can be reused in any context in Haystack, just like any other view

Haystack, MIT

Property Lens View Designer

Haystack, MIT

Property Lens View Designer (contd.)

• Captures – RDF semantics only (property name and value)

• Resource View• Literal Read-only/Editable

– Layout

• Has design and usage views• Design Intentionally Abstract

– Multi-valued properties are supported– Provides baseline view extensibility without developer

support or additional view designers

Haystack, MIT

(Information) Channels

• What is a channel?– Query

– Dynamic set of items (no ordering) URIs only

– Organizing mechanism for dynamic corpora

• What are the implementation components?– Channel Agent

– Query Primitives• Query

• View of Query Primitive

Haystack, MIT

Channel Query Primitives

<Set of URIs> <Query Primitive Arguments>queryPrimitive

<URI1><URI2><URI3>

…

RDF Query Primitives Operators

Domain Specific Query

PrimitivesProperty Constraints

Haystack, MIT

Channel Query Primitives

RDF Query PrimitiveOperator

Domain Specific Query Primitive

Haystack, MIT

Discussion

Haystack, MIT

Discussion

• Transferable and Personalizable• Capture process in an interface• Low overhead task switching• Extensible by developers• Seamless integration of user/developer extensions• Consistent interaction modality• Domain specific view designers• Benefits of Channels• How much work is this?

Haystack, MIT

Evaluation

Haystack, MIT

Evaluation

• By Example– Our tools/Specialized tools– Paper Writing Workspace bridged multiple

domains– Project Management Workspace

• Preliminary End-User evaluation with Massachusetts General Hospital researchers

Haystack, MIT

Evaluation (contd.)

• Future User Study– Is such “just in time” information management

desirable?– Can users succeed with such an task interface

paradigm (not whether the particular interface implementation is usable)?

Haystack, MIT

Conclusion

Haystack, MIT

Conclusion

• We advocated user specified task workspaces– We identified three primary aspects of tasks

a simple ontology for building applications– We provide tools that allow composing

fragments of functionality into task workspaces• Content Channels• Presentation Views• Manipulation Operations

Haystack, MIT

Conclusion (contd.)

• We demonstrated the usefulness and validity of the ideas (although a field study is currently lacking)

• We have proposed a solution to the general problem of information management that will be even more critical for the Semantic Web– Tools to take advantage of Semantic Web content

immediately– Tools to build applications for the Semantic Web

• We have proposed an application of the Semantic Web

Haystack, MIT

Future Work

• Better UI for Tools

• Other enabling components for Semantic Web– Ontology Translation– View Servers

Haystack, MIT

Acknowledgements

• Project Oxygen and Biomedical Informatics Research Network (www.nbirn.net) for funding

• David Karger for Advising

• Various audiences and friends for helpful feedback on this work and presentation

Haystack, MIT

Thank You!

http://people.csail.mit.edu/kbakshi

Haystack, MIT

Q&A