Date post: | 13-Jan-2016 |
Category: |
Documents |
Upload: | norma-owens |
View: | 223 times |
Download: | 0 times |
Creating novel experiences for Creating novel experiences for interacting with mediainteracting with media
Dr. Steven M. DruckerDr. Steven M. DruckerLead ResearcherLead ResearcherNext Media Research GroupNext Media Research GroupMicrosoft CorporationMicrosoft Corporation
OverviewOverview
Role of Design and software developmentRole of Design and software development
New experiences, new interfaces: an New experiences, new interfaces: an exampleexample
Rapid prototyping of novel interfacesRapid prototyping of novel interfacesWhy novel interfaces?Why novel interfaces?
Why rapid prototype?Why rapid prototype?
How to do it now?How to do it now?
Baby steps: Generalized List WidgetsBaby steps: Generalized List Widgets
Overview Overview (continued)(continued)
Application domain: media browsing: Application domain: media browsing: DEMOSDEMOS
Finding media: Movie Variations, Finding media: Movie Variations, MediaBrowser MediaBrowser
Browsing media: TimeQuilt:Browsing media: TimeQuilt:EvaluationEvaluation
Moving within media: SmartSkipMoving within media: SmartSkipEvaluationEvaluation
Vision Pieces: Vision Pieces: SpectatorSpectator
Current software developmentCurrent software development“Projects get a green light right at the start, and go directly to engineering. The next phase is when they ship—usually late, with bugs, over budget and missing functionality.“
EngineeringEngineering SalesSales
Proceedings of the Second International Conference on Usage-Centered Design.Portsmouth, NH, 26-29 October 2003, pp. 1-15.http://www.foruse.com/2003/ Bill Buxton: www.billbuxton.com Bill Buxton: www.billbuxton.com
“… demonstrable myth that we know what we want at the start, and how to get it, and therefore build our process assuming that we will take an optimal, direct path to get there. Nonsense … the sooner we make errors and detect and fix them, the less (not more) the cost.”
The role of designThe role of design
From Bill Buxton: From Bill Buxton: www.billbuxton.com
EngineeringEngineering SalesSales
“Inserting an explicit design process at the front end, prior to green lighting the project. The process is represented as a funnel, since the number of concepts to emerge is always anticipated to be fewer than enter.”
DesignDesign
Components of DesignComponents of Design
Iterative DesignIterative Design
PickPickEvaluateEvaluate
CullCull
Novel ExperiencesNovel Experiences
SketchesSketchesStoryboardsStoryboards
VideosVideosPrototypesPrototypesEvaluationEvaluation
Adapted from Bill Buxton: www.billbuxton.com Adapted from Bill Buxton: www.billbuxton.com
Why novel interfaces?Why novel interfaces?
Need both next versions AND new Need both next versions AND new experiencesexperiences
Next version gets harder with each iterationNext version gets harder with each iteration
New capabilities: New capabilities: Moore’s law: how can we improve Moore’s law: how can we improve interaction?interaction?
New problems: New problems: Information overflowInformation overflow
Ubiquitous computingUbiquitous computing
New expectations: New expectations: Emotional design: iPod exampleEmotional design: iPod example
An example vision: Blitz UXAn example vision: Blitz UX
Joint collaboration Joint collaboration between the Windows between the Windows Shell UX team and Next Shell UX team and Next Media Group Media Group
Vision piece for how a Vision piece for how a broadband website broadband website might take advantage of might take advantage of rich 3d and multi layered rich 3d and multi layered graphics compositing on graphics compositing on the desktop. the desktop.
Why rapid prototype?Why rapid prototype?
Need to quickly try many new ideas.Need to quickly try many new ideas.
Need to iterate on design concepts.Need to iterate on design concepts.
Need to test ideas with diverse users.Need to test ideas with diverse users.
Subtle interactive refinement can be Subtle interactive refinement can be crucial.crucial.
Rapid prototyping of novel Rapid prototyping of novel interfaces: How to do it now?interfaces: How to do it now?
• Existing tools: Existing tools: • Flash, DirectorFlash, Director
• Difficult to integrate with databases and other componentsDifficult to integrate with databases and other components• Don’t handle bitmaps or video wellDon’t handle bitmaps or video well• Not 3DNot 3D• Scattered programming modelScattered programming model
• Anark: 3D but no same problems as aboveAnark: 3D but no same problems as above• C++: difficult to create novel interfacesC++: difficult to create novel interfaces• Toolkits/Languages: Piccolo, Python, and beyond…Toolkits/Languages: Piccolo, Python, and beyond…
• In house tool:In house tool:• ““Game-engine” for user interfaces.Game-engine” for user interfaces.• Designer as “mod developer”Designer as “mod developer”• Object/behavior separationObject/behavior separation• Dynamic Languages, faster iteration of designsDynamic Languages, faster iteration of designs• Don’t start from scratch every timeDon’t start from scratch every time
Example: Generalized List ControlExample: Generalized List Control
Creation of Creation of rich reusable rich reusable componentscomponents
Let designers Let designers iterate without iterate without developers in developers in the loopthe loop
With Kentaro ToyamaWith Kentaro Toyama
Domain for applying novel interfacesDomain for applying novel interfaces
Text doesn’t help Text doesn’t help (no, or limited metadata)(no, or limited metadata)
Challenging: Challenging: Large numbers Large numbers
Immediately understandableImmediately understandable
Emphasis on usability Emphasis on usability (won’t get used otherwise)(won’t get used otherwise)
Media Interaction:Media Interaction:
Movie Variations:Movie Variations:Browsing multidimensional information using movies as an example content.
Explores browsing Explores browsing local arealocal area
Uses “6 degrees of Uses “6 degrees of separations”separations”
Applicable to many Applicable to many different domainsdifferent domains
With Asta RosewayWith Asta Roseway
SqlConnector
Movie Variations: ImplementationMovie Variations: Implementation
Uses DirectX9Uses DirectX9
Written entirely in C# Written entirely in C# on .NET Frameworkon .NET Framework
Accesses a database of Accesses a database of 17000 films originally built 17000 films originally built for a universal metadata for a universal metadata project all stored within project all stored within SQL.SQL.
Can use other techniques Can use other techniques for clustering data for clustering data (collaborative filtering, (collaborative filtering, metadata, etc.)metadata, etc.)
Can be applied to other Can be applied to other domains.domains.
Media Variations Client
SQLserver
DX9
.NET Framework
Graphics Display(needs GeForce2
equiv or better)
Effective browsing and annotation of personal photos
Media BrowserMedia Browser
Leverage Leverage clustering and clustering and analysis of photos analysis of photos for annotationfor annotation
Effective filtering to Effective filtering to find imagesfind images
Use animation and Use animation and 3D effectively3D effectively
Leverage Leverage clustering and clustering and analysis of photos analysis of photos for annotationfor annotation
Effective filtering to Effective filtering to find imagesfind images
Use animation and Use animation and 3D effectively3D effectively
Drucker, S. C. Wong, A. Roseway, S. Glenner, S. De Mar, MediaBrowser: Reclaiming the Shoebox. in Proceedings of AVI2004, Gallipoli, Italy, 2004.
Media Browser: ImplementationMedia Browser: ImplementationMedia Browser Client
VIZUALIZERS:(Timeline & Tabular)
FILTERS
METADATA - manual - automatated
KeywordsTimeSliderAttributesFaces
ORGANIZERS
ACTIONS
ODBC
Media Browser Client
Access (later winFS)
DX9
.NET Framework
Graphics Display(needs GeForce2
equiv or better)
TimeQuiltTimeQuilt
Dealing with 10,000’s Dealing with 10,000’s of photosof photos
Just use implicit Just use implicit metadatametadata
Use temporal Use temporal clustering and clustering and representative representative thumbnailsthumbnails
Scaling up Zoomable Photo Browsers for Large, Unstructured Photo Collections
Huynh, D., Drucker, S., Baudisch, P., Wong, C.Time Quilt: Scaling up Zoomable Photo Browsers for Large, Unstructured Photo Collections. CHI 2005. Portland, OR. Apr. 2005
Basic AlgorithmBasic Algorithm
Basic AlgorithmBasic Algorithm
Basic AlgorithmBasic Algorithm
Basic AlgorithmBasic Algorithm
Basic AlgorithmBasic Algorithm
Basic AlgorithmBasic Algorithm
Temporal ClusteringTemporal Clustering
Algorithm from Platt et al …Algorithm from Platt et al …
gi is gap between picture i and picture i+1d is window size (chosen to be 10 pictures)K is empirical threshold (chosen to be 17)
Compares a gap to the a local geometric average of gap times, and declares an new event when the difference is large enough.
Representative Photo SelectionRepresentative Photo Selection
We used “dumb algorithm” We used “dumb algorithm” first or middle picture from a clusterfirst or middle picture from a cluster
Could do other things:Could do other things:See Lim et al. “Content based See Lim et al. “Content based summarization for personal image summarization for personal image library”, Proceedings of the 3library”, Proceedings of the 3rdrd ACM/IEEE- ACM/IEEE-CS joint conference on digital libraries, CS joint conference on digital libraries, 2003.2003.
Experiment: Retrieving PhotosExperiment: Retrieving Photos
Within subjects design:Within subjects design:3 different interfaces tried:3 different interfaces tried:
Space filling, timeline and timequilt layoutsSpace filling, timeline and timequilt layouts
Space filling tried with and without representative Space filling tried with and without representative photosphotos10 subjects (8 male, 2 female)10 subjects (8 male, 2 female)Subjects had between 2863 and 5708 photosSubjects had between 2863 and 5708 photosEach participant selected 28 favorite photosEach participant selected 28 favorite photosRandomly divided selected photos into 4 groupsRandomly divided selected photos into 4 groups2 Training photos and 5 photos for actual test 2 Training photos and 5 photos for actual test
ResultsResultsRetrieval Time
0
20
40
60
80
100
120
Space-Filling TimeQuilt TimeLine
Se
co
nd
s
Without and With representative Photos
SmartSkip:SmartSkip:Convenient skipping and browsing of digital video from a remote control.
Thumbnail view Thumbnail view interface for skipping interface for skipping and browsing digital and browsing digital video. video. Adjust time variation Adjust time variation with shot detectionwith shot detectionUser study comparing User study comparing this system with this system with systems alternativessystems alternatives
Drucker, S., Glatzer, A., De Mar, S and Wong, C. SmartSkip: Consumer level browsing and skipping of digital video content. In Proceedings of CHI 2002, Minneapolis, Minnesota, 2002
BackgroundBackground
Video Browsing and SkimmingVideo Browsing and SkimmingInformedia projectInformedia project
Video surrogates: [Elliot ’93], [Komlodi ’98]Video surrogates: [Elliot ’93], [Komlodi ’98]
Skimming [Li ’00], [Christel ’98]Skimming [Li ’00], [Christel ’98]
Most tend to be for computer monitor (close Most tend to be for computer monitor (close up) and not for consumer use.up) and not for consumer use.
But many show the utility of having a But many show the utility of having a storyboard (spatial) layout as opposed to storyboard (spatial) layout as opposed to temporal layouts [Tse et al ’99]. temporal layouts [Tse et al ’99].
ImplementationImplementation
Combine shot detection with evenly Combine shot detection with evenly spaced temporal skipping.spaced temporal skipping.
Use rapid changes in overall color and Use rapid changes in overall color and brightness. Doesn’t currently detect fades.brightness. Doesn’t currently detect fades.
Use thumbnails from moments after the Use thumbnails from moments after the detected shot.detected shot.Used infrared keyboard and Used infrared keyboard and programmable remote controls.programmable remote controls.
10 20 30 40 50 60 70 80
a b c d e
User study: Experimental DesignUser study: Experimental Design
Independent variables:Independent variables:Interfaces (3 treatments): Skip, Multiple Interfaces (3 treatments): Skip, Multiple levels of fast-forward, SmartSkiplevels of fast-forward, SmartSkip
Tasks (2 treatments): Commercial Tasks (2 treatments): Commercial skipping and weather segment finding.skipping and weather segment finding.
Dependent variables:Dependent variables:Task performance: accuracy, time to Task performance: accuracy, time to completion, # of clicks to completioncompletion, # of clicks to completion
Subjective satisfaction: ranking, ease of Subjective satisfaction: ranking, ease of use, ease of learning, frustration, funuse, ease of learning, frustration, fun
User study results: quantitative,User study results: quantitative,commercial skipping taskcommercial skipping task
Quantitative performance in seconds and # of clicks for commercial skipping.Statistically significant for time between skip and other 2 interfacesStatistically significant difference between SmartSkip and other 2 for # clicks
User study results: quantitative,User study results: quantitative,weather segment finding taskweather segment finding task
Quantitative performance in seconds and # of clicks for weather segment finding.Statistically significant for time between skip and other 2 interfacesStatistically significant difference between all pairs for # clicks
User study: subjective User study: subjective questionsquestions
## QuestionsQuestions
11 I found this interface easy to use.I found this interface easy to use.
22 I found this interface easy to learn.I found this interface easy to learn.
33 I could skip commercials easily with this interface.I could skip commercials easily with this interface.
44 I could find the weather section easily with this I could find the weather section easily with this interface.interface.
5*5* I thought this interface was fun to use.I thought this interface was fun to use.
6*6* I thought this interface was frustrating to use. (this is I thought this interface was frustrating to use. (this is the only question where lower numbers indicate more the only question where lower numbers indicate more satisfactory experience).satisfactory experience).
User study: subjective User study: subjective responsesresponses
Subjective Response
12345678
1 2 3 4 5 6
Questions
Ag
reem
ent
(Lo
w->
Hig
h)
SKip
FFwd
SmtSkip
Statistical significance for questions 5 (fun) and 6 (frustrating).
User study: subjective rankingUser study: subjective ranking
Ranking
0%
10%
20%30%
40%
50%
60%
70%80%
90%
100%
Skip FFwd SmtSkip
Interface
Worst
Medium
Best
Smart Skip conclusionsSmart Skip conclusions
Make sure you do both quantitative and qualitative Make sure you do both quantitative and qualitative evaluations, especially for consumer applications! evaluations, especially for consumer applications! What is the appropriate quantitative measure?What is the appropriate quantitative measure?
Untested hypothesis: the interfaces that require the Untested hypothesis: the interfaces that require the user to attend closely to the display were found less user to attend closely to the display were found less ‘satisfying’.‘satisfying’.
Some problems were identified:Some problems were identified:Some people “got lost”Some people “got lost”
Some found the thumbnails too small.Some found the thumbnails too small.
Not clear what the best resolution to zoom in.Not clear what the best resolution to zoom in.
SmartSkip is a promising interface that combines the SmartSkip is a promising interface that combines the benefits of temporal and spatial layout of display.benefits of temporal and spatial layout of display.
Spectator (from vision to features):Spectator (from vision to features):Web community based on Spectator enabled games
Enable community Enable community around video gamesaround video games
Use fame and learningUse fame and learning
Vision, don’t worry Vision, don’t worry about implementationabout implementation
Keep in mind feasibilityKeep in mind feasibility Spectator Games: A New Entertainment Modality for Networked Multiplayer Games, 2000. http://research.microsoft.com/~sdruckerhttp://research.microsoft.com/~sdrucker/papers/spectator.pdf /papers/spectator.pdf
Spectator (automated game cameras):Spectator (automated game cameras):Realtime in-game cinematic camera control
Chicken and egg Chicken and egg problemproblem
Automate Automate expensive tasksexpensive tasks
Demonstrate in Demonstrate in real applicationreal application
Chicken and egg Chicken and egg problemproblem
Automate Automate expensive tasksexpensive tasks
Demonstrate in Demonstrate in real applicationreal application
Drucker, S.M. Intelligent Camera Control for Graphical Environments PhD Thesis, MIT Media Lab. 1994.
Technical approaches for Spectator Technical approaches for Spectator ((VideoVideo))
Video – Video – Encode the one view of game in Encode the one view of game in progress and broadcast that as progress and broadcast that as streaming videostreaming video
Broadcast video of games in Broadcast video of games in progressprogress
• Requires second computer for encoding• Requires fast connection• Requires central server for rebroadcast• Scalability issues• Single choice of view
Game machine
VideoEncoder
• Conventional encoding and streaming media technology can be used.
• No pre-distribution required
• Plays on simple media player
• No modification of source code
• Easily done
PROS CONS
Technical approaches for Spectator Technical approaches for Spectator ((Game EngineGame Engine))
Distributed Game EngineDistributed Game Engine
Pre-Distribute spectator only game Pre-Distribute spectator only game engine and selected contentengine and selected content
Broadcast game state data to playersBroadcast game state data to playersBroadcast to one client, cascade to Broadcast to one client, cascade to othersothers
Use index server to locate free Use index server to locate free resourcesresources
Game machine
Index server
Spectator
Requires special purpose code for spectator engines Requires optimized platform for receiving games (already owned)Security concerns for distribution of content May requires central server for rebroadcast Scalability issues but easier to overcome
Flexible game presentation (POV, replay, etc)High quality playback with low bandwidthMay be best suited for Xbox business plan (advertising to people who already have the box)Pre-Distribution may be through magazine or online service
PROS CONS
Technical Tradeoff Issues (Technical Tradeoff Issues (VideoVideo vs. vs. EngineEngine))
Developer work (Developer work (VideoVideo))Most developers do not have time for any extra development for non-playersMost developers do not have time for any extra development for non-players
Amount of dedicated infrastructure and scalability (Amount of dedicated infrastructure and scalability (EngineEngine))Dedicated any central services on a “per game” basis will be prohibitive Dedicated any central services on a “per game” basis will be prohibitive except for large scale tournamentsexcept for large scale tournaments
Bandwidth (Bandwidth (EngineEngine))Current bandwidth constraints favor delivery of game engine commands as Current bandwidth constraints favor delivery of game engine commands as opposed to video for much better quality.opposed to video for much better quality.
Security (Security (VideoVideo))Though Xbox may be tougher to hack into games then PC environment.Though Xbox may be tougher to hack into games then PC environment.
Content/Engine Distribution (Content/Engine Distribution (VideoVideo))Though there may be satisfactory mechanisms for predistributing engines Though there may be satisfactory mechanisms for predistributing engines and content. Can be value-add for magazine or xbox internet service. May and content. Can be value-add for magazine or xbox internet service. May make good use of existing hard disk on xbox.make good use of existing hard disk on xbox.
Client Flexibility (Client Flexibility (EngineEngine))Can allow for greater freedom in pausing, replaying, changing point of view. Can allow for greater freedom in pausing, replaying, changing point of view. Though having a central video broadcast server can use manual directorial Though having a central video broadcast server can use manual directorial control.control.
Preferred Approach (P2P fan out)Preferred Approach (P2P fan out)
1.1. Individual game registers at Individual game registers at central index server.central index server.
2.2. Spectators access index Spectators access index server to determine other server to determine other spectators which are available spectators which are available to rebroadcast game data. to rebroadcast game data.
3.3. Optionally, a spectator master Optionally, a spectator master can be used for tournaments can be used for tournaments which can do analysis, which can do analysis, optimization and stream optimization and stream information without peer-to-information without peer-to-peer scheme.peer scheme.
Index Server
Optional Game SpectatorMaster Server
America 1900:America 1900:Enhanced television concept prototype of enhanced content from closed captions.
Vision pieceVision piece
Analyze closed Analyze closed captionscaptions
Automatically Automatically generate related generate related linkslinks
Deep News:Deep News:Working prototype of automated generation of links to news web pages
Video of working Video of working prototypeprototype
Uses Headline Uses Headline NewsNews
TV Tuner card -> TV Tuner card -> extract closed extract closed captionscaptions
MindNet NLP -> MindNet NLP -> extract keywordsextract keywords
Feed to news Feed to news search enginessearch engines
Some closing words:Some closing words:
Interaction design can make or break Interaction design can make or break an experience.an experience.Start early on in the process, often the Start early on in the process, often the earlier, the better.earlier, the better.Need to iterate on design concepts.Need to iterate on design concepts.Need to build and test ideas with Need to build and test ideas with diverse users.diverse users.Ideas encourage other ideas, Ideas encourage other ideas, experiment!experiment!Web site: Web site: http://research.microsoft.com/~sdruckerhttp://research.microsoft.com/~sdrucker
© 2004 Microsoft Corporation. All rights reserved.© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.