Project 1: CoMediAnnotate: towards more usablemultimedia content annotation by adapting the UI
“Final” Presentation: Friday August 6, 2010
Christian Frisson 1, Sema Alaçam 2, Emirhan Coşkun 2,Dominik Ertl 1, Ceren Kayalar 4, Lionel Lawson 1,
Florian Lingenfelser 5, Johannes Wagner 5
1 University of Louvain-la-Neuve (Belgium),2 Istanbul Technical University (Turkey),3 Technical University of Vienna (Austria), 4 Sabanci University, Istanbul (Turkey),
5 University of Augsburg (Germany)
Friday August 6, 2010eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 1 / 20
eNTERFACEbook (reminder)
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 2 / 20
Outline
1 ContextTeamAnnotation tools: current problems, proposed solutions
2 AchievementsImprovement of an annotation tool (SSI)Design and Prototyping of a new UIIntegration of input devices into OpenInterface
3 To DoFinalize our (OpenInterface) componentsProduce more pipeline prototypes towards usability testingPort CoMediAnnotate into the MediaCycle framework
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 3 / 20
Annotated Team
input devices integrator
Ceren
UI developer
Christian
UI developer
Emirhan
annotation core integrator
Dominik
UI designer
Sema
Ismail
interviewed expert user
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 4 / 20
Annotation tools: current problems
2 Scale: number and/or length of media elements in the database2 Reusability: isolated tools, rather than toolboxes/frameworks2 Genericity: different tools, different media, different languages...2 Accessibility: local media databases, rather than client/server apps2 Multimedia: better file formats support2� Interactivity: generally WIMP2� Workflow: lack of support of the full annotation workflow
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 5 / 20
Annotation tools: proposed solutions
Interactivity: a single visual user interface that allows:1 2� to monitor signal feeds while recording datasets,2 2� optionally to add annotations while recording,3 2� to edit or correct annotations;4 2� a more natural, usable, pleasurable user interface (pen and touch).
Workflow:1 2 one administrator prepares (design of a template and choice of
coders);2 2� several coders record;3 2� several coders annotate;4 2 the administrator analyses results (coder agreement...).
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 6 / 20
Outline
1 ContextTeamAnnotation tools: current problems, proposed solutions
2 AchievementsImprovement of an annotation tool (SSI)Design and Prototyping of a new UIIntegration of input devices into OpenInterface
3 To DoFinalize our (OpenInterface) componentsProduce more pipeline prototypes towards usability testingPort CoMediAnnotate into the MediaCycle framework
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 7 / 20
Timeline-based Annotation Tools: SSI winsA detailled comparison in 3 tables is on the wiki:
1 development (quantitative)2 context, usage (quantitative)3 eNTERFACE participants feedback (qualitative)
Advene
Python
AmiGram
Deprecated
Anvil
Not opensource
Elan
Finalist
Lignes de Temps
Flash/Flex
On The Mark
JavaFX, simple
SSI VCode/VData
OSX-only
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 8 / 20
Timeline-based Annotation Tools: why SSI?
close contact with its developers(also project participants)core separated from UIlightweight GUI:
1 simple to understand2 easy to replace
toolkit first aimed at multimodalsignals annotation, towardsadaptive multimodal interfacesby training
More info:http://mm-werkstatt.informatik.uni-augsburg.de/ssi.html
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 9 / 20
Improved SSI UI
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 10 / 20
Integration of SSI into OpenInterface
More info: http://www.openinterface.orgeNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 11 / 20
UI design proposal
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 12 / 20
UI design proposal (annotated)
Fixed Play Zone
Past Events Future Events
Audio Track: Waveform
Video Track: Keyframes
Sensor Tracks:
Frames sliding right-to-left
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 13 / 20
UI prototype in Processing
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 14 / 20
Integration of input devices into OpenInterface
Previously available:Ï Wii RemoteÏ 3D mice
In progress:Ï multitouch devices, with two options available:
F WM_TOUCH high-level events fromWindows 7 using MT4j
F cross-platform low-level HID usingGenericHID
Ï jog wheels (also reusing GenericHID)Ï pens
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 15 / 20
Outline
1 ContextTeamAnnotation tools: current problems, proposed solutions
2 AchievementsImprovement of an annotation tool (SSI)Design and Prototyping of a new UIIntegration of input devices into OpenInterface
3 To DoFinalize our (OpenInterface) componentsProduce more pipeline prototypes towards usability testingPort CoMediAnnotate into the MediaCycle framework
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 16 / 20
Finalize our (OpenInterface) components
Input modalities, towards:Ï finger-based navigation / annotationÏ pen-based annotationÏ manual navigation using jog wheels
“On-demand” gesture recognition and rawdevice event filteringGranularized SSI core componentsOur timeline-based multitrack audio/videovisual user interface
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 17 / 20
Produce more pipeline prototypes towards usability testing
3D Mice
Pen+Touch
Jog Wheels Multitouch screen
Mice
Keyboards
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 18 / 20
Port CoMediAnnotate into the MediaCycle framework
More info:http://www.numediart.org/projects/08-3-multimediacycleeNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 19 / 20
Thanks for your attention, once again!
We stayed okay! ™ →
← ...and liked the social events!
eNTERFACE 2010 (Project 1) CoMediAnnotate Friday August 6, 2010 20 / 20