ViPERVideo Annotation and

Performance Evaluationviper-toolkit.sf.net

The ViPER Toolkit

• Unified video performance evaluation resource, including:– ViPER-GT – a Java toolkit for marking up

videos with truth data.– ViPER-PE – a command line tool for

comparing truth data to result data.– A set of scripts for running several sets

of results with different options and generating graphs.

The Video Performance

Evaluation Resource

Ground Truth Editor

PerformanceEvaluation Tool

TruthData

Video AnalysisAlgorithm

ResultData

SchemaMapping

Metrics Filters

EvaluationResults

Video AnalysisAlgorithm

ResultDataVideo Analysis

AlgorithmResultData

ViPERGround Truth Editing

viper-toolkit.sf.net

Ground Truth Editing

Ground Truth Editing

• Lets the user specify the task and the interpretation.

• Improved (Verson 4)– Schema editor.– Timeline view.– Supports undo/redo.– All new video annotation widget.– MPEG-2 support on Windows.

Latest Version in Series

Time Line View

• Provides summary and editing of annotation.

• Very useful for tracking annotation, great for event or activity annotation.

Time Line View

• Provides summary of ground truth.• Direct manipulation.

– Quick editing of activities, events, and other descriptors.

– Some ability to modify descriptors with dynamic attributes directly, if not the attribute values.

• Feedback for indirect manipulation.– Easier to notice massive changes.

Play When Valid Feature

• New feature divides movie into ‘valid’ segments.

• Useful for dividing movie and for playback.

• Combined with ‘Generate I-Frame Descriptor’ plug-in, supports i-frame-only mark-up.

Enhanced Keyboard Editing

• Support for real-time mark-up of events and activities.– Keys for creating and deleting activities.– Keys for controlling rate of display (jog dials).

• Enhance mark-up of spatial data.– Keys for creating, editing of a single descriptor's

attribute.

• Overall attempt to minimize effort in a GOMS model.– Mouse events unnecessary except for polygon editing.

Video Frame View Enhancements

• Display with Respect To– Play back the movie with a spatial object

remaining static and the video frame shifting behind it.

– Good for editing several things that move together.

• Enhanced editing:– More direct manipulation controls.– Zooming enhancements, including zoom history,

scroll-wheel-zoom, and center-click-panning.

Schema Editor

• Works on live data.• Don’t have to muck about with the

file format as much.

Demonstration

ViPER-GT Internals

ViPER-GT:A Video

Ground Truth Annotation

Tool

Schema Editor

ViPER Metadata API

Pure Java MPEG Decoder

AppLoaderPlug-In Manager

Jena

Core GT API

Plug-InsPlug-Ins

Plug-Ins Native Decoders:VirtualDubQuickTime

JMF

• VideoAnnEx– IBM AlphaWorks MPEG-7 Editor

• OntoLog (OWL)– Jon Heggland’s RDF Video Ontology Editor

• Informedia– CMU Digital Video Library

• PhotoStuff– Still image annotation for the semantic web

• ANVIL• Any one-off tool a developer has created

to test their stuff