MBARI’s Video Annotation and Reference System (VARS)

transcript

Brian Schlining and Nancy Jacobsen-Stout

Monterey Canyo

• More than 300 ROV dives per year• More than 12,000 hours of video observations

generated in 16 years• More than 1.6 million individual observations

MBARI Video Data

Video observations and related data• Used by MBARI researchers in well over 200 professional publications and

presentations, in addition to countless education and outreach projects• Span multiple sub-disciplines within biology, geology, chemistry

“A picture can be worth a thousand words”(if you can find it!)

Interesting Taxa (gorgonocephalus)Benthic Diversity

Scientific Applications

• Trained science staff identify animal taxonomy, animal behavior, geological features, evidence of human impact, and other objects recorded in the ROV video stream

• Annotations are stored in a database shared between MBARI scientists.

• Database continues to evolve as our researchers learn more

• Work closely with and rely on scientists to keep updated on new discoveries and publications in a timely fashion

Annotations

1st generation – Free Text– Inconsistent annotations– Difficult to search

2nd generation – VICKI/VIMS– Constrained annotations– Written in Smalltalk– Generated files

3rd generation – VARS– Constrained annotations– Written in Java– Writes information directly to database

Annotation History

Knowledge Base = hierarchical constraint lexicon & referenceMore than 3500 taxonomic, geologic, technical concepts. Reference media and descriptions.

Annotation = catalog video observations, inc. visual mediaUses the KB data to create and edit structured annotations for objects in the video stream.

Query = retrieve video-related data Uses the KB to structure simple to complex queries against annotations database

VARS - System Components

VARS - Knowledge BaseManages constraint lexicon and references

VARS - AnnotationGenerates video-annotations and frame-grabs

VCR Control(RS-422)

Frame capture(QuickTime for Java)

VARS - AnnotationGenerates video-annotations and frame-grabs

Customizable Interface

Knowledge Base

VARS - QuerySearches for and retrieves information and frame-grabs.

Vestimentiferan Tubeworms

Vesicomyid Clams

Example – “Cold Seeps”

“Cold Seep” Map Vintage 1999

“Cold Seep” Map Vintage 2004

Monterey bay is one of the most observed continental margins in the world

Visited 0.9% pixels in this map

• Previous “seep” models assumed seeps are associated with faults (fluid low conduits)

Example – “Cold Seeps”

Chemosynthetic Biological Communities occur preferentially on steep slopes?

Steep slopes imply areas of recent erosion.

VARS user goals include:1) Provide responsive user interfaces to maximize user efficiency and

accommodate real- and greater than real-time analyses.

2) Provide a robust system.

3) Provide clear and intuitive user interfaces.

VARS developmental goals include:1) Create maintainable (modular, understandable) software for future

developers.

2) Maximize use of existing software (e.g. old VIMS query and commercially available software)

3) Provide a solution that can be exported to other institutions.

4) The database should be simple to maintain by MBARI IS administrators.

VARS Development Goals

Vars system requirements:

1) Java 1.4 or greater

2) Java Comm (or RXTX)

3) Quicktime for Java (For frame-capture)

4) Relational Database

VARS Requirements

VARS Deployment

Point Lobos

Western Flyer

VARS Deployment

Point Lobos

Western Flyer

Database replication

VARS - Annotation data

A related group of tapes.e.g. tapes from a single expedition

A video source, such as a tape or QuickTime movie

A single video frame

An object observed in a particular video-frame. The name is constrained by the knowledgebase

Descriptive information. Can also be used to link ‘concepts’. For example, ‘nanomia eating krill’

Position, physical data, expedition information,and camera data,

VARS - Knowledgebase data

~ Hierarchical, Allows representation of relations or phylogeny~ Linked to descriptive info. (History, authors, images, video, etc. [not shown])

VARS - Knowledgebase data~ A concept may have one or more concept-names.~ One concept-name is designated as a primary name.~ User may annotate with any concept-name. However, only the primary name is stored in the database.

VARS - Knowledgebase data

VARS – Available to you

VARS is available to other science institutions.

See http://www.mbari.org/vars/

Fall 2005 – Release VARS as open-source

December 2005 – MBARI will host a developer workshop

If you are interested in using VARS contact:

Judith Connor (conn@mbari.org)

Director of Information & Technology Dissemination

MBARI’s Video Annotation and Reference System (VARS)

Documents