1
30.10.2002 INF 312, Ifi, UiO, høst2002 1
Database Systems and WWW Applications
Digital Libraries
Multimedia Database Systems
Earl F. Ecklund, Jr.
30 October 2002
30.10.2002 INF 312, Ifi, UiO, høst2002 2
Contents
• Database Systems and WWW Applications– Internet DB Architecture– Internet Applications
• Digital Libraries
• Multimedia Database Systems
2
30.10.2002 INF 312, Ifi, UiO, høst2002 3
Internet Application Architecture: Today
Application messages
Browser Browser
Physical Middle Tier
Data Sources
Client Tier
ORDBMS
WEB/APP Server
Middle-TierApplication
Data Integration,Storage, Query,Management
OtherDataSources
Gateways
OLE/DBData source
authoringtools, etc.
HTTPHTTP
Remote messages
Nori, A., Databases in Internet Applications: Case Studies, in: Postmodern DBS, UC Berkeley, Spring 1999
30.10.2002 INF 312, Ifi, UiO, høst2002 4
Internet Applications• Entertainment
– Games, Music, Films, Multi-person chat• Public information
– Maps, Tax return helper• Advertisement
– Interactive catalogues for products and services• Medicine
– Diagnosis, Consultation, Remote surgery• Education
– Learning-on-demand (for a degree),virtual museums, tour remote spaces
• Engineering– Collaborative design, remote parallel simulation services
• Publishing– Submit, Review, Proof-editing (text and graphics)
• Tele-communication– Conferencing
• ...
3
30.10.2002 INF 312, Ifi, UiO, høst2002 5
Contents
• Database Systems and WWW Applications
• Digital Libraries– Definitions
– Underlying concepts
– Digital Libraries Initiative
– Digital Libraries (examples)
• Multimedia Database Systems
30.10.2002 INF 312, Ifi, UiO, høst2002 6
DefinitionsIn the Stanford Digital Library project, we view long-term digital library systems as collections of widely distributed, autonomously maintained services. Of course, a digital library system must include services that allow users to search over collections of information objects. Examples of searchable collections include traditionallibrary collections, digital images, e-mail archives, video, on-line books, and scientific article citation catalogs (containing onlymetadata about the articles, not the articles themselves).
While searching services are valuable, they are not the only kind of service in the digital library of the future. Remotely usable information processing facilities are also important digital library services. These services provide support for activities such as document summarization, indexing, collaborative annotation, format conversion, bibliography maintenance, and copyright clearance.
The Stanford Digital Library Technologies Project
4
30.10.2002 INF 312, Ifi, UiO, høst2002 7
Definitions
Digital libraries are organizations that provide the resources, including the specialized staff, to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use by a defined community or set of communities.
The Digital Library Federation (DLF)
Note:CS researchers tend to focus on digital libraries as content collected on behalf of user communities, while librarians focus on digital libraries as institutions or services.
30.10.2002 INF 312, Ifi, UiO, høst2002 8
Definitions1. Digital libraries are a set of electronic resources and associated technical capabilities for creating, searching and using information, including the full life cycle of the resources. In this sense they are an extension and enhancement of information storage and retrieval systems that manipulate digital data in any medium (text, images, sounds; static or dynamic images) and exist in distributed networks. The content of digital libraries includes data, metadata that describes various aspects of the data (e.g., representation, creator, owner, reproduction rights), and metadata that consists of links or relationships to other data or metadata, whether internal or external to the digital library.
2. Digital libraries are constructed -collected and organized- by [and for] a community of users,and their functional capabilities support the information needs and uses of that community. They are a component of communities in which individuals and groups interact with each other, using data, information, and knowledge resources and systems. In this sense they are an extension, enhancement, and integration of a variety of information institutions as physical places where resources are selected, collected, organized, preserved and accessed in support of a user community. These information institutions include, among others, libraries, museums, archives and schools, but digital libraries also extend and serve other community settings including classrooms, offices, laboratories, homes and public spaces)
Borgman et al. 1996
5
30.10.2002 INF 312, Ifi, UiO, høst2002 9
Contents
• Database Systems and WWW Applications
• Digital Libraries– Definitions
– Underlying concepts
– Digital Libraries Initiative
– Digital Libraries (examples)
• Multimedia Database Systems
30.10.2002 INF 312, Ifi, UiO, høst2002 10
Notions
• Content – the items in the library collection• Annotation – information added to (associated with) an image• Subject matter – focus of a collection, topics used in classification• Catalog – database (card file) of bibliographic records• Classification – assigning call number, adding keywords
• Rights to use - permissions• License agreements – contractual right to use• Copyright• Watermark – a subliminal pixal pattern to identify a digital work• Copy detection – verifying copying, searching for copies
• Search (40% of search queries on the web are reported to be single words)
• Metasearchers (services that provide unified query interfaces to multiple search engines. Thus users have the illusion of a single combined document source. Three main tasks: choosing the best sources to evaluate a query; evaluating the query at these sources; merging the query results from these sources.)
6
30.10.2002 INF 312, Ifi, UiO, høst2002 11
File Formats– Image formats
• TIFF Tagged Image File Format• GIF Graphics Image File Format• JFIF JPEG File Format• SPIFF Still Picture Interchange File Format• PICT Macintosh Picture• TGA TrueVision Targa file (bit mapped images)• EPS Encapsulated PostScript• CGM Computer Graphics Metafile• PhotoCD (Kodak)
– Picture and video formats• JPEG Joint Photographic Expert Group• Motion JPEG• MPEG Moving Pictures Expert Group
– Document formats• PostScript (Adobe)• PDF Portable Document Format (Adobe)
30.10.2002 INF 312, Ifi, UiO, høst2002 12
Compression
• Compression
– lossless • color 25%-50%-67%
• B/W 50%-90%;
– lossy up to 95%
• Compression formats
– CCITT Group III or Group IV
– JPEG
– JBIG An international compression standard– LZW.Subsampling (lossy)
• Compression schemes
– LZW Lempel-Ziv-Welch (lossless)
– MPEG Group of Pictures: IBBBPBBBPB…I– QuickTime (Apple)
7
30.10.2002 INF 312, Ifi, UiO, høst2002 13
Images in the Digital LibraryMost image-database systems store descriptive information about the images in a traditional text-based information retrieval system. An additional field containing the filename of the image is added to each record in this text-based system to link it to an image file. Images are selected by querying the text-based system. When the query is specific enough, the user requests a selected image (or a set of images) to view. Extensions to user-interface software look up the filename field(s) in the text record(s) and display the image(s), often in a new window. Each system handles the text/image relationship in its own way, and standards need to be developed to enable the interchange of image files among systems
Much research remains in the field of image databases, particularly with respect to image-quality needs. Further studies need to stratify types of collections, as well as users and uses of those collections, relating each to a series of required image qualities.
Howard Besser and Jennifer TrantIntroduction to Imaging: Issues in Constructing an Image Databasehttp://www.getty.edu/research/institute/standards/introimages/
30.10.2002 INF 312, Ifi, UiO, høst2002 14
Metadata and Cataloging
• Metadata Data about data (structure and access)
• OPAC On-line Public Access Catalog
• Content description structured vocabulariesdata-structure guidelines
• MARC MAchine Readable Cataloging
• Dublin Core a classification scheme
• Indexing abbreviation for works
organized for reference• MPEG7 metadata about MPEG data
8
30.10.2002 INF 312, Ifi, UiO, høst2002 15
The Dublin Core metadata standard• Document A work that is mostly textual in nature, but may include images, maps, tables or other formats.
– Abstract
– Article (a short published or pre-published piece) o Preprint Refereed Unrefereed
– Bibliography
– Correspondenceo Email Letter Listserv Newsgroup Postcard
– Essay
– Factsheet
– Manuscript
– Minutes
– Monograph (books and book-like objects) o Autobiography Biography Collection Manual Novel Play
– Pamphlet (and booklets) – Paper (conference paper or other short work) – Poem
– Serial (items published on a serial basis) o Journal Magazine Newsletter Newspaper Proceedings
– Story
– TechReport
– Thesiso Doctoral Masters
– Webpageo Organizational Personal
30.10.2002 INF 312, Ifi, UiO, høst2002 16
The Dublin Core (continued)• Image
– Interactiveo 3DObject VR (Virtual Reality)
– Movingo Animation Film
– Photographo Aerial Landscape Portrait
– Graphic
• Sound All sound formats, including MIDI files. – Ambient Effect Music Narration Speech
• Software Binary executables and source code. Both may be further subdivided with the name of the programming language used.
– Executableo Game
– Source
• Dataset Alphanumeric collections of data. – Bibliographic
– Cartographic
– Spectral
– Spreadsheet
– Statistical
o Miscellaneous Work of another or undetermined type. This is the default type if not explicitly stated.
9
30.10.2002 INF 312, Ifi, UiO, høst2002 17
Information Retrieval - Text
• Basic searching techniques:
– Linear search (can do regular expressions)– Inverted files– Hash tables
– Signature files (compressed linear search)
• Linear search requires no extra space, linear complexity in size, no preprocessing
• Inverted files cannot search for arbitrary expressions, (usually) must start at the beginnings of words, building index takes n log n time length of file (n words). Index overhead ranges from 25% to 200%.
• Hash coding is sensitive to the exact spelling of the word, and tends to scatter words nearly spelled the same; requires preprocessing and has slight storage overhead.
30.10.2002 INF 312, Ifi, UiO, høst2002 18
Information Retrieval - Images
• Indexes made for other purposes
– Citations
– Reviews– …
• Thumbnails
• Exploit layout formats (e.g., newspaper columns)
• Image alignment
– Centering
– Feature analysis, normalize rotation and X-Y orientation)
• Complementation: an image of a red rose will not normally have the keyword "red". Thus image features and associated words can complement and even disambuguate each other.
10
30.10.2002 INF 312, Ifi, UiO, høst2002 19
Contents
• Database Systems and WWW Applications
• Digital Libraries– Definitions
– Underlying concepts
– Digital Libraries Initiative
– Digital Libraries (examples)
• Multimedia Database Systems
30.10.2002 INF 312, Ifi, UiO, høst2002 20
US Digital Libraries Initiative (Phase I)
• University of California, Berkeley – Work-centered digital information services
• University of California, Santa Barbara – Spatially referenced map information
• Carnegie Mellon University – Full-content search and retrieval of video
• University of Illinois at Urbana-Champaign – Federating repositories of scientific literature
• University of Michigan – Intelligent agents for information location
• Stanford University – Interoperation mechanisms among heterogeneous services
Shared vision: an entire Net of distributed repositories, where objects of any type can be searched within and across indexed collections
11
30.10.2002 INF 312, Ifi, UiO, høst2002 21
US Digital Libraries Initiative (Phase I)
• University of California, Berkeley
– http://elib.cs.berkeley.edu/
– also see http://sunsite.berkeley.edu/• University of California, Santa Barbara
– http://www.alexandria.ucsb.edu/
• Carnegie Mellon University
– http://www.informedia.cs.cmu.edu/
• University of Illinois at Urbana-Champaign – http://dli.grainger.uiuc.edu/idli/idli.htm
• University of Michigan
– http://www.si.umich.edu/UMDL/
• Stanford University
– http://www-diglib.stanford.edu/
30.10.2002 INF 312, Ifi, UiO, høst2002 22
Examples of Technology Impact
• University of California, Berkeley
– Multivalent Documents
– Robust Hyperlinks and Robust Locations– TilePics
– …
• Carnegie Mellon University
– Informedia Digital Video Library System
– …• Stanford University
– Archival Digital Libraries Repositories
– Large Scale Copy Detection
– Google Search Engine
– …• …
12
30.10.2002 INF 312, Ifi, UiO, høst2002 23
Multivalent Documents
• Multivalent Annotations– Stored separately from the document they annotate– Appear in situ – as if part of the content of the document
• Hyperlinks• Highlights• Notes• Copy editor markup (executable)
– Three classes of behavior• Spans (anchored to points or intervals)
– E.g., Hyperlinks, Rollovers
• Lenses (anchored to geometric regions)– E.g., Bit Magnify, Optical Character Recognition
• Structures (within the document tree)– E.g., Extracting references in bibsys format
– Combining Annotations• Notemarks
– E.g., outlining, man pages
30.10.2002 INF 312, Ifi, UiO, høst2002 24
Robust Hyperlinks and Robust Locations
• URLs can be made robust
– if a web page moves to another location anywhere on the web, you can find it.
• Even if that page has been edited.
• Robust Hyperlinks
– URLs are augmented with a five or so words to make a content-based lexical signature
– If the URL's address-based portion breaksFeed the signature into any web search engine to find the new site of the page.
13
30.10.2002 INF 312, Ifi, UiO, høst2002 25
TilePics
• A file format designed to store tiled data of arbitrary type in a hierarchical, indexed format in order to provide fast retrieval.
• a fixed sized header
• tile index data• an optional gap• contiguous tile data
• optional attribute data
• Encapsulate a large amount of related, static data in a single file. • A one or two-dimensional dataset• At multiple scales of resolution or abstraction. • Tileable, based on x,y coordinates for quick localized access
• Store data at multiple levels of resolution • in multiple layers of tiles• each layer relates to the next by the same scale factor
• Zoom by drawing just the relevant tiles at the next layer down
30.10.2002 INF 312, Ifi, UiO, høst2002 26
Informedia Digital Video Library System
• IDVLS attempts to automate cataloging by:
– Recognizing speech
– Understanding text and language– Segmenting text
– Recognizing text within imagery
– Segmenting video
– Analysing video structure
– Image matching based on perceived color– Region matching for content-based image retrieval
– Detecting video shot boundaries
14
30.10.2002 INF 312, Ifi, UiO, høst2002 27
Archival Digital Libraries Respositories
File System
InfoMonitor
usersusers
Archival Repository Web Server
30.10.2002 INF 312, Ifi, UiO, høst2002 28
Large Scale Copy Detection
• CDS: Content Detection System– content publishers register their valuable digital content– CDS crawls the web
• compares the web content to the registered content
– notifies the content owners of illegal copies.
• Key challenges– accuracy, in terms of high precision and recall, – scalability, in terms of coping with several terabytes of data
(or several tens of millions of web pages)– resiliency to “attacks”
• Two prototypes– SCAM (Stanford Copy Analysis Mechanism, for text)– FRAUD (Finding Replicas of AUDio)
15
30.10.2002 INF 312, Ifi, UiO, høst2002 29
Google Search Engine
• PageRank: A Citation Importance Ranking
– Number of backlinks (~ citations)• Approximation of importance
• Citation analysis literature– Citation indexes
• Extreme variation in importance
– Large database of links: propagation
• Idealized Model
B
C
A
1 2
B and C arebacklinks of A
l1,2 = 1
l2,1 = 0
N number of outgoing linksni = li,j on page i (includes multiple
j=1, i≠j links to the same page)
N WiWj = (li,j — ) PageRank of page j
i=1, i≠j ni
Σ
Σ
30.10.2002 INF 312, Ifi, UiO, høst2002 30
Papers on the Creation of Google
• The Anatomy of a Large-Scale Hypertextual Web Search Engine, by Sergey Brin, Lawrence Page
• Dynamic Data Mining: Exploring Large Rule Spaces by Sampling, by Sergey Brin, Lawrence Page
• Computing Iceberg Queries Efficiently, by Min Fang, Narayanan Shivakumar, Hector Garcia-Molina, Rajeev Motwani, and Jeffrey D. Ullman
• The PageRank Citation Ranking: Bringing Order to the Web, by Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd
• Extracting Patterns and Relations from the World Wide Web,by Sergey Brin
• Finding near-replicas of documents on the web, by Narayanan Shivakumar, Hector Garcia-Molina
• Efficient Crawling Through URL Ordering, by Junghoo Cho, Hector Garcia-Molina, Lawrence Page
16
30.10.2002 INF 312, Ifi, UiO, høst2002 31
Contents
• Database Systems and WWW Applications
• Digital Libraries– Definitions
– Underlying concepts
– Digital Libraries Initiative
– Digital Libraries (examples)
• Multimedia Database Systems
30.10.2002 INF 312, Ifi, UiO, høst2002 32
ACM Portal: ACM Digital Library
Bibliographic information, abstracts, reviews, and the full-text for articles published in ACM periodicals and proceedings since its founding in 1947 are available in the library together with selected works published by affiliated organizations.
As of October 15, 2002, the Digital Library contains: – over 102,500 full-text articles from journals, magazines, and
conference proceedings.
– Tables of Contents with over 33,000 citations from articles published in journals and magazines from 1954 forward.
– Tables of contents with more than 69,000 citations from articlespublished in over 1100 volumes of conference proceedings since 1970.
17
30.10.2002 INF 312, Ifi, UiO, høst2002 33
ACM Digital Library
• The Digital Library presents all material associated with an article:– Bibliographic data
includes the title, author(s), publication, volume, issue, and page numbers of an article.
– Index termscompiled using article keywords and the ACM Computing Classification System.
– Abstracts available for most articles in the Digital Library.– Reviews from ACM Computing Reviews (Selected articles) – Full-text view or download complete articles.
Most articles are available in PDF -- some are available in other formats including HTML, postscript, and LaTeX.
– DOIWhen ACM submits a reference query and it is matched, a Universal Resource Name (URN) in the form of a Digital Object Identifier (DOI) is returned and inserted as an external link from ACM's site to the source for the material.
30.10.2002 INF 312, Ifi, UiO, høst2002 34
University of Oslo Digital Library Project
• “Post graduate theses in the digital library” (Hovedfagsoppgaver i digitalt bibliotek) is a pilot project where theses will be published in full text on the world wide web.
• A step in establishing a digital library where the University ofOslo shall keep electronic teaching materials and documents.
• A joint project between the USIT SGML group, the University of Oslo library, and other institutions, including the Institute ofInformatics
• Students are to use Microsoft Word and a template file providedby the project.
• Microsoft Word documents using the template styles can by automatically converted to HTML and SGML.
• http://www.digbib.uio.no/ (in norwegian)
18
30.10.2002 INF 312, Ifi, UiO, høst2002 35
Contents
• Database Systems and WWW Applications
• Digital Libraries
• Multimedia Database Systems– Definitions
– Example Application
• MM QoS Requirements
– MMDBMS Requirements
– MMDBMS Concepts
30.10.2002 INF 312, Ifi, UiO, høst2002 36
Definitions
• Multimedia (MM); loosely: any system that can be used to present information in more than one form: text, graphics, still images, animation, sound, video, special computer-generated effects.
The system should have user-friendly interactive interfaces that helpthe communication of complexly structured data.
• MMDBSs: are the DBSs that manage MM data, facilitate MM for presentations, and use specific tools for the storage, management, and retrieval of MM data.
19
30.10.2002 INF 312, Ifi, UiO, høst2002 37
Data Flow for a Multimedia Network Server
Network
StorageMultimedia server
Buffers Buffers
Graphics/videohardware
Audiohardware
Client
30.10.2002 INF 312, Ifi, UiO, høst2002 38
Contents
• Database Systems and WWW Applications
• Digital Libraries
• Multimedia Database Systems– Definitions
– Example Application
• MM QoS Requirements
– MMDBMS Requirements
– MMDBMS Concepts
20
30.10.2002 INF 312, Ifi, UiO, høst2002 39
Multimedia-Supported Learning of Practical Medical Procedures
• Provide realistic visualization of required practical skills• Proven to be pedagogically beneficial to view the multimedia lesson on
a procedure in a “learning on demand” setting before observing it in the clinic
• Lessons involve realistic multimedia elements (video and audio) recorded in Oslo hospitals, with expert commentary,
• Over 17,000 multimedia elements in OKSE-basen database. • Mostly on CD-ROM.• LoD over the Internet would enable
– Greater flexibility (time and location) for students– Other applications
• Paramedics review skills on demand in emergency situations• Doctors take courses in their office for lifelong learning
– Incremental release and revision of lessons or skill segments
2
30.10.2002 INF 312, Ifi, UiO, høst2002 40
Selective Multimedia Quality is Critical
• Quality of Service (QoS): – The collective effects of service performance which determine
the degree of satisfaction of a user of the service.– Performance, not operation (non-functional requirements,
independent of functional requirements)
• Video accuracy, for example, when draining the chest. – The video must accurately show location of arteries, ribs,
where the drain can safely be inserted to avoid arteries.• Audio fidelity, for example, when breathing is difficulty.
– The audio must be clear enough to differentiate between stridor, an obstruction of the large airways, and asthmatic breath sounds.
• Timing accuracy. – Some procedures should be viewed in near real time, possibly at
reduced video resolution and reduced audio fidelity.
• The critical quality focus may shift within a lesson. – The infrastructure should shift resources to the critical qualities
(and ignore others if necessary). 3
21
30.10.2002 INF 312, Ifi, UiO, høst2002 41
Use Case Model• A Use Case specifies a sequence of actions that the system can perform
and that yields an observable result of value to a particular actor.
– A “use case” is a story of how an actor achieves its goal using the system under design.
– The purpose of a use case is to define a piece of coherent behavior without revealing or dictating the internal structure of the system.
– A use case model structures a system’s use cases, representing its requirements and interface behavior to serve as input to its designers and implementers.
Record Element Recording Tech
Create Lecture Lecture Producer
Store Data
<<include>>
<<include>>
4
30.10.2002 INF 312, Ifi, UiO, høst2002 42
Use Case Model• A Use Case specifies a sequence of actions that the system can perform
and that yields an observable result of value to a particular actor.
– A “use case” is a story of how an actor achieves its goal using the system under design.
– The purpose of a use case is to define a piece of coherent behavior without revealing or dictating the internal structure of the system.
– A use case model structures a system’s use cases, representing its requirements and interface behavior to serve as input to its designers and implementers.
Record Element Recording Tech
Create Lecture Lecture Producer
Store Data
<<include>>
<<include>>
4
22
30.10.2002 INF 312, Ifi, UiO, høst2002 43
Main MSLoPP Use Cases
• Record multimedia elements
• Produce a lecture (a skills lesson, a presentation)• View a lecture
Use Case 3. View a lectureGoal: A medical student reviews a procedure over the internet, with acceptable QoS. Comments: The skills lab is linked to the LoD server by an 155 Mbyte/second ATM NetworkScenario: A student goes to the skills lab at the medical school and starts an internet browser.
(Alternatively, the student could connect to the internet from home.) The student logs in to the MSLoPP LoD server, then finds and selects the lesson on draining the chest. The system accesses the multimedia database for the selected lesson to determine what multimedia elements are to be included, what resources are required and what levels of QoS should be used to presentthe lesson. The system confirms that the student’s environment has adequate resources for playing the presentation. Then the system sets up the environment and starts playing the presentation. The student views the procedure, using pause, rewind, and play controls to review and replay key parts of the lesson, such as the location of the insertion point relative to the ribs and arteries. The student logs off from the LoD server, ending this use case.
Other Requirements: The LoD server must be able to detect the multimedia software available on theviewing station, and determine the QoS capabilities of the software and the viewing environment.
The student must be able to follow the focus to the emphasized elements of the presentation. Timing critical elements must be presented in near real time. Concurrent elements specified to be synchronized must be presented in synchrony.
30.10.2002 INF 312, Ifi, UiO, høst2002 44
Medical LoD Use Case Model
7
{ Notify of Event - Ask Question < 1 minute }
Paramedic
Conferee
{ Data Sequence strict in-order }
{ Time Delay < 10 msec }
Editor
Recording Tech
Set Synch Point
Producer
Remote Procedure
Doctor
View PresentationViewer
Teleconference
Lecture Producer
Edit Lecture
Record Stream<<extend>>
Setup BindingPlay Element
Create LectureStore Data
<<inc lude>>
<<include>>
<<inc lude>>
Oracle
Answer Question
<<include>>
Find Element
<<include>>
<<include>>
<<include>>
Play One Element
<<include>><<include>>
<<include>>
<<include>><<include>>
Find Lecture
<<include>>
Play Lecture
<<include>>
<<include>>
<<include>>
Ask Question
<<include>>
Notify of Event
<<include>>
<<include>>
View Answer
<<include>>
<<include>>
<<extend>>
View Lecture
<<include>>
<<include>>
<<extend>>
Student Produce Element
<<include>>
<<include>>
<<include>>
<<include>>
23
30.10.2002 INF 312, Ifi, UiO, høst2002 45
{ Notify of Event - Ask Question < 1 minute }
Paramedic
Conferee
{ Data Sequence strict in-order }
{ Time Delay < 10 msec }
Editor
Recording Tech
Set Synch Point
Producer
Remote Procedure
Doctor
View PresentationViewer
Teleconference
Lecture Producer
Edit Lecture
Record Stream<<extend>>
Setup BindingPlay Element
Create LectureStore Data
<<inc lude>>
<<include>>
<<inc lude>>
Oracle
Answer Question
<<include>>
Find Element
<<include>>
<<include>>
<<include>>
Play One Element
<<include>><<include>>
<<include>>
<<include>><<include>>
Find Lecture
<<include>>
Play Lecture
<<include>>
<<include>>
<<include>>
Ask Question
<<include>>
Notify of Event
<<include>>
<<include>>
View Answer
<<include>>
<<include>>
<<extend>>
View Lecture
<<include>>
<<include>>
<<extend>>
Student Produce Element
<<include>>
<<include>>
<<include>>
<<include>>
Medical LoD Use Case Model
7
30.10.2002 INF 312, Ifi, UiO, høst2002 46
Contents
• Database Systems and WWW Applications
• Digital Libraries
• Multimedia Database Systems– Definitions
– Example Application
• MM QoS Requirements
– MMDBMS Requirements
– MMDBMS Concepts
24
30.10.2002 INF 312, Ifi, UiO, høst2002 47
Requirements for MMDBSs
• represent arbitrary data types and specification of programs thatinteract with arbitrary data sources;
• query and modify (update, insert, delete) MM data; includingretrieval of MM data via associative search within MM data (minimally, text);
• specify and execute abstract operations on MM data, e.g., play, fast forward, pause, and rewind one-dimensional data like audioor text; to display, expand, and compress two-dimensional data like bit-mapped images;
• deal with heterogeneous data sources in a uniform manner; thisincludes access to data in these sources and migration of data from one data source to another.
Ability to ...
30.10.2002 INF 312, Ifi, UiO, høst2002 48
Requirements - 2
• MM & object-oriented data modeling concepts;• management of several kinds of magnetic and optical storage
devices appropriate for MM data handling;
• uniform management of very large data volumes => management of tertiary storage and multi-level storagehierarchies;
• support for realtime data processing =>appropriate scheduling and resource allocation techniques;
• support for storage and processing parallelism (performancerequirements);
• support for distribution => appropriate distributed DBMS concepts.
MM data storage and retrieval:
25
30.10.2002 INF 312, Ifi, UiO, høst2002 49
Storage space requirements for uncompressed digital multimedia data (examples)
Media type Specifications Data rate per sec.
Voice-quality audio 1 channel, 8-bitsamples at 8 kHz
64 Kbits
MPEG-encoded audio Equiv. to CD quality 384 Kbits
CD-quality audio 2 channels, 16-bitsamples at 44.1 kHz
1.4 Mbits
MPEG2-encoded video 640x480 pixels/frame,24 bits/pixel
0.42 Mbytes
NTSC-quality video 640x480 pixels/frame,24 bits/pixel
27 Mbytes
HDTV-quality video 1280x720 pixels/frame,24 bits/pixel
81 Mbytes
30.10.2002 INF 312, Ifi, UiO, høst2002 50
Requirements - 3
• “soft” realtime transfer requirements• “hard” transaction deadlines
• synchronization between different data streams (data types)
• user interactions (synchronous and asynchronous)
Realtime and synchronization issues:
=> dependent on data distribution, storage devices, compression techniques for the various data types,buffer management techniques, scheduling algorithms,data placement techniques, and communication bandwidth
26
30.10.2002 INF 312, Ifi, UiO, høst2002 51
Contents
• Database Systems and WWW Applications
• Digital Libraries
• Multimedia Database Systems– Definitions
– Example Application
• MM QoS Requirements
– MMDBMS Requirements
– MMDBMS Concepts
30.10.2002 INF 312, Ifi, UiO, høst2002 52
DBMS Concepts
• Data modeling: temporal object-oriented modeling and presenting (HCI) of multimedia data+ extra data types & operations
• Query processing and optimization: browsing, contentaddressing
• Storage management: optimization techniques
• Transaction management: realtime processing for readtransactions (presentations),write transactions (authoring) use a advanced transaction model(e.g., checkout-checkin with versioned data)
27
30.10.2002 INF 312, Ifi, UiO, høst2002 53
User Interface Design for MM Applications
• User interaction and user interfaces become much more complex if MM data is involved.
• State-of-the-art: buttons, text entry, scrollable areas, ...-> does not support interaction with continuous media
• New devices (e.g., cameras, microphones, loudspeakers, ...) have to be taken into account in addition to keyboard, mouse, monitor, and external devices (e.g., VCRs, ...) for input and output handling:
- simultaneous control of different devices- efficient handling of user interrupts- standardized interaction paradigms- support for pen + voice input- ...
30.10.2002 INF 312, Ifi, UiO, høst2002 54
Object-Oriented Data Modeling + ...
• text• graphic
• image
• audio
• speech
• video• generated media
Data types and operations for:
Temporal relationships:- Synchronization and realtime processing
Quality-of-Service:- to handle average delay, speed ratio, utilization,
jitter, skew, and reliability.
28
30.10.2002 INF 312, Ifi, UiO, høst2002 55
Required Data Model Concepts and Related Work
• Time independent data types
• Time dependent data types (continuous types)
• Temporal concepts: valid, transaction, and play time
• Temporal data models: TIGUKAT, T_Chimera, Mediadoc, SGML/HyTime, ...
• Multimedia data models: AMOS, SGML/HyTime, LMDM, ...
30.10.2002 INF 312, Ifi, UiO, høst2002 56
Concepts of TOOMM
Logical Data Model
Video 1
Video 2
Audio 1
Multimedia Objects
CPO1
Atomic Presentation Objects
Presentation Model
P_Video 13
P_Video 14
P_Video 15
P_Audio 11CompositePresentationObject
29
30.10.2002 INF 312, Ifi, UiO, høst2002 57
Example: Modeling a Video Object
Frame 0
Timestamp 0
Frame 1
Timestamp 1
Frame n
Timestamp n
TA 0
TA 1
TA n
Video 1
30.10.2002 INF 312, Ifi, UiO, høst2002 58
Type Hierarchy
MMDT
PTI_MMDT
PictureText Graphics
CGM
Video
PTD_MMDT
Stream
Audio Music Animation
LDU
Anim.
Component
Event
NoteSampleFrame
30
30.10.2002 INF 312, Ifi, UiO, høst2002 59
Type Hierarchy
MMDT
PTI_MMDT
PictureText Graphics
CGM
Video
PTD_MMDT
Stream
Audio Music Animation
LDU
Anim.
Component
Event
NoteSampleFrame
30.10.2002 INF 312, Ifi, UiO, høst2002 60
Play Time
Components of a stream multimedia object
TA 0
TS 0 LDU 0
TA 1
TS 1 LDU 1
TA n
TS n LDU n
Play Time
Components of a CGM multimedia object
TA 0
event 0 TS 0
TA 1
event 1 TS 1
TA n
event n TS n
31
30.10.2002 INF 312, Ifi, UiO, høst2002 61
EER Diagram
P_MMDT
P_Text P_Picture P_Graphics
P_PTI_MMDT
P_Music P_Anim.
P_CGM
P_Audio P_Video
P_Stream
P_PTD_MMDT
Temporal_reference
start stop1:11:1
1:11:11:10:1
MMDT1:1 0:M
Effect1:1 0:M
CPO
Parallel
Temporal_Relationships
Serial
0:M1:1
1:1
1:M
1
2
30.10.2002 INF 312, Ifi, UiO, høst2002 62
EER Diagram
P_MMDT
P_Text P_Picture P_Graphics
P_PTI_MMDT
P_Music P_Anim.
P_CGM
P_Audio P_Video
P_Stream
P_PTD_MMDT
Temporal_reference
start stop1:11:1
1:11:11:10:1
MMDT1:1 0:M
Effect1:1 0:M
CPO
Parallel
Temporal_Relationships
Serial
0:M1:1
1:1
1:M
1
2
32
30.10.2002 INF 312, Ifi, UiO, høst2002 63
Example: Using Temporal References
P_Video 1
P_Video 2
P_Text 3
Composite multimedia presentation
Recursive temporalreference list
Reference: Truedeviation: 0time_point: NA
Reference: Truedeviation: -5time_point: NA
Reference: Truedeviation: NAtime_point: 15
1
2
3
Multimediapresentation
objects
timeActual play time valueTemporal references
30.10.2002 INF 312, Ifi, UiO, høst2002 64
Building Multimedia Presentations
P_Video
Video
CPO
Actual play time valueTemporal references
time
P_start P_stop
Start Stop
33
30.10.2002 INF 312, Ifi, UiO, høst2002 65
Type: CPOName: Lecture_19_2_1998MTU_duration:1/44100
Type: P_AudioName: P_Audio 1Speed: 1Start: 0Stop: 31752000p_start.get_time_point()=0p_stop.get_time_point() = 31752000
Type: AudioName: PMC_Lecture_hour1_clip_1LDU_duration: 1/44100Duration: 31752000Content description:- (0, 4988, “Lecturer talks about files”)- (4989, 12134, “Lecturer talks about
directories”)
Type: P_VideoName: P_Video 1Speed: 1Start: 0Stop: 18000p_start.get_time_point()=0p_stop.get_time_point() = 31752000
Type: VideoName: PMC_Lecture_hour1_scene1LDU_duration: 1/25Duration: 1800Content description:- (0, 4988, “Lecturer talks about files”)- (4989, 12134, “Lecturer talks about
directories”)
Type: ParallelName: TR 1Temporal relationshiptype: EqualSkew tolerance: 80 ms
Type: P_HTMLName: P_HTML 1p_start.get_time_point() =3987233p_stop.get_time_point() = 7234443
Type: P_HTMLName: P_HTML 2p_start.get_time_point() =10234234p_stop.get_time_point() = 16230933
Type: P_Light_PenName: P_Light_Pen 1p_start.get_time_point() =4457111p_stop.get_time_point() = 6283324
Type: HTMLName: File System
Type: HTMLName: Directory Example
Type: Light_PenName: Drawing_objectsLDU_duration: 200Content description:- (0, 100, “Draw a bow in File System”)- (101, 200, “Draw a dot”)
ExampleCPO
30.10.2002 INF 312, Ifi, UiO, høst2002 66
Query Processing and Optimization
- Browsing: efficient location of data elements in very large amounts of data,exact-match (pattern-matching) queries (e.g., text)and similarity-based queries (e.g., images, ...)-> query refinement-> set-oriented and navigation-oriented browsing techniques
- Content addressing: efficient location of data with complex data types like images (difficult to access in realtime using pattern-recognition techniques) comprises: natural language understanding,speech processing,vision,and user modeling
34
30.10.2002 INF 312, Ifi, UiO, høst2002 67
Meta-Data Management
• Meta-data needed especially for continuous data to support retrieval
• Textual data describing contents of audio and video segments
• Content search mostly performed on meta-data• Problems:
– Modeling of meta-data
– Meta-data acquisition
– Association of meta-data to “real” data
30.10.2002 INF 312, Ifi, UiO, høst2002 68
Storage Management Issues
• addressing techniques
• access paths
• data placement techniques:clustering, partitioning, allocation
• system buffer management:paging, ...
• disk scheduling:sweeping, deadline-driven, …
35
30.10.2002 INF 312, Ifi, UiO, høst2002 69
Data Placement
• Clustering and partitioning:
– data striping and data interleaving
• Allocation:
– contiguous placement
– constrained placement
– log-structured placement
Controller
Sector 0Sector 1Sector 2
Sector 0Sector 1Sector 2
Sector 0Sector 1Sector 2
Logical sector 0
30.10.2002 INF 312, Ifi, UiO, høst2002 70
Disk Scheduling
• Traditional algorithms
– FIFO (first come, first served)
– SSTF (shortest seek time first)– SCAN (elevator algorithm)
1.Generation MM algorithms
– EDF (earliest deadline first)
– SCAN-EDF
– GSS (grouped sweeping scheme)2.Generation MM algorithms
– two-phase algorithms
- reduce seek time- reduce rotational latency- increase throughput- fair stream access- real-time constraints?
36
30.10.2002 INF 312, Ifi, UiO, høst2002 71
MMDBS: Conclusions
• investigated functionality needed to support MM applications
• illustrated how object-oriented and other modern DBMS technologies can be applied to realize MMDBMS
• alternative “levels” of application support by DBMS
• open issues:- effective storage models- MM query languages and processing techniques(handling of imprecise queries)
- ...
• Role of (MM)DBS in distributed MM systems
30.10.2002 INF 312, Ifi, UiO, høst2002 72
Conclusions - State-of-the-Art
• Multimedia file systems and multimedia storage servers for special multimedia applications exist today
• Implement the presented concepts
• Acceptable performance• Multimedia database systems are still under development,
certain aspects are solved
• Retrieval problems not yet solved in a satisfying manner