Date post: | 13-Jan-2016 |
Category: |
Documents |
Upload: | bama-raja-segaran |
View: | 11 times |
Download: | 0 times |
O. Abou Khaled, D. Lalanne, J. BapstAssistants : Tadeusz Senn, Sandro Gerardi, Florian Evéquoz, Bruno Dumas
[5] Multimedia content representation &
Information Visualization
Multimodal Interfaces
The MMI team 2MMI_05
Multimodal Interfaces OverviewOverview
Multimedia Content RepresentationIntroduction, definitions, etc.
Audio, Image, VideoMultimedia content protection (Watermarking)Multimedia storage and transmissionDigital library, content-based multimedia systems.
Multimedia Information RetrievalAudio, image, video
Examples & research Projects
Goal: have a wide idea about Multimedia fundamental and multimedia system construction/access (over the networks)Multimedia Information Retrieval
The MMI team 3MMI_05
Multimodal Interfaces OverviewOverview
Information VisualizationIntroductio, DefinitionsThe Power of Information VisualizationVisualization for What ?
Examples of Information Visualization
Goal: have a wide idea about Visualization techniques
The MMI team 4MMI_05
Multimodal Interfaces
WhatWhat’’s Multimedia?s Multimedia?Multi: ManyMedia:
A means to distribute and represent information: Text, graphics, pictures, voice, sound and music..
Perception media (how do humans perceive information?)– Audio/visual media
Representation media (how is information encoded?)– ASCII, JPG, MPEG, PAL.
Presentation media (medium used for output/input)– Input/output media (keyboards, papers)
Storage media (Where is information stored?)– Magnetic disk, optical disk
Multimedia:To distribute and present information coded as
Text, Graphics, animation, audio and video..By Computer, TV, phone, etc.
Multimedia: a working definitionA combination of two or more categories of information having different transport signal characteristics. Typically, one medium is a continuous medium while another is discreteImage, audio, video and graphics are usually the examples of media
The MMI team 5MMI_05
Multimodal Interfaces Why we need Media?Why we need Media?
Our words cannot exactly describe the images.
Speaking is faster than writingListing is easier than readingShowing is easier and clearer than describing
The MMI team 6MMI_05
Multimodal Interfaces The Types of MediaThe Types of Media
Perception MediaThe nature of information perceived by humans (How do humans perceive information?)Auditory media and Visual media
Representation MediaHow information is represented internally to the computer (How is information encoded in the computer?)Character (ASCII) , image (JPEG), audio (PCM) , video (TV signal, MPEG)
Presentation MediaPhysical means used by systems to reproduce information for humans (Which medium is used to output information from the or input in the computer?)Monitors, keyboard, cameras (Input output devices)
The MMI team 7MMI_05
Multimodal Interfaces The Types of MediaThe Types of Media
Storage MediaPhysical means for storing computer data (where is information stored?)Magnetic tapes, magnetic disks, optical disks.
Transmission MediaPhysical means that allow the transmission of signals. (which medium is used to transmit data?)Cables, Radio tower, satellite...
Information Exchange MediaAll data media used to transport information. (which data medium is used to exchange information between different locations?)
The MMI team 8MMI_05
Multimodal Interfaces ContextContext
Recent advances in the technologies of communication, computer science and electronics have facilitated production, and distribution of multimedia dataHuge quantity of multimedia data is accessible in different domains (education, entertainment, communication, etc.)Rich content of multimedia data
Integration of different media (audio, video, still images, text)Complex relationships (spatio-temporal, composition, etc.)
Professional/non professional users need to access multimedia data
=> Important need for elaborate multimedia indexing and retrieval techniques.
The MMI team 9MMI_05
Multimodal Interfaces Multimedia IR vs. Text IRMultimedia IR vs. Text IR
People are used to express their needs using natural language Natural language queries are frequently used for text information retrievalMatching between text queries and text documents is more or less straightforwardMultimedia documents contain non-textual dataTo provide text queries on top of multimedia documents, the document content should be described (annotated) textually
The MMI team 10MMI_05
Multimodal Interfaces
Two approaches in retrieving multimedia Two approaches in retrieving multimedia documentdocument
Text-based retrievalApproach
Annotating the multimedia content with text descriptions and allowing text queries on top of these descriptions
ProsSome multimedia contain text zones which can be used for description
ConsAutomatic description is very hard to achieveManual description is time consumingSome contents are difficult to describe.
The MMI team 11MMI_05
Multimodal Interfaces
Content-based retrievalApproach
Querying the content based on similarities with a given multimedia example, a sketch, etc.
ProsCan be useful for restricted/specialised content (such as logo databases)
ConsExamples are not always easy to find/createPerformances are very limitedSemantic retrieval is almost impossible.
Two approaches in retrieving multimedia Two approaches in retrieving multimedia documentdocument
The MMI team 12MMI_05
Multimodal Interfaces What is a Multimedia System?What is a Multimedia System?
A system that involves:Generation
Representation
Storage
Transmission
search and retrieval
delivery of multimedia information
A system that involves:production/authoring tools
compression and formats
file system design
networking issues
database management
server design, streaming
The MMI team 13MMI_05
Multimodal Interfaces
How to Build Multimedia Database Systems?How to Build Multimedia Database Systems?
How to build text database?
Text document Natural language processing
Tree-based indexingText database
Multimedia data Multimedia analysis
Multimedia Indexing
Multimedia database
Yahoo, Google
TransmissionActions
Transmission
Actions
The MMI team 14MMI_05
Multimodal Interfaces AudioAudio
Sound FundamentalsSound is a continuous wave that travels through the air. The wave is made up of pressure differences.Sound is detected by measuring the pressure level at a location.Sound waves have normal wave properties (reflection, refraction,diffraction, etc.).
Sound reflects off walls if small wave length– reflection
Sound bends around walls if large wave lengths– diffraction
Sound changes direction due to temperature shifts– refraction
Not covered subjectsSignal Fundamental, Human Perception, Sound Quality Measures, Sound Codec Standards, etc.
The MMI team 15MMI_05
Multimodal Interfaces Audio Information RetrievalAudio Information Retrieval
The Basics of Audio Search and Audio Information Retrieval
Audio Information Retrieval is the process of retrieving audio information by using various available resources:
If the available resources are series of keywords annotated manually -> Text-based retrievalText based searching for audio information is most commonIf the available resource is a piece of audio information (ex: a melody of a song) -> Content-based retrieval.Content based audio research is promising and attractive, but there is a long way to go …
The MMI team 16MMI_05
Multimodal Interfaces Audio Information RetrievalAudio Information Retrieval
What are some Audio IR Mechanisms?Annotation based Audio RetrievalContent-based Audio Retrieval
Annotation based Audio RetrievalPeer-to-peer file sharing softwareFTP, Streaming audio, WebsitesOnline network drives, Clip Art
ProblemsSpyware, virus, unrelated results returned
The MMI team 17MMI_05
Multimodal Interfaces Audio Information RetrievalAudio Information Retrieval
Content-based Audio RetrievalAudio feature extractionAudio classification and Retrieval
What’s content-based retrieval? The retrieval is facilitated by the information content in contrast to simple retrieval based on manual index terms or keywords.
What’s the content?The semantic concept meaning of the information.
Why key-words annotation is not good.Subjective and expensiveInsufficient
How to describe the content?Features:Audio: Loudness, bandwidth, pitch..Image: Color, Texture, Objects..Video: Temporal information change, Image+Audio
The MMI team 18MMI_05
Multimodal Interfaces General Audio Retrieval FrameworkGeneral Audio Retrieval Framework
Audio Repository Features Extraction
Classification: Male, Laughing,
Indexing:Using feature describe
audio unit
Audio DatabaseRetrievalAudio Example Features
ExtractionUser
InterfaceKeywords
Browsing
The MMI team 19MMI_05
Multimodal Interfaces ContentContent--based Audio Retrievalbased Audio Retrieval
General Audio Features for information retrieval.Time-domain FeaturesFrequency-domain Features
Audio classification Goal
Classify audio into speech, music, and other categories and subcategories
Motivation Different audio types require different processing and indexing retrieval techniquesDifferent audio types have diff signification to different application Search space after classification is reduced to a particular audio class
The MMI team 20MMI_05
Multimodal Interfaces ContentContent--based Audio Retrievalbased Audio Retrieval
A demo on Speech and Music Classificationhttp://www.musclefish.comhttp://www.soundfisher.com/download/
Content-based Audio database browse and retrievalhttp://www.soundfisher.com/index_flash.htmlhttp://www.musclefish.com/frameset.html
Conclusions Audio Information Retrieval
Annotation Based:– Peer-to-peer system, ftp
Content Based:– Audio Feature extraction– Audio Classification and Retrieval– Music Retrieval.
The MMI team 21MMI_05
Multimodal Interfaces Image Image
What’s an Image?An image is a 2D rectilinear array of Pixels
Image Data StructurePixel
Picture elements in digital images, it usually indicate a “point” in an image.Image Resolution
The number of pixels in a digital image.Depth
The number of bit used to characterize each pixel information.– Bit Map: 1 bit/pixel, Gray scale: 2-8bits/pixel, Full color: 24 bits/ pixel, Color
mapped: 8 bits/ pixel
The Quality of the imageResolution (The number of pixel)Image DepthAdopted compression algorithm (if adopted)
Not covered subjectsImage Depth, Monochrome/Bit-Map, Dithering, Gray Scale Images, 8-bit/24-bit Color Images, Image Format, etc.
The MMI team 22MMI_05
Multimodal Interfaces Image Retrieval Image Retrieval
Text-based retrieval Using the text surrounding the image
Text close to the image in the containing document – URL: http://www.host.com/animals/dogs/poodle.gif– Alt text: <img src=URL alt="picture of poodle">– Hyperlink text: <a href=URL>Sally the poodle</a>
Using the text inside the imageRequired OCR technique
Some image search engines use this techniqueGoogle, Altavista, www.ditto.com
ProsEasy to implement and useUseful for simple and non-professional image retrieval
ConsIt is incomplete and subjectiveSome features are difficult to define in text such as texture or object shape
The MMI team 23MMI_05
Multimodal Interfaces Image retrievalImage retrieval
The MMI team 24MMI_05
Multimodal Interfaces Image retrievalImage retrieval
Almost impossible to describe all the contents.Some contents are difficult to describe.
The MMI team 25MMI_05
Multimodal Interfaces Image retrievalImage retrieval
Content-based image retrieval (CBIR)The commonly acceptable way is
To show a sample image, or draw a sketch of the desired images to computer, and ask the system to retrieve all the images similar to that sample image or sketch.It relies on features such as colour, shape, texture
ExamplesIBM’s Query By Image Content (QBIC)
Retrieves based on visual content, including properties such as color percentage, color layout and texture.Fine Arts Museum of San Francisco uses QBIC.
Virages’s VIR Image Engine Can search based on color, composition, texture and structure.
ChallengesThe term “similarity” has different meaning for different people.Even the same person uses different similarity measures in different situations.
The MMI team 26MMI_05
Multimodal Interfaces Images containing similar colorsImages containing similar colors
The MMI team 27MMI_05
Multimodal Interfaces Images containing similar shapeImages containing similar shape
The MMI team 28MMI_05
Multimodal Interfaces Images containing similar contentImages containing similar content
The MMI team 29MMI_05
Multimodal Interfaces Variety of SimilarityVariety of Similarity
• Similar color distribution
• Similar texture pattern
• Similar shape/pattern
• Similar real content
Degree of difficulty
Histogram matching
Texture analysis
Image Segmentation,Pattern recognition
Eternal goal :-)
The MMI team 30MMI_05
Multimodal Interfaces IndexingIndexing
Use any text available: Title, Subject, CaptionUse content information: Colour histogram, Shape, Texture
The MMI team 31MMI_05
Multimodal Interfaces Image retrievalImage retrieval
Demonstrationshttp://zomax.wins.uva.nl:5345/ret_user/http://www.ifp.uiuc.edu/~nakazato/CBIR/
Other demoshttp://eidetic.ai.ru.nl/egon/cogw/co440/CBIR_Demo-s.htmlhttp://www.ee.surrey.ac.uk/Research/VSSP/imagedb/demo.htmlhttp://www.fb9-ti.uni-duisburg.de/rotdemo.htmlhttp://mmdb.ece.ucsb.edu/~demo/corelacm/
ConclusionsImage RetrievalContent-based Image Retrieval (CBIR)General Measures:
Gray intensity, Color, Texture, ShapeDistances Measures:
Color similarity, Texture similarity, Shape similarity, Object and relationship similarity.
The MMI team 32MMI_05
Multimodal Interfaces VideoVideo
Video consists of imagesWhat’s the interval of images?
Sampling rates must be high enough to avoid motion "aliasing”.1. At least 15 frames/Sec2. 30 frames/ Sec appears smoothly3. At least 50 frames/ sec needed in the ideal case
Not covered subjectsVideo standards, Broadcast System Elements, Analog Video Representations, human perception, Compression Standards, Video Processing Techniques, etc.
The MMI team 33MMI_05
Multimodal Interfaces ContentContent--based Video Retrievalbased Video Retrieval
ApplicationsVideo Surveillance
Find where else the person appearsExperience On-Demand
Help to remember previous eventsProvide useful information on traveling
Equipment on cars to retrieve useful multimedia information according to your location/preference
Typical Retrieval FrameworkUser : provide query information that represents his informationneeds Database: store a large collection of video dataGoal: Find the most relevant shots from the database
Shots: “paragraph” in video, typically 20 – 40 seconds, which is the basic unit of video retrieval
The MMI team 34MMI_05
Multimodal Interfaces Sample QuerySample Query
Text : Find pictures of George Washington
Image: Video:
The MMI team 35MMI_05
Multimodal Interfaces Bridging the Gap Bridging the Gap
Video Database User
Result
The MMI team 36MMI_05
Multimodal Interfaces Automatically Structure Video DataAutomatically Structure Video Data
The first step for video retrieval: Video “programmes” are structured into logical scenes, and physical shots If dealing with text, then the structure is obvious:
paragraph, section, topic, page, etc.
All text-based indexing, retrieval, linking, etc. builds upon this structure;Automatic shot boundary detection and selection of representative keyframes is usually the first step;
The MMI team 37MMI_05
Multimodal Interfaces Typical automatic structuring of videoTypical automatic structuring of video
A set of shots
a video document
Keyframe browser combined with transcript or object-based search
The MMI team 38MMI_05
Multimodal Interfaces Bridging the Gap Bridging the Gap
Video DatabaseUser
Video Structure Information Need
Result
The MMI team 39MMI_05
Multimodal Interfaces Ideal solution Ideal solution
Video DatabaseUser
Video Structure Information NeedUnderstanding the semantic meaning and retrieve
Result
The MMI team 40MMI_05
Multimodal Interfaces Ideal solution Ideal solution
Video DatabaseUser
Video Structure Information NeedUnderstanding the semantic meaning and retrieve
Result
However, 1. Hard to represent query in natural
language and for computer to understand2. Computers have no experience3. Other representation restriction like
position, time
The MMI team 41MMI_05
Multimodal Interfaces Alternative SolutionAlternative Solution
Video DatabaseUser
Video Structure Information Need
Match and combine
Result
Provide evidence of relevant information ( text, image, audio)
The MMI team 42MMI_05
Multimodal Interfaces EvidenceEvidence--based Retrieval Systembased Retrieval System
General framework for current video retrieval systemVideo retrieval based on the evidence from both users and database, including
Text information Image informationMotion informationAudio information
Return a relevant score for each evidenceCombination of the scores
The MMI team 43MMI_05
Multimodal Interfaces More Evidence in Video RetrievalMore Evidence in Video Retrieval
Video DatabaseUser
Video Structure Information Need
Text Information Keyword
Image Information Query
Images
MotionInformation
Audio Information
Motion
Audio
The MMI team 44MMI_05
Multimodal Interfaces Combination of multiCombination of multi--modal resultsmodal results
Difference characteristics between multi-modal informationText-based Information: better for middle and high level queries
e.g. Find the video clip of dancing women wearing dresses Image-based Information: better for low and middle level queries
e.g. Find the video clip of green trees
Combination of multi-modal information
The MMI team 45MMI_05
Multimodal Interfaces Video Video retrievalretrieval
Primitives of Color Moments Methodhttp://debut.cis.nctu.edu.tw/Demo/ContentBasedVideoRetrieval/CBVR/PrimitivesE/index.html
Dominant Colors Methodhttp://debut.cis.nctu.edu.tw/Demo/ContentBasedVideoRetrieval/CBVR/DominantE/index.html
Combination Methodhttp://debut.cis.nctu.edu.tw/Demo/ContentBasedVideoRetrieval/CBVR/demoE.html
The MMI team 46MMI_05
Multimodal Interfaces Ideal solution: TSR Case study Ideal solution: TSR Case study
Video DatabaseUser
Video Structure Information NeedUnderstanding the semantic meaning and retrieve
Result
The MMI team 47MMI_05
Multimodal Interfaces TSR StudyTSR Study
Indexing
Retrieval
TV news
Production
The MMI team 48MMI_05
Multimodal Interfaces Problems (I): Problems (I): IndexingIndexing
Non-optimal reuse of information
Production
Indexing
Retrieval
ScriptSubtitleTeletextEdited VideoDescribed rushesJournalist commentaries
Video segments described following the TSR scheme (places, events, persons, dates, etc.)
Ineffective information exchange
Described video relevant to the
query
The MMI team 49MMI_05
Multimodal Interfaces MPEGMPEG--7 in Practice7 in Practice
Library of audiovisual descriptions Coverage
Providing comprehensive set of descriptions needed in known audiovisual applications
InteroperabilityData Definition Language based on the W3C XML Schema
The MMI team 50MMI_05
Multimodal Interfaces Example of MPEGExample of MPEG--7 Description7 Description
TV news audiovisual data
The MMI team 51MMI_05
Multimodal Interfaces Example of MPEGExample of MPEG--7 Description7 Description
<Mpeg7>…
<StillRegion id = “news”></StillRegion>
…</Mpeg7>
Title
The MMI team 52MMI_05
Multimodal Interfaces Example of MPEGExample of MPEG--7 Description7 Description
<Mpeg7><Mpeg7>……
<<StillRegionStillRegion id = id = ““newsnews””> > <<SpatialDecompositionSpatialDecomposition>>
<StillRegion id = <StillRegion id = ““backgroundbackground””> > <<VisualDescriptorVisualDescriptor
xsi:typexsi:type==““DominantColorTypeDominantColorType””>>110 108 140 110 108 140
</</VisualDescriptorVisualDescriptor>><StillRegion id = <StillRegion id = ““speakerspeaker””>>
</</SpatialDecompositionSpatialDecomposition>></</StillRegionStillRegion>>……
</Mpeg7></Mpeg7>
Back ground features
The MMI team 53MMI_05
Multimodal Interfaces
Example of MPEGExample of MPEG--7 Description7 Description
<Mpeg7><Mpeg7>……
<<StillRegionStillRegion id = id = ““speakerspeaker””>><TextAnnotation><TextAnnotation>
<<FreeTextAnnotationFreeTextAnnotation> Journalist > Journalist Anna BlancoAnna Blanco……
</</FreeTextAnnotationFreeTextAnnotation> > </</TextAnnotationTextAnnotation>><<MaskMask xsi:typexsi:type="="SpatialMaskTypeSpatialMaskType">">
<<SubRegionSubRegion>><Poly><Poly>
<<CoordsCoords> 80 288, 100 200, > 80 288, 100 200, ……, , 352 288 352 288
</</CoordsCoords>></Poly></Poly>
</</SubRegionSubRegion>></</MaskMask>>
</</StillRegionStillRegion> > ……
</Mpeg7></Mpeg7>
More features
The MMI team 54MMI_05
Multimodal Interfaces Semantic Views Model (I)Semantic Views Model (I)
GoalProvide a common TV news retrieval platform for professional and non-professional usersCover a rich combination of content descriptions and AV structure via a simple model
DesignAnalyzing queries asked by different users in Télévision Suisse Romande(TSR) revealed a set of common description types
Example
A news item in the context of Euro 2000 football games containing a shot of at least 5 seconds
showing a French football supporter saying « que le meilleur gagne »
Find
The MMI team 55MMI_05
Multimodal Interfaces Semantic Views Model (II)Semantic Views Model (II)
=> Users describe AV information following a set of “Views”
Its duration isat least 5 seconds
It isin the contextof EURO 2000 football games
It isa news item
It contains a shot showing an French football supporter
I can hear« Que le meilleur
gagne! »
A video segment
PhysicalView
ThematicView
Production View
VisualView
AudioView
The MMI team 56MMI_05
Multimodal Interfaces
TV news indexing and retrieval platform of COALTV news indexing and retrieval platform of COAL
COALA
AudiovisualRepository
system
Descriptionsystem
TV newsMPEG-7 corpus
Retrieval system
Visualizationsystem
COALA
AudiovisualRepository
system
Descriptionsystem
TV newsMPEG-7 corpus
Retrieval system
Visualizationsystem
The MMI team 57MMI_05
Multimodal Interfaces Indexing toolIndexing tool
Demo
The MMI team 58MMI_05
Multimodal Interfaces Indexing toolIndexing tool
The MMI team 59MMI_05
Multimodal Interfaces
Querying tool based onQuerying tool based onSemantic Views ModelSemantic Views Model
BasicViewEntities ViewDescriptions
IntraViewRelations
InterViewRelations
Five Views
The MMI team 60MMI_05
Multimodal Interfaces Hierarchical browsing toolHierarchical browsing tool
Demo
The MMI team 61MMI_05
Multimodal Interfaces Semantic views browsing toolSemantic views browsing tool
The MMI team 62MMI_05
Multimodal Interfaces Conclusion and discussionConclusion and discussion
Recent approaches to the problem of multimedia IR are mostly based on the extraction of text/audiovisual featuresExtraction/creation of descriptions is hard and expensive
Manual approaches are time consumingAutomatic approaches are not always possible, some are not sufficiently accurate
Multimedia descriptions are very preciousApplications need to exchange themCreated descriptions should be conserved
Important need for a standard multimedia description language
The MMI team 63MMI_05
Multimodal Interfaces Information visualizationInformation visualization
What is Information Visualization?Visualize: to form a mental image or vision of …
Visualize: to imagine or remember as if actually seeing.American Heritage dictionary, Concise Oxford dictionary
“Transformation of the symbolic into the geometric” (McCormick et al., 1987)
“... finding the artificial memory that best supports our natural means of perception.'‘ (Bertin, 1983)
The depiction of information using spatial or graphical representations, to facilitate comparison, pattern recognition, change detection, and other cognitive skills by making use of the visual system (Hearst 03).
The MMI team 64MMI_05
Multimodal Interfaces
The Power of VisualizationThe Power of Visualization
Visualization for Problem SolvingVisualization for ElicitingKnowledge from Data
statistics
Visualization for ClarificationMappy, etc.
The MMI team 65MMI_05
Multimodal Interfaces
Two Different Primary Goals:Two Different Primary Goals:Two Different Types of Two Different Types of VizViz
Explore/CalculateAnalyzeReason about Information
CommunicateExplain Make DecisionsReason about Information
In more detail, visualization should:Make large datasets coherent
(Present huge amounts of information compactly) Present information from various viewpoints Present information at several levels of detail
(from overviews to fine structure) Support visual comparisons Tell stories about the data
The MMI team 66MMI_05
Multimodal Interfaces Human Perceptual FacilitiesHuman Perceptual Facilities
Use the eye for pattern recognition; people are good atScanning, recognizing, remembering images
Graphical elements facilitate comparisons via Length, shape, orientation,texture
Animation shows changes across time Color helps make distinctionsAesthetics make the process appealing
The MMI team 67MMI_05
Multimodal Interfaces Information visualization: contextInformation visualization: context
Large amount of information vs. relatively small computer screen.
locating a given item of information? interpreting an item of information? relating an item with some other items?
Two Category of ApproachesNon-distortion-oriented approaches.
Displaying a portion of the information at a time;Scrolling or paging accessProviding hierarchical accessStructure-specific presentation
Distortion-oriented Approaches:Distort an image of large amount of information so that it can fit in screen.Allow the user to examine a local area in detail;At the same time, present a global view of the information space;Provide navigation mechanism.
The MMI team 68MMI_05
Multimodal Interfaces Information visualization: contextInformation visualization: context
The MMI team 69MMI_05
Multimodal Interfaces DistortionDistortion--based Techniquesbased Techniques
Idea of Distortion-based TechniquesCo-existence of local details with global context at reduced magnification.A focus region to display detailed information.Demagnified view of the peripheral areas is presented around the focus area.A distorted view is created by applying a transformation function to an undistorted image.A magnification function, provides a profile of the magnification factors for the entire area of image.
Ex. Bifocal DisplayDistortion at one or two dimensions with linear transformation function.Combination of detailed view and two distorted side views.
The MMI team 70MMI_05
Multimodal Interfaces DistortionDistortion--based Techniquesbased Techniques
Ex. Polyfocal DisplayPerspective Wall
A conceptual descendent of the Bifocal display.Smoothly integrated detailed and contextual views.Side panels are demagnified directly proportional to their distance from the viewer.
The MMI team 71MMI_05
Multimodal Interfaces DistortionDistortion--based Techniquesbased Techniques
Fisheye ViewBasic idea: more relevant information presented in great detail; the less relevant information presented as an abstraction.Relevance is computed on basis of the importance of information
elements and their distance to the focus.
Graphical Fisheye ViewAn extension of the fisheye view concept.Could be also considered as a special case of polyfocal display.
The MMI team 72MMI_05
Multimodal Interfaces Why Visualize Text?Why Visualize Text?
To help with Information Retrievalgive an overview of a collectionshow user what aspects of their interests are present in a collectionhelp user understand why documents retrieved as a result of a query
The MMI team 73MMI_05
Multimodal Interfaces Exploiting Visual PropertiesExploiting Visual Properties
Analyzing retrieval resultsKartOO http://www.kartoo.com/
Grokker http://www.groxis.com/service/grok
The MMI team 74MMI_05
Multimodal Interfaces Exploiting Visual PropertiesExploiting Visual Properties