Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 212 times |
Download: | 0 times |
1
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
JOANNEUM RESEARCH
a TRADITION of INNOVATION
Herwig Rehatschek
Institute of Information Systems & Information Management
Cross Media Aspects in the Areas of Media Monitoring
and Content Production
2
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Contents
Cross media aspects in media production (NM2 project) in media monitoring (DIRECT-INFO project)
Conclusions
Outlook
3
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
NM2 (New Media for a New Millennium) - MotivationTraditional distribution channels have developed their specific compelling media forms art, science and technology of production, direction and editing in
non-linear narrative are poorly formed
NM2 investigates on non-linear, interactive narratives tools for production and delivery
Main Objectives of NM2 proof of concept to produce compelling non-linear narratives production of compelling media
aesthetically pleasing truly engaging (30+ minutes) plausible from a business perspective
4
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Basic Technical Concepts
Narrative Objects (NO) MPEG-7 to describe audiovisual features OWL to describe semantics of NO
Narrative Structure Language internal representation of a „story world“ basic narrative structures
5
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Media Semantic Mapping Utility (Monsoon)
Tabs for objects, concepts and relations
Definition of an object by low level features
Definition of an object by using other objects
6
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
System Architecture
metadata essence stories
Database interface
Middleware API
user info
Management Realisation Engine Profile Learning
Inge
stio
nDescription•Manual•Automated•Ontology-based
AuthoringUsing narrative
structure language
Session ManagementFor high volumes
End-user ApplicationIncluding application preview
tools delivery
middleware
JRS
7
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Main Project Results
Definition of a Narrative Structure Language describing multimedia objects in narrative context
Bridging semantic gap MPEG-7 & OWL linking lower level features to higher level concepts
Six NL productions with compelling media
Targeted for three end-user devices Windows XP Media Centre and TV card Game consoles Mobile phones
8
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
DIRECT-INFO - Background Media Monitoring Companies
Measurement of advertisement expenditure
Find out how much a company has spent for advertising on a product level over different media
Typical business cases advertisement information creativity (new campaigns) (targeted in MediaCampaign) sponsorship tracking (targeted in DIRECT-INFO)
9
18/0
4/23
ISO
9001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Sponsorship Tracking
Sport teams need sponsors to finance their activities
Measure efficiency of sport sponsorship broadcasting & print media convince sponsors to invest again currently done manually
Technically very complex task context information needed multimodal analysis necessary
10
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Main Technical Innovations
Identification of logos within visual content
Multimodal event modelling
Genre classification
Text analysis (pos. / neg. mentions)
Com
bin
ati
on
of
Semantic correlation & fusion of multi-modal content analysis tosemantically meaningful reports
11
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Technical Workflow (1/2)
12
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Technical Workflow (2/2)
13
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
BrandDetector – Detection of Logos
14
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
MPEG-7 Result Viewer/Editor
Player
Key framesshots
Basic events
Module parameters
15
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
ConclusionsStandardized metadata (MPEG-7, OWL) is a good choice otherwise a lot of time will be lost with specification and
extensions MPEG-7, OWL already provided the needed structures one expert for MPEG-7, OWL is necessary
Web services not so straightforward as always thought managed vs. unmanaged code not all SOAP protocol features supported
Do not underestimate integration efforts! take security issues into account remote access essential
16
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Outlook: MediaCampaignR&D project, Started: 04/2006
Identification and tracking of new campaigns in different media (Press, TV, Internet) utilizing different modalities (audio, text, image, video)
Semantic cross-market advertis. interlinking relate similar advertisements over different media &
countries
Define Ontology for cross-media modeling of media presence and campaigns Media Presence and Campaign Ontology (MEPCO) includes media presence information
17
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Relevant LinksNM2 website http://www.ist-nm2.org/
DIRECT-INFO website http://www.direct-info.net
MediaCampaign website http://www.media-campaign.eu
JOANNEUM RESEARCH MPEG-7 Library available for free! full support for parts 3, 4 and 5 of
the MPEG-7 standard http://mpeg-7.joanneum.at
Available inJune 2006!
18
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
JOANNEUM RESEARCHInstitute of Information Systems &
Information ManagementSteyrergasse 17
A-8010 Graz / AUSTRIAVoice: +43 316 876-1194
Mobile: +43 699 1876-1194Fax: +43 316 876-1191
E-mail: [email protected]: http://www.joanneum.at/iis
Dr. Herwig Rehatschek
20
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Delivery
Optimiser
InferenceEngine
Realisation Engine
Delivery SystemRendering
Engine
StreamingServer
Client
MediaPlayer
MediaStorage
playlists
video
21
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
MPEG-7 Automatic Detection Preview (UI)
22
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
System Architecture
23
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Genre Classification
Fully automatic classification of commercials news studio/speaker soccer credits movie
Real-time analysis
Semantic block classification ~90% precision ~90% recall
24
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Multimodal Event Modelling
Event detection in audio and video
Integration of basic events visual shots audio segments overlaid text visible frontal faces
Pseudo-scene/story segmentation
Classified segments
s1 s2 c1 s2a2+s2 s3 c2 c3 s4 u s4 s5
25
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Text AnalysisDetect positive & negative mentions of brand names teams
SCHUG text analysis engine “Shallow and Chunk-based Unification Grammar” robust chunk parser for linguistic analysis
Topic detection
Sources transcripts and text captions of TV audio newspaper texts supported languages: English, Italian
26
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Semantic Data Fusion
Combing data from various analysis subsystems
Based on time-stamped analysis results of all analysis modules
Rule-based derivation of new facts basic appearancescomplex appearances
Two main user interfacesconsolidation of appearancesset-up application for delivery system
27
18/
04/2
3IS
O 9
001
cert
ified
Herwig RehatschekLREC Workshop, Genoa23 May 2006
Semantic Data FusionDisplay of Results