Date post: | 02-Jan-2016 |
Category: |
Documents |
Upload: | abraham-saunders |
View: | 35 times |
Download: | 0 times |
1
Presented by Jari Korhonen
Centre for Quantifiable Quality of Service in Communication Systems (Q2S)Norwegian University of Science and Technology (NTNU)
Research Activities for Quality of Experience in
Networked Multimedia within Q2S
2
Outline
About Q2S Research topics and vision at Q2S
o Research directions and visiono Subjective / Objective assessment of Audio / Video /
Audiovisual content
Research highlights: comparing apples and orangeso Subjective comparison of source and channel distortion in
video streaming
Conclusions
3
• Centre of Excellence in Quantifiable Quality of Service in Communication Systems – Q2S
• Funded by Norwegian Research Council• Supported by Telenor R&I• Hosted by Norwegian University of
Science and Technology• 52 people (7 profs., post-docs, PhD
students and administration)• Basic research and laboratory
experimentation• Main Goals:
– Network Media Handling – QoS and QoE assessment and monitoring – QoS mechanisms for dynamic networks.
Q2S
4
Q2S Research
Networked Media Handlingo Technology to present, manipulate and evaluate multimedia contento Focus on media representation, error protection, perceived quality,
functional placement
Quality assessment and monitoringo Measuring methods and models for QoS and QoEo Measurement of perceived QoS and QoE, measurement methods and
architecture, and traffic performance
QoS mechanisms for dynamic networkso Mechanisms that affect QoS, emphasis on heterogeneous and dynamic
environmento Focus on dependability, security, resource allocation
5
Research Directions (QoE)
QoE
Applications
Virtual worldsSerious gamingBroadcastingMedicalTourism...
Content
User generatedInteractivityIdentity and security
Technology
Network Media Handling3D Media
Perception
Cognitive technologies
MarketsBusiness Models
Creativity
Implem
entation
Modeling and
assessment Su
bjec
tive
test
met
hods
7
QoS and QoE modeling and assessment
Development of Subjective Test Methods and Objective Models- Multimodal perception of audio and video- Content dependency
- Perceived audio-visual quality assessment for future media (multimodal media with complex scenes (HDTV, UHDTV, NHX, 3D, etc.))- Design of quantifiable metrics for perceived quality (audio, video, audiovisual, …)
Use of Objective Models- Automated monitoring of end-user perceived quality- Using “Perceived QoS” to adapt and enhance system performance- Objective network measurements of end-to-end QoS
8
Research highlights: Comparing apples to oranges In multimedia communications applications (such as video
streaming), both source and channel distortion may appearo Source distortion is derived from lossy compressiono Channel distortion is caused by transmission errors (packet losses
and/or bit errors)
Encoding (compression)
Transmission channel
Decoding and error
concealment
Original quality video
Compressed video with
source distortion
Compressed video with source and
channel distortion
Reproduced video with source and
channel distortion
Video quality experienced by user
9
Motivation – comparison of source and channel distortion
Qualitative characteristics of source and channel distortion are differento Source distortion impacts the
overall quality (see the upper image)
o Channel distortion typically appear in spatially and temporally limited areas (see the lower image)
o Classical example of ”comparing apples and oranges” applies
Source distortion
Channel distortion
10
Motivation – limitations of known quality assessment methods It is challenging to compare the perceptual impact of
source and channel distortiono Objective metrics, such as PSNR, typically give reliable
results only when same type of distortions are comparedo Traditional subjective metrics and methodologies require
quite a lot of work and reliability may be questionable• Different persons may understand subjective scales differently
There is a need for new subjective quality assessment method!
11
Proposed test method Instead of rating the difference between test sequence
and anchor sequence (traditional double-stimulus impairment scaling), we use adjustable stimuluso Named as double-stimulus adjustable quality fixed anchor
(DSAQFA)o User adjusts one of the stimuli so that its quality is (as closely
as possible) similar to the non-adjustable anchor sequenceo Allows comparison between different types of distortionso Minimizes the need for training and eliminates the personal
differences how quality labels are understood
12
Functionality of the test program Anchor sequence and
adjustable sequence played in sync
Adjustable sequence: different source distortion levels the user can choose between
When ready with adjustment, user presses ’ok’, result is recorded and system moves to the next test case
Quality level 2
...
Quality level 1
Quality level 16
Adjustable video player
Anchor video player
Anchor video
Sync
Quality level adjustment
13
UI of the test program
Adjustable sequence
Anchor sequence
Slider for adjusting quality,
and ok-button
14
Generation of test sequences Adjustable sequences: original video clip is encoded
with H.264/AVC using 16 different quantization parameter (QP), varying from 24 (best quality) to 51 (worst quality)
Anchor sequences: bit errors inserted in encoded files using Gilbert-Elliot model, to create channel distortion
H.264/AVC encoder
Original video
Channel error
simulator
H.264/AVC decoder
Resulting video
Quantization Parameter (QP)
Gilbert-Elliot model parameters
15
Voted and anchor PSNR in selected cases
H M1M2 L H M1M2 L H M1M2 L H M1M2 L25
30
35
40
45
PS
NR
[d
B]
Test Case
Comparison of voted and anchor PSNR
Akiyo Harbour Ice Bus
O = Anchor PSNR
16
Discussion of results
PSNR seems to overestimate the quality, when source distortion in anchor is low and there is some channel distortion
However, this difference seems to disappear when the source distortion in anchor gets lower
Content matters: channel distortion seems to be less annoying (compared to PSNR) for sequences with high temporal and spatial activity
Also a lot of individual variance: some test subjects give constantly higher or lower ratings than average
Ongoing research: other objective metrics than PSNR included
17
Concluding summary
General introduction of NTNU-Q2S Research areas and vision
o Emphasis of presentation on quality metrics and assessment for networked video
Research highlightso Comparing apples and oranges: novel subjective quality
assessment method for comparing video sequences with different types of distortion (source and channel)
18
QoMEX ’10The Second International Workshop on Quality of
Multimedia Experience
June 21-23, 2010Trondheim, Norway
Topics of interest (not limited to):o User experience assessment and enhancemento Visual and auditory user experienceo QoE for virtual, augmented and mixed realitieso Link between QoS, QoE and acceptanceo Psychological and social dimensions of QoEo Standardization acitivities in multimedia quality evaluation
Important dates:o Submission deadline: January 31, 2010o Notification of acceptance: April 1, 2010
More information: qomex2010.org
QoMEX