Quality of Experience characterization and provisioning in ... semes… · + In-service,...

Mobile and wireless networks

Quality of Experience characterization and provisioning in

wireless and mobile networks

Dr. Eirini Liotou14/12/2018

[email protected]


Subjective: Crowdsourcing

* T. Hossfeld, C. Keimel, M. Hirth, B. Gardlo, J. Habigt, K. Diepold, and P. Tran-Gia, “Best Practices for QoE Crowdtesting: QoE Assessment With Crowdsourcing,” IEEE Trans. Multimed., vol. 16, no. 2, pp. 541–558, Feb. 2014.

Q: Have you ever used such

a tool?



Crowdsourcing example 1 (BSc thesis)

• http://gain.di.uoa.gr/kyr/s2b/qoe.html

• http://gain.di.uoa.gr/kyr/s1c/qoe.htmlHands on

http://gain.di.uoa.gr/kyr/s2b/qoe.html

http://gain.di.uoa.gr/kyr/s1c/qoe.html


Crowdsourcing example 1

X

X



X

X



Video ID

Inst

ance

s o

f h

igh

er

pe

rcei

ved

qu

alit

y

1 stalling of 6s vs 2 stallings of 3s each



• http://gain.di.uoa.gr/kyr/357/qoe.html

• http://gain.di.uoa.gr/kyr/753/qoe.html

Descending vs ascending quality

Video ID

Inst

ance

s o

f h

igh

er

pe

rcei

ved

qu

alit

y

http://gain.di.uoa.gr/kyr/357/qoe.html

http://gain.di.uoa.gr/kyr/753/qoe.html


QoE monitoring example (MSc thesis)


QoE monitoring example

• Initial delay

• Number and duration of stalling events

• Total video duration

• Actual user viewing time

• Video size in bytes

• GPS info

• Internet connection type


Electroencephalography (EEG)

• Human influence factors (HIFs) characterize the user’s perception, emotional and mental state with respect to a service

– Facial expressions, body posture, voice, eye measurement, electrocardiography (ECG), electrodermal activity (EDA)

– EEG measures electrical activity in the brain

* R. Gupta, K. Laghari, H. Banville, T. H. Falk, “Using affective brain-computer interfaces to characterize human influential factors for speech quality-of-experience perception modelling”, Human-centric Computing and Information Sciences, 2016.

* http://myscienceschool.org/index.php?/archives/3208-What-is-Electroencephalography-EEG.html


Objective: evaluation methods

• Media-layer

• Packet-layer / Bitstream

• Parametric planning

NetworkMedia-layer

model

Input

Output

Reference

QoE

estimate

QoE

estimateNetwork

Parametric model

Input

Output

QoE

estimate

Input

NetworkPacket-layer

model

Input

Output

QoE

estimate

Q: Why is this difficult/impossible to implement in a real-time network?


Objective: evaluation methodsModel Advantages Disadvantages

Media-layer:

Full Reference

(e.g. PESQ)

+ Do not require any a-priori knowledge or assumptions about

the underlying network

+ Highly accurate and robust (based on psycho-acoustics)

- Require the reference signal (intrusive)

- Very high computational effort

- Practically impossible to implement at network midpoint

- Do not enable insight into the internal system functionality &

degradation causes (black-box) => diagnosis not possible

- Neglect human dimensions, pure technical

Parametric planning:

E-model

+ Ease of use and respect of privacy

+ The network is characterized by the technical specifications

of its constituent elements, (non-intrusive approach )

+ Quantifies the human factor through the “Advantage

factor”, & contextual factor

+ Mouth-to-ear complete transmission chain => conversational

+ No restrictions on the network with respect to size,

configuration, hierarchy, technology used, nor on the

components of the network

- Intended only for the planning phase of a system (extended format)

- Good in theory, but difficult to include all the model parameters

online

- Accurate only under strict application scenarios: new subjective

tests and regression analysis needed for different conditions

- Speech independent

- A-priori information requirement

Packet-layer:

ITU-T P.564

+ Enables insight into the internal system functionality (glass-

box)

+ Light in terms of computational effort

+ Multiple monitoring points help identify the root of a

network impairment

+ Used not only for speech quality predictions but also for the

production of diagnostic outputs

+ In-service, non-intrusive (privacy)

+ Quality followed and pooled over time

- Not standardized, models need to be created that comply with

these recommendations

- The model doesn’t know the characteristics of speech content to

evaluate (speech level, echo, background noise etc.): assumes a

generic voice payload

- Only concerns impairments on the IP network (no end-to-end

evaluation)

- Large volume of QoE data

- Models deployed require strict conformance testing


Examples of objective models

➢ VoIP: = 𝟗𝟒. 𝟐 − 𝟎. 𝟎𝟐𝟒𝒅 + 𝟎. 𝟏𝟏 𝒅 − 𝟏𝟕𝟕. 𝟑 𝑯 𝒅 − 𝟏𝟕𝟕. 𝟑 −[𝟏𝟏 + 𝟒𝟎 𝒍𝒏 𝟏 + 𝟏𝟎𝒑 ]

➢ YouTube (TCP): 𝑸𝒐𝑬 = 𝟑. 𝟓 ∗ 𝒆− 𝟎.𝟏𝟓𝑳+ 𝟎.𝟏𝟗 ∗𝜨 + 𝟏. 𝟓

➢ HTTP Adaptive Streaming (TCP): 𝑸𝒐𝑬 = 𝟎. 𝟎𝟎𝟑 ∗ 𝒆𝟎.𝟎𝟔𝟒∗𝒕 + 𝟐. 𝟒𝟗𝟖

➢ Real-time video (UDP): 𝑽𝒒 = 𝟏 + 𝑰𝒄𝒐𝒅𝒊𝒏𝒈 ∗ 𝑰𝒕𝒓𝒂𝒏𝒔𝒎𝒊𝒔𝒔𝒊𝒐𝒏

➢ FTP: QoE = 𝜶 𝒍𝒐𝒈𝟏𝟎(𝜷𝑹) , 𝟏𝟎𝒌𝒃𝒑𝒔 < 𝑹 < 𝟑𝟎𝟎𝒌𝒃𝒑𝒔

packet loss ratedelay

#of stallsduration of stalls

time on highest quality level

data rate

FR, BR, PLR


A. ITU-T G.107 “E-model” for voice

• Computes the transmission quality of VoIP by estimating the mouth-to-ear conversational quality as perceived by the receiver

• A parametric model that produces the so-called Rating factor 𝑅:

𝑹 = 𝑹𝟎 − 𝑰𝒔 − 𝑰𝒅 − 𝑰𝒆−𝒆𝒇𝒇 + 𝑨

➢ Ro→ basic signal-to-noise ratio, Ro = 100

➢ Is→ impairments due to the voice signal travelling in the network

➢ Id→ impairments caused by delay from end-to-end travelling signal

➢ Ie-eff→ equipment impairment factor & impairments due to packet loss

➢ A → advantage/expectation factor, in exchange for some user benefits or other factors difficult to quantify


𝑅 = 𝑅0 − 𝐼𝑠 − 𝐼𝑑 − 𝐼𝑒−𝑒𝑓𝑓 + 𝐴

𝑅 = 94.2 − 0.024𝑑 − 0.11 𝑑 − 177.3 𝐻 𝑑 − 177.3 − 11 − 40 𝑙𝑛 1 + 10𝑝

E-model: simplified version

0 50 100 150 200 250 300 350 4000

20

40

One-way delay (msec)

Id

0 50 100 150 200 250 300 350 4003

4

5

QoE

(M

OS

)

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.160

20

40

60

Packet loss rate

Ie-e

ff

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.162

3

4

5

QoE

(M

OS

)

packet loss ratedelay


B. ITU-T G.1070 “E-model” for video

• A computational model for point-to-point interactive videophone applications over IP networks (UDP-based - lossy video)

• Network, Application & Terminal parameters incorporated

• Video quality =

𝑽𝒒 = 𝟏 + 𝑰𝒄𝒐𝒅𝒊𝒏𝒈 ∗ 𝑰𝒕𝒓𝒂𝒏𝒔𝒎𝒊𝒔𝒔𝒊𝒐𝒏

➢ 𝐼𝑐𝑜𝑑𝑖𝑛𝑔 = the video quality affected by the coding distortion

➢ 𝐼𝑡𝑟𝑎𝑛𝑠𝑚𝑖𝑠𝑠𝑖𝑜𝑛 = the video quality affected by the transmission process

➢ Ultimately everything is a function of: • the video frame rate (fps) - FR

• the video bit rate (kbps) - BR

• the video packet loss rate - PLR

• 12 coefficients


C. QoE for YouTube

• Video on Demand (VoD), TCP-based connection (no losses)

• Quality influence factors (by crowdsourcing & lab tests):

➢ Number of stalling events, N

➢ Duration of stalling events, L

➢ Total video duration, T (total stalling duration over video duration)

➢ Initial delay (video start-up delay) → cache redirections’ impact

Q: Why does a stalling happen?


QoE for YouTube

• Some conclusions:

➢ The user demographics have no significant influence (!)

➢ Initial delays have almost no influence on MOS for videos of duration 60s and 30s compared to the influence of stalling length

➢ The user ratings are statistically independent from video motion, type of content, the usage pattern of the user, access speed, etc.

➢ The number of stalling events together with the stalling length are clearly dominating the user perceived quality

➢ The video duration only plays a role if there are only a very few stalling events

Q: Is this your impression too?


QoE for YouTube

IQX hypothesis validation:

𝑸𝒐𝑬 𝑳, 𝑵 = 𝜶 ∗ 𝒆−𝜷 𝑳 ∗𝜨 + 𝜸,

α = 3.5, β(L) = 0.15L+ 0.19, γ = 1.5

* T. Hossfeld, R. Schatz, E. W. Biersack, and L. Plissonneau, “Internet Video Delivery in YouTube: From Traffic Measurements to Quality of Experience,” in Data Traffic Monitoring and Analysis, Eds. Springer Berlin Heidelberg, pp. 264–301, 2013.


D. HTTP Adaptive Streaming (HAS)

Comparison of HTTP video streaming and HTTP adaptive video streaming

Q: Why is this better?


HTTP Adaptive Streaming (HAS)

Based on how fast the current (and previous)

segments are downloaded, the bit rate

of the next segment is selected

* M. Seufert, S. Egger, M.Slanina, T. Zinner, T. Hoßfeld, and P. Tran-Gia, “Survey on Quality of Experience of HTTP Adaptive Streaming”, IEEE Communication Surveys & Tutorials, Vol. 17, No. 1, First Quarter 2015.


HTTP Adaptive Streaming (HAS)

• Other influence factors: Adaptation frequency (number of switches), adaptation amplitude, adaptation direction, segment length, buffer size, etc.

𝑸𝒐𝑬= 𝟎. 𝟎𝟎𝟑 ∗ 𝒆𝟎.𝟎𝟔𝟒∗𝒕 + 𝟐. 𝟒𝟗𝟖

𝒕 = time on highest layer


E. QoE for file download services

• Elastic service, for which the utility function is an increasing, strictly concave, and continuously differentiable function of throughput

• The user satisfaction of a file transfer service is solely dependent on the provided data rate

• Logarithmic relationship between MOS and throughput:

𝑴𝑶𝑺 = ቐ

𝟏, 𝑹 < 𝟏𝟎𝒌𝒃𝒑𝒔𝜶 𝒍𝒐𝒈𝟏𝟎(𝜷𝑹) , 𝟏𝟎𝒌𝒃𝒑𝒔 < 𝑹 < 𝟑𝟎𝟎𝒌𝒃𝒑𝒔

𝟒. 𝟓, 𝟑𝟎𝟎𝒌𝒃𝒑𝒔 < 𝑹

➢ R is the data rate of the service ➢ α and β obtained from the upper and lower

user perceived quality expectations

* S. Thakolsri, S. Khan, E. Steinbach, and W. Kellerer, “QoE-Driven Cross-Layer Optimization for High Speed Downlink Packet Access,” J. Commun., vol. 4, no. 9, 2009


F. Netflix – challenges

• Understanding the impact of QoE on user behavior (compression artifacts, scaling artifacts, rebuffering rate, bitrate, etc.)

• Creating a personalized streaming experience

• Determining what movies and shows to cache on the edge servers based on member viewing behavior

• Improving the technical quality of the content using viewing data and member feedback

* https://medium.com/netflix-techblog/optimizing-the-netflix-streaming-experience-with-data-science-725f04c3e834

Q: How could Netflix infer that something is wrong?


Other QoE metrics

PSNR=31dB

PSNR=34dB

DMOS=82

DMOS=96

DMOS=27

DMOS=58

DMOS is 100 for the reference video

Q: Does this or… Q: this look better?


Netflix Video Multimethod Assessment Fusion (VMAF) quality metric

* https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652

Q: Which one is better and why?


SOS – The MOS is not enough

• Standard deviation of Opinion Scores (SOS)

• Reflects the level of rating diversity

• A square function of MOS → SOS hypothesis

• No diversity at the edges and maximal diversity at MOS = 3

* T. Hossfeld, R. Schatz, and S. Egger, “SOS: The MOS is not enough!,” in 2011 Third International Workshop on Quality of Multimedia Exper ience, 2011, pp. 131–136.


QOE MANAGEMENT


QoE management framework

QoEMANAGER

(1) Instructions controlling the QoE-input data generation are sent to the network(2) Input data from all data sources are collected by the QoE-Controller(3a) Processed QoE-data per flow are sent to the QoE-Monitor(3b) Information regarding the current network state is sent to the QoE-Manager(4) Estimated QoE scores are reported to the QoE-Manager per flow (5a) Customer Experience Management procedures are performed(5b) Corrective actions are triggered, if required(6) The QoE-Controller actualizes these corrective actions

QoEMONITOR

(4)

(3a) (3b)

(5b)

NETWORK

QoECONTROLLER

Periodical

Conditional

(5a)

Network specific control & optimization

Network specific collection & feedback

mechanisms

Appropriate QoE estimation models

OBJECTIVE:

Enable a QoE-centric network management framework to:

1. Monitor the end-users’ QoE

2. Enhance their experience

3. Improve the network’s efficiency (spectrum, energy)

* E. Liotou, D. Tsolkas, N. Passas and L. Merakos, “Quality of Experience management in mobile cellular networks: Key issues and design challenges,” IEEE Communications Magazine, Network & Service Management Series, July 2015.


QoE research stages & management

Goal: Optimize end-user QoE, while making efficient use of network resources & maintaining a satisfied customer base

Function 1

QoE Controller(data collection)

Function 2

QoE Monitor (modeling)

Function 3

QoE Manager(control)

DIAGNOSIS

MONITORING

ACTIONS

What to collect?

From where?

How?When?

Which model?

How to react?

Where?Where?How

often?

How to transfer?

How to diagnose?

How to deliver?


Realization issues

• Selection of the physical/virtual location of the QoE managementframework inside the operator’s infrastructure

• Identification of required QoE data sources, configuration of datacollection periodicity, signaling between network and QoE-Controller

• Selection of appropriate QoE models and KPIs for the QoE-Monitor

• Traffic/service classification performed in the QoE-Monitor,especially in the content-encrypted domain

• Network-specific type of decisions taken by the QoE-Manager andtheir actualization through the QoE-Controller


Driving network decisions – D2D

D2D1

D2D2

QoED2D

D2D data Cellular data

QoEcell

➢ QoE awareness can control the operational mode of users in LTE-A:

• Drive cellular Device-to-Device (D2D) mode transitions

➢ Enhance QoE, ↑ throughput, offload network, ↓ power, allow for profits

Q: Why is D2D beneficiary?


Driving network decisions – D2D

* E. Liotou, E. Papadomichelakis, N. Passas and L. Merakos, “Quality of Experience-centric management in LTE-A mobile networks: The Device-to-Device communication paradigm”, in 6th International Workshop on Quality of Multimedia Experience (IEEE QoMEX), Singapore, September 2014.


Improving QoE – Admission control

1

1.5

2

2.5

3

3.5

4

4.5

Progress of time

Inst

anta

neo

us

syst

em Q

oE

(M

OS)

Real-time operation of the QoE management framework

With QoE-management framework

Without QoE-management framework

AREA OF NON-ACCEPTABLE QoE

AREA OF ACCEPTABLE QoE

QoE Manager triggerscorrective actions in

the NetworkMonitored system QoE

less than theacceptable QoEthreshold (=3.5)


Improving QoE – Admission control

* E. Liotou, D. Tsolkas, N. Passas and L. Merakos, “Quality of Experience management in mobile cellular networks: Key issues and design challenges,” IEEE Communications Magazine, Network & Service Management Series, July 2015.

80 90 100 110 120 130 140 150 160 170 180 190 200 210 2201

1.5

2

2.5

3

3.5

4

4.5

Number of total concurrent VoIP flows in the system

QoE

: Me

an O

pin

ion

Sco

re (

MO

S)

c) QoE-driven admission control

Without QoE-management,UEs served by the small-cell

With QoE-management,UEs served by the small-cell

With QoE-management,UEs non-admitted by thesmall-cell, served by themacro-cell

QoE-basedadmission

triggerred by theQoE Manager

130 flows onwards

60flows

90flows

30 flows1flow


Saving resources through QoE awareness

➢ QoE awareness can drive a Power-Controlled Interference Managementscheme in femto-overlaid LTE-A

➢ How: Reduce HeNB’s transmit power, with no loss in femto-UEs’ QoE,provided that this is optimal

Optimum point of

operation

MUE

HeNBeNB

victimMUE

VictimFUE

MACROCELL

FEMTOCELL

Communication linkInterference link

IQX hypothesis

Q: Why is this the optimum operation point?


Saving resources through QoE awareness

20 40 60 80 100 120 140 160-6

-4

-2

0

2

4

6

8

10

12

HeNB-MUE distance (m)

HeN

B T

ransm

it P

ow

er

(dB

m)

3GPP PC

QoE-aware PC

* D. Tsolkas, E. Liotou, N. Passas, and L. Merakos, “The Need for QoE-driven Interference Management in Femtocell-Overlaid Cellular Networks ”, in 10th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (Mobiquitous), Tokyo, December 2013.


Extra: Another idea to save resources

QoE gain

5

Area 1Area 3

x1 x2

4

3

2

1

Area 2

ΔQ

oS1

ΔQ

oS2

ΔQoE1

ΔQoE2

q1 q2 q4q3

Qo

S g

ain

Qo

E (

MO

S)

QoS degradation

Preferred QoS reduction!

Q: Which ΔQoS reduction is preferred?


QOE CHALLENGES IN MOBILE NETWORKS


Key challenges in the QoE domain

Technical challenges

Economic challenges

Legal issues

QoE integration in communication networks

QoE needs to be managed on a per-user, per-application, and per-terminal basis in a real-time way


Technical challenges

• Monitoring: Network-centric vs. user agent-based approaches

➢ Agent-based:

+ Capture the HUMAN, CONTEXT, and WIRELESS medium aspects

- Do not offer diagnosis information

- Depend on manufacturer, not scalable

- Privacy, security, energy concerns

• Scalability & complexity issues

➢ QoE feedback, control and modeling per user session

• Network diversity

➢ Different operators or vendors, networks, mobile technologies (e.g., 2G or 3G), or even different countries or continents

• Energy consumption

➢ QoE-awareness and provisioning: monitoring, signaling, processing, memory requirements, new network entities


Heterogeneity (LTE-A example)


Economic challenges

QoE estimation may heavily depend on the expected price itself!

Charging for QoS vs. charging for QoE

𝑝𝑟𝑖𝑐𝑒 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛: 𝑝 = 𝑝 𝑞𝑑𝑒𝑚𝑎𝑛𝑑 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛: 𝑑 = 𝑑 𝑝𝑄𝑜𝑆 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛: 𝑞 = 𝑞 𝑑

𝑑𝑒𝑚𝑎𝑛𝑑 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛: 𝑑 = 𝑑 𝑝𝑄𝑜𝑆 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛: 𝑞 = 𝑞 𝑑𝑝𝑟𝑖𝑐𝑒 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛: 𝑝 = 𝑝 𝑥𝑄𝑜𝐸 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛: 𝑥 = 𝑥 𝑞, 𝑝

* P. Reichl, P. Maillé, P. Zwickl, and A. Sackl, “A Fixed-Point Model for QoE-based Charging,” in the ACM SIGCOMM 2013 Workshop on Future Human-Centric Multimedia Networking (FhMN2013), Hong Kong, 2013.


Legal challenges

• Network Neutrality

➢ “Quality” may be considered as a public good

➢ Differentiation/prioritization may be legally not allowed

• Double selling

➢ Sold as an add-on service to existing network connections?

➢ How profits will be distributed to involved parties?

• Service/Experience Level Agreements (SLAs/ELAs)

➢ Define the delivered quality in terms of QoE

➢ Find a “common vocabulary”

• Agreements among operators

➢ Collaborations, especially at interconnections

➢ Violations’ responsibility and handling

• Privacy and fidelity

➢ Transfer of user-sensitive information in an E2E path


MNO versus OTT

For mobile network operators (MNO) For Over-The-Top players (OTT)

• MNOs face a tremendous increase of data traffic

• Much of the traffic is originating from OTT Apps

• MNOs lose money: profits are unaffected while cost is higher

• Increased App types with large QoS diversity “compete” for the same pool of resources

• OTT Apps are served without QoS guarantees, over the default best-effort bearer

Then, why not go for a win-win paradigm of MNO-OTT “interfacing”, where:

• OTT Apps are served with better QoE →

(introducing some OTT control)

• QoS differentiation per App type

• App prioritization possible

• Resources shared more efficiently

• MNOs get into the revenue loop


SYNOPSIS


Revision

• The multidimensional definition of QoE

• The relationships between QoS and QoE

• QoE modeling evaluation methods

• QoE management required building blocks

• QoE exploitation possibilities

• Main challenges


Thank you!

Date post:	18-Jul-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Quality of Experience characterization and provisioning in ... semes… · + In-service,...

Documents