School of Computing, Communications and Electronics

University of Plymouth

Dr. Lingfen Sun

Voice over IP and Voice Quality Measurement

4/2/2005 SoCCE, UoP 2

Outline of Talk

Introduction VoIP Networks What is QoS or Perceived QoS? How to Measure/Predict Voice Quality?

Subjective Objective (intrusive and non-intrusive methods)

QoS Prediction and Control Research in Plymouth

4/2/2005 SoCCE, UoP 3

Introduction – the problem

Internet Protocol (IP) networks On a steep slope of innovation – long term carriers

of all traffic including voice traffic. IP is now the “universal” communications protocol

because it facilitates convergence of networks and the ability to offer multiple services on the same networks.

Not designed to carry real-time traffic, such as voice and video, because of their variable characteristics (e.g. delay, delay variation and packet loss) . These have adverse effects on voice quality.

4/2/2005 SoCCE, UoP 4

Introduction – Voice Quality in IP networks

User perceived quality is the key QoS metric in VoIP applications - The end-user of a VoIP service expects: voice quality to be as good as in traditional

networks, and the service to be as reliable.

This is not the case at present. This makes it necessary to be able to predict/measure, and if appropriate, control voice quality in order to deliver the desired QoS.

4/2/2005 SoCCE, UoP 5

VoIP Network and Perceived QoS

Network QoS Perceived QoS is measured from ‘mouth to ear’, i.e. end-to-end and depends on the

performance of IP network and terminal/gateway.

IP NetworkSCN SCN

Gateway Gateway

SCN: Switched Communication Networks (PSTN, ISDN, GSM …)

Network QoS

Perceived QoS

IP phone

IP softphone

4/2/2005 SoCCE, UoP 6

VoIP – New Applications

IP Network/MPLS

IAD

DSLAM

VoDSLEnterprise LAN

Dual-mode handset

VoWLAN

MGW

Mobile network

AP

IP access network

PSTN

GW

IAD: Integrated Access Device

DSLAM: DSL Access Multiplexer

MGW: Media Gateway

MPLS: Multi-protocol Label Switching

4/2/2005 SoCCE, UoP 7

VoIP Protocol Stack

e.g. Ethernet/SDH

IP

UDP TCP

Audio

/video

RTP RTCP SIP H.323

Physical layer

Network layer

Transport layer

Application layer

4/2/2005 SoCCE, UoP 8

What is QoS?

The ISO standard defines QoS as a concept for specifying how “good” the offered networking services are. QoS can be characterised by a number of specific parameters.

For Multimedia Communication System (MCS), QoS concept can be extended to “User QoS” or “Perceived QoS”.

For VoIP, Perceived QoS – user perceived voice quality (e.g. MOS)

4/2/2005 SoCCE, UoP 9

Factors affect voice quality

IP Network

Receiver

Voice source

Encoder

Sender

PacketizerJitter

bufferDecoder

De-packetizer

End-to-end perceived voice quality (MOS)

packet loss

network delay

jitter

coding distortion

codec delay

delay delay buffer-delay

buffer-loss

codec impairment

delay

• Other impairments: echo, sidetone, background noise …

• Other factors: language, gender, FEC, packet loss concealment

Voice receiver

4/2/2005 SoCCE, UoP 10

Inter-relationships between the QoS Parameters [1]

Network Packet Loss

Network Jitter

Network Delay

Codec Performance

Overall Packet Loss

Perceived Quality

OverallDelay

Jitter Buffers

Network Factors Application FactorsQoS Service

Level

4/2/2005 SoCCE, UoP 11

QoS parameters [1]

QoS Service Class

Codec Performance, VAD, Frames perPacket, Jitter Buffer, Codec Delay,

FEC (Redundancy)

Max Packet Loss, Max Mean delay,Max Delay Variation

SERVICE

APPLICATION

TRANSPORT

4/2/2005 SoCCE, UoP 12

Key QoS parameters and how they arise Packet Loss

Network packet loss (as a result of congestion or rerouting in the IP network)

Late arrival loss (dropped at receiver) Link failures and system errors.

End-to-end Delay Network delay (transmission and queuing delay) Buffer delay Codec processing delay Packetizing/depacketizing delay

Jitter (delay variation) Caused by queuing delays within the IP network

4/2/2005 SoCCE, UoP 13

Delay impact on multimedia quality [7]

0%

Pac

ket

Los

s

Conversationalvoice and video

Voice/videomessaging

Streamingaudio/video Fax

5%

100 msec 1 sec 10 sec 100 sec

Interactive Responsive Timely Non-critical

Delay

For VoIP applications, delay < 150 ms, imperceptible, delay > 400 ms, quality unacceptable for most users.

4/2/2005 SoCCE, UoP 14

How to Enhance QoS?

Application-level QoS mechanisms Packet loss compensation (e.g. FEC, loss

concealment) Jitter compensation (e.g. buffer algorithms) Adaptive source coding …

Network-level QoS mechanisms How to guarantee IP network performance Diffserv (Differentiated Services) Intserv (Integrated Services) …

4/2/2005 SoCCE, UoP 15

How to Measure Voice Quality?

Why need to measure voice quality? For QoS monitoring and/or control

purposes to ensure that the technical and commercial requirements (e.g. SLA) are met.

How to measure voice quality? Subjective methods (e.g. MOS) Objective methods (e.g. PESQ or E-model)

4/2/2005 SoCCE, UoP 16

Subjective or objective measurement

Subjective Voice Quality Measurement Subjective listening tests by a group of people Provides a benchmark for objective test methods Expensive and time-consuming

Objective Speech Quality Measurement Repeatable, automatic, and predicts subjective score Suitable for online quality measurement/monitoring Can be used for intrusive and Non-intrusive

measurements.

4/2/2005 SoCCE, UoP 17

Voice quality measurement

SCN SCNIP Network

Gateway

SCN: Switched Comm. Networks (PSTN, ISDN, GSM …)

Non-intrusiveMeasurement

(parameter-basede.g. E-model)

MOSc

Gateway

IntrusiveMeasurement(e.g. PESQ)

Reference speech

Degraded speech

Non-intrusiveMeasurement(signal-based

e.g. P.563)

(e.g. loss, delay, jitter)

MOS-LQMOS-LQ

MOS-LQ

MOS-LQ: MOS-Listening quality

MOSc: Conversational MOS score

4/2/2005 SoCCE, UoP 18

Voice quality measurement (cont.)

Voice quality measurement

Subjective methods

Objective methods

Non-intrusivemethods

Intrusivemethods

Parameter-based methods

Signal - based methods

Comparison-based methods

Calibration

4/2/2005 SoCCE, UoP 19

Mean Opinion Score (MOS) The most widely used subjective measure of voice quality. Provides a direct link to voice quality as perceived by the end user. Gives average opinion of quality based on asking people to grade

the quality of speech on a five-point scale: Excellent, Good, Fair, Poor and Bad.

Slow, time-consuming, expensive, not repeatable and cannot be used to monitor voice quality on-line in a large network.

Different Categories of MOS Test (ITU P.800[2]) Absolute Category Rating (ACR): only listen to the degraded

speech signals (most commonly used) Degradation Category Rating (DCR): rate annoyance or

degradation level between the reference and degraded signal

Subjective voice quality measurement

4/2/2005 SoCCE, UoP 20

MOS Test Based on ACR

Category Speech Quality

5 Excellent

4 Good

3 Fair

2 Poor

1 Bad

Absolute Category Rating (ACR)

4/2/2005 SoCCE, UoP 21

MOS Test based on DCR

Category Degradation level

5 Inaudible

4 Audible but not annoying

3 Slightly annoying

2 Annoying

1 Very annoying

Degradation Category Rating (DCR)

4/2/2005 SoCCE, UoP 22

Online MOS Test Website

http://www.tech.plymouth.ac.uk/spmc/people/lfsun/mos

This is our research on subjective tests. The aim is to provide a more efficient method to carry out subjective tests compared to standard MOS test (e.g. ITU P.800).

Standard MOS measurement requires a stringent test requirement (e.g. sound proof room, a large number of subjects, test procedures). Thus, it is very time consuming, expensive, and difficult to organise a test.

4/2/2005 SoCCE, UoP 23

Objective voice quality measurement

Automated measure of speech quality using an appropriate model.

Conventional methods, e.g. SNR-based approach, are not appropriate as they fail to reveal quality as perceived by the end user.

Emerging methods for voice quality prediction are based on models of human auditory perception or psychologically-derived computational models.

Can be intrusive (e.g. ITU P.862, PESQ [3]) or Non-intrusive (e.g. ITU P.563 [4] formerly P.SEAM) .

4/2/2005 SoCCE, UoP 24

Intrusive measurement

PESQ (Perceptual Evaluation of Speech Quality), ITU P.862, Feb, 2001

Intrusive (active) test, listening-only quality uses test stimuli, such as speech signal

System

under testPESQ

Reference signal/speech

Degraded signal/speech

PESQ quality score

(MOS)

4/2/2005 SoCCE, UoP 25

Perceptual Evaluation of Speech Quality

Transforms the original and degraded speech signals into a psychophysical representation that approximates human perception.

Calculates their perceptual distance and maps this into an objective MOS score.

4/2/2005 SoCCE, UoP 26

PESQ (perceptual difference)

reference speech

degraded speech

Loss position

PSQM

PESQ

4/2/2005 SoCCE, UoP 27

OPTICOM- Opera system

                                     

Opera system "Digital Ear“

http://www.opticom.de

Perceptual Voice/Audio Quality

PESQ/PSQM/PEAQ

4/2/2005 SoCCE, UoP 28

Non-intrusive measurement

Non-intrusive (passive) test Output-based (speech signal based) or

parameter-based Low accuracy if compared to the

intrusive methods Adequate for real-time, online

monitoring purposes

4/2/2005 SoCCE, UoP 29

Non-intrusive Speech Quality Prediction

Signal-based (output-based): to predict/measure voice quality directly from degraded speech signal (e.g. from T1/E1).

Parameter-based: to predict/measure voice quality directly from IP network impairment parameters (e.g. loss, delay, jitter).

PSTN

Gateway

MOS

Parameter-based method

IPT1/E1

Signal-based method

MOS

IP Network

Signal-based method

MOS

4/2/2005 SoCCE, UoP 30

Signal based (output-based) Method

Assess/predict speech quality non-intrusively from degraded speech signal only

Need to extract speech features (e.g. unnaturalness voice, noises, time clipping)

Mapping to MOS via quality prediction model

ITU P.563 – May 2004 (single-end, signal-based or output-based)

From T1/E1 link or end terminal

Speech

speech feature parameters extract/

analysis

Speech quality model

MOS

Pre-processing

4/2/2005 SoCCE, UoP 31

Parameter based Method

IP packets

RTP header / network parameter

analysis

Parameters (e.g. loss, jitter, delay)

Quality prediction models (e.g. NN or non-linear models)

MOS

Access/predict speech quality from IP network impairments (e.g. loss, delay) and codec etc.

Neural network model, non-linear regression model, ITU-T E-model [5]

External or built-in approach (be located before/after jitter buffer)

4/2/2005 SoCCE, UoP 32

E-model (ITU G.107, G.108)

Computational model – can be used to compute the “Mouth-to-ear” transmission quality.

Overall Transmission Quality Rating given by model is referred to as the R factor. R lies in the range 0-100 and can be mapped to MOS.

Designed for network planning, but may be used for non-intrusive quality monitoring/measurement.

Based on the principle that “Psychological factors on the psychological scale are additive”

4/2/2005 SoCCE, UoP 33

E-model equation

AIIIRR esd 0

Ro: base R value (noise level) Id: impairments that are delayed with respect to speech

(e.g. talker/listener echo and absolute delay) Is: impairments that occur simultaneously with speech

(e.g. quantization noise, received speech level and sidetone level)

Ie: equipment impairment (e.g. codec, packet loss, jitter) A: Advantage factor (e.g. 0 for wireline and 10 for GSM)

4/2/2005 SoCCE, UoP 34

Loss model - maps loss to Ie

0

10

20

30

40

50

0 5 10 15

Packet Loss Rate

Ie (

pac

ket

loss

)Curve is CODECdependant

4/2/2005 SoCCE, UoP 35

Delay model

0

10

20

30

0 100 200 300 400

End to end delay (ms)

R FactorReduction

4/2/2005 SoCCE, UoP 36

E-model (a simplified version)

Delay model

Loss model

RMOSMOS

Packet loss rate

Codec type

Delay (d)

Id

Ie

ed IIR 2.93

4/2/2005 SoCCE, UoP 37

E-model (R factor) and MOS

TIA 2000

4/2/2005 SoCCE, UoP 38

Extended E-model

Simplified E-model consider only effects from codec, packet

loss (random packet loss) and end-to-end delay.

Extended E-model [6] Further consider burst loss effects (e.g. 2-state

Gilbert model, 3 or 4 states Markov models) Further consider recency effects. Telchemy (http://www.telchemy.com/)

4/2/2005 SoCCE, UoP 39

Burst Loss vs. Random Loss

Burst packet loss

Non-bursty packet loss

Packet lost Packet received

4/2/2005 SoCCE, UoP 40

“Recency” Effect [6]

“Good” 4.3MOS“Bad” 1.8 MOS(3dB SNR)

MOS 3.82

MOS 3.28

MOS 3.18

Source AT&TT1A1.7/98-031

60 second call

4/2/2005 SoCCE, UoP 41

Extended E Model [6]

Delay, measuredusing RTCP

NetworkR FactorIe

PacketLoss

Jitter

Codectype

LossModel

JitterModel

CodecModel

Burstmodel

Recencymodel

UserR Factor

Delaymodel

4/2/2005 SoCCE, UoP 42

VQmon – Embedded Monitoring[6]

IPNetwork

GatewayGateway

VQmon Agent embedded into VoIP Gateway

QoSmetrics

NMS

Telchemy (http://www.telchemy.com/)

4/2/2005 SoCCE, UoP 43

Voice and Video quality Assessment in Psytechnics

Psytechnics – spin off from BT http://www.psytechnics.com Intrusive model (e.g. PESQ) Non-intrusive model

psyVoIP (parameter-based) E-model NiQA (signal-based) CCI (Call Clarity Index)/INMD (In-service Non-

intrusive Measurement Device)

4/2/2005 SoCCE, UoP 44

QoS Prediction and Control - Research in Plymouth

Aims and objectives To research and develop novel, generic methods

for objective measurement, prediction and control of user-perceived quality.

To apply the methods to real world problems in communications, audio and healthcare.

Examples Non-intrusive voice quality prediction and

measurement for VoIP QoS prediction and control for wireless VoIP Multimedia quality prediction (voice, audio and

video)

4/2/2005 SoCCE, UoP 45

Signal Processing & Multimedia Communications Group

Research within the Group is concerned with the development of novel, generic signal and informationprocessing methods and their applications to real worldproblems.

Main application areas: Multimedia communications – quality of service

prediction and control Audio – sound synthesis, audio quality assessment Biomedicine – intelligent biosignal analysis, biomedical

informatics, decision support.

4/2/2005 SoCCE, UoP 46

About my PhD project

Voice source

Voice receiver

IP Network

Receiver

Encoder

Sender

PacketizerJitter

bufferDecoder

De-packetizer

Non-intrusivemeasurement

MOS

End-to-end perceived voice quality (MOS)

To develop novel and efficient method/models for non-intrusive quality prediction, To apply the models for perceptual optimization control( e.g. buffer optimization and

adaptive sender-bit-rate QoS control)

4/2/2005 SoCCE, UoP 47

A New Methodology

VoIP Network

New model

(packet loss, delay, codec …)

Predicted MOSc

PESQ

E-model Measured MOScdelay

MOS(PESQ)

Reference speech Degraded speech

Intrusive method

(regression or ANN models)Non-intrusive method

Based on intrusive quality measurement (e.g. PESQ) to predict voice quality non-intrusively which avoids subjective tests.

A generic method which can be easily applied to audio, image and video.

4/2/2005 SoCCE, UoP 48

Two Non-intrusive Models

Artificial neural network models for predicting listening and conversational voice quality

Simplified regression models to predict voice quality

4/2/2005 SoCCE, UoP 49

Three Applications

Voice quality monitoring/prediction for real Internet VoIP traces

Perceived voice quality driven jitter buffer optimization

Perceived voice quality driven QoS control (combined adaptive sender-bit-rate and priority marking control)

4/2/2005 SoCCE, UoP 50

References

1. M. Buckley, End-to-end QoS control in VoIP systems, Workshop on QoS and user perceived transmission quality in evolving networks, Oct. 2002.

2. ITU-T Rec. P.800, Methods for subjective determination of transmission quality, Aug.1996.

3. ITU-T Rec. P. 862, Perceptual evaluation of speech quality (PESQ), an objective method for end-to-end speech quality assessment of narrow‑band telephone networks and speech codecs, Feb. 2001

4. ITU-T Rec. P.563, Single-ended method for objective speech quality assessment in narrow-band telephony applications, May 2004.

5. ITU-T Recommendation G.107, The E-model, a computational model for use in transmission planning, 2000.

6. A. Clark, Modeling the Effects of Burst Packet Loss and Recency on Subjective Voice Quality, 2nd IPTel Workshop, 2001, pp.123 – 127.

7. H. Schink, Characterising end to end quality of service in TIPHON systems, IP Networking & Mediacom Workshop, April 2001.

4/2/2005 SoCCE, UoP 51

References L Sun and E Ifeachor, "New Models for Perceived Voice Quality

Prediction and their Applications in Playout Buffer Optimization for VoIP Networks“ Proceedings of IEEE ICC 2004, Paris, France, June 2004, pp.1478 - 1483.

Z Qiao, L Sun, N Heilemann and E Ifeachor "A New Method for VoIP Quality of Service Control Based on Combined Adaptive Sender Rate and Priority Marking“ Proceedings of IEEE ICC 2004, Paris, France, June 2004, pp.1473 - 1477.

L Sun and E Ifeachor, "New Methods for Voice Quality Evaluation for IP Networks"  Proceedings of the 18th International Teletraffic Congress (ITC18), Berlin, Germany, 31 Aug - 5 Sep 2003, pp. 1201 - 1210.

L Sun and E Ifeachor, "Prediction of Perceived Conversational Speech Quality and Effects of Playout Buffer Algorithms“, Proceedings of IEEE ICC 2003, Anchorage, USA, May 2003, pp. 1- 6.

L Sun and E Ifeachor, "Perceived Speech Quality Prediction for Voice over IP-based Networks" Proceedings of IEEE ICC 2002, New York, USA, April 2002, pp.2573-2577.

L Sun, G Wade, B Lines and E Ifeachor, "Impact of Packet Loss Location on Perceived Speech Quality“, Proceedings of 2nd IP-Telephony Workshop (IPTEL '01), New York, April 2001, pp.114-122. 

4/2/2005 SoCCE, UoP 52

Contact details:

SPMC Group website: http://www.tech.plymouth.ac.uk/spmc

Professor Emmanuel Ifeachor, Head of Group,

E-mail:E.Ifeachor@plymouth.ac.uk Dr. Lingfen Sun

E-mail:L.Sun@plymouth.ac.uk

Homepage: http://www.tech.plymouth.ac.uk/spmc/people/lfsun/

4/2/2005 SoCCE, UoP 53

Thank you!