VoIP Performance Management - Telchemy Telephony Fal… · • From a service provider perspective...

Internet Telephony Fall 20051

VoIP Performance Management

Alan Clark

CEO, Telchemy

Internet Telephony - Fall 2005


Outline

• Problems affecting VoIP performance• Tools for Measuring and Diagnosing Problems• Protocols for Reporting QoS• VoIP Performance Management Architecture• Application to Enterprise and Service Provider

Networks


Enterprise VoIP Application

Branch Office

IP Phone

IP VPN

IP Phone

Teleworker

IP Phones

Gateway


Residential VoIP Application

Internet TrunkingGateway

PSTN

ResidentialGateway


Potential Issues

IP VPN

IP Phone

IP Phones

GatewayLineEcho

Access LinkCongestion

LAN congestion,Long Ethernet segments,Duplex mismatch

Route flapping,Link failures,Delay

CODECdistortion

AcousticECHO


Call Quality Problems

• Packet Loss• Jitter (Packet Delay Variation)• Codecs and PLC• Delay (Latency)• Echo• Signal Level• Noise Level


Packet Loss and Jitter

CodecIPNetwork

JitterBuffer

Packets lostin network

Packets discardeddue to jitter

DistortedSpeech


Jitter measurements can be misleading!!!

50

75

100

125

150

0 0.5 1 1.5 2

Time (Seconds)

De

lay (

mS

)

Average jitter level (PPDV) = 4.5mSPeak jitter level = 60mS


WiFi can also cause jitter

0

50

100

150

200

250

300

0 25 50 75 100

125

150

175

200

225

250

275

300

325

350

375

400

425

450

Time

Dela

y (

mS)

& R

SSI

Recvd Signal Strength

Delay


Effects of Jitter

• Low levels of jitter absorbed by jitter buffer

• High levels of jitter– lead to packets being discarded– cause adaptive jitter buffer to grow - increasing delay but

reducing discards

• If packets are discarded by the jitter buffer as they arrive too late they are regarded as “discarded”

• Simple jitter metrics such as PPDV can be misleading


Packet Loss

0

10

20

30

40

50

30 35 40 45 50 55 60 65 70

Time (seconds)

50

0m

S A

vg

e P

ack

et

Loss

Rate

Average packet loss rate = 2.1%Peak packet loss = 30%


0

50

100

150

200

0 100 200 300 400 500Burst length (packets)

Bu

rst

we

igh

t (p

ack

ets

)Example Packet Loss Distribution

20 percent burst density (sparse burst)

Con

secu

tive

loss


Loss and Discard

• Loss is often associated with periods of high congestion

• Jitter is due to congestion (usually) and leads to packet discard

• Hence Loss and Discard often coincide

• Other factors can apply - e.g. duplex mismatch, link failures etc.


Example Loss/Discard Distribution

0

50

100

150

200

0 100 200 300 400 500Burst length (packets)

Burs

t w

eig

ht

(pack

ets

)

25 percent burst density (sparse burst)C

onse

cutiv

e lo

ss


Leads To Time Varying Call Quality

12

34

5

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Time

MO

S

0100200300400500

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Ba

nd

wid

th (

kb

it/

s)

Voice

Data

High jitter/ loss/ discard


Packet Loss Concealment

• Mitigates impact of packet loss/ discard by replacing lost speech segments

• Very effective for isolated lost packets, less effective for bursty loss/discard

• But isn’t loss/discard bursty?– Need to be able to deal with 10-20-30% loss!!!

Estimated by PLC algorithm


Effectiveness of PLC

1

2

3

4

5

0 5 10 15 20

Packet Loss/Discard Rate

AC

R M

OS

G.711 no PLCG.711 PLCCodec

distortionImpact of loss/ discard and PLC

Typical burst packetLoss/discard rate





Effect of Delay on Conversational Quality

1

2

3

4

5

0 100 200 300 400 500 600

Round trip delay (milliseconds)

MO

S S

core

Low echo level (55DB)

Significant echo level (35dB)


Interaction of echo and delay

• Echo with very low delay sounds like “sidetone”

• Echo with some delay makes the line sound hollow

• Echo with over 50mS delay sounds like…. Echo

• Echo Return Loss – 55dB or above is good– 25dB or below is bad


Cause of Echo

IP

EchoCanceller

Gateway

LineEchoRound trip delay - typically 50mS+

Additional delay introduced by VoIP makes existing echo problems more obvious

AcousticEcho


Causes of Delay

CODEC Echo Control

RTP

IPUDPTCP

CODEC Echo Control

RTP

IPUDPTCP

External delayAccumulate and encode

Network delay Jitter buffer, decode and playout





Signal Level Problems

Temporal Clipping occurs with VAD or Echo Suppressors -- gaps in speech, start/end of words missing

Amplitude Clipping occurs -- speech sounds loud and “buzzy”

0 dBm0

-36 dBm0


Noise

• Noise can be due to– Low signal level– Equipment/ encoding (e.g. quantization noise)– External local loops– Environmental (room) noise

• From a service provider perspective - how to distinguish between – room noise (not my problem)– Network/equipment/circuit noise (is my problem)


Measuring VoIP performance

VQmon

ITU P.VTQITU P.862 (PESQ)

VQmon

ITU P.VTQITU P.563

Active Test- Measure test calls

Passive Test- Measure live calls

VoIP SpecificAnalog signal based


“Gold Standard” - ACR Test

• Speech material– Phonetically balanced speech samples 8-10 seconds in length– Test designed to eliminate bias (e.g. presentation order different

for each listener)– Known files included as anchors (e.g. MNRU)

• Listening conditions– Panel of listeners– Controlled conditions (quiet environment with known level of

background noise)

23 2

4


Example ACR test results

• Extract from an ITU subjective test

• Mean Opinion Score (MOS) was 2.4

• 1=Unacceptable• 2=Poor• 3=Fair• 4=Good• 5=Excellent

0

10

20

30

40

50

Votes

1 2 3 4 5

Opinion Score


Measuring VoIP Performance

• VQmon– Most widely used algorithm for VoIP performance monitoring. Fast efficient,

supports narrow and wideband codecs, listening and conversational quality. Incorporates P.VTQ and G.107 as subsets, original model for RTCP XR.

• ITU P.VTQ– In development - expected completion in mid-2006. Lightweight algorithm

for narrowband use, currently only listening quality, may extend to conversational.

• ITU G.107 E Model– Network planning tool, used as a basis for some monitoring applications.

Inaccurate under conditions of bursty packet loss.

• P.862– Intrusive speech quality algorithm. Slow - takes a PC to process one speech

file in approx real time.

• P.563– Non-intrusive algorithm that operates on analog speech data. Highly

MIPS/Memory intensive and very inaccurate for individual calls.


Reminder - loss/jitter are time varying

50

75

100

125

150

0 0.5 1 1.5 2

Time (Seconds)

De

lay (

mS

)

Average jitter level (PPDV) = 4.5mSPeak jitter level = 60mS


VQmon algorithm

Arrivingpackets

Discarded

CODEC

Jitterbuffer

Loss/ Discardevents

MetricsCalculation

4 State Markov ModelGather detailedpacket loss infoin real time

Signal levelNoise levelEcho levelDelay

Call Quality ScoresDiagnostic Data


Modeling transient effects

10 15 20 25 30 35Time (seconds)

MeasuredCall quality

User ReportedCall quality

Ie(gap)

Ie(burst)

Ie(VQmon)


Computational model

Burst lossrate

Gap lossrate

Ie mapping

Perceptual model

CalculateR-LQMOS-LQ

CalculateRo, Is

Signal levelNoise level

CalculateId

EchoDelay

CalculateR-CQMOS-CQ

Recencymodel

ETSI TS101 329-5

ModifiedITU-T G.107


Accuracy: Non-bursty conditions

Comparison of VQmon vs ACR MOS - ILBC 15.2k

1

1.5

2

2.5

3

3.5

4

4.5

5

0 5 10 15 20Packet Loss Rate (%)

MO

S S

co

re

ACR MOS

VQmon MOS-LQ

Comparison of VQmon vs PESQ - ILBC 15.2k

1

1.5

2

2.5

3

3.5

4

0 5 10 15 20 25 30Packet Loss Rate (%)

PE

SQ

Sco

re

PESQ

VQmon MOS-PQ


1.5

2

2.5

3

3.5

4

1.5 2 2.5 3 3.5 4ACR MOS

Est

imate

d M

OS

Comparison of VQmon and E Model

• VQmon– Extended G.107– Transient impairment model– Wide range of codec models– Narrow & Wideband– Jitter Buffer Emulator– Listening and Conversational

Quality

• G.107– Well established model for

network planning– No way to represent jitter– Few codec models– Inaccurate for bursty loss– Conversational Quality only

VQmon

E Model

Comparison of VQmon and E Modelfor severely time varying conditions


ITU P.563 - Passive monitoring

• Analyses received speech file (single ended)

• Produces a MOS score

• Correlates well with MOS when averaged over manycalls

• Requires 100MIPS per call

• NOT suitable for individual calls

1.00

2.00

3.00

4.00

5.00

1 2 3 4 5

P563 Score

AC

R M

OS

Comparison of P.563 estimated MOS scores with actual ACR test scores.Each point is average per file ACR MOS with 16listeners compared to P.563 score


Active or Passive Testing?

• Active testing – works for pre-deployment testing and on-demand

troubleshooting

• But!!!!– IP problems are transient

• Passive monitoring – Monitors every call made - but needs a call to

monitor– Captures information on transient problems– Provides data for post-analysis

• Therefore - you need both


VoIP Performance Management Framework

Media Path Reporting(RTCP XR)

Call Server andCDR database

VoIPEndpoint

VoIPGateway

SNMPReporting

NetworkManagementSystem

Signaling Based QoS Reporting

Embedded Monitoring

Network Probe,Analyzer orRouter

VQVQ

Embedded Monitoring

VQ

RTP stream (possibly encrypted)


RFC3611 - RTCP XR

Loss Rate Discard Rate Burst Density Gap Density

Burst Duration (mS) Gap Duration (mS)

Round Trip Delay (mS) End System Delay (mS)

Signal level RERL Noise Level Gmin

R Factor Ext R MOS-LQ MOS-CQ

Rx Config - Jitter Buffer Nominal

Jitter Buffer Max Jitter Buffer Abs Max


RTCP XR Application

“A” “B”

RTCP XR

RTCP XR

Quality of stream from A to Band Echo level on trunk side

Trunkside

Quality of stream from B to Aand acoustic echo at A (if known)

Residential Subscriber


SIP Service Quality Reporting Event

PUBLISH sip:[email protected] SIP/2.0

Via: SIP/2.0/UDP pc22.example.com;branch=z9hG4bK3343d7………

Content-Type: application/rtcpxrContent-Length: ...

VQSessionReportLocalMetrics:TimeStamps=START:10012004.18.23.43 STOP:10012004.18.26.02SessionDesc=PT:0 PD:G.711 SR:8000 FD:20 FPP:2 PLC:3 SSUP:[email protected]………

Signal=SL:2 NL:10 RERL:14QualityEst=RLQ:90 RCQ:85 EXTR:90 MOSLQ:3.4 MOSCQ:3.3

QoEEstAlg:VQMonv2.1DialogID:38419823470834;to-tag=8472761;from-tag=9123dh311


SIP QoS Reporting Application

“A” “B” Trunkside

SIP QoS “Collector”

SIP QoS report sent at end of call.Can report on both directions if RTCP XR ispresent in both endpoints, otherwise onlyon received direction.

Residential Subscriber


Enterprise Application

Branch Office

IP Phone

IP VPN

IP Phone

Teleworker

VQ

IP Phones

Gateway

NMS

VQ

VQ

VQ

VQ

VQ

VQ

VQ

VQ

VQ

VQ

VQ

RTCP XR

SIP QoS ReportSNMP

VoIP SLA


Actual (typical?) VoIP SLA

Jitter < 20mS

Loss < 0.1% per month

Latency < 100mS

Availability 99.9%

What does this mean in practice?


A Better VoIP SLA

99.9% of calls/intervals haveMOS-LQ > 3.9MOS-CQ > 3.8

Degraded Service QualityEvents < 0.1/ hour[DSQ = ….]

Latency < 100mS

Availability 99.9%

Based on either referenceor actual endpoint

Also reflected in MOS-CQ

Availability of media ANDSignaling path

Transient quality problems


Enterprise Applications

• Ensure network is VoIP ready before deployment!!

• Use VQmon/RTCP XR/ SIP QoS in IP phones and gateways

• Use passive monitoring on every call to catch transient problems for post analysis

• Develop meaningful SLAs



Internet TrunkingGateway

PSTN

VQ

VQ

VQ

VQ

VQ

ResidentialGateway

RTCP

XR

SIP QoS



Internet

TrunkingGateway

PSTN

VQ

VQ

VQ

VQ

ResidentialGateway

RTCP

XR

SIP QoS


Residential VoIP Management

• Use RTCP XR between IP endpoints to provide more detailed call quality metrics and bidirectional reporting

• Use SIP QoS reports to get data back to management systems

• Insist that peer networks (either VoIP or PSTN) support RTCP XR and defined SLAs


Summary

• Problems affecting VoIP performance• Tools for Measuring and Diagnosing Problems• Performance Management Architecture• Applications

Date post:	24-Sep-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

VoIP Performance Management - Telchemy Telephony Fal… · • From a service provider perspective...

Documents