23
Service Level Agreements for VoIP Alan Clark CEO, Telchemy Burton Group Catalyst 2005
1 Burton Group Catalyst 2005
Service Level Agreements for VoIP
Alan ClarkCEO, Telchemy
Burton Group Catalyst 2005
2 Burton Group Catalyst 2005
Agenda
• VoIP SLAs– What’s typical– Why typical isn’t ok
• Approaches to SLA measurement• What to measure• The “trust” problem• Final thoughts
3 Burton Group Catalyst 2005
What is an SLA?
• Agreed set of performance parameters that:– Are measurable– Can be easily related to application performance
• In theory - if the SLA is met then the customer is happy
• How well does this work for Voice over IP?
4 Burton Group Catalyst 2005
Service Provider Perspective
• Contractual SLA– Keep SLA as general as possible– Easily measured metrics– Measure at service demarcation point– SLA monitoring
• But……– Want customer to have good experience overall but
minimize level of contractual commitment to this
5 Burton Group Catalyst 2005
Customer Perspective
• Voice is a mission critical application
• SLA should report all issues that affect service quality (i.e. don’t want service provider to claim they meet SLA when service unsatisfactory)
• Need service provider to “make it work”
• SLA helps to focus service provider on delivering agreed quality levels
• Highly reliable voice service more important than refunds
6 Burton Group Catalyst 2005
Actual (typical?) VoIP SLA
Jitter < 20mS
Loss < 0.1%
Latency < 100mS
Availability 99.9%
What does this mean in practice?
7 Burton Group Catalyst 2005
Jitter
0
50
100
150
200
0 0.5 1 1.5 2
Time (Seconds)
De
lay (
mS
)
Average jitter level (PPDV) = 4.5mS
8 Burton Group Catalyst 2005
Packet Loss - also time varying
0
10
20
30
40
50
30 35 40 45 50 55 60 65 70
Time (seconds)
50
0m
S A
vg
e P
ack
et
Loss
Rate
Average packet loss rate = 2.1%
9 Burton Group Catalyst 2005
Leads To Time Varying Call Quality
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Time
MO
S
0
100
200
300
400
500
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Ban
dw
idth
(kb
it/
s)
Voice
DataHigh jitter/ loss/ discard
Degraded Service Quality (DSQ) Event = MOS less than X for more than N mS
10 Burton Group Catalyst 2005
Delay
2
3
4
5
0 100 200 300 400 500 600
Round trip delay (milliseconds)
MO
S S
core
55dB Echo Return Loss
35dB Echo Return Loss
Conversational problemsEcho problems
11 Burton Group Catalyst 2005
Why does echo make a difference?
IP
EchoCanceller
Gateway
Echo
Round trip delay - typically 50mS+
Additional delay introduced by VoIP makes existing echo problems more obvious
12 Burton Group Catalyst 2005
Customer expectations of service quality
• Listening quality– Clarity, no distracting noise/ pops/ distortion
• Conversational quality– No noticeable delay or echo
• Availability– Always available, does not drop calls
• Signaling quality– Low call setup delay, features work
13 Burton Group Catalyst 2005
A Better VoIP SLA?
99.9% of calls/intervals haveMOS-LQ > 3.9MOS-CQ > 3.8
Degraded Service QualityEvents < 0.1/ hour[DSQ = ….]
Latency < 100mS
Availability 99.9%
Based on either referenceor actual endpoint
Also reflected in MOS-CQ
Availability of media ANDSignaling path
Transient quality problems
14 Burton Group Catalyst 2005
Enterprise Scenario
IP Phones
IP Phones
IP Phones
IP VPN
Gateway
PSTN
15 Burton Group Catalyst 2005
Measuring at ISP Demarcation - active test
Active Test Functions
Test call
16 Burton Group Catalyst 2005
Active test for IP Service SLA
• Uses VoIP calls, to ensure packets are treated identically to “real” VoIP calls
• Use a Reference endpoint - I.e. a fixed configuration, known, virtual IP endpoint
• Test:– Peak times - to understand quality under load
conditions– Off-peak times - to detect problems before they
impact users
17 Burton Group Catalyst 2005
Measuring at user desktop - passive test
RTCP XR
SIP QoSReport
EmbeddedMonitoringFunction
18 Burton Group Catalyst 2005
Passive test for IP SLA
• Most effective for end-to-end measurement
• Embedded quality monitoring function in IP endpoint
• Can measure service quality, signaling reliability……
• Collect data via RTCP XR or SIP QoS reports
19 Burton Group Catalyst 2005
Service Level Metrics - “mine” or “yours”
• Common problem with SLAs– Service provider measures SLA and reports that they
meet SLA– Customer uses different tools and finds that service
provider “does not” meet SLA
• Who is right?
20 Burton Group Catalyst 2005
Overcoming the “somebody else’s problem” problem
• Need– Common measurement methodology– Ability to make the same measurements at the same
time in the same way
• Otherwise– Results will differ and fingers will point
• Solution?– A common (trusted) measurement methodology– A shared (trusted) measurement function that is
accessible to both service provider and customer
21 Burton Group Catalyst 2005
A “Trusted” SLA Monitoring Function
SummaryStats
SummaryStats
ServiceProvider
Enterprise
Non-intrusivemonitoring
Active test agent
Edge router
22 Burton Group Catalyst 2005
SLA Monitoring Function
• Ideally - locate in edge router on customer premise
• Measurement data available to both service provider and customer
• Provides both – Non-intrusive per-call monitoring of live traffic– Active test agent for scheduled testing and
troubleshooting
• Measures SLA in terms of– Estimated call quality level (MOS, R)– DSQ (Degraded Service Quality) events– Loss, jitter, discard, delay………
23 Burton Group Catalyst 2005
VoIP SLA Management
• VoIP SLAs– What’s typical– Why typical isn’t ok
• Approaches to SLA measurement• What to measure• The “trust” problem• Final thoughts?
– VoIP does not have such distinct service boundaries as traditional telephony - some form of shared data model is an essential addition to an SLA for collaborative troubleshooting across network boundaries.
