Analyze Assure Accelerate
TMC Developers ConferenceSan FranciscoAug 03rd, 2005
Andy Huckridge
Spirent Communications.
Chair, Interop WG, MSF
ST-09 Network Assurance and Testing During the Migration to VoIP
Analyze Assure Accelerate
Agenda
• Spirent overview
• Key implementation issues
• What is Triple Play / Converged networks?
• Specifics on testing SIP
• Network Impairments and Parameters that Voice and Video Affect Quality
• Metrics for Measuring Voice and Video Quality and Performance
• Good test methodology
Analyze Assure Accelerate
Spirent Communications
• Spirent is the test solution leader
– 1,800 employees in 14 countries
– More than 1,500 customers
– Sales and service capabilities in 30 countries
Analyze Assure Accelerate
PerformancePerformanceAnalysisAnalysis
Industry standards
PerformanceTesting Functionality &
Conformance Testing
ManufacturingQuality Assurance
ServicesServicesDeploymentDeployment
• Characterize system before trial
• Validate system scalability
• Identify capacity limits
• Measure call performance
• Automate regression testing
• Characterize system before trial
• Validate system scalability
• Identify capacity limits
• Measure call performance
• Automate regression testing
Implementation steps - Lab
Analyze Assure Accelerate
ServiceServiceAssuranceAssurance
Pilot Networks
Network Certification
Initial Deployment
WidespreadNetwork Deployment
• Facilitate vendor selection
• Identify performance ceilings
• Enable accurate capacity planning
• End-to-end service assurance testing
• Improve operational performance
• Improve customer satisfaction
• Facilitate vendor selection
• Identify performance ceilings
• Enable accurate capacity planning
• End-to-end service assurance testing
• Improve operational performance
• Improve customer satisfaction
Implementation steps - Network
Analyze Assure Accelerate
Key Implementation Issues
• Circuit to packet migration
• Scalability and Performance
• Voice quality
• Interoperability and conformance
• Budget pressures
Analyze Assure Accelerate
Before you deploy!
• Network Equipment Manufacturers (Chips, IP-PBX, Gateways, MSs & SSs)
– Characterize your system before trial
– Validate system scalability
– Identify capacity limits
– Measure call performance
• Service Providers(NSPs, SPs, ITSPs)
– Define criteria for vendor selection
– Identify performance ceilings
– Accurately plan for your capacity needs
– End-to-end service assurance testing
Analyze Assure Accelerate
Data Transmission“Non-Real-Time” Applications
Name ResolvingDNS
File TransferFTP
Data BaseMS SQLOracle
WebHTTP
Email and MessagingPOPSMPTExchange
Telenet
Data Examples: Internet access, Email, File Transfer, Portals, Database Applications, Gaming, Government Services, Online Commerce
Music DownloadingHome control
Analyze Assure Accelerate
Voice and Video “Real-Time Applications”
VoIP, IP Telephony, Video TelephonyG.711, G.729, G.728, G.726, G.723
H.261, H.263, SIP, SIP-T, H323, Skinny, MGCP,
MEGACO/H.248
VoDIP Music/Audio/Radio
IPTV ServicesBroadcast, On-Demand, Bi-directional / InteractiveMPEG1, MPEG2, MPEG4, VC1, H264
Multi-MediaRTP
H.264,Microsoft AVI, QuickTime (.mov)Windows Media (.wmv, .asf), RealMedia (.rm),
Voice Applications: Phone service integrated with videoVideo Applications: Broadcast TV, video on demand, distance learning
Real-Time Online CommunicationsInstant MessengerWebexNetmeetingSIPH.323
GamingSingle / Multiplayer
Analyze Assure Accelerate
Off HookDial #
Dialed Digits
ConfigureConfigure
ConnectConnect
Notify
Ring Off HookHello
Hello
Voice Conversation
Good Bye
Good ByeOn Hook
On Hook
Disconnect
Configure
Signaling Path
Ring Back
Converged Triple Play: Data, Voice and Video With Network Impairments
Data
Video
Impairments can be heard in the voice conversation
Analyze Assure Accelerate
Testing SIP Conformance
• Comprehensive and scriptable SIP call flows
• Complete configurable SIP signaling messages
• SIP protocol analysis
• Simplified flow diagrams with visual analysis
• Comprehensive conformance test suites
Analyze Assure Accelerate
Testing SIP Conformance
• ETSI TS 102-027-1 v2.12,Tiphon:
– RFC 3261 user agent, proxy and redirect server compliancecompliance
• Graphical SDL and TTCN tools
– Create, edit, compile and execute simulation scripts and conformanceconformance tests
• Additional SIP messages beyond RFC 3261
– Included in torture teststorture tests
• Additional tests as defined by the SIP Forum
Analyze Assure Accelerate
Testing with Configurable SIP
Configurable SIPConfigurable SIP call setup and call teardown
• Configurable call flows and messagesConfigurable call flows and messages
• Incoming message filterIncoming message filter
–Adaptive signaling syntax for SIP
– Improves interoperabilityinteroperability with new drafts and non-conformant proprietary implementations
Analyze Assure Accelerate
Testing with Configurable SIP
• Configurable messagesConfigurable messages:
Invite, ACK, bye, register
Responses: 1xx, 2xx, 3xx, 4xx, 5xx, 6xx
• Configurable timers,Configurable timers, message intervalsmessage intervals
• Enable and disable optional messages: Re-invite, cancel, options, message, info, notify,
subscribe, unsubscribe, update, refer, Prack
Fix erroneous incoming messages “on the fly” with the “search and replace” method
Allows interoperability with SIP devices (including drafts, non conformant, prototype)
Analyze Assure Accelerate
SIP Message Registration Screen Shots
Analyze Assure Accelerate
SIP Message Origination
Analyze Assure Accelerate
SIP Message Termination
Analyze Assure Accelerate
Incoming Message Filters
Analyze Assure Accelerate
TOS for SIP Signaling
Analyze Assure Accelerate
Diffserv for SIP Signaling
Analyze Assure Accelerate
Testing SIP Robustness
• Robustness testingRobustness testing
– Passed: does not crash, stable, or acceptable results
– Failed: crashes, unstable, or unacceptable results
• Security testingSecurity testing
– It is crucial to identify SIP security holesIt is crucial to identify SIP security holes
• SIP testing toolSIP testing tool
– Tests SIP robustness and security
– Comprehensive negative test suites for SIP
Analyze Assure Accelerate
Real Signaling with real RTP
• Capability to do signaling with audio
• Capability to perform real time measurements
• Capability of using signaling without audio
• Problems of not using real signaling
• Problems of not using real RTP streams
• Real time objective metrics
Analyze Assure Accelerate
Testing SIP-T
PSTN
SIP-T
SIP-TSIP-T
SS7 SS7
SIP ProxyH.248/Megaco H.248/Megaco
Trunking Gateway
Trunking Gateway
RTP/RTCP
POTS
MGC
SS7GR303ISDNCASV5
1000Base-SX/LX10/100/1000Base-T
PSTN
VoIP Network
POTS
MGC
Analyze Assure Accelerate
SIP-T Performance Testing Suites
Performance testing
– Validate and stress-test SS7 ISUP and SIP interworking with optional media, over thousands of emulated user agents
• SIT-T testing
– Configurable SIP-T calls with intelligent protocols• QoS and CoS testing
– Optional TOS/Diffserv and VLAN options in SIP-T media calls, used to measure QoS with PESQ and e-model
• Feature testing
– Automated and configurable SIP call set-up, teardown, flows, messages
Analyze Assure Accelerate
Network Impairments and Parameters that Affect Voice and Video Quality
• Network Architecture
• Types of Access Links
• QoS controlled Edge Routing
• MTU Size
• Packet Loss (Frame Loss)
• Out of order packets
• One Way Delay (Latency)
• Variable Delays (Jitter)
• Background Traffic (Congestion, Bandwidth, Utilization, Network Load, Load Sharing)
• Timing Drift
• Route Flapping
• Signaling protocol mismatches
• Network faults
• Link Failures
• Voice Only Impairments– Echo
– Voice coding algorithms
– A/D and D/A Conversion
– Noise – Circuit and External
• Video Only Impairments– Video coding algorithms
– Fixed vs Variable Frame Rate
Analyze Assure Accelerate
IP Network Architecture
Core IP NetworkLAN A
Local AccessB
1000BaseX* 100BaseT Switch100BaseT Hub10BaseT* WLAN (~4 Mbit/s)----------------------Occupancy levelPacket loss
64 kbit/s*128 kbit/s256 kbit/s*384 kbit/s512 kbit/s*768 kbit/s
*T1 (1.536 kbit/s)E1 (1.920 kbit/s)E3 (34 Mbit/s)*T3 (44 Mbit/s)
ADSL (~256 kbit/s)*Cable (~256 kbit/s)Fiber (1-10 Gbit/s)
--------------------Occupancy levelQoS edge router
LAN B
Route flappingOne-way delay
JitterPacket loss
DestinationDevice B
Local AccessA
64 kbit/s*128 kbit/s256 kbit/s*384 kbit/s512 kbit/s*768 kbit/s
*T1 (1.536 kbit/s)E1 (1.920 kbit/s)E3 (34 Mbit/s)*T3 (44 Mbit/s)
ADSL (~2 Mbit/s)*Cable (~3 Mbit/s)Fiber (1-10 Gbit/s)
--------------------Occupancy levelQoS edge router
SourceDevice A
* Case used in impairment tables
1000BaseX* 100BaseT Switch100BaseT Hub10BaseT* WLAN (~4 Mbit/s)----------------------Occupancy levelPacket loss
Affects Data, Voice and Video Quality
Analyze Assure Accelerate
Network Operating With Constant Delay
Affects Voice and Video Quality
Analyze Assure Accelerate
End to End Delay Sources
• Algorithmic delay
• Serialization delay
• Propagation delay
• Component delay
Affects Voice and Video Quality
Fixed• Look ahead
• Encoding
• Buffer
• VAD
• Packetizing
Fixed• Switching
Variable• Voice contention
• Data Contention
• Video Contention
Fixed• Serialization
WAN
Fixed• Switching
•Propagation
•Serialization
Variable• Voice contention
• Data Contention
•Video Contention
Fixed• Switching
Variable• Voice contention
• Data Contention
• Video Contention
Fixed• Decoding
Variable • De-jitter buffer
• Packet loss Concealment
Originating LAN
Core Network Terminating
LANOriginating Gateway
Edge Router Core
Network Routers
Terminating Gateway
Edge Router
Fixed• Serialization
WAN
Analyze Assure Accelerate
Echo Impairment on Converged network
VV
T1 Link
Echo Canceller in MG reduces Echo Level
IP Network
ERLERLE
TELR
PBX POTSPhone
ERLE – Echo Return Loss EnhancementERL – Echo Return LossTELR – Talker Echo Loudness Rating
2 Wire
PBX
Analog 4-Wire Link
E&M
POTSPhone
Hybrid Transformer
Analog 2-Wire Link
TX
RX
TX
RX
T1 Link
Impedance Mismatch
MG
Tail CircuitIP
Phone
IP Phone
Delay in IP Network makes Echo sound worse
Affects Voice Quality
Analyze Assure Accelerate
Echo Impairment on Converged network
Converged Network
Echo Path Side A (250ms)Echo Path Side B (250ms)
Path A to BPath B to A
Echo is caused by impedance mismatches in hybrid circuits (2w to 4w) and feedback between the telephone mouth piece and ear piece
Electrical Coupling• Impedance Mismatch (Hybrid)
Acoustical Coupling• Speakerphone
Affects Voice Quality
Analyze Assure Accelerate
Effect of Delay on Voice Quality
PSTN
> 25ms Echo Cancellation Required
<150 ms (with echo cancellation): acceptable
> 400 ms unacceptable for most applications
150-400 ms: acceptable if delay expected
Voice Quality
Analyze Assure Accelerate
Effect of Echo Level on Voice Quality
TELR – Talker Echo Loudness Rating(Signal to Echo Ratio)
More Echo
Less Echo
Affects
Voice Quality
Less Echo
More Echo
Analyze Assure Accelerate
Network with Variable Delays (Jitter)
• Variable processing delay
– A busy router or switch will take longer to look up the routing (address) table
• Queuing delay
– Network congestion
Time (s)
Del
ay (
ms)
Affects
Voice and Video Quality
Analyze Assure Accelerate
Jitter Characteristics
Time (s)
Del
ay (
ms)
Del
ay (
ms)
Del
ay (
ms)
Good
Bad
Severe
Affects Voice and Video Quality
Analyze Assure Accelerate
Packet LossExample: Queue Management
Bit Bucket
Threshold
RED (Random Early Discard)
Affects Voice and Video Quality
Analyze Assure Accelerate
Speech Compression Impairment
G.711 Best Quality
Common Compression Types: G.711, G.729, G.728, G.726, G.723, AMR, EVRC
Voice Quality
Analyze Assure Accelerate
VAD – Voice Activity Detection
No VAD
VAD
Data is intentionally not sent during times of Silence
Affects Voice Quality
Timing may be different
Analyze Assure Accelerate
= 80 ms Speech= 80 ms Speech
= 40 ms Speech= 40 ms Speech
= 20 ms Speech= 20 ms Speech
80 Bytes80 Bytes
40 Bytes40 Bytes
20 Bytes20 Bytes
Impact Of Packet Size
• Typically Packets are kept small for best results
• Many equipment manufacturers use dynamic packet size to optimize for network conditions
Normal size for VoIP applications
Affects Data, Voice and Video Quality
10 Bytes10 Bytes
= 10 ms Speech= 10 ms Speech
Analyze Assure Accelerate
Mechanisms for Assuring QOS
• Class of Service (COS) ITU-T Y.1541 defines the 5 classes of service and their application
• Type of Services (TOS)
• TOS and COS are both elements with in an IP Packet
• DIFSER and RSVP provide mechanisms to improve QOS
QoS Class(Y.1541)
Applications (Examples) Node Mechanisms Network Techniques
0Well
Managed
Real-Time, loss sensitive, Jitter sensitive, high interaction (VoIP, VTC, IPTV
Strict QoS. Guaranteed no over subscription on links.
Constrained Routing and Distance
1Best
Effort
Real-Time, Jitter sensitive, interactive (VoIP, VTC).
Separate Queue with preferential servicing, Traffic grooming
Less constrained Routing and Distances
2 Transaction Data, Highly Interactive, (Signaling) Separate Queue, Drop
priority
Constrained Routing and Distance
3 Transaction Data, Interactive
Less constrained Routing and Distances
4 Low Loss Only (Short Transactions, Bulk Data, Video Streaming)
Long Queue, Drop priority Any route/path
5Internet
Traditional Applications of Default IP Networks
Separate Queue (lowest priority)
Any route/path
Video
Data
Voice
Triple Play
Affects Data, Voice and Video Quality
Analyze Assure Accelerate
TIA-921 and ITU-T G.NIMMTest Profiles Based on QoS (Y.1541) Classes
Impairment Type Units Range
Jitter ms 0 to +/- 250
One Way Latency ms 50 to 400
Sequential Packet Loss #sequential packets
2 to 500
Rate of Sequential Loss sec-1 < 10-1
Random Packet Loss % 0 to 20
Out of Sequence Packets % 0 to 20
Impairment Type Units Range
Jitter ms +/- 75
One Way Latency ms 50 to 200
Sequential Packet Loss #sequential packets
2 to 5
Rate of Sequential Loss sec-1 0 to 2
Random Packet Loss % 0 to 2
Out of Sequence Packets % 0 to 0.1
Profile C Un-Managed Network
Table 4
Profile BBest Effort Managed Network
Table 3
Profile AWell Managed Network
Table 2
Different test profiles for different Service Level Agreements (SLAs)
Impairment Type Units Range
Jitter ms +/- 50
One Way Latency ms 50 to 100
Sequential Packet Loss #sequential packets
Random loss only
Rate of Sequential Loss sec-1
Random Packet Loss % 0 to 0.05
Out of Sequence Packets % 0 to 0.001
Analyze Assure Accelerate
Early Voice Quality Testing
Analyze Assure Accelerate
Voice Quality Testing
• Active (Intrusive) Testing– Sends, Receives and compares Wave Files to measure voice quality
– MOS (Mean Opinion Score)
– PSQM, PSQM+ (Perceptual Speech Quality Measurement)
– PESQ (Perceptual Evaluation of Speech Quality)
– R-Value and J-MOS derived from PESQ
• Passive Testing– R-Value – ITU-T P.VTQ
• Measures Voice Quality on RTP Packets
• Based on E-model
• Japan – J-MOS
• Similar Techniques can be used to measure Video Quality
– P.563 (ITU-T recommendation) 3SQM, P-Stream• Measures Voice Quality of Voice traffic based on Audio Siginal
• Provides an estimate of PSQM
Analyze Assure Accelerate
Active (Intrusive) Voice Quality Testing
DUTSend Wave Files
Example: (ITU-T Female Nice File with Pilot Tone)
Receive Wave Files
Measures Voice Quality by Comparing Sent and Received Wave files
MOS, PSQM, PSQM+, PESQ, R-Factor (PESQ Derived)
Sent (Green) and Received (Orange) wave files
Expanded Sent (Green) and Received (Orange) wave files
PESQ Score vs Number of PESQ Measurements
Values are different for Male, Female, different Wave Files and different Languages
Analyze Assure Accelerate
Passive Voice and Video Quality Testing
R-Factor/Emodel
TrunkingGateway
E1/T1/E3/T3/PRI/GR303,
V5,SLC96
RTPPSTNPSTN
IP Network
IP Telephone
Measure:Video QualityMOS-LQMOS-CQMOS-PQJ-MOSNetwork RUser RBurst statisticsDiagnostic data
Measure: Video QualityMOS-LQMOS-CQMOS-PQJ-MOSNetwork RUser RBurst statisticsDiagnostic data
RTP
ITU-T P.VTQ
Analyze Assure Accelerate
Passive Voice Quality TestingP.563 (P-Stream, 3SQM)
DUT Receive Audio
Estimates Voice Quality based on 3 Characteristic of Received Audio
RTP
Analyze Assure Accelerate
Voice Quality Measurements
P.861PSQM/PSQM+
0
6.5
PAMS
1
5
Emodel P.862PESQ
~3.88
~3.65
~3.40
~3.13
~2.84
-0.5
4.5
Analyze Assure Accelerate
Sample Voice Quality Test Results
G.711 G.723.1 (6300 bps)
Comparison of Scores for G.711 and G.723.1 (6300 bps)
PSQM
PESQ
MOS
Created by inducing packets lost
Analyze Assure Accelerate
• Video Compression– Video Compress schemes affect the video quality
– H.261, H.263, H.264, VC1, MPEG-1, MPEG-2, MPEG-4, Microsoft AVI, Windows Media (.wmv, .asf), RealMedia (.rm), QuickTime (.mov)
• Interactive real-time applications (e.g., video conferencing, voice over IP) are sensitive to latency and Frame Rate
• Typical Video Quality Metrics– Objective MOS
– Blockiness
– Blur
– PSNR
– Spatial Resolution
– Temporal Resolution
Video Quality Measurement
– SNR
– Edge Noise
– Jerkiness
– Error Blocks
– Object Retention
– Color Reproduction Accuracy
Analyze Assure Accelerate
Video Quality MeasurementBlockiness
OriginalBlockiness
Analyze Assure Accelerate
Video Quality MeasurementReference and Blocky Video
Original(Reference)
Blocky
Analyze Assure Accelerate
Video Quality MeasurementBlur
OriginalBlur
Analyze Assure Accelerate
Video Quality MeasurementNoise
OriginalNoise
Analyze Assure Accelerate
Video Quality MeasurementSpatial (Pixel) Resolution
128X128 32X32 8X8
Spatial Resolution
Department of Computer ScienceUniversity of Canterbury http://www.cosc.canterbury.ac.nz/people/mukundan/covn/Imgresl.htm
Analyze Assure Accelerate
Video Quality Measurement Temporal (Motion) Resolution
Video Frames
Fk Fk+1 Fk+2 Fk+3 Fk+4 Fk+5
Temporal-Width (t)
Vertical-Width (v) Horizontal-Width (h)
Analyze Assure Accelerate
Video Quality MeasurementModels
• Video Quality Metrics (VQM)– ITU-T SG9 and VQEG are working on standard
• TV Model - optimized for higher bit-rate digital television systems with no frame dropping (e.g., MPEG-2)
• Videoconferencing Model - optimized for lower bit-rate videoconferencing systems that drop frames (e.g., H.261, H.263).
• General Model - optimized for a wide range of video quality (videoconferencing, TV)
• Developer Model - optimized for a wide range of video quality (videoconferencing, TV) with the added constraint of fast computation.
• PSNR Model - based on the traditional peak signal-to-noise-ratio (PSNR) calculation.
Analyze Assure Accelerate
Video Quality MeasurementTechniques
• Full Reference (ITU-T J.144R and BT.1683)– Video quality is calculated by comparing the received video with
the complete original video
• Reduced Reference (ANSI T1.801.03-2003 and ITU-T J.143) – Spatial and temporal information are calculated from original
video and transmitted to the receiving end
– Video quality is calculated by comparing the received video with the reduced reference
• No Reference – Passive – Video quality on based only on the received information (picture
content)
– Video quality is derived from RTP packet information similar to R-Factor (E-Model) for voice quality
Analyze Assure Accelerate
Video Quality Measurement Full Reference
Full Reference
Video Quality Score
•MOS
•Blockiness
• Blur
• PSNR
Processor Processor
Original Video Received Video
Original Video
Transmitted Video Signal
Analyze Assure Accelerate
Video Quality Measurement Reduced Reference
Reduced-Reference •Spatial (Pixel) and Temporal (Motion) Information
Reduced-Reference SignalLow Rate
Processor Processor
Transmitted Video Received Video
Video Quality Score
•MOS
•Blockiness
• Blur
• PSNR
Analyze Assure Accelerate
Video Quality Measurement No-Reference
No-ReferencePicture Content
or Passive monitoring of RTP
Processor Processor
Transmitted Video Received Video
Video Quality Score
•MOS
•Blockiness
• Blur
• PSNR
Analyze Assure Accelerate
Types of TestingType of tests
• Video Quality
• Voice quality
• Functional
• Scalability
• Troubleshooting
• Conformance
• Interoperability
• Triple Play (Data, Voice and Video)
Payload types
• Video
• Voice
• Data
• Fax
• Modem
With these protocols
• IP
– SIP
– H.323
– MGCP
– Megaco/H.248
– Skinny
• PSTN
– CAS
– PRI
– SS7
– NFAS
– V5
– GR303
With these interfaces
• GigE 1000Base-SX, 1000Base-LX
• 10/100/1000Base-T
• Analog
• T1/E1
• T3/E3
Test these DUTs
• IP PBX
• Gateways
• IP Phone
• Servers
• Firewalls
• IAD
Analyze Assure Accelerate
Types of Testing
•Call Establishment
–Start Dial Signal Delay
–Post Dial Delay
–Call Duration
–Ring Duration
•Call Disconnect
–Connection Disconnect Delay
–Release on Request
•Call Statistics
–Connection set-up failures
–Connection premature disconnect
–Call completion percentages
•Speech Quality Measurements
–PESQ
–PSQM
–MOS
–R Factor
–Echo Delay
–Round Trip Delay
–Echo Return Loss
–Signal Pass Noise
–Noise Level
•Video Quality Measurements
–MOS
–Blockiness
– Blur
– PSNR
•Transport Layer Measurements
–One-way Transmission Time
–Roundtrip Transmission Time
–Jitter
–Packets out of order
–Packet Loss
Analyze Assure Accelerate
Call Quality Summary
Fair24%
Poor2% Bad
1%
Good 73%
Good
Fair
Poor
Bad
Call Quality Summary
Not Recommended1%
Nearly All Users Dissatisfied
2%Many Users Dissatisfied
10% Very Satisfied20%
Some Users Dissatisfied
14%
Satisfied53%
Video Telephony TestingDistributed Testing
75
80
85
90
95
100
% C
om
ple
te
% Complete
Call Completion Rate by Day
% Complete 98 97 99 99 96 99 99
Mon Tue Wed Thu Fri Sat Sun
0
20
40
60
80
100
120
140
160
180D
ela
y (
ms)
One Way Delay by Call Group
Delay (ms) 122 98 106 173 132 112
NYC - DAL NYC - CHI NYC - SJ DAL - CHI DAL - SJ CHI - SJ
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40%
Lo
ss
Packet Loss by Call Group
% Loss 0.10 0.43 0.40 1.23 0.35 0.64
NYC - DAL NYC - CHI NYC - SJ DAL - CHI DAL - SJ CHI - SJ
0
50
100
150
200
250
300
350
Cal
l S
etu
p T
ime
(ms)
Call Setup Time by Call Group
Call Setup Time (ms) 150 175 130 313 110 105
NYC - DAL NYC - CHI NYC - SJ DAL - CHI DAL - SJ CHI - SJ
0
10
20
30
40
50
60
70
80
Jitt
er (
ms)
Jitter by Call Group
Jitter (ms) 43 51 41 73 54 45
NYC - DAL NYC - CHI NYC - SJ DAL - CHI DAL - SJ CHI - SJ
0
20
40
60
80
100
Jitt
er (
ms)
Jitter (ms)
Jitter by HourCHI-to-DAL
Jitter (ms) 77 69 67 65 68 68 69 70 73 75 76 93 92 100 82 83 81 80 79 79 79 79 79 78
12 AM
1 AM
2 AM
3 AM
4 AM
5 AM
6 AM
7 AM
8 AM
9 AM
10 AM
11 AM
12 PM
1 PM
2 PM
3 PM
4 PM
5 PM
6 PM
7 PM
8 PM
9 PM
10 PM
11 PM
1.00
2.00
3.00
4.00
5.00
MO
S
MOS
Call Quality Summary by HourCHI-to-DAL
MOS 3.5 3.7 3.6 3.4 3.8 3.4 3.6 3.8 3.5 2.9 2.8 2.3 1.9 1.8 2.9 2.9 3.1 2.9 3.4 3.5 3.5 3.6 3.6 3.7
12 AM
1 AM
2 AM
3 AM
4 AM
5 AM
6 AM
7 AM
8 AM
9 AM
10 AM
11 AM
12 PM
1 PM
2 PM
3 PM
4 PM
5 PM
6 PM
7 PM
8 PM
9 PM
10 PM
11 PM
• Isolate Network Problems
• Results Over Time
• Results by Group
Analyze Assure Accelerate
Good test methodology
Implementation, Validation & Observation– Conformance testing
• IETF 3261 & new SIP RFC’s
– Stress testing• Scriptable call flow
• Bulk signaling with real RTP
– Robustness testing• SIPPING Torture Test
• PROTOS / ETSI TIPHON
– Visual protocol analysis• Application & content decoding
Analyze Assure Accelerate
Analyze Assure Accelerate