The authoritative, unbiased source for ITcertification, research and testing
""DDiissppeelllliinngg VVooIIPP MMyytthhss""wwiitthh HHPP PPrrooCCuurrvveeNNeettwwoorrkkiinngg Head-to-Head Testing of 3Com, Ciscoand Mitel VoIP Solutions Shows HPProCurve Switching Infrastructure is aViable Alternative to a HomogenousVendor Approach
For additional information on acceptable usage of this document (TollyGroup Document # 204114) contact The Tolly Group at (561) 391-5610 orvia E-mail at [email protected].
Reproduction of this publication in any form without prior written permissionis forbidden. The information contained herein is believed to be accurateand reliable. The Tolly Group shall have no liability for errors, omissions orinadequacies in the information contained herein or for interpretationsthereof.
All excerpts from this report must be approved by The Tolly Group inadvance of publication or use in any public materials.
Tolly Group Services
With more than 15 years experience validating leading-edge InformationTechnology products and services; The Tolly Group has built a global reputation for producing accurate and unbiased evaluations and analysis.We employ time-proven test methodologies and fair testing principles tobenchmark products and services with the highest degree of accuracy.
Launched in 2003, The Tolly Group's "Tolly Verified" service provides in-depth, vendor-neutral certification of an array of features, functionsand performance characteristics in technology disciplines as diverse asWLAN Switching and Anti-spam. See our "Tolly Verified" Home Page.
Our "Up-to-Spec" service provides the custom testing complement to the"standard", granular tests offered in "Tolly Verified". See our "Up-to-Spec"Home Page.
Plus, unlike narrowly focused testing labs, The Tolly Group combines its vasttechnology knowledge with focused marketing services to help clients better position product benchmarks for maximum exposure.
Figure 8. 3Com NBX-100 PSQM scores over HP ProCurve network
Figure 9. 3Com NBX-100 PESQ scores over HP ProCurve network
Figure 10. HP ProCurve Switch 2650-PWR PSQM scores
Figure 11. HP ProCurve Switch 2650-PWR PESQ scores
Figure 12. HP ProCurve Switch 4104gl PSQM scores
Figure 13. HP ProCurve Switch 4104gl PESQ scores
Figure 14. HP ProCurve Switch 5304xl PSQM scores
Figure 15. HP ProCurve Switch 5304xl PESQ scores
List of Figures
5
6
7
8
8
9
10
10
11
12
13
13
14
14
15
Executive SummaryExtensive call-quality tests conducted independently by The Tolly Groupin September 2003 show that Hewlett-Packard Co.'s ProCurve enterpriseLAN switching infrastructure is a fertile network transport for voice over IP (VoIP) traffic generated by a diverse set of IP PBX products from multi-ple vendors. HP ProCurve Networking products gracefully supports call-quality levels on par with, or better than, those provided by a pure Cisco Catalyst network or a pure 3Com Corp. switch fabric whentransporting VoIP alongside data traffic. Tests results offer conclusive evidence that Cisco AVVID traffic transported over an HP ProCurveswitching infrastructure achieves virtually the same toll quality as AVVIDtraffic that rides over a Cisco Catalyst network. What's more, HP ProCurvesupports VoIP traffic generated by other third-party solutions, such as IPPBXes from Mitel Corp. Hands-on testing shows that these devices candeliver VoIP calls that are either on par with, or better than, the call quality delivered by Cisco's AVVID – either in a pure Catalyst environ-ment, or over an HP ProCurve switched network. This demonstrates that users should investigate the full extent of the VoIP options availablethat provide the best price/performance and flexibility to meet userrequirements.
Meeting the ChallengeNever before have enterprise local area networks been so complex.Enterprise LANs have evolved far beyond the pipeline to ferry E-mail andconduct file transfers for integrated workgroups of users. Today, enterpriseLANs have evolved into complex networks that handle traditional datatypes alongside voice and video. Enterprise LANs now serve companyemployees, suppliers and customers – forcing network architects to dealwith a range of issues such as network flexibility, security, reliability, reducedcomplexity – all while offering a competitive return on investment.
The net result is an underlying network infrastructure that delivers varyinglevels of access and service, depending upon user type and application.These network infrastructures must embrace services that address strategic corporate needs for security, mobility and convergence thatmake the network flexible enough to support existing services while alsoaccommodating innovations that are absorbed into the network.
In today's business climate, enterprise LANs must adapt, quickly, to support business requirements and to extend business services andaccess to resources worldwide, or to local business partners.
In essence, enterprise LANs are being transformed into public-like networksthat provide flexible access to corporate resources on an anytime, anywhere basis – provided authenticated users are permitted to accessthe services and resources they desire. While there are many ways to deliver such a network infrastructure, HP has built an architecture thatpushes intelligence to the edge of the network where network administratorsprovide centralized control over network services.
In older enterprise networks, decision-making was highly centralized toparcel out network access, oversee traffic prioritization, direct traffic flows and groom bandwidth. However, as new applications emerge, it isvital that decision-making gets pushed out to the edge of the network –the junction where users connect – to address resource requirementswhere they are needed most. Depending upon user privileges, networkdevices at the edge of the enterprise can unlock or lock entry points tovarious network resources, making the network edge the gateway to theenterprise resources.
The HP ProCurve Networking Adaptive EDGE ArchitectureTM embracesthat type of distributed edge services concept. In a nutshell, theAdaptive EDGE Architecture focuses on:
Traffic monitoring to control bandwidth optimization throughout the network
Integration of industry-standard VLAN and other security capabilitiesto establish access management controls at edge devices
Implementation of standard traffic routing and Layer 2 mesh capabilities to provide path failover and multi-path load balancingto improve "always-on" reliability
Extensive traffic prioritization to provide traffic-type coexistence and Quality of Service (QoS) functions to support current and futurevoice, video and content delivery applications.
It is this very control to the edge philosophy that makes HP ProCurveNetworking well suited to support VoIP. VoIP applications utilize Layer2/3/4 traffic prioritization to ensure voice packets receive the bandwidththey need and packets are not discarded, and that latency is minimizedfrom network edge to network edge. Moreover, HP's extensive support for standards ensures that the ProCurve infrastructure is open and interoperable with major IP telephony gear, so that VoIP traffic createdon any third-party VoIP gateway will be transported over the HP ProCurveswitching infrastructure.
T H E
TO L LYG R O U P
HP ProCurve Networking Adaptive EDGE Architecture focuses on:
Traffic monitoring VLAN integration
Implementation of standard traffic routingExtensive traffic prioritization/QoS
(For a more detailed understanding, check out the HP ProCurve Networking Adaptive EDGE ArchitectureTM white paper at:http://www.hp.com/rnd/pdfs/Adaptive_EDGE_Arch_wp.pdf)
Recent testing conducted by The Tolly Group, investigates the voicequality delivered by HP ProCurve Networking as it carries a variety of VoIPtraffic created by different brand VoIP solutions.
One of the urban myths circulating in the IT community is that VoIP traffic generated by a specific brand of IP PBX must run over that samevendor's switching infrastructure to maintain acceptable toll quality toend users. VoIP traffic, however, is founded upon the bedrock of IP, whichmeans that whatever IP-based switching infrastructure is employed, VoIPtraffic should flow across any IP switching network equally well.
The purpose of the testing effort was not to prove necessarily that oneLAN switching infrastructure platform or voice solution was better thanthe other. Rather testing was designed to demonstrate that HP ProCurveprovides an acceptable switching infrastructure, regardless of the voicesolution vendor. Specifically, The Tolly Group set out to prove the HPProCurve switching infrastructure is a fertile switching environment for the transport of VoIP traffic created by virtually any brand of VoIP IP PBX.Moreover, tests sought to prove that HP ProCurve can deliver sufficientvoice quality on par or better than VoIP running across switching infrastructures from Cisco Systems and 3Com Corp.
Dispelling VoIP MythsWhen it comes to the uncharted waters of Voice over IP (VoIP), fear,uncertainty and doubt play a central role at many organizations embarkingupon a VoIP infrastructure deployment. Given the strategic importanceof converged networks, few organizations want to take the VoIP plungewithout assurance that the underlying switching infrastructure can deliver effective voice call quality. Unfortunately, verbal assurancessometimes sound credible, but when put to the test, can be debunkedas simple myths.
To hear channel partners of market-leading switch maker Cisco Systemstell the story, VoIP is complex, it requires heavy doses of Quality of Service(QoS) and voice traffic supporting Cisco's AVVID (Architecture for Voice,Video and Integrated Data) must run over a homogenous Cisco switchinginfrastructure to deliver the levels of call quality required for enterprise-class service. In fact, Cisco fosters the perception that AVVID traffic willnot run well over anything but a Cisco switching network to attain business-class call quality.
Consider this typical position statement from the Cisco white paper"Cisco AVVID Network Infrastructure Overview:"
"Cisco AVVID Network Infrastructure provides the technological foundationfor Cisco AVVID. It is analogous to a foundation of poured concrete andsteel-reinforcement over which an industrial-strength structure stands.Without this foundation, there is no unifying system tying the applicationstogether…"
Talk like that is enough to strike fear into users who already have significantinvestments in Cisco hardware/software for enterprise LANs, and likely will convince them not to explore lower cost alternatives, or solutions thatpromise to provide greater functionality and flexibility than the CiscoCatalyst switching infrastructure delivers. Talk alone, however, is cheap.What users need is good, solid testing-based research that gives themthe type of empirical data required to understand the true facts aboutVoIP and IP switching.
Recent testing conducted by The Tolly Group, compares the voice qualitydelivered by a Cisco LAN switching infrastructure versus Hewlett-PackardCo.'s ProCurve enterprise LAN switching fabric, providing strong evidencethat there is more myth than fact to Cisco's AVVID claims. The Tolly Groupalso tested the VoIP call quality of a pure 3Com Corp. switched network asit supported VoIP traffic generated by a 3Com NBX-100 IP PBX.
(HP commissioned The Tolly Group to compare the voice quality offeredby several VoIP gateway offerings that generate real-time voice traffictraveling over a ProCurve switching infrastructure. The Tolly Group compared the toll quality supported by ProCurve to the toll quality ofCisco AVVID traffic running over a pure Cisco Catalyst network and the3Com NBX -generated VoIP running over a pure 3Com switched network.)
AVVID transport: HP and Cisco head-to-head comparison
Does Cisco's Catalyst, alone, provide the right transport underpinningsto guarantee high toll quality for voice traffic supporting Cisco's AVVIDblueprint? Or can HP's ProCurve switching line provide a transport forCisco AVVID and other VoIP solutions by delivering call quality eitheron par or better than a Cisco switching infrastructure? That's preciselywhat The Tolly Group set out to determine in its head-to-head tests.
Tolly Group engineers measured call quality of the AVVID traffic onboth switching infrastructures by using two ITU standard metrics topaint an accurate picture – the ITU P.861 Perceptual Speech QualityMeasurement (PSQM) and the ITU P.862 Perceptual Evaluation ofSpeech Quality (PESQ). PESQ is actually a replacement to the p.861standard, but so much of existing marketing materials out there have PSQM that The Tolly Group used both to assist readers with anapples-to-apples comparison.
T H E
TO L LYG R O U P
Tolly Group engineers measured thecall quality of AVVID traffic runningover an HP ProCurve network by usingthe ITU P.861 PSQM and ITU P.862PESQ metrics.
PSQM represents an attempt to create an automated means forassessing objectively the quality of speech flowing across a telephonynetwork. It measures the distortion of a speech signal taking intoaccount the human perception and processing of speech. PSQMscores can range from 0 to 6.5, where the lower the number the better the score. A toll-quality call is usually considered anything lessthan a 1.29 with a business audio quality score falling in the range of 1.30 – 1.59.
By contrast, PESQ is an enhanced perceptual quality measurementfor voice quality in telecommunications. PESQ was specifically developed to be applicable to end-to-end voice quality testingunder real network conditions. With PESQ scores ranging from 1 to 4.5, with the high range representing a perfect score; a score of 3.8or higher represents toll quality, with business audio quality falling inthe range of 3.30 – 3.79.
Tolly Group engineers also measured the one-way delay (latency)associated with a voice call.
To add another wrinkle to the reported results, The Tolly Group measured call quality across a network of three different ProCurveaccess switches: an HP ProCurve 2650-PWR-PWR (Power overEthernet), ProCurve 4104 and ProCurve 5304. End users have differentneeds and requirements on the edge of their networks. Engineerstested these three different ProCurve access switches to demonstratethat HP can serve a variety of unique access point requirements andstill deliver excellent toll quality. Each of those devices fed trafficupstream to a ProCurve 5308 aggregation switch, which in turnpassed it to a ProCurve 9308 core switch and on to the IP phone to the core device. (See Figure 1.) An Agilent Voice Quality Tester
(VQT) was used to measure call quality.
T H E
TO L LYG R O U P
Figure 1. Call quality test bed diagram
VoIP calls earning a PSQM score ofless than 1.29 qualify as 'toll quality.'
VoIP calls earning a PESQ score of 3.8or higher qualify as 'toll quality.'
Conversely, for the AVVID over Cisco infrastructure tests, a CiscoCatalyst 3500 was used as the access switch, a Catalyst 4506 fit in asthe aggregation switch and a Catalyst 6509 represented the coreswitch in a comparable test bed scenario.
Engineers recorded call quality scores for a baseline scenario inwhich engineers established a 64-Kbps voice call using the G.711voice encoder – which is the equivalent to a standard dial-up phonecall placed across the PSTN. (G.711 voice encoders create the equivalent of an uncompressed 64 Kbps voice call.) Measurementswere taken across the network with no other traffic except for theAVVID traffic flowing between the origination and destination phones.VoIP calls originated from an IP phone connected to the accessswitch and terminated at an IP phone connected to the core switch.
Following the baseline, engineers introduced congestion onto thenetwork and then measured call quality again, with QoS activated.The goal was to determine what effect QoS would have on providinga level of call quality consistency with the baseline scores.
In the congestion scenario, engineers connected ports from an Ixia1600T traffic generator to both the access switch and the aggregationswitch and fully subscribed the pipe. Thus, when engineers placedthe single G.711 VoIP call on a fully utilized network, it created a condition of oversubscription on the network and QoS facilities kickedin to effectively prioritize the latency-sensitive voice traffic.
In every scenario, across both the PSQM andPESQ tests, the ProCurve switching infrastructurecarried AVVID traffic with call quality that eitherwas on par or better than provided by theCisco Catalyst network.
AVVID over Cisco
The market perception is that in order toachieve satisfactory call quality with Cisco'sAVVID, the solution has to run over a CiscoCatalyst switching infrastructure. So, that'swhere The Tolly Group began its testing, as well, to establish a performance baseline forcomparison of VoIP traffic generated by AVVIDand non-AVVID devices as it flows over the HPProCurve switch infrastructure.
AVVID over Catalyst: PSQM results
First, we measured the voice quality of a singleG.711 voice call traversing the Cisco switchinginfrastructure. Testing conducted on AVVID
T H E
TO L LYG R O U P
Perceptual Speech Quality Measurement (PSQM) MeasurementSingle G.711 Call Across a Cisco Catalyst Infrastructure
(lower scores are better)as Reported by Agilent VQT
traffic traversing a Catalyst switching infrastructure yielded a baseline PSQM score of 0.99 – well within the 1.29 ceiling for toll-quality voice. (See Figure 2.) Next, we established a single G.711
call over the network in the presence of congestion with QoS active. Here, the Catalystinfrastructure achieved a PSQM score of 1.0 –this is virtually identical to the baseline result.
AVVID over Catalyst: PESQ results
Here, the Cisco AVVID traffic posted a baselineperformance of 4.23 using PESQ and 4.35 whensubjected to a congestion scenario with QoSactive. These scores are well within toll qualityand represent near-perfect audio. (See Figure 3.)
AVVID over Catalyst: Latency results
In the baseline tests, the AVVID traffic, travelingover the Catalyst infrastructure offered an average one-way delay of 70 milliseconds forthe baseline test and 74.4 ms. for the conges-tion test. These delay results are consistent withlow latency normally associated with real-timevoice traffic.
AVVID over ProCurve
Next, we set out to evaluate how Cisco AVVID traffic performs whenit is transported over a non-Catalyst IP network. The Tolly Group testedthis scenario by injecting Cisco AVVID voice traffic across an underlyingHP ProCurve infrastructure.
Engineers measure call quality of the AVVID traffic for a single G.711voice call as it traversed a ProCurve network with an access switch,mid-level aggregation switch and core switching element. For the pur-poses of testing, engineers established call quality measurements withthree different ProCurve access switches (ProCurve 2650-PWR, ProCurve4104 and ProCurve 5304) since each address different user needs.
AVVID over ProCurve: PSQM results
Cisco's AVVID voice traffic running over the ProCurve switchinginfrastructure yielded an average PSQM score of 1.01 in baselinetesting – well within the parameters of meeting toll-quality audio.Interestingly, the 1.01 PSQM score was consistent across the threeaccess switches tested and it compares almost identically to the0.99 that AVVID traffic scored when traversing the Catalyst network.
Next, engineers passed AVVID traffic over the network during a
T H E
TO L LYG R O U P
Perceptual Evaluation of Speech Quality (PESQ) Measurement Single G.711 Call Across a Cisco Catalyst Infrastructure
(higher scores are better) as Reported by Agilent VQT
period of peak bandwidth congestion andQoS active. The PSQM quality when the AVVIDtraffic passed over a ProCurve 2650-PWRswitch was 1.01, versus 1.0 when it traveled across the ProCurve 4104and ProCurve 5304 devices. Incidentally, thesescores were virtually identical in call quality tomeasurements taken when AVVID traffic wasrunning over the Catalyst infrastructure.
AVVID over ProCurve: PESQ results
In PESQ tests, AVVID voice traffic running overthe ProCurve switching infrastructure yieldedan average PESQ score of 4.28 when runningacross the ProCurve 2650-PWR – in PESQ, ascore of 3.8 or higher represents toll quality.Testing conducted on AVVID voice traffictransported across a pure Cisco switching fabric yielded baseline PESQ score of 4.23 –slightly below the baseline delivered by theAVVID/ProCurve scenario. (See Figure 5.)
In the congestion with QoS tests, the ProCurveswitching network supporting AVVID trafficyielded a high PESQ score of 4.29 on theProCurve 4104. That compares to a PESQscore of 4.35 for AVVID running across aCatalyst infrastructure. While the pure Ciscoscenario delivers a slightly better score, theaudio difference would be indiscernible to the human ear – both scenarios would delivertoll quality.
The PESQ tests confirm what The Tolly Groupuncovered in the PSQM tests – HP's ProCurveswitching infrastructure delivers consistentlyhigh toll-quality for AVVID voice calls placedover the network and the toll quality offeredby ProCurve is on par, or sometimes better,than what is delivered in the Catalyst network.
AVVID over ProCurve: Latencyresults
When ProCurve was paired with the CiscoCatalyst/AVVID solution, an average one-way
delay of 75.6 ms was recorded regardless of infrastructure configu-ration. When QoS was enabled, one-way delay was recorded as
T H E
TO L LYG R O U P
Perceptual Speech Quality Measurement (PSQM) MeasurementSingle AVVID G.711 Call Across an HP ProCurve Infrastructure
(lower scores are better)as Reported by Agilent VQT
1.011.011.01 1.001.011.00
0.00
0.25
0.50
0.75
1.00
1.25
HP ProCurve5304xl
HP ProCurve 2650-PWR
HP ProCurve 4104gl
Switch Utilized
PSQ
M S
core
Baseline Congestion with QoS
Figure 4. AVVID over ProCurve PSQM scores
Perceptual Evaluation of Speech Quality (PESQ) Measurement Single AVVID G.711 Call Across an HP ProCurve Infrastructure
(higher scores are better) as Reported by Agilent VQT
average of 72.1 ms across the different ProCurve access switchestested. Again, with the ProCurve switching fabric handling CiscoAVVID traffic, the one-way delays were consistent across the infra-structure offerings with the delay typical in the Cisco switchinginfrastructure.
NBX over 3Com
3Com is a well-recognized brand in the VoIP market so The TollyGroup sought to establish voice call quality measurements of VoIPtraffic generated by a 3Com NBX-100 IP PBX as it traverses a pure3Com network.
Engineers measured call quality of the NBXVoIP traffic for a single G.711 voice call as ittraversed a 3Com network with a 3ComSwitch 4400 access switch, a 3Com Switch4950 mid-level aggregation switch and 3ComSwitch 7700 core switching element.
NBX VoIP over 3Com: PSQM results
First, we measured the voice quality of a singleG.711 voice call traversing the 3Com switchinginfrastructure. Testing conducted on NBX-generated VoIP traffic traversing a 3Comswitching infrastructure yielded a baselinePSQM score of 0.97 – well within the 1.29 ceiling for toll-quality voice. (See Figure 6.)Next, we established a single G.711 call overthe network in the presence of congestionwith QoS active. Here, the 3Com infrastructureachieved a PSQM score of 0.99 – this is virtuallyidentical to the baseline result.
NBX VoIP over 3Com: PESQ results
In PESQ tests, NBX-generated VoIP traffic running over the 3Comswitching infrastructure yielded an average PESQ score of 2.16when running across the 3Com infrastructure – in PESQ, a score of3.8 or higher represents toll quality. (See Figure 7.)
In the congestion with QoS tests, the 3Com switching network supporting VoIP traffic yielded a PESQ score of 2.15 – well belowacceptable toll quality levels for PESQ.
These tests show that using PESQ, which factors in significantlymore network factors than does PSQM, toll quality of VoIP trafficrunning over the pure 3Com infrastructure lags that of otherdevices tested.
T H E
TO L LYG R O U P
Perceptual Speech Quality Measurement (PSQM)Single G.711 Call Across a 3Com Switching Infrastructure
(lower scores are better)as Reported by Agilent VQT
Latency introduced by the 3Com switchingfabric was low at 69 milliseconds, which wason par with latency for other test scenarios.
NBX VoIP over ProCurve: PSQMresults
Tolly Group engineers also measured the callquality of VoIP traffic generated by an NBX-100 as it traversed the HP ProCurve network.NBX-100 traffic running over the ProCurveswitching infrastructure yielded an averagePSQM score of 1.03 in baseline testing – wellwithin the parameters of meeting toll-qualityaudio. (See Figure 8.) Interestingly, the PSQMscore was fairly consistent across the threeProCurve access switches tested (1.01 to 1.05)and it compares almost identically to the 0.99that VoIP traffic scored when traversing the3Com network.
Next, engineers passed NBX-generated VoIPtraffic over the network during a period ofpeak bandwidth congestion and QoS active.The PSQM quality when the NBX traffic passedover a ProCurve 2650-PWR switch was 1.02, versus 1.01 when it traveled across theProCurve 4104 and 1.02 when it ran across theProCurve 5304 devices. Incidentally, thesescores compare virtually identical to measure-ments taken when NBX traffic was running overthe pure 3Com switching infrastructure (0.99).
NBX VoIP over ProCurve: PESQresults
In PESQ tests, NBX-generated VoIP traffic running over the ProCurve switching infrastruc-ture yielded an average PESQ score of 2.36when running across the ProCurve 2650-PWR –in PESQ, a score of 3.8 or higher represents toll quality. However, this PESQ score is actuallybetter when NBX traffic runs over HP ProCurvethan when it runs over a pure 3Com switchingfabric.
T H E
TO L LYG R O U P
Perceptual Evaluation of Speech Quality (PESQ)Single G.711 Call Across a 3Com Switching Infrastructure
(higher scores are better)as Reported by Agilent VQT
2.16 2.15
0.0
0.5
1.0
1.5
2.0
2.5
3Com NBX-100VoIP system utilized
PESQ
sco
re
Baseline Congestion with QoS
Figure 7. 3Com NBX-100 PESQ scores
Perceptual Speech Quality Measurement (PSQM) Measurement for Hewlett Packard Infrastructure with 5304 Access Switch
Single G.711 Call(lower scores are better)
as Reported by Agilent VQT
1.021.051.01 1.011.021.02
0.00
0.25
0.50
0.75
1.00
1.25
HP ProCurve 5304xl HP 2650-PWR HP ProCurve 4104gl
Access Switch Utilized
PSQ
M S
core
Baseline Congestion with QoS
Figure 8. 3Com NBX-100 PSQM scores over HP ProCurve network
In the congestion with QoS tests, theProCurve switching network support-ing NBX-generated VoIP traffic yield-ed a high PESQ score of 2.39 on theProCurve 2650-PWR. (See Figure9.)That compares to a PESQ score of2.15 for VoIP running across a pure3Com infrastructure. Thus, testingshows that the HP ProCurve switchinginfrastructure delivers slightly bettertoll quality than attained in the pure3Com switching network – thoughboth scores still lag acceptable tollquality levels.
NBX VoIP over ProCurve:Latency results
Latency introduced by the 3Comswitching fabric ranged from 79 to 83ms., which represents very low latencyand is comparable to the latencyobserved in the pure 3Com switchingscenario.
ProCurve as a Viable VoIP TransportTests discussed in earlier sections demonstrate how VoIP traffic generatedby Cisco AVVID and 3Com NBX-100 solutions provide toll-quality VoIP –regardless if the data runs over a pure Cisco network, a 3Com switchinginfrastructure or across an HP ProCurve switching fabric.
Many users, today, adopt best-of-breed solution strategies, where theymix and match components from several manufacturers that representthe best performers in their class. HP has partnered with Mitel Corp. –long considered one of the premiere VoIP IP PBX makers – to integratethe Mitel Corp. 3300-Integrated Communications Platform (ICP), an IPPBX designed for VoIP applications that HP also markets, into an HPProCurve infrastructure.
This chapter explores the performance of that hybrid solution and compares its performance to the 3Com and Cisco call quality results previously discussed. In each case, all VoIP gateway solutions were test-ed in an apples-to-apples scenario as either NBX-100-generated orAVVID-generated VoIP traffic traversed the HP ProCurve switching fabric.Engineers established a single G.711 VoIP call from the switch to any ofthree ProCurve access switches, which then passed the call into the
T H E
TO L LYG R O U P
Perceptual Evaluation of Speech Quality (PESQ) Measurement for Hewlett Packard Infrastructure with 5304 Access Switch
Single G.711 Call(higher scores are better)
as Reported by Agilent VQT
2.31 2.36 2.302.33 2.39 2.30
0.00
0.50
1.00
1.50
2.00
2.50
3.00
HP ProCurve 5304xl HP 2650-PWR HP ProCurve 4104gl
Access Switch Utilized
PESQ
Sco
re
Baseline Congestion with QoS
Figure 9. 3Com NBX-100 PESQ scores over HP ProCurve network
ProCurve network to an aggregation switch and on to a core switchwhere the call was terminated.
The objective of this chapter is to demonstrate conclusively that the HPProCurve switching fabric is a hospitable host for VoIP traffic, regardless ofthe brand of VoIP gateway used. The ensuing sections report on the callquality of the Mitel, 3Com NBX-100 and Cisco AVVID VoIP traffic as it passes over the ProCurve infrastructure. Also, each section is based uponone of three HP ProCurve access switches employed; HP elected to testthe variety of access switch choices to demonstrate the breadth of itsofferings that appeal to diverse user requirements.
ProCurve 2650-PWR scenario
PSQM results
Engineers tested VoIP traffic traversing a ProCurve infrastructure inwhich a G.711 call originated from a phone attached to a Mitel
3300-ICP. Baseline tests revealed that the Mitel3300-ICP voice solution combined with theProCurve switched network delivered a PSQMscore of 0.56, which equates to excellent tollquality, significantly surpassing the call qualityoffered by the AVVID over Cisco Catalystinfrastructure scenario.
Tolly Group engineers also measured the call quality of a single G.711 voice call thatemanated from a 3Com NBX-100 and trav-eled across the ProCurve switching infrastruc-ture. Even the NBX-100's PSQM score of 1.05was competitive in toll quality with the callquality of AVVID traffic running over theCatalyst network (1.01). (See Figure 10.)
In the congestion with QoS scenario, againthe Mitel 3300-ICP delivered superior call quality to the Cisco AVVID solutions with aPSQM score of 0.58. Again, the 3Com NBX-100achieved a PSQM score of 1.02, which is onpar with the PSQM delivered by the CiscoAVVID solution.
PESQ results
In PESQ tests, the Mitel 3300-ICP passed VoIP traffic across aProCurve 2650-PWR to yield a PESQ score of 4.26, which is on parwith the AVVID/ProCurve scenario. (See Figure 11.) Such scoresreflect the fact that HP's ProCurve delivers consistent, high-quality
T H E
TO L LYG R O U P
Perceptual Speech Quality Measurement (PSQM) Measurement Single G.711 Call Across an HP Infrastructure
with a ProCurve 2650-PWR (lower scores are better)
as Reported by Agilent VQT
1.051.01
0.56
1.021.01
0.58
0.00
0.25
0.50
0.75
1.00
1.25
Mitel 3300-ICP Cisco AVVID 3Com NBX-100
VoIP Solution Utilized
PSQ
M S
core
Baseline Congestion with QoS
Figure 10. HP ProCurve Switch 2650-PWR PSQM scores
voice traffic, regardless if it is handling VoIPtraffic generated by AVVID-based or Mitel3300-ICP-based systems.
Even in the congestion with QoS tests, theMitel 3300 ICP delivered PESQ scores that wereconsistent with the Cisco solution traveling overthe ProCurve network and even on par withthe pure Cisco configuration.
ProCurve 2650-PWR latency results
The single G.711 voice call traveling over theProCurve 2650-PWR switch resulted in latencyranging from 79 ms. for the NBX-100 to 80.4ms. for the Mitel 3300-ICP – both of which are on par with latency offered by the Ciscosolutions.
ProCurve 4104 scenario
PSQM results
Engineers tested VoIP traffic traversing aProCurve 4104 infrastructure utilizing a Mitel3300-ICP. Baseline tests revealed that the Mitel3300-ICP voice solution combined with theProCurve switched network delivered a PSQMscore of 0.56, the same excellent toll qualityattained across the ProCurve 2650-PWR. Thistoll quality compares to the 0.99 score of theCisco AVVID traffic running over a CiscoCatalyst network and underscores the factthat the HP ProCurve network is a fertile infrastructure for the transport of real-time,latency-sensitive VoIP traffic.
Tolly Group engineers also measured the call quality of a single G.711 voice call thatemanated from a 3Com NBX-100 and traveled across the ProCurve switching infra-structure. The NBX-100's PSQM score of 1.02was competitive in toll quality with the 1.01call quality score of AVVID traffic runningover the ProCurve network. (See Figure 12.)
In the congestion with QoS scenario, again theMitel 3300-ICP delivered superior call quality tothe Cisco AVVID solutions with a PSQM scoreof 0.57. Again, the 3Com NBX-100 achieved a
T H E
TO L LYG R O U P
Perceptual Evaluation of Speech Quality (PESQ) MeasurementSingle G.711 Call Across an HP Infrastructure
with a ProCurve 2650-PWR (higher scores are better)
as Reported by Agilent VQT
4.26 4.28
2.36
4.25 4.24
2.39
0
1
2
3
4
5
Mitel 3300-ICP Cisco AVVID 3Com NBX-100
VoIP Solution Utilized
PESQ
Sco
re
Baseline Congestion w ith QoS
Figure 11. HP ProCurve Switch 2650-PWR PESQ scores
Perceptual Speech Quality Measurement (PSQM) Measurement Single G.711 Call Across an HP Infrastructure with a ProCurve 4104gl
(lower scores are better)as Reported by Agilent VQT
PSQM score of 1.01, which is on par with thePSQM delivered by the Cisco AVVID solution.
PESQ results
Engineers tested the PESQ score of a single voicecall originating from a phone attached to aProCurve 4104 switch, passing across theProCurve network to a core switch where it wasterminated. Here, again, the Mitel 3300-ICP deliv-ered excellent toll quality with a PESQ score of4.29 in baseline tests, on par with the 4.28 for theAVVID traffic running across the HP infrastructure.
In the congestion with QoS tests, the scenariowas the same with the Mitel 3300-ICP scoring4.26 PESQ and the Cisco AVVID solutionachieving a 4.31. (See Figure 13.)
ProCurve 4104 latency results
The single G.711 voice call traveling over the ProCurve 4104 switch resulted in latencyranging from a low of 59 ms. for the Mitel 3300-ICP to 83 ms. for the NBX-100. In the congestionscenario with QoS, the latency on the Mitelperked up to 83 ms. All the latency results were well within line of acceptable real-time VoIP latency.
ProCurve 5304 scenario
PSQM results
The Mitel 3300-ICP made it a perfect trio ofPSQM results by posting a baseline score of0.55 in the ProCurve 5304 test – meaning ineach of the three ProCurve access switch sce-narios, the Mitel 3300-ICP delivered PSQM callquality of 0.56 or better versus 0.99 for the pureCisco scenario. While both the Mitel and Cisco PSQM results indicate excellent tollquality, it is important to note that the Mitelsolution beats the Cisco alternative time after time in both call quality and latency performance.
With the ProCurve 5304 used as an accessswitch, the NBX-100 achieved a PSQM score
T H E
TO L LYG R O U P
Perceptual Evaluation of Speech Quality (PESQ) Measurement Single G.711 Call Across an HP Infrastructure with a ProCurve 4104gl
(higher scores are better) as Reported by Agilent VQT
of 1.01 for the baseline test, well on par with the Cisco AVVID overProCurve results. (See Figure 14.)
For the congestion with QoS tests, again the Mitel 3300-ICP lead alldevices with a PSQM of 0.62, versus 1.02 for the NBX-100 and 1.00for the Cisco solution.
PESQ results
In PESQ tests, the Mitel 3300-ICP distanceditself from other devices tested by posting aPESQ score of 4.27 versus 4.03 for the Ciscoand 2.31 for the NBX-100 when tested with the ProCurve 5304 as the access switch. (See Figure 15.)
When the test was conducted with congestionand active QoS, the Mitel 3330-ICP earned ascore of 4.26, versus 4.0 for the Cisco and 2.33for the NBX-100.
ProCurve 5304 latency results
The single G.711 voice call traveling over theProCurve 5304 switch resulted in latency ranging from a low of 67 ms. for the Mitel3300-ICP and the Cisco AVVID solutions to 79ms. for the NBX-100 in the baseline test. In thecongestion scenario with QoS, the latency onthe Mitel 3300 was 65 ms. And the NBX-100
rose to 81.5 ms. All the latency results were well within line ofacceptable real-time VoIP latency.
Call-Quality AnalysisTests provide ample evidence that no matter what VoIP gateway generatesvoice traffic, HP 's ProCurve switching infrastructure can transport it at callquality levels that are either on par with, or better than, those supportedby a pure Cisco or pure 3Com switching network.
Tests results show that AVVID VoIP traffic runs just as well across an HPProCurve enterprise LAN switching infrastructure as it does across ahomogenous Cisco switch fabric. These results discount any contentionthat a homogenous Cisco LAN switching platform is required to obtainconsistently high call quality with AVVID.
Moreover, tests demonstrate that both the HP ProCurve and the CiscoCatalyst switches administer QoS effectively, and that Cisco does notwield any advantage from a QoS perspective vis a vis VoIP. Call quality,
T H E
TO L LYG R O U P
Perceptual Evaluation of Speech Quality (PESQ) Measurement Single G.711 Call Across an HP Infrastructure
in fact, remained constant for both the ProCurve and Cisco Catalystswitching infrastructures when QoS was active, as compared to baselinecall quality scores. HP's ProCurve switch fabric, for instance, delivered aslightly higher call quality for baseline tests when compared to the Ciscoinfrastructure. And while Cisco held a slight edge in call quality in thecongestion scenario, the difference would be largely imperceptible tothe human ear since both the Cisco and the ProCurve switch fabricsdelivered excellent toll quality.
Another variable to consider is the impact that delay has on call quality.The ProCurve switching/VoIP solutions tested demonstrate that HP candeliver one-way delay on par or better than Cisco. Moreover, even inheavily loaded oversubscription scenarios with QoS active, delay doesnot fluctuate wildly, but actually decreases in two of the three switching/VoIP solution sets, while Cisco's delay increased slightly by about 6% fromthe baseline to the congestion tests.
This shows that HP's ProCurve line and its QoS facilities can effectivelymanage voice traffic over congested links without permitting delay toimpact the quality of voice calls. In fact, the Mitel 3300-ICP resultsdemonstrate that HP ProCurve can support a level of call quality that even exceeds that offered by Cisco AVVID on a pure Catalyst infrastructure. Users would be wise to contemplate ProCurve and any of its third-party VoIP gateways as viable alternatives to a pure Cisco VoIP play.
Also, such results demonstrate the implementation of QoS is not aHerculean effort, but rather one of several steps organizations mustimplement to guarantee voice traffic receives the preferential treatmentit deserves.
In fact, QoS goes hand in hand with careful bandwidth planning in thedeployment of VoIP services. QoS alone cannot guarantee effectivetreatment of VoIP traffic. Users need to measure the bandwidth require-ments for the network and apportion adequate reserves of bandwidth tohandle surges in traffic. This becomes the bandwidth foundation of theconverged network infrastructure. QoS comes into play as an insurancepolicy in the event that bandwidth reserves become depleted due tooversubscription of links.
Of course, that means that switch pricing comes into play as a gatingfactor in terms of how much bandwidth users can afford to deploy. WithHP, the aggressive price/performance of its ProCurve switches enablesusers to implement plentiful amounts of bandwidth at significant savingsor lower than a Cisco switch fabric. (See The Tolly Group/HP white paper,"Gaining an Intelligent Edge for VoIP with HP ProCurve Networking.") Todownload the complete document from the Web, go to:http://www.tolly.com/DocDetail.aspx?DocNumber=203505.
Cisco, on the other hand, demands a premium for its switching products– meaning that users may not be able to afford to deploy the samebreadth of switches, and ultimately bandwidth, that ProCurve users can.From that perspective, it's no surprise that Cisco would promote a QoS-centric view of VoIP management. HP's approach, however, is more balanced. It stresses judicious bandwidth planning, with reserve capacitiesfigured in, and high-quality QoS as an insurance policy that kicks in whenbandwidth becomes saturated.
No matter what your needs, large or small, everyone wants to get themost for their money. Many users have been programmed to believethat buying from a single source has advantages, but many times itcomes at a very high price.
Best-of-breed mentality is the way to go to help ensure that you get notjust what you want, but what you need. Every company's needs areslightly different, and by selecting the switching and VoIP infrastructureindividually based on verifiable performance, features and price will help prevent you from being 'married' to a vendor that might be great at some things and marginal at others. The goal is to work with a vendor that delivers to you what is needed at a cost that you and yourcompany can afford.
Information technology is an area of rapid growth and constant change. The Tolly Group conductsengineering-caliber testing in an effort to provide the internetworking industry with valuable informationon current products and technology. While great care is taken to assure utmost accuracy, mistakescan occur. In no event shall The Tolly Group be liable for damages of any kind including direct, indirect, special, incidental, and consequential damages which may result from the use of informationcontained in this document. All trademarks are the property of their respective owners.
