Network World WAN Acceleration Product Review Highlights
Executive Summary In Network World’s “WAN Acceleration Clear Choice” product review published on August 13, 2007, Cisco WAAS is validated as the only enterprise-class solution designed for large scale deployments:
No.1 Enterprise-class Scalability - Cisco WAAS outperformed all vendors by a large margin, sustaining 50,113 TCP connections:
Vendor Marketing Claim vs. RealityMaximum Accelerated TCP Connections
19,499
50,113
12,202
43,306
0
10,000
20,000
30,000
40,000
50,000
Blue Coat Cisco Riverbed Silver Peak
19,499
50,000 50,113
15,000 12,202
128,000
43,306
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
Bluecoat Cisco Riverbed Silver Peak
ClaimReality
Maximum Accelerated TCP Connections Vendor Marketing Claim vs. RealityMaximum Accelerated TCP Connections
19,499
50,113
12,202
43,306
0
10,000
20,000
30,000
40,000
50,000
Blue Coat Cisco Riverbed Silver Peak
19,499
50,000 50,113
15,000 12,202
128,000
43,306
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
Bluecoat Cisco Riverbed Silver Peak
ClaimReality
Maximum Accelerated TCP Connections
Not published
No.1 Bandwidth Optimization Performance – Using percentage, which is easier to understand than factor and used as the industry norm, Cisco WAAS outperforms the group for a large margin for fresh file transfer and comes 2nd with 0.7% striking distance to the highest vendor. A large-margin win plus a undifferentiated loss makes Cisco the top performer.
WAN Bandwidth Reduction Percentage
024.4%
96.4%44.8%
27.2%97.9%98.1%98.8%
96.2%94.8%94.9%96.9%
94.6%
0 0.2 0.4 0.6 0.8 1 1.2
No accelerationBlue Coat cold
Cisco coldRiverbed cold
Silver Peak coldBlue Coat warm
Cisco warmRiverbed warm
Silver Peak warmBlue Coat 10% changed
Cisco 10% changedRiverbed 10% changed
Silver Peak 10% changed
Vendor Marketing Claim vs. Reality
Maximum Accelerated TCP Connections
19,499
50,113
12,202
43,306
0
10,000
20,000
30,000
40,000
50,000
Blue Coat Cisco Riverbed Silver Peak
19,499
50,000 50,113
15,00012,202
128,000
43,306
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
Bluecoat Cisco Riverbed Silver Peak
ClaimReality
Maximum Accelerated TCP
ConnectionsVendor Marketing Claim vs. Reality
Maximum Accelerated TCP Connections
19,499
50,113
12,202
43,306
0
10,000
20,000
30,000
40,000
50,000
Blue Coat Cisco Riverbed Silver Peak
19,499
50,000 50,113
15,00012,202
128,000
43,306
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
Bluecoat Cisco Riverbed Silver Peak
ClaimReality
Maximum Accelerated TCP
Connections
Cisco outperforms by a large margin
33
Illusion: Cisco 52.97
Riverbed 83.95Reality:
Cisco 98.1% Riverbed 98.8%
11
22
Network World Product Review Highlights
No.1 QoS Performance – Using enterprises’ existing router and therefore protecting their investment, Cisco scored 92.03 out of the maximum 93 QoS R-value measuring VoIP audio quality. Note that this test was using a single VoIP call and hence not a realistic real world scenario. In a more realistic scenario many VoIP calls are present, Cisco router QoS will perform substantially better than other vendors’ limited QoS functionality.
No.1 or No.2 Acceleration – Under low bandwidth and high delay hence the most demanding test conditions where acceleration is needed the most, Cisco performs 9.59% faster than Riverbed for CIFS cold transfer, 3.39% faster for MAPI, and 12.99% slower for CIFS (10% changed content).
Response Time % Better or Worse Than Riverbed
-12.99%
3.39%
9.59%
-8.32%-15.00%
-10.00%
-5.00%
0.00%
5.00%
10.00%
15.00%
CIFS (Pull 10%changed)MAPICIFS (pull cold)
CIFS (pullwarm)
Low bandwidth, high delay
In addition, all vendors hit diminished return on acceleration, especially for MAPI when response time improvements are measured in download time (second).
MAPI Download Time (Seconds)
388
214
289
269
198
31
17
16
14
18
122
83
98
90
83
33
14
15
16
18
0 100 200 300 400 500
No Acceleration
Blue Coat
Cisco
Riverbed
Silver Peak
High bandwidth, high delayLow bandwidth, low delayHigh bandwidth, low delayLow bandwidth, high delay
Put “Acceleration” Into Right Perspective –While it’s tempting to raise an eyebrow on Cisco’s 12.39 CIFS response time improvement factor vs. the highest vendor’s 45.46, a seemingly big difference, the reality is that the same measurement really means that the response time is improved by Cisco by 91.9% vs. the highest vendor’s 97.8%. Cisco WAAS clearly turns in top notch performance under this single user, “drag race” test. Performance under large load, which is scalability, becomes far more important and Cisco scores No.1 in that area by a large margin.
Illusion: Cisco 12.39
Riverbed 45.46
CIFS Download Response Time Reduction Percentage
81.5%
44.5%
86.8%
76.6%
64.1%
91.9%
97.8%
91.1%
54.7%
29.8%
77.4%
72.6%
76.9%
84.4%
97.0%
87.6%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
Blue Coat
Cisco
Riverbed
Silver Peak
High bandwidth, high delayLow bandwidth, low delayHigh bandwidth, low delayLow bandwidth, high delay
Reality: Cisco 91.9%
Riverbed 97.8%
Network World Response Time Improvement Factor
Response Time Improvement Percentage
Outstanding Manageability – Cisco is one of the 3 vendors that fully support centralized management, with outstanding management interface: “Cisco's display bristles with features and commands… Cisco's redesigned dashboard offers whizzy graphics, useful pie charts on CIFS application performance and real-time connection monitoring and per-device reporting on connection statistics”.
Conclusion
By achieving No.1 scalability in the Network World test, and the “Best Value” award from Network Computing, and “Product Leadership Silver Award” from Search Networking, Cisco WAAS is clearly validated as the only enterprise-class solution that meets all of the following requirements:
• No.1 scalability, No.1 or No.2 acceleration performance, and outstanding manageability (Network World) • “Product Leadership Silver Award” for best network integration and interoperability (Search Networking) • “Best Value Award” (Network Computing)
