22
2
The Best of the Web 2010 is Compuware’s second annual award for the best performing Web and mobile sites. The awards provide a benchmark for performance across six industries and highlight the leaders who provide superior Web and mobile site experiences that enhance customer loyalty, reduce costs and increase revenue.
The Gomez Benchmarks have provided insight into Web performance for more than a decade. The Benchmarks measure key customer interaction points — the home page (“the front door” to the site) and key business process transactions like placing an order or making a trade — on both Web and mobile sites. Today, Gomez monitors over 3000 Web and mobile sites with over 20 million tests conducted on an extensive network of 150 backbone locations and over 150,000 “Last Mile” desktop computers.
Why Web Performance Matters . . . . . . . . . . . . . . . . . . . . .3
How Much Does Mobile Site Performance Matter? . . . . . . .4
Best of the Web 2010 | Compuware Gomez Web and Mobile Performance Top Award Winners . . . . . . . . . . . . . . . . . . . .5
Best of the Web 2010 — Financial Services . . . . . . . . . . . .6
Banking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Brokerage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Best of Web 2010 — Health Information . . . . . . . . . . . . . .9
Best of the Web 2010 — Media . . . . . . . . . . . . . . . . . . . 10
Best of the Web 2010 — Retail. . . . . . . . . . . . . . . . . . . . 11
Best of the Web 2010 — Travel . . . . . . . . . . . . . . . . . . . 13
Airlines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Hotels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Best of the Web 2010 — US Government . . . . . . . . . . . . 15
Web Performance Best Practices . . . . . . . . . . . . . . . . . . . 16
Benchmark the Best, Your Competition and Your Own Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Optimize for Your End Users’ Experience . . . . . . . . . . . . . . . 16
Think Mobile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20
Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
About Gomez . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22
TABLE OF CONTENTS
The 2010 winners in each industry — retail, financial services, media, travel, healthcare and government — were determined by measuring their Web and mobile sites across three key metrics — response time, availability and consistency. Three website winners in each industry were awarded with Gold, Silver and Bronze and Most Improved awards. In select industries, there are also Mobile Leader and One Web winners, which take into account home page and business process transaction performance from the Internet backbone and from the Gomez Last Mile and mobile home page performance.
3
WHY WEB PERFORMANCE MATTERSWeb performance is a business issue. Web and mobile sites that don’t perform well — are slow to load, have periods of unavailability or have inconsistent performance — negatively impact customers’ experience, their satisfaction, and the likelihood that they will continue to spend time on the site or make a purchase. The business impact has been documented in numerous studies. Compuware’s own research found a direct correlation between an improvement in page response time and an increase in conversions and page abandonment in two separate studies.
Figure 1: Compuware found that conversion rate increases 74% when page load time improves from eight to two seconds. The study was based upon conversion data for a key non-Commerce transaction on the sites of 33 major retailers and over 3 million page views.
Figure 2: In another Compuware study across more than 200 sites and more than 177 million page views, the page abandonment rate increases steeply as page load time increases.
4
HOW MUCH DOES MOBILE SITE PERFORMANCE MATTER?A lot. According to Morgan Stanley, the volume of mobile users will outstrip the number of desktop Internet users by 2014. By 2011 Nielsen expects the U.S. market will own more smart phones than ordinary “feature” phones.
As more consumers do more online with their fully-featured smart phone — shop, read the news and search for information — there is revenue at risk if the site doesn’t perform well. As an indication of the explosive growth in revenue generated from mobile sites, eBay reported $600 million in mobile commerce revenues in 2009 and just one year later the company expects $1.5 billion in mobile sales. And, within just a hundred days of launching their mobile sites, both Marriott and Pizza Hut realized more than $1 million in revenue from those sites.1
Figure 3: According to Morgan Stanley, the volume of mobile users will outstrip the number of desktop Internet users by 2014.
Figure 4: Not meeting customers’ expectations for mobile site performance negatively impacts revenue and brand equity.
Consumers’ expectations for performance are pretty high for mobile sites. According to a 2009 study by Equation Research, the majority of mobile phone users expect websites to load as quickly, almost as quickly or faster on their mobile phone, compared to the computer they use at home. 2
Bottomline: Web and mobile site performance is a business issue. Maintaining a site that performs well is critical to customer satisfaction, maintaining loyalty and driving revenue.
5
The winners have worked hard to ensure that their Web and mobile site performance is a differentiator and drives business. With the lightening-speed of technology change and marketing innovation, they have made a commitment to continuous improvement as they evolve their sites to meet their customers’ needs and expectations and beat the competition.
CONGRATULATIONS TO THE 2010 BEST OF THE WEB WINNERS!
The winners below include the One Web, Web Gold and Mobile Leader award winners. To see who the Silver, Bronze and Most Improved winners are turn to the industry pages later in the report.
TOP AWARD WINNERS
6
In financial services, good Web performance is paramount. A majority of US adults now bank online (57 percent in 2009) and 36 percent have an online investment account.3 These consumers have high expectations for performance. Seventy-five percent expect 99 percent website availability or higher and 56 percent expect Web pages to load in 2 seconds or less3 — that expectation is significantly higher than for ecommerce sites where 47 percent of consumers expect pages to load in 2 seconds or less. 9
When a site performs poorly, the impact to the business is felt by:
• Customer migration to more costly services — Fifty-percent of online brokers would use the phone or branch when faced with a poorly performing site3;
• Reduced loyalty — Twenty-nine percent of customers researching a product would go to a competitors’ site and forty-eight percent would be less likely to recommend a firm. 3
FINANCIAL SERVICES
The 2010 Best of the Web award winners in Banking Web performance were analyzed across multiple benchmarks:
• a key business transaction, the process of checking account details;
• the performance of the home page on the Internet Explorer browser accessed from the Internet backbone;
• home page performance as experienced on the end users’ desktop from the Gomez Last Mile; and,
• mobile site performance.
Key performance metrics included response time, availability and consistency. The One Web Winner, Branch Banking & Trust, performed the best across all of these dimensions.
“Investors demand low commissions, fast trade executions and a secure trading experience, and our web performance is paramount to delivering an exceptional customer experience. For more than 30 years Scottrade has worked to provide individuals with the latest tools, resources and education they need to take control of their investments, and we are proud to once again be recognized by Gomez as a Best of the Web Award Winner for the industry-leading speed and performance of our website.”
— Ian Patterson, Scottrade’s Chief Information Officer
“We are very proud of what we’ve accomplished with regions.com from both a performance, content and functionality standpoint. We believe it’s due to a strong focus on serving customers in ways that are meaningful, the hard work of our associates to provide a flexible platform and to our ongoing commitment to improving our web performance.”
— Chris Cox, Executive Vice President of Regions E-business division
“BB&T strives to deliver the Perfect Client Experience, regardless of whether clients choose to bank with us in one of our financial centers or by using our online services. We are committed to providing our clients with one of the most respected and useful online services in the industry. In doing so, availability and performance are top priorities. We are proud to be recognized by Gomez as a Best of the Web Award winner; it validates that we are accomplishing our objectives.”
— Gary Coleman, EVP, Chief Technology Officer, BB&T Information Technology
7
FINANCIAL SERVICES
BANKING
WINNERS
One Web Winner Branch Banking &Trust
WEB PERFORMANCE INDEX
Our Web Performance Index aggregates benchmark data for the Account Detail Transaction; home page on an IE agent; and on the Gomez Last Mile across response time, availability and consistency. The table below provides only average response time for each of these benchmarks.
Account Detail
Transaction
Home Page Last Mile Home Page
Average Response Time (Sec)
Gold Regions Bank 5.062 1.258 4.197
Silver BB&T 6.989 1.379 5.158
Bronze (Tie)
HSBC 7.819 1.955 7.401
Bronze (Tie)
US Bank 9.184 2.271 4.470
BANKING MOBILE LEADER
The Banking Mobile Leader was determined by the site that had the best response time, availability and consistency across four carriers and their top devices. See the Methodology section for more detail on measurement.
BANKING MOBILE LEADER
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
Capital One 4.219 99.70% 2.272
MOST IMPROVED
TD Bank was Most Improved on the Banking Account Detail Transaction, with an average response time higher than the benchmark average and the most improved response time in the second half of the year versus the first half.
Most Improved TD Bank 44% response time improvement
ONE WEBWINNER
Banking — Web Gold Regions Bank
8
BROKERAGE
WINNERS
Brokerage — Gold Fidelity
FINANCIAL SERVICES
The brokerage winners were determined by the sites with the best performance on a key transaction — generating an order.
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
Gold Fidelity 3.815 99.91% 1.211
Silver TradeKing 5.655 99.81% 2.162
Bronze Scottrade 4.778 99.81% 3.096
MOST IMPROVED
Charles Schwab was the Most Improved brokerage with an average response time higher than the benchmark average and the most improved response time in the second half of the year versus the first half.
Most Improved Charles Schwab 11% response time improvement
“At Schwab, we know that the availability and performance of our web site, schwab.com, is paramount to ensuring a great client experience. This is why our commitment to constantly improving our web site is uncompromising; we strive to ensure the strong performance and rich experience that Schwab clients have come to expect from us. Receiving a Best of the Web 2010 award from Gomez is a strong indicator that we are achieving our goals.”
— Jared Price, Vice President Web Solutions at Charles Schwab, Inc
“Fidelity is honored to be recognized as a repeat Gold Winner in Gomez’s Best of the Web 2010 awards. In addition to broad investment choices, industry-leading low prices and award-winning customer service, we consider a fast, simple, consistent and secure experience each time a customer visits Fidelity.com to be critically important. We will continue to expand and enhance the leading brokerage tools and services we make available to our customers via the Web and Mobile.
— Richard Blunck, Executive Vice President, Web Distribution at Fidelity Investments
9
A majority of Internet users (83 percent) look for a variety of health information online ranging from information about a specific medical problem or disease to treatment, procedures and drugs. Users are also researching doctors, insurance providers, hospitals and other medical facilities.4 Healthcare reform is increasing traffic to health insurance sites. In fact, the number of people clicking on health insurance company links rose 56 percent year over year.5
Both insurers and health information sites need to ensure that Web applications are available and fast enough to support users’ objectives on the site and not drive them to competing sites or, for the insurers, more expensive channels such as call centers.
WINNERS
Health Information — Gold United Health
HEALTH INFORMATION
The Health Information winners were determined by the sites with the best performing home pages taking into account response time, availability and consistency.
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
Gold United Health 1.915 99.99% 1.142
Silver Puritans Pride 1.638 99.99% 2.133
Bronze Yahoo Health 0.962 99.96% 1.039
MOST IMPROVED
AAOS was the Most Improved health information site with an average response time higher than the benchmark average and the most improved response time in the second half of the year versus the first half.
Most Improved AAOS 43% response time improvement
10
Online consumer behavior continues to shift in consumption of media online. While portals still take the largest portion of time spent by consumers, consumers are increasing their time spent on social media and entertainment. Consumers are also watching more video online. In December 2010, 88.6 million watched video daily, a 32 percent increase from December 2009. 6
The online advertising market rebounded in 2010 with the number of ads growing 23 percent and users receiving a total of 4.9 trillion display ads. 6
As more viewers migrate to media sites for content and online advertising revenue continues to grow with it, it is more important than ever that third-party CDNs and ad networks don’t impact site performance and visitor traffic. Benchmarking the best will provide a barometer for good Web and mobile application performance.
WINNERS
Media — Gold CBS News
MEDIA
The Media Winners were determined by the sites that had the best performing Media Article Search transaction.
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
Gold CBS News 7.082 99.98% 5.143
Silver Yahoo News 4.287 99.21% 6.174
Bronze New York Times 7.756 99.64% 5.030
MOST IMPROVED
MSNBC was the Most Improved media property with an average response time higher than the benchmark average and the most improved response time in the second half of the year versus the first half.
Most Improved MSNBC 39% response time improvement
MOBILE NEWS LEADER — PREVIEW
For the first month of 2011, US News was the best performing mobile site.
MOBILE LEADERAverage
Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
US News 6.276 99.75% 3.617
CBSNews.com puts enormous focus on creating a rich user experience that offers depth and context to our award-winning broadcast shows through original content created specifically for an online environment. We’re very proud to be recognized with a Gold Award in the Media category by Gomez for our work in delivering a top-performing news destination.
— Mark Larkin, Vice President, CBSNews.com
“At msnbc.com having low availability numbers is not an option for our customers because they have many options to choose from if our site is not available. We rely heavily on 3rd party measuring tools such as Gomez to provide us with accurate and real-time performance information. From an Operations standpoint our customers are what’s most important to our business and providing them with a high performing web site is what we strive to do on a daily basis.”
— Doug Shore, Sr. Director — msnbc.com Operations
11
RETAIL
After a flat 2009, 2010 retail ecommerce revenue was up 10 percent with spending reaching $142.5 billion.6 Unique visitors to ecommerce sites rose 15 percent versus a year ago. 8 Cyber Monday 2010 was the first time ever that online spending reached $1 billion for one day. 6
With increased online spending come growing consumer expectations that Web and mobile site applications perform optimally. In 2006, shoppers expected sites to load in four second or less but by 2009 that number had dropped to 2 seconds or less. Forrester Research found that when these expectations are not met, consumers head for the competition — in fact, 79 percent are likely to no longer buy from a site.9
The 2010 Best of the Web award winners in Retail Web performance were analyzed across multiple benchmarks:
• a key business transaction, the process of searching for a product and putting it in a cart;
• the performance of the home page on the Internet Explorer browser accessed from the Internet backbone;
• home page performance as experienced on the end users desktop from the Gomez Last Mile; and,
• mobile site performance.
Key performance metrics included response time, availability and consistency. The One Web award category resulted in tie between Newegg and QVC — both performing the best across all of the dimensions.
WINNERS
One Web Winners (Tie) Newegg and QVC
ONE WEBWINNER
ONE WEBWINNER
“Gomez is a well-respected arbiter that has proven its expertise in evaluating website performance, availability and quality. To be honored with their GOLD award for the second year in a row designates Newegg as a clear leader in delivering a high-quality, efficient and satisfying online experience for our customers,” Bernard Luthi, vice president of marketing, web management and customer service for Newegg. “We look forward to continuing to innovate and improve the Newegg shopping experience even more as time goes on. Everything we do at Newegg is geared toward making our customer experience simple, efficient, informative and entertaining, and being ranked so highly by Gomez both emphasizes and reflects our commitment to our customers.”
12
RETAIL
Retail — Web Gold NeweggWEB PERFORMANCE INDEX
Our Web Performance Index aggregates benchmark data for the Product Order Transaction; the home page on an IE agent; and on the Gomez Last Mile across response time, availability and consistency. The table below provides only average response time for each of these benchmarks.
Product Order Transaction — Average Response Time (Sec)
Home Page — Average Response Time (Sec)
Last Mile Home Page — Average Response Time (Sec)
Gold Newegg 7.432 1.008 7.558
Silver QVC 14.314 1.089 6.958
Bronze Amazon 19.210 2.534 5.349
RETAIL MOBILE LEADER
The Retail Mobile Leader was determined by the mobile site that had the best response time, availability and consistency across four carriers and their top devices. See the Methodology section for more detail on measurement.
RETAIL MOBILE LEADER
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
QVC 2.457 99.90% 1.698
MOST IMPROVED
Overstock was Most Improved on the Product Order Transaction, with an average response time higher than the benchmark average and the most improved response time in the second half of the year versus the first half.
Most Improved Overstock 27% response time improvement
“Overstock.com is proud to receive this award from Gomez. Our goal as a savings engine is to save people money and having a website that performs quickly and functions efficiently is a key part in achieving that goal.”
— Jonathan Johnson, President, Overstock.com
13
TRAVEL
Online travel spending grew 6 percent in 2010 to reach $85.2 billion. 6 The industry continues to evolve how it leverages the online channel. These changes are being driven by younger consumers, those in Generation Y, who travel more, make more travel purchases online and have less patience for technology problems. As an example of the generational difference of online behavior, a consumer survey found that 7:
• Sixty-five percent of 18-24 year olds expect a site to load faster than the older demographics — in two seconds or less.
• Only 6 percent of 18-24 year olds expect to wait four seconds or more for a page to load; whereas,
• A third of seniors expect to wait four seconds or longer.
AIRLINES
WINNERS
Airlines — Gold Delta
The Airline winners were determined by the sites that had the best performing Airline Flight Search Transaction.
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
Gold Delta 4.794 99.95% 3.089
Silver AirTran 8.079 99.93% 3.487
Bronze JetBlue 12.366 99.46% 3.073
MOST IMPROVED
AirTran was most improved on the Flight Search Transaction with an average response time higher than the benchmark average and the most improved response time in the second half of the year versus the first half.
Most Improved AirTran 20% response time improvement
“Delta’s ongoing investments in technology and booking experience on delta.com bring speed and convenience to our customer’s travel planning,” said Bob Kupbens, Delta’s Vice President — eCommerce. “Our customers can continue to expect innovation from Delta and count on a reliable, fast web experience.”
“We are honored that the efforts of our E-Commerce and IT teams are being recognized by Gomez for the second consecutive year,” said Michael Stromer, director e-commerce & interactive marketing at JetBlue Airways. “Our team is constantly looking for ways to improve the jetblue.com web and mobile interfaces to deliver efficient, intuitive and reliable service to our customers at the starting point of their JetBlue Experience, and we will continue to expand and simplify our offerings in the coming year to deliver on that promise.”
14
TRAVEL
HOTELS
WINNERS
Hotels — Gold Radisson
The Hotel winners were determined by the sites that had the best performing Hotel Search Transaction.
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
Gold Radisson 3.436 99.64% 3.126
Silver Choice Hotels 7.121 99.72% 4.004
Bronze Starwood 8.310 99.90% 4.611
MOST IMPROVED
HotelGuides was Most Improved on the Hotel Search Transaction with an average response time higher than the benchmark average and the most improved response time in the second half of the year versus the first half.
Most Improved HotelGuides 34% response time improvement
TRAVEL MOBILE LEADER
The Travel Mobile Leader was determined by the mobile site that had the best response time, availability and consistency across four carriers and their top devices. See the Methodology section for more detail on measurement.
TRAVEL MOBILE LEADER
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
JetBlue 3.542 99.80% 2.002
“Carlson and Radisson.com are honored to be selected as Gomez’ Best of the Web in the hotel industry. As part of our Ambition 2015 strategy, Carlson is investing over USD 10 million in our ecommerce and mobile platforms. With that investment, we are providing our guests with a reliable, intuitive and informative experience. Additionally, by utilizing our distributed content network, we can ensure Radisson.com has a consistent, highly responsive guest experience anywhere in the world. This award, coupled with the recent successful launch of our Radisson® and Country Inn & SuitesSM mobile sites, signifies that we are achieving our objectives.”
— Bill Baltes, Vice President, Enterprise Architecture, Carlson
“At Choice Hotels International, we are passionate about providing a consistently fast, reliable, and informative web site that simplifies the hotel reservation process. The Gomez platform has proven very helpful in revealing key insights on site performance so that we continue to deliver a positive experience to our customers. We are honored to be a Gomez Best of the Web 2010 award winner!”
— John Kinnane, Senior Director, Internet Systems Development, Choice Hotels International, Inc.
15
US GOVERNMENT
In December 2009, the Obama administration issued the Open Government Directive that focused on improving transparency, collaboration and participation. This was to be accomplished through technology, where possible. Agencies were charged with developing websites to publish high value data that could be downloaded and searched and to provide citizens with the opportunity to provide feedback on the agencies’ core mission.
It is critical that these sites and the applications supporting them perform well — are fast and available — to support the initiative’s goal of an open and transparent government.
WINNERS
US Government — Gold FEMA
The Government winners were determined by the sites with the best performing home pages taking into account response time, availability and consistency.
Average Response Time (Sec)
Average Availability
(%)
Average Consistency
(Sec)
Gold FEMA 0.617 99.98% 0.864
SilverUS House of
Representatives0.867 99.98% 1.271
Bronze IRS 0.648 99.96% 0.977
MOST IMPROVED
USA Jobs was Most Improved Government site with an average home page response time higher than the benchmark average and the most improved response time in the second half of the year versus the first half.
Most Improved USA Jobs 45% response time improvement
16
BENCHMARK THE BEST, YOUR COMPETITION AND YOUR OWN SITEGomez Benchmarks compare Web and mobile performance metrics of leading companies in an industry to determine who the winners are and what others can do to improve. The process of benchmarking is leveraged across business to measure everything from share price, to branding strategies, to manufacturing processes.
Comparing your Web and mobile site performance metrics against the best performers gives you a tangible and quantitative measure of site performance that you can integrate with other key data, such as Web analytics metrics (e.g., traffic, page views, and unique visitors) and ecommerce metrics (e.g. sales, average order size and profitability). Benchmarking also provides a good balance for more qualitative measures of customer experience like usability studies.
The benchmarks in this report can give you a good starting place to compare how your Web and mobile sites perform against others in your industry. The keys to a solid benchmarking practice include:
• Track your performance against the best and your competition — Make sure you keep an eye on how you stack up. Since consumers’ have little flexibility in their willingness to wait for a site to load, ensure that you are not losing visitors and revenue due to a slow Web or mobile site.
WEB PERFORMANCE BEST PRACTICES
• Baseline your performance and track it over time — It is critical that you track performance before and after you add new marketing widgets to your site such as ad-serving networks, video and images, dynamic promotions and interactive applications such as ratings and reviews. Marketers are continuously looking to make sites more engaging but with that goal comes technology complexity and potentially, site performance problems.
• Use benchmarks to measure success for IT and business stakeholders — With all the new technologies available to support innovative marketing campaigns on your Web or mobile site, it is important not to lose sight of the fundamentals. Using Web performance benchmarks as a key indicator of success for both the business and IT owners of the website is a good start.
OPTIMIZE FOR YOUR END USERS’ EXPERIENCEMaking sure Web and mobile sites are fast enough and are consistently available for customers is not easy. Performance monitoring and analysis tools are only effective if they take an outside-in, end user perspective. This means monitoring performance across the Application Delivery Chain from the browser on a real user’s computer or mobile device, across the Internet or a corporate WAN, across third party and cloud providers to the infrastructure running inside the data center.
Figure 5: Web and applications must be monitoring across the Application Delivery Chain to provide a true indicator of the end users experience — from browser to datacenter.
17
The end users’ experience on your Web or mobile site is dependent upon the following factors:
• Third Parties and Rich Technologies — The most engaging websites are a mix of video, dynamic images and rich interactive applications. They include technologies such as AJAX and Flash and have third-parties delivering content, images and ads. It is important to monitor the impact of these factors to the overall performance of the site and your end users’ experience.
• Browser usage — The browser market is changing rapidly and versions are updated all the time. Browsers have varied performance profiles — impacting how users of those browsers will experience your site. Your site must be optimized for the browsers your customers use.
• Customers’ location — End users are distributed globally. If you are a retailer located on the east coast of the U.S. with your datacenter on the east coast, you need to ensure your customers on the west are also experiencing acceptable performance. Otherwise you are losing customers.
Figure 7: Browser usage is changing over time and is unique for your customer base.
Figure 6: Compuware analyzed data from 3000 companies and found that the average number of hosts accessed by the browser per user transaction was 10.5.
18
THINK MOBILESmart phone usage is proliferating. More and more of your customers are accessing your site on their mobile device or using smart phone applications. You must deliver a consistent experience to your customers independent of whether they are accessing your site on their computer or on a mobile device. And, new platforms such as the iPad tablet must be considered when you are optimizing site experience.
Two key principles of mobile site performance are:
Figure 8: There are significant performance differences across browsers and devices. Compuware analyzed data across more than 520 pages and 200 sites and found a range of almost 8 seconds in page load time differences.
Figure 9: A customers’ location should not impact their experience but it will if you don’t optimize for it. In this example, a retailer that is located with their datacenter on the east coast has not optimized for users across the country and is losing sales from west coast customers due to poor performance.
1.) Keep the end users’ experience in mind — Optimize your mobile site for the devices and networks your customer use taking into account both where you customers are located and their usage patterns.
2.) Don’t sacrifice speed — Mobile users are typically goal directed. They need to perform a task immediately or in a short period of time such as checking flight status, confirming reservations, comparing price options and making appointments. According to a study of 1000 U.S. mobile phone users, 52 percent are unlikely to return to a website if they had trouble accessing it by phone.
19
NEXT STEPSWhat are the next steps to improving your Web or mobile site performance and implementing best practices? We recommend that you:
1.) Benchmark your results. Use the Gomez Benchmarks to see where your site ranks.
• If your site is measured on a published benchmark and you are not a winner, contact us and we can provide you with your rank for the year.
• If your site is not measured on a benchmark, assess your performance over the next week for free with the Gomez Custom Benchmark Report.
2.) Determine how poor performance is impacting your business results. For some actual case studies to guide your analysis, read Why Web Performance Matters.
3.) Use performance monitoring tools to provide insight into your end users’ experience. Performance monitoring can help you determine whether your Web or mobile site and application performance is impacting your end users. Specifically, they can help you determine whether your performance problems are in your datacenter behind the firewall, are due to the ISP, or your end users’ browser, location or device. Obtaining visibility into the entire application delivery chain and the end user’s experience will ensure that you are delivering a superior experience to all your end users.
CREATE A FREE CUSTOM BENCHMARK TO SEE HOW YOUR WEBSITE RANKS.
http://www.gomez.com/my-benchmark/
20
BANKING & RETAIL AWARDS
• Gold, Silver and Bronze Winners on the Website Performance Index — The winners had the best aggregate response time, availability and consistency ranks for 2010 across multiple benchmarks — a business transaction, an Internet Explorer agent on the internet backbone measuring home page performance and home page performance measured from the Gomez Last Mile.
• One Web Winner — This winner had the best aggregate ranking for the Gomez Performance Index Website and mobile performance combined for 2010.
• Mobile Leader —This leader had the aggregate best response time, availability and consistency ranking for the period the data was available in 2010.
• Most Improved — This award winner had a response time in the second half of 2010 that was above the benchmark average for the year and had the biggest improvement in response time in the second half of the year vs. the first half of the year.
OTHER INDUSTRIES AWARDS
• Gold, Silver and Bronze Winners — The winners had the best aggregate response time, availability and consistency rankings across the indicated benchmark for 2010.
• Most Improved — This award winner had a response time in the second half of 2010 that was above the benchmark average for the year and had the biggest improvement in response time in the second half of the year vs. the first half of the year.
• Mobile Leader — This leader had the best aggregate response time, availability and consistency ranking for the period the data was available in 2010.
Home Page Backbone BenchmarksThese benchmarks measure the performance of the website’s front door — the home page — from the Internet Backbone.
Home Page Last Mile BenchmarksThese benchmarks measure the performance of the home page from the end user’s desktop taking into account the real user’s connection speed. These benchmarks leverage the Gomez Network of 150,000 desktop computers connected to 2500 local ISPs in 168+ countries around the globe.
Transaction BenchmarksThese benchmarks measure the performance of a key business process such as ordering a product or making a stock trade.
Mobile Benchmarks These benchmarks measure the performance of the home page of a mobile site for four carriers with their top devices. The site / carrier combinations are: AT&T iPhone, Verizon Droid, T-Mobile HTC Dream and Sprint HTC Hero.
Frequency and Location of TestsFor our homepage and transactional benchmarks, we ran tests from twelve geographically dispersed co-location facilities across multiple network providers within the United States at regular 30- or 60-minute intervals throughout each business day (several benchmarks include predetermined maintenance windows that limit the reporting period for the benchmark).
Gomez backbone locations used to collect benchmark data are located in: NY, DC, GA, FL, IL, CO, TX, MO, AZ, WA and CA.
Response TimeResponse time measures the end-to-end time elapsed while downloading each Web page or the entire multistep transaction process. This includes the base HTML page and any referenced objects, third-party content, JavaScript and Cascading Style Sheets and other Web content. The reported metric represents the average time required to download the homepage or execute the multistep transaction without factoring in user latency.
AvailabilityAvailability measures the percentage of successfully completed tests out of total test attempts for the period. Availability rates are dependent upon a number of variables including the ability of our testing agent to access the website, its ability to process a transaction without error, and its ability to download each step of the transaction within a reasonable 60-second time frame. Reported availability rates do not include errors related to the Gomez measurement system or scripting problems.
ConsistencyConsistency measures the standard deviation of the response times of successful tests. The goal is to have a low number (in seconds), which indicates how consistent the response time is each time a user visits a website. Highly variable response times (in seconds) indicate inconsistent performance that may be caused by poor website performance as measured across diverse geographies, issues with Internet peering or an under-powered technical architecture which leads to slowdowns during periods of peak traffic.
Financial Services Benchmark ParticipantsThe companies tracked for the One Web award were: Bank of America, Branch Bank and Trust, Capital One, JP Morgan Chase, KeyBank, PNC Bank, USBank, Wachovia and Wells Fargo.
METHODOLOGY
21
Web performance was tracked on the following banks for the Website Performance Index and Most Improved Awards for this report: Bank of America, Branch Bank and Trust, Capital One, Citizens Bank, HSBC, JP Morgan Chase, KeyBank, M&T Bank, PNC Bank, Regions Bank, Sovereign Bank, SunTrust, TD Bank, USBank, Wachovia and Wells Fargo.
Mobile performance was tracked for the following banks for this report: Bank of America, Branch Banking and Trust, Capital One, CitiBank, Eastern Bank, Fifth Third, Huntington Bank, JP Morgan Chase, Keybank, Northeast Bank, PNC Bank, US Bank, Wachovia and Wells Fargo.
The brokerages tracked for this report were: Charles Schwab, E*Trade, Fidelity, FirstTrade, Options Xpress, Scottrade, TD Ameritrade, TradeKing, Vanguard and Wells Fargo Brokerage.
Health Information Benchmark ParticipantsThe companies tracked for this report were: AAOS, AARP, Abbott Laboratories, About Health, Aetna, AMA, American Diabetes Association, American Heart, AOL Health, Beachbody, bmj.com, Cancer.org, Discovery Health, Drugs.com, eDiets, Family Doctor, FootSmart, Health Communities, HealthAtoZ, Healthcentral, Healthline, Healthology, HealthScout, HealthSquare, HealthyPlace, InteliHealth, iVillage, Health, Jenny Craig, LifeScript, Mamashealth, Mayo Clinic, Medco Health, Medhelp, Mercola, NetDoctor, Prevention.com, Puritans Pride, Realage, RevivalSoy, Science Daily, Waterfront, WebMD, Weight Watchers, WholeHealthMD and Yahoo Health.
Media Benchmark ParticipantsWeb performance was tracked for the following media properties for this report: ABC News, AOL News, Associated Press, Atlanta Journal, BBC News, Boston.com, CBS News, Chicago Tribune, CNN, Dallas News, FOX News, Miami Herald, MSNBC, NPR, NY Times, SF Gate, Slate Magazine, Time Magazine, Topix.net, truTV , USA Today, Washington Post, WNBC and Yahoo News.
Mobile performance was measured for January 2011 for the following media properties: About.com, Aolnews, APnews, BBCnews, Canada.com, Foxnews, Msnbc, NPR.org, Slate, Telegraph, Usatoday, USnews and Yahoonews.
Retail Benchmark ParticipantsThe companies tracked for the One Web award were: Amazon, Dell, HSN, Newegg, Office Depot, Office Max, QVC, Sears, Target, Walmart and Williams-Sonoma.
Website performance was tracked for the following companies for the Website Performance Index and Most Improved Awards: The companies tracked on the Website Performance Index for this report were: Amazon, CDW, Costco, Dell, HP, HSN, J C Penney, LL Bean, Newegg, Office Depot, Office Max, Overstock, QVC, Sears, Staples, Target, Walmart and Williams-Sonoma.
Mobile site performance was tracked for the following companies for this report: Amazon, BestBuy, Dell, HSN, Macys, Netflix, Newegg, OfficeDepot, OfficeMax, QVC, Sears, Target, VictoriasSecret, Walmart and Williams Sonoma.
Travel Benchmark ParticipantsThe following companies were tracked for the Airline winner for this report: AirTran, American Airlines, Continental, Delta, Frontier, JetBlue, Priceline Air, Southwest, Spirit Airlines, Travelocity Air, United and USAirways.
The following companies were tracked for the Hotel winner for this report: AOL Travel, Best Western, Choice Hotels, Days Inn, Expedia, Hilton, Hotel-Guides, Hotels.com, Hyatt, InterContinental Hotels, One Travel, Radisson, Ramada, Starwood, Travelocity and Yahoo Travel.
Mobile performance was tracked for the following companies for this report: Airtran, Alaska Airlines, American Airlines, Avis, Best Western, Choice Hotels, Continental Airlines, Delta Airlines, Harrahs, Hertz, Hilton, ICHotelsgroup, Jetblue, Kayak, Lowfares.com, Marriott, Orbitz, Priceline, Southwest, Travelocity, United Airlines, US Airways and Zipcar.
U.S. Government Benchmark ParticipantsThe following U.S. Government agencies were tracked for this report: Bureau of Citizenship and Immigration, Bureau of Labor Statistics, Census Bureau, Centers for Disease Control, Centers for Medicare and Medicaid Services, CIA, Department of Defense, Dept of Agriculture, Dept of Education, Dept of Health and Human Services, Dept of Housing and Urban Development, Dept of Justice, Dept of Labor, Dept of State, Dept of Transportation, Dept of Treasury, Dept of Veterans Affairs, EPA, Federal Aviation Administration, Federal Bureau of Investigation, Federal Trade Commission, FEMA, Food and Drug Administration, General Services Administration, Government Printing Office, IRS, Library of Congress, NASA, National Institutes of Health, National Oceanic and Atmospheric Administration, Office of Personnel Management, OSHA, Securities and Exchange Commission, Small Business Administration, Social Security Administration, Thrift Savings Plan, US Air Force, US Army, US Courts, US Geological Survey, US House of Representatives, US Mint, US Navy, US Park Service, US Patent and Trademark Office, US Senate, USA Jobs, USA Search, USA.gov and White House.
BENCHMARK PARTICIPANT INCLUSION METHODOLOGY
Companies on the benchmarks with less than 9 months of testing data were excluded from the evaluation set.
Compuware Corporation, the technology performance company, provides software, experts and best practices to ensure technology works well and delivers value. Compuware solutions make the world’s most important technologies perform at their best for leading organizations worldwide, including 46 of the top 50 Fortune 500 companies and 12 of the top 20 most visited U.S. web sites. Learn more at: compuware.com.
Compuware Corporation World Headquarters • One Campus Martius • Detroit, MI 48226-5099
© 2011 Compuware Corporation
Compuware products and services listed within are trademarks or registered trademarks of Compuware Corporation. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
02.11
SOURCES1 Mobile Marketer: http://www.mobilemarketer.com/cms/news/commerce/4533.htmlhttp://www.mobilemarketer.com/cms/news/commerce/2358.html
2 Equation Research Gomez Mobile Web Experience Survey - Why the Mobile Web is Disappointing End-Users , 2009
3 Forrester Research, the Impact of Poor Web Site Performance in Financial Services, January 15, 2010.
4 Pew Internet, the Social Life of Health Information, June 2009
5 comscore Marketer, October 2010.
6 comscore 2010 U.S. Digital Year In Review
7 PhoCusWright, Consumer Response to Travel Site Performance, 2010
8 comscore State of the Internet, October 2010
9 eCommerce Web Site Performance Today, Forrester Research, August 2009
ABOUT GOMEZ
The Gomez platform is the industry’s leading solution for optimizing the performance, availability, and quality of web, non-web, mobile, streaming and cloud applications. The Gomez approach to application performance management starts by measuring your end user’s experiences and all the components that contribute to it to proactively detect performance issues, quantify their business impact and accelerate resolution. The Gomez solution works for any type of application, including enterprise applications accessed by employees, e-commerce web sites visited by customers or applications running on mobile devices. Only the Gomez “First Mile to Last Mile” solution eliminates blind spots across the entire application delivery chain, from the browser on a user’s computer or mobile device, across the Internet or a corporate WAN, across third-party and cloud providers, to the complex infrastructure inside data centers. Business managers, IT operations personnel and application development/QA engineers benefit from the insight provided by the Gomez solution. More than 4,000 customers worldwide, ranging from small companies to large enterprises and managed service providers, use Gomez to increase revenue, build brand loyalty and decrease costs.
