Website Latency Diagnostics alex.burciu @bad robot.amazon.com

Agenda

• Web Performance Optimization

• Measuring Web Latency

• Latency Monitoring and Analysis

• Testing Latency

Web Performance Optimization

• Customer satisfaction & loyalty – White page of death – “World Wide Wait”

• Faster pages = longer sessions

– Sales opportunities – Ads impressions

• Competitive Advantage

– Better customer experience – Factor in SEO rankings

Latency Metrics

• Understand what you’re optimizing and quantify impact. – serverRenderTime (webserver render time)

• time delay on the webserver since receiving the HTTP request to when generating and emitting the last byte of content to the client.

– clientPageLoaded (browser render time) • time delay spent on the browser since receiving the first

HTML response byte to when the DOM load event is fired.

– clickToPageLoaded (complete rendering time) • time delay experienced by the customer since when

initiating navigation to when the page loaded.

Metrics Relations

(?) clickToPageLoaded = serverRenderTime + clientPageLoaded

(?) serverRenderTime > clientPageLoaded

Browser-webserver Interaction

serverRenderTime

Response Transfer RTT Unload Redirect Cache DNS TCP SSL

DOM PL Click

clientPageLoaded

serverRenderTime

clientPageLoaded

Response transfer RTT Unload Redirect Cache DNS TCP SSL

DOM PL Click

HTTP Flushing

HTTP Caching

clientPageLoaded

Unload Redirect Cache

DOM PL Click

Back-Forward Rendering Cache

Redirect Cache

PL Click

Pre-rendering

serverRenderTime

Transfer RTT Unload Redirect Cache DNS TCP SSL

DOM PL Click

clientPageLoaded

Navigation Timing (pre-request)

Click Cache DNS TCP Connection SSL Redirect Req... Unload

Navigation Timing (pre-request)

Click Unload Cache DNS TCP Connection SSL Redirect Req...

secureConnectionStart unloadEventStart

connectStart requestStart

navigationStart

redirectStart

unloadEventEnd

redirectEnd

fetchStart

domainLookupStart

domainLookupEnd

connectEnd

DOM Processing

Navigation Timing (post-request)

Request onLoad PL Response

Navigation Timing (post-request)

DOM Processing Request onLoad

domLoading

domInteractive

loadEventStart

requestStart

responseEnd

responseStart

domComplete

loadEventEnd

domContentLoadedEventStart

domContentLoadedEventEnd

PL Response

PageLoad Lifetime

0 2,000 4,000 6,000 8,000 10,000 12,000 14,000

navigationStart

redirect

fetchStart

connect

dns

requestStart

responseStart

domLoading

unloadEvent

domInteractive

domContentLoaded

domComplete

loadEvent

Navigation Timing JavaScript API >> window.performance.timing PerformanceTiming

connectEnd: 1344530318281 connectStart: 1344530318281 domComplete: 1344530330212 domContentLoadedEventEnd: 1344530321367 domContentLoadedEventStart: 1344530320211 domInteractive: 1344530320048 domLoading: 1344530318567 domainLookupEnd: 1344530317143 domainLookupStart: 1344530317143 fetchStart: 1344530318281 loadEventEnd: 1344530330214 loadEventStart: 1344530330212 navigationStart: 1344530317144 redirectEnd: 1344530318281 redirectStart: 1344530317149 requestStart: 1344530318283 responseEnd: 1344530319479 responseStart: 1344530318566 secureConnectionStart: 0 unloadEventEnd: 1344530319045 unloadEventStart: 1344530318567

Page Responsiveness clickToFirstByte = responseStart - navigationStart

Influencing factor

Network Server Backend

Application Browser

Front-end Application

Unload ✓ ✓

Redirect ✓ ✓ ✓

Cache ✓ ✓

DNS ✓ ✓

Connection ✓ ✓ ✓ ✓

SSL ✓ ✓

Request ✓ ✓ ✓

Response ✓ ✓ ✓

End-to-end Latency clickToPageLoaded = loadEventEnd - navigationStart

Influencing factor

Network Server Backend

Application Browser

Front-end Application

clickToFB ✓ ✓ ✓ ✓ ✓

DOM Interactive

✓ ✓ ✓ ✓ ✓

DOM Content Loaded

✓ ✓ ✓

DOM Complete

✓ ✓ ✓

DOM Load Event

✓ ✓

Testing Web Performance

Synthetic Benchmarking

– Generated traffic

– Limited context

– Subset of pages

– On-demand

– Expensive

Real Users Monitoring

– Prod traffic

– Context-aware

– All visited pages

– Continuous

– Cheap

Average Daily Latency in 3 months

Average Daily Latency in 3 months

Avg Daily Latency vs Traffic

Avg Daily Latency vs Traffic

Avg Hourly Latency vs Traffic

Prev Week Avg Hourly Latency &Traffic

Percentile Metrics

• P(x) = variable value for which x% of the variables are smaller

P25 = first quartile (Q1) P50 = median (Q2) P75 = third quartile (Q3)

• Example

– 10, 770, 40, 60, 50, 880, 9999, 30, 30, 20, 10 – 10, 10, 20, 30, 30, 40, 50, 60, 770, 880, 9999

• Average vs P50: 1,081 > 40

Prev Week Hourly Latency Percentiles

Understats and overstats

• The inverse of a percentile.

• Percentiles – Constant percentage (e.g. 50%) – Variable measurement (e.g. latency in ms)

• Understatistics and overstatistics

– Constant measurement (e.g. latency <1000ms) – Variable percentage (e.g. percentage of hits)

• Understat x = percentage of hits with latency under x ms

Prev Week Traffic% under 2 seconds

Prev Week Traffic% under 2 seconds

Avg vs P50 5-minute Latency

P50 for components before response

P50 for components after response

domContentLoaded per browser

DOM Content Loaded domLoadingToDomContentLoaded = domContentLoadedEventStart - domLoading

Influencing Factor

Network Server Backend

Application Browser

Front-end Application

DOM Content Loaded

✓ ✓ ✓

Root Cause: Content Change

“Jeff Letter” on Gateway Page

Date Press Release Title

7/31/2012 Updated Amazon Cloud Player Includes New Scan and Match Technology, Free Audio Quality Upgrades, and More

7/26/2012 Amazon.com Announces Second Quarter Sales up 29% to $12.83 Billion

7/23/2012

Amazon Launches Innovative New Education Initiative, Paying 95% of Tuition Costs for Employees to Pursue Their Aspirations – Whether at Amazon, or in Another Industry

7/23/2012 Amazon.com Books Editors Announce the Big Fall Books Preview

7/20/2012

On Prime Instant Video First: Amazon Announces Digital Video License Agreement with Warner Bros. Domestic Television Distribution, Bringing Acclaimed TV Shows Exclusively to Kindle Fire and Prime Customers

7/19/2012 Amazon Web Services Introduces New Amazon EC2 High I/O Instance Type

7/12/2012 Amazon.com to Webcast Second Quarter 2012 Financial Results Conference Call

Example Latency Root Causes

A/B Latency Testing

Key Takeaways

• Invest in Latency Monitoring – 80% of the latency is on the client – Navigation Timing cross-browser API

• Browser rendering is complex

– Latency easily influenced by context – Lab traffic is different from the mix from real customers

• Look at your data

– Average poorly represents a population – Control releases using A/B testing

Q&A

“Premature optimization is the root of all evil.” – Donald Knuth