Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront and Amazon Route 53

transcript

Dynamic Content Acceleration

Nihar Bihani, Amazon Web Services

July 10, 2014

Fundamental Facts

• Any web application must have…

– Tight security

– High availability

– High performance

Why Does Availability Matter?

• If your application is not available, your revenue loss is

• Impact to customer loyalty and your brand image.

How AWS Helps?

• Use Amazon Route 53 to health-check your origin

webservers with automatic failover.

• Use Amazon CloudFront to front your origins to reduce load

on your origins.

• Use Amazon CloudFront to customize your error pages.

• Amazon CloudFront will automatically serve stale content

when origin is unavailable.

Why Performance Matters?

• Performance translates to…

– Higher page views

– Better customer experience

– Higher conversion rates

Why Performance Matters?

• Great amount of time and money spent improving back-end

infrastructure performance.

• 80% of user’s perceived latency comes from front-end.

How do we Improve Performance?

• A typical web application has…

– Static or Re-Usable content

• High TTLs

• Low TTLs (Customized Content)

• Can be cached

– Dynamic or Unique content

• Zero TTL

• Cannot be cached BUT affects 100% of your viewers!

Elastic Load

Balancing

Dynamic Content

Amazon EC2

Static Content

Amazon S3 Custom Origin

Custom Origin

www.example.com/*.php

cdn.example.com/*.jpg

Typical Architecture

Elastic Load

Balancing

Dynamic Content

Amazon EC2

Static Content

Amazon S3 Custom Origin

Custom Origin*.php

Amazon CloudFront

www.example.com

Why Not?

Why Don’t Customers Use CDNs for Dynamic Content?

I don’t see the value - each request is unique and must go

back to the origin web server.

I see the value, but my current CDN charges premium rates

for dynamic content acceleration, with many additional fees.

Configuring a CDN for dynamic content acceleration requires

expensive professional services and is not self-service.

How Can Amazon CloudFront Help?

• TCP/IP optimizations for the network path

• Keep-Alive connections to reduce RTT

• SSL termination close to viewers

• POST/PUT upload optimizations

• Latency based routing

• Low prices; same as static content delivery!

Re-Usable or Unique Content?

• Static or Re-Usable

– A given content where the state of the content does NOT change for

a given period of time.

• Dynamic or Unique

– A given content where the state of the content changes as soon as it

gets created.

Example

Index.jsp (Dynamic)

Images (Static)

Example

Page Load Time?

Improving Web Application Performance

• Accelerating static content

• Accelerating dynamic content

Brief Introduction to Waterfall Graphics

• What happens?

What happens in betweenTyping the address Browser renders

Waterfall Graphs

• Most important tool in web performance measurement

• Most browsers provide Waterfall Graph plugins

Understanding Waterfall Graphs

TCP connection Content download

DNS lookup

Time to first byte

Understanding Waterfall Graphs

index.jsp

Optimizing Static Content

• Content is static: images, JS, CSS

– It can be distributed to more than one user.

– State of the object does not change: sec, minute, hours, etc.

– Caching is a way to server static content to more than one user.

Amazon S3 Custom

Optimizing Static Content with Caching

User request A Edge location

Origin

Amazon S3 Custom

Origin

Get image

Amazon S3 Custom

Origin

Get image Get image

Amazon S3 Custom

Origin

Get image Get image

Amazon S3 Custom

Origin

Get image Get image

ImageImage

Amazon S3 Custom

User request B Edge location

Origin

Amazon S3 Custom

Origin

Get image

Amazon S3 Custom

Origin

Get image

• Bring content closer to the users

• Improves the experience and performance

• Offloads your infrastructure

• Before caching = 1.46 seconds

• After caching = 770 milliseconds

• Are we done? Not so fast! Goal is 0.5 seconds.

index.jsp

• Cache as much as you can.

• How? I’m caching all my images, CSS, and JS.

• Find cacheable content.

– Collect web (w3c) logs from your web-tier

– Run a report on your logs (EMR, RDS, or Redshift)

– Identify top N URLs

• Steps to find cacheable content

– Example query

Select count(url) count, url from logs_tableGroup by url sort by count;

• Report example

• Static or Re-Usable

– A given content where the state of the content does NOT change for

a given period of time.

Caching for Smaller Time Units

• Goal: find content that can be cached for any given period

of time.

– Hours

– Minutes

– Seconds

• CloudFront can cache content for any period of time.

• Content with query strings

• Reusable?

• CloudFront can cache content with query strings.

• Every unique query-string combination is a new object in

CloudFront’s cache.

110 /factor/create_image?name=book1&size=10x10

• API calls

• Reusable?

• CloudFront can cache content with query strings.

• Every unique query-string combination is a new object in

CloudFront’s cache.

100 /api/GetBooks?category=math

Caching for Smaller Time Units

• Imagine your have a read heavy API GETS hit 100 or 1,000

• Offload your web-tier from handling 1,000 RPS.

• Offload your load balancer; Elastic Load Balancing or any

other LB.

• Provision less capacity and reduce costs.

100 /api/GetBooks?category=math

Caching Personalized Content Just Launched!

• Optionally configure CloudFront to forward request headers

to your origin.

• Enables caching for personalized content:

• Mobile Device Detection

• Geo Targeting

• Multi-Site Hosting

• Cross Origin Resource Sharing (CORS)

• Protocol Detection

Base Page (First HTML Page)

• Re-usable?

220 /index.jsp

Optimizing Dynamic Content

index.jsp

• Can dynamic content be optimized?

– Dynamic content is not cacheable.

– Content can be “proxied” by CDN to the origin and back.

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Response

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Response

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Response

Amazon S3 Custom

User request A

Edge location

Origin

User request B

Response

• Can dynamic content be optimized?

– That adds latency?

– How to optimize dynamic content?

– Response time = ∑ Time (DNS + Connection + First Byte + Content

Download)

DNS lookup

Content downloadTCP connection

Time to first byte

Faster Response Time =

Reduced DNS Time

Reduced Connection Time

Reduced First Byte Time

Reduced Content Download Time

Reduced DNS Time

Keep-Alive connections

& SSL termination

Reduced DNS Time

& SSL termination

Reduced DNS Time

& SSL termination

Reduced DNS Time

TCP/IP optimization

• After caching/before CloudFront dynamic acceleration =

770 milliseconds

• How to optimize DNS response time? – with Route 53

Optimizing DNS Response Time

DNS lookup

• Amazon Route 53 managed DNS offering

• Designed for high availability

• Low latency DNS resolution

• Global network of DNS servers

• Queries routed to nearest DNS server

Optimizing DNS Response Time

Without Route 53

With Route 53

• How to optimize TCP connections? – with CloudFront Keep

Alive connections.

Optimizing TCP Connections and First Byte

TCP connection

Amazon CloudFront

Keep-Alive Connections

• TCP/IP handshake

– HTTP runs on TCP/IP

– TCP has the concept of TCP handshake

– Every HTTP connection has to complete TCP handshake

– TCP/IP handshake penalizes dynamic content

Optimizing TCP Connections

Two Users without CloudFront

Region

SYN-ACK

Region

SYN-ACK

GET /index.jsp

Region

SYN-ACK

GET /index.jsp

Region

SYN-ACK

GET /index.jsp

Region

SYN-ACK

GET /index.jsp

SYN-ACK

Region

SYN-ACK

GET /index.jsp

SYN-ACK

GET /index.jsp

Region

• Every user is a new connection.

• More users = more TCP connections.

Without CloudFront

Region

Without CloudFront

Region

Without CloudFront

Region

Without CloudFront

Region

• Without Keep-Alive connections

– Puts load on memory/CPU

– Puts load on your web servers

– Load on your web servers, increases the time to first byte.

Two Users with CloudFront Keep-Alive

60ms30ms

Region

SYN-ACK

60ms30ms

Region

SYN-ACK

GET /index.jsp

60ms30ms

Region

SYN-ACK

GET /index.jsp

60ms30ms

Region

SYN-ACK

GET /index.jsp

SYN-ACK

60ms30ms

Region

SYN-ACK

GET /index.jsp

SYN-ACK

GET /index.jsp

60ms30ms

Region

SYN-ACK

GET /index.jsp

SYN-ACK

GET /index.jsp

Region

SYN-ACK

GET /index.jsp

SYN-ACK

GET /index.jsp

SYN-ACK

Region

SYN-ACK

GET /index.jsp

SYN-ACK

GET /index.jsp

SYN-ACK

GET /index.jsp

Region

GET /index.jsp

• CloudFront Keep-Alive connections

– More users does not have to equal more TCP connections

– Without CloudFront two users equal two connections

– With CloudFront two users equal one connection

– Offloads your web tier’s CPU/memory

– Improves response time

– Without CloudFront two users equal 720ms

– With CloudFront two users equal 540ms

Test CPU Util. %

Without CloudFront 20%

With CloudFront 6%

• How to optimize SSL connections? – with CloudFront SSL

termination.

Optimizing SSL Connections

TCP connection

Amazon CloudFront

SSL termination

• CloudFront has the ability to support SSL traffic.

• Use CloudFront cert or bring your own.

• SSL traffic gets terminated at the closet CloudFront

location.

• Taking advantage of Keep-Alive connections

– SSL introduces additional TCP handshake packets.

– Keep-Alive eliminates additional SSL TCP handshake packets

– Offloading your infrastructure from terminating 1,000s of end-users

SSL connections.

• Two optimization patterns:

– Half bridge SSL termination

– Full bridge SSL termination

SSL Optimization Patterns with CloudFront

Half Bridge SSL Termination

CloudFront

• Better performance by leveraging HTTP connections to

origin

Region

Full Bridge SSL Termination

CloudFront

Region

• How to optimize content download time? – with CloudFront

Slow Start Optimization

Optimizing Content Download

Content download

Amazon CloudFront

Slow Start optimization

TCP Slow Start

Packet 1

TCP Slow Start

Packet 1

Packet 1 ACK

TCP Slow Start

Packet 1

Packet 1 ACK

Packet 2

Packet 3

TCP Slow Start

Packet 1

Packet 1 ACK

Packet 2

Packet 3

Packet 3 ACK

TCP Slow Start

Packet 1

Packet 1 ACK

Packet 2

Packet 3

Packet 3 ACK

Packet 4

Packet 5

Packet 6

• CloudFront can optimize slow start for new connections.

• CloudFront uses existing connections so users can skip slow

start.

• Users benefit from TCP window optimized with an existing

connection.

• More packets transferred in a single round trip.

Optimizing Content Download

TCP Slow Start

Region

TCP Slow Start

Packet 1

Packet 1 ACK

Region

TCP Slow Start

Packet 1

Packet 1 ACK

Packet 2

Packet 3

Region

TCP Slow Start

Packet 1

Packet 1 ACK

Packet 2

Packet 3

Packet 3 ACK

Region

TCP Slow Start

Packet 1

Packet 1 ACK

Packet 2

Packet 3

Packet 3 ACK

Packet 4

Packet 5

Packet 6

Region

TCP Slow Start

Packet 1

Packet 1 ACK

Packet 2

Packet 3

Packet 3 ACK

Packet 4

Packet 5

Packet 6

Packet 1

Packet 2

Packet 3

Packet 4

Region

TCP Slow Start

Packet 1

Packet 1 ACK

Packet 2

Packet 3

Packet 3 ACK

Packet 4

Packet 5

Packet 6

Packet 1

Packet 2

Packet 3

Packet 4

Packet 4 ACK

Packet 5

Packet 6

Packet 7

Region

• Without CloudFront

• With CloudFront

Performance Tests

Oregon Virginia

Performance Tests Results

Test # Of Packets Response Time Per RequestResponse Time For 200

Requests

Without CloudFront

2605 170 ms 33.876 s

WithCloudFront

896 96 ms 19.24 s

• How to optimize content PUT/POST? – with CloudFront

PUT/POST Verb optimization

Optimizing PUT/POST

TCP connection

Amazon CloudFront

PUT/POST Verb optimization

• CloudFront can support verbs: PUT, POST, DELETE,

OPTIONS, and PATCH.

• Data won’t get cached.

• CloudFront can proxy data to origin.

Optimizing PUT/POST

• Dynamic content optimizations apply

– Keep-Alive connections

– TCP Slow Start optimization

– Close proximity connection termination

Optimizing PUT/POST

• Optimizing form POSTs

• Optimizing AJAX POST requests

• Optimizing content upload

– Uploading to Amazon S3

Optimizing PUT/POST

Optimizing PUT/POST Performance Tests

Oregon Virginia

Upload

Oregon Virginia

Upload

• Uploading 10MB data from an instance in US East region to

US West region: average result is 5 seconds.

Oregon Virginia

• Uploading 10MB data from an instance in US East region to

closest CloudFront location: average result is 3.5 seconds.

• How to optimize content download even more? – with

Route 53

Optimizing Content Download – Even More!

Content download

• Latency Based Routing (LBR)

• Run multiple stacks of your application in different AWS

regions around the world.

• Create LBR records for each location and tag the location

with GEO information.

• Route 53 will route end users to the endpoint that provides

the lowest latency.

• LBR Benefits

– Better performance than running instances in single region.

– Improved reliability relative to running in a single region.

– Easier implementation than traditional DNS solutions.

– Much lower prices than traditional DNS solutions.

• Use CloudFront for dynamic content optimization

• Host your origin at multiple AWS locations (or data centers)

– US

– Europe

• Create origin DNS records in Route 53 at each location.

• Route 53 measures the latency between CloudFront and all

configured origins.

• Route 53 resolves origin’s hostname to the closest location.

• Reduce download time.

Lower Latency with CloudFront & Route 53

• Optimization equals 55 milliseconds

• CloudFront & Route 53

• Normal interaction

– Users connect to CloudFront

– CloudFront connects to origin

Design For Failure

CloudFront

Region

• What happens if the origin fails to respond to CloudFront?

Design For Failure

CloudFront

Region

• Failures can be detected by Route 53 health checks.

Design For Failure

Region

Health checks

• The traffic shifts to the healthy instances or load balancers.

Design For Failure

Region

CloudFront

Health checks

• Can mix health checks and LBR.

• Can apply the same logic to multi-region deployments.

• Users get connected to the closest region if both regions

are healthy.

• Route 53 detects failures via health checks.

• Users get connected to the healthy region if the closest

region is not passing the health check.

Design For Failure

Region

Design For Failure

Region2

CloudFront

Health checksHealth checks

Region1

• Caching improves performance.

• Caching can also improve availability.

• If your infrastructure is experiencing failure, CloudFront can

server cached content instead of 5xx, 4xx, etc.

Design For Failure

Region

Amazon S3 Custom

Design for Failure

Origin

Amazon S3 Custom

Design for Failure

Origin

Get image

Amazon S3 Custom

Design for Failure

Origin

Get image Get image

Amazon S3 Custom

Design for Failure

Origin

Get image Get image

Amazon S3 Custom

Design for Failure

Origin

Get image Get image

ImageImage

Amazon S3 Custom

Design for Failure

Origin

Amazon S3 Custom

Design for Failure

Origin

Get image

Amazon S3 Custom

Design for Failure

Origin

Get image Get image

Amazon S3 Custom

Design for Failure

Origin

Get image Get image

Amazon S3 Custom

Design for Failure

Origin

Get image Get image

• Accelerate all your content with CloudFront.

• Use CloudFront with Route 53 LBR to improve your

performance.

• Design for failure with CloudFront and Route 53.

Summary

Region

Customer Stories

• Low TTLs

Customer Stories

• Query Strings

Customer Stories

• Cookies

• PUT/POST

– “We are excited to use CloudFront's new POST, PUT, PATCH, and DELETE

capabilities to accelerate our RESTful APIs on Amazon EC2. With these

new HTTP methods we can now take advantage of CloudFront’s global

footprint and optimized connections back to our origin servers in AWS.

Routing our customers’ API requests via a CloudFront edge location near

them will help improve their experience by minimizing packet loss and

upload latency. This will help provide a streamlined experience for our

customers.” Ilan Rabinovitch, Tech Lead, Site Reliability Engineering

Customer Stories

Region

• Custom SSL

Customer Stories

Region

• Health Checks and Failover

– “Amazon Route 53’s DNS Failover feature provides high availability

across our multiple AWS regions and gives us the ability to offload

our origins.”

Customer Stories

Region

• AWS Free Usage Tier

• 50 GB CloudFront Data Transfer per Month

• 2,000,000 HTTP/HTTPS Requests per Month

• Learn More: http://aws.amazon.com/free/

• Office Hours with CloudFront Engineers

• July 30th, 2014

• 10 – 11am (PST)

• Register Here: http://aws.amazon.com/cloudfront/

Get Started with CloudFront

Region

Dynamic Content Acceleration: Lightning Fast Web Apps with Amazon CloudFront and Amazon Route 53

Technology