Date post: | 08-Sep-2014 |
Category: |
Technology |
Upload: | amazon-web-services |
View: | 487 times |
Download: | 1 times |
© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Dynamic Content Acceleration
Nihar Bihani, Amazon Web Services
July 10, 2014
Fundamental Facts
• Any web application must have…
– Tight security
– High availability
– High performance
Why Does Availability Matter?
• If your application is not available, your revenue loss is
100%.
• Impact to customer loyalty and your brand image.
How AWS Helps?
• Use Amazon Route 53 to health-check your origin
webservers with automatic failover.
• Use Amazon CloudFront to front your origins to reduce load
on your origins.
• Use Amazon CloudFront to customize your error pages.
• Amazon CloudFront will automatically serve stale content
when origin is unavailable.
Why Performance Matters?
• Performance translates to…
– Higher page views
– Better customer experience
– Higher conversion rates
Why Performance Matters?
Why Performance Matters?
• Great amount of time and money spent improving back-end
infrastructure performance.
• 80% of user’s perceived latency comes from front-end.
How do we Improve Performance?
• A typical web application has…
– Static or Re-Usable content
• High TTLs
• Low TTLs (Customized Content)
• Can be cached
– Dynamic or Unique content
• Zero TTL
• Cannot be cached BUT affects 100% of your viewers!
Elastic Load
Balancing
Dynamic Content
Amazon EC2
Static Content
Amazon S3 Custom Origin
OR
OR
Custom Origin
www.example.com/*.php
cdn.example.com/*.jpg
Typical Architecture
Elastic Load
Balancing
Dynamic Content
Amazon EC2
Static Content
Amazon S3 Custom Origin
OR
OR
Custom Origin*.php
*.jpg
Amazon CloudFront
www.example.com
Why Not?
Why Don’t Customers Use CDNs for Dynamic Content?
I don’t see the value - each request is unique and must go
back to the origin web server.
I see the value, but my current CDN charges premium rates
for dynamic content acceleration, with many additional fees.
Configuring a CDN for dynamic content acceleration requires
expensive professional services and is not self-service.
How Can Amazon CloudFront Help?
• TCP/IP optimizations for the network path
• Keep-Alive connections to reduce RTT
• SSL termination close to viewers
• POST/PUT upload optimizations
• Latency based routing
• Low prices; same as static content delivery!
Re-Usable or Unique Content?
• Static or Re-Usable
– A given content where the state of the content does NOT change for
a given period of time.
t
0
t
1
Re-Usable or Unique Content?
• Dynamic or Unique
– A given content where the state of the content changes as soon as it
gets created.
t
0
t
1
Example
Example
Index.jsp (Dynamic)
Images (Static)
Example
. Sec
Page Load Time?
. Sec
Improving Web Application Performance
• Accelerating static content
• Accelerating dynamic content
Brief Introduction to Waterfall Graphics
• What happens?
What happens in betweenTyping the address Browser renders
Waterfall Graphs
• Most important tool in web performance measurement
• Most browsers provide Waterfall Graph plugins
Understanding Waterfall Graphs
TCP connection Content download
DNS lookup
Time to first byte
Understanding Waterfall Graphs
Understanding Waterfall Graphs
index.jsp
Optimizing Static Content
• Content is static: images, JS, CSS
– It can be distributed to more than one user.
– State of the object does not change: sec, minute, hours, etc.
– Caching is a way to server static content to more than one user.
Amazon S3 Custom
OR
Optimizing Static Content with Caching
User request A Edge location
Origin
Amazon S3 Custom
OR
Optimizing Static Content with Caching
User request A Edge location
Origin
Get image
Amazon S3 Custom
OR
Optimizing Static Content with Caching
User request A Edge location
Origin
Get image Get image
Amazon S3 Custom
OR
Optimizing Static Content with Caching
User request A Edge location
Origin
Get image Get image
Image
Amazon S3 Custom
OR
Optimizing Static Content with Caching
User request A Edge location
Origin
Get image Get image
ImageImage
Amazon S3 Custom
OR
Optimizing Static Content with Caching
User request B Edge location
Origin
Amazon S3 Custom
OR
Optimizing Static Content with Caching
User request B Edge location
Origin
Get image
Amazon S3 Custom
OR
Optimizing Static Content with Caching
User request B Edge location
Origin
Get image
Image
Optimizing Static Content with Caching
• Bring content closer to the users
• Improves the experience and performance
• Offloads your infrastructure
Optimizing Static Content with Caching
• Before caching = 1.46 seconds
Optimizing Static Content with Caching
• After caching = 770 milliseconds
• Are we done? Not so fast! Goal is 0.5 seconds.
Optimizing Static Content with Caching
index.jsp
Optimizing Static Content with Caching
• Cache as much as you can.
• How? I’m caching all my images, CSS, and JS.
• Find cacheable content.
– Collect web (w3c) logs from your web-tier
– Run a report on your logs (EMR, RDS, or Redshift)
– Identify top N URLs
Optimizing Static Content with Caching
• Steps to find cacheable content
– Example query
Select count(url) count, url from logs_tableGroup by url sort by count;
Optimizing Static Content with Caching
• Report example
Re-Usable or Unique Content?
• Static or Re-Usable
– A given content where the state of the content does NOT change for
a given period of time.
t
0
t
1
Caching for Smaller Time Units
• Goal: find content that can be cached for any given period
of time.
– Hours
– Minutes
– Seconds
• CloudFront can cache content for any period of time.
Optimizing Static Content with Caching
• Content with query strings
• Reusable?
• CloudFront can cache content with query strings.
• Every unique query-string combination is a new object in
CloudFront’s cache.
110 /factor/create_image?name=book1&size=10x10
Optimizing Static Content with Caching
• API calls
• Reusable?
• CloudFront can cache content with query strings.
• Every unique query-string combination is a new object in
CloudFront’s cache.
100 /api/GetBooks?category=math
Caching for Smaller Time Units
• Imagine your have a read heavy API GETS hit 100 or 1,000
RPS.
• Offload your web-tier from handling 1,000 RPS.
• Offload your load balancer; Elastic Load Balancing or any
other LB.
• Provision less capacity and reduce costs.
100 /api/GetBooks?category=math
Caching Personalized Content Just Launched!
• Optionally configure CloudFront to forward request headers
to your origin.
• Enables caching for personalized content:
• Mobile Device Detection
• Geo Targeting
• Multi-Site Hosting
• Cross Origin Resource Sharing (CORS)
• Protocol Detection
Base Page (First HTML Page)
• Re-usable?
220 /index.jsp
Optimizing Dynamic Content
index.jsp
Optimizing Dynamic Content
index.jsp
Optimizing Dynamic Content
• Can dynamic content be optimized?
– Dynamic content is not cacheable.
– Content can be “proxied” by CDN to the origin and back.
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Get
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Get
Response
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Get
Response
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Get
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Get
Response
Amazon S3 Custom
OR
Optimizing Dynamic Content
User request A
Edge location
Origin
User request B
Get
Response
Optimizing Dynamic Content
• Can dynamic content be optimized?
– That adds latency?
– How to optimize dynamic content?
– Response time = ∑ Time (DNS + Connection + First Byte + Content
Download)
DNS lookup
Content downloadTCP connection
Time to first byte
Faster Response Time =
Reduced DNS Time
+
Reduced Connection Time
+
Reduced First Byte Time
+
Reduced Content Download Time
Optimizing Dynamic Content
Faster Response Time =
Reduced DNS Time
+
Reduced Connection Time
+
Reduced First Byte Time
+
Reduced Content Download Time
Optimizing Dynamic Content
Keep-Alive connections
& SSL termination
Faster Response Time =
Reduced DNS Time
+
Reduced Connection Time
+
Reduced First Byte Time
+
Reduced Content Download Time
Optimizing Dynamic Content
Keep-Alive connections
& SSL termination
Faster Response Time =
Reduced DNS Time
+
Reduced Connection Time
+
Reduced First Byte Time
+
Reduced Content Download Time
Keep-Alive connections
Optimizing Dynamic Content
Keep-Alive connections
& SSL termination
Faster Response Time =
Reduced DNS Time
+
Reduced Connection Time
+
Reduced First Byte Time
+
Reduced Content Download Time
Keep-Alive connections
TCP/IP optimization
Optimizing Dynamic Content
Optimizing Dynamic Content
• After caching/before CloudFront dynamic acceleration =
770 milliseconds
• How to optimize DNS response time? – with Route 53
Optimizing DNS Response Time
DNS lookup
• Amazon Route 53 managed DNS offering
• Designed for high availability
• Low latency DNS resolution
• Global network of DNS servers
• Queries routed to nearest DNS server
Optimizing DNS Response Time
Without Route 53
With Route 53
• How to optimize TCP connections? – with CloudFront Keep
Alive connections.
Optimizing TCP Connections and First Byte
TCP connection
Amazon CloudFront
Keep-Alive Connections
• TCP/IP handshake
– HTTP runs on TCP/IP
– TCP has the concept of TCP handshake
– Every HTTP connection has to complete TCP handshake
– TCP/IP handshake penalizes dynamic content
Optimizing TCP Connections
Two Users without CloudFront
90ms
Region
Two Users without CloudFront
SYN
90ms
Region
Two Users without CloudFront
SYN
SYN-ACK
90ms
Region
Two Users without CloudFront
SYN
SYN-ACK
ACK
GET /index.jsp
90ms
Region
Two Users without CloudFront
360ms
SYN
SYN-ACK
ACK
GET /index.jsp
90ms
Region
Two Users without CloudFront
360ms
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
90ms
Region
Two Users without CloudFront
360ms
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
SYN-ACK
90ms
Region
Two Users without CloudFront
360ms
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
SYN-ACK
ACK
GET /index.jsp
90ms
Region
360ms
• Every user is a new connection.
• More users = more TCP connections.
Without CloudFront
Region
• Every user is a new connection.
• More users = more TCP connections.
Without CloudFront
Region
• Every user is a new connection.
• More users = more TCP connections.
Without CloudFront
Region
• Every user is a new connection.
• More users = more TCP connections.
Without CloudFront
Region
• Without Keep-Alive connections
– Puts load on memory/CPU
– Puts load on your web servers
– Load on your web servers, increases the time to first byte.
Optimizing TCP Connections
Two Users with CloudFront Keep-Alive
SYN
60ms30ms
Region
Two Users with CloudFront Keep-Alive
SYN
SYN-ACK
60ms30ms
Region
Two Users with CloudFront Keep-Alive
SYN
SYN-ACK
ACK
GET /index.jsp
60ms30ms
Region
Two Users with CloudFront Keep-Alive
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
60ms30ms
Region
Two Users with CloudFront Keep-Alive
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
SYN-ACK
60ms30ms
Region
Two Users with CloudFront Keep-Alive
360ms
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
SYN-ACK
ACK
GET /index.jsp
60ms30ms
Region
Two Users with CloudFront Keep-Alive
360ms
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
SYN-ACK
ACK
GET /index.jsp
60ms
SYN
30ms
Region
Two Users with CloudFront Keep-Alive
360ms
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
SYN-ACK
ACK
GET /index.jsp
60ms
SYN
SYN-ACK
30ms
Region
Two Users with CloudFront Keep-Alive
360ms
SYN
SYN-ACK
ACK
GET /index.jsp
SYN
SYN-ACK
ACK
GET /index.jsp
60ms
SYN
SYN-ACK
ACK
GET /index.jsp
30ms
Region
GET /index.jsp
180ms
• CloudFront Keep-Alive connections
– More users does not have to equal more TCP connections
– Without CloudFront two users equal two connections
– With CloudFront two users equal one connection
– Offloads your web tier’s CPU/memory
– Improves response time
– Without CloudFront two users equal 720ms
– With CloudFront two users equal 540ms
Optimizing TCP Connections
Optimizing TCP Connections
Test CPU Util. %
Without CloudFront 20%
With CloudFront 6%
• How to optimize SSL connections? – with CloudFront SSL
termination.
Optimizing SSL Connections
TCP connection
Amazon CloudFront
SSL termination
• CloudFront has the ability to support SSL traffic.
• Use CloudFront cert or bring your own.
• SSL traffic gets terminated at the closet CloudFront
location.
Optimizing SSL Connections
• Taking advantage of Keep-Alive connections
– SSL introduces additional TCP handshake packets.
– Keep-Alive eliminates additional SSL TCP handshake packets
– Offloading your infrastructure from terminating 1,000s of end-users
SSL connections.
Optimizing SSL Connections
• Two optimization patterns:
– Half bridge SSL termination
– Full bridge SSL termination
SSL Optimization Patterns with CloudFront
Half Bridge SSL Termination
CloudFront
HTTP
• Better performance by leveraging HTTP connections to
origin
Region
Full Bridge SSL Termination
CloudFront
HTTPS
Region
• How to optimize content download time? – with CloudFront
Slow Start Optimization
Optimizing Content Download
Content download
Amazon CloudFront
Slow Start optimization
TCP Slow Start
TCP Slow Start
Packet 1
TCP Slow Start
Packet 1
Packet 1 ACK
TCP Slow Start
Packet 1
Packet 1 ACK
Packet 2
Packet 3
TCP Slow Start
Packet 1
Packet 1 ACK
Packet 2
Packet 3
Packet 3 ACK
TCP Slow Start
Packet 1
Packet 1 ACK
Packet 2
Packet 3
Packet 3 ACK
Packet 4
Packet 5
Packet 6
• CloudFront can optimize slow start for new connections.
• CloudFront uses existing connections so users can skip slow
start.
• Users benefit from TCP window optimized with an existing
connection.
• More packets transferred in a single round trip.
Optimizing Content Download
TCP Slow Start
Region
TCP Slow Start
Packet 1
Packet 1 ACK
Region
TCP Slow Start
Packet 1
Packet 1 ACK
Packet 2
Packet 3
Region
TCP Slow Start
Packet 1
Packet 1 ACK
Packet 2
Packet 3
Packet 3 ACK
Region
TCP Slow Start
Packet 1
Packet 1 ACK
Packet 2
Packet 3
Packet 3 ACK
Packet 4
Packet 5
Packet 6
Region
TCP Slow Start
Packet 1
Packet 1 ACK
Packet 2
Packet 3
Packet 3 ACK
Packet 4
Packet 5
Packet 6
Packet 1
Packet 2
Packet 3
Packet 4
Region
TCP Slow Start
Packet 1
Packet 1 ACK
Packet 2
Packet 3
Packet 3 ACK
Packet 4
Packet 5
Packet 6
Packet 1
Packet 2
Packet 3
Packet 4
Packet 4 ACK
Packet 5
Packet 6
Packet 7
Region
• Without CloudFront
• With CloudFront
Performance Tests
Oregon Virginia
Oregon Virginia
Performance Tests Results
Test # Of Packets Response Time Per RequestResponse Time For 200
Requests
Without CloudFront
2605 170 ms 33.876 s
WithCloudFront
896 96 ms 19.24 s
• How to optimize content PUT/POST? – with CloudFront
PUT/POST Verb optimization
Optimizing PUT/POST
TCP connection
Amazon CloudFront
PUT/POST Verb optimization
• CloudFront can support verbs: PUT, POST, DELETE,
OPTIONS, and PATCH.
• Data won’t get cached.
• CloudFront can proxy data to origin.
Optimizing PUT/POST
• Dynamic content optimizations apply
– Keep-Alive connections
– TCP Slow Start optimization
– Close proximity connection termination
Optimizing PUT/POST
• Optimizing form POSTs
• Optimizing AJAX POST requests
• Optimizing content upload
– Uploading to Amazon S3
Optimizing PUT/POST
Optimizing PUT/POST Performance Tests
Oregon Virginia
Upload
Optimizing PUT/POST Performance Tests
Oregon Virginia
Upload
• Uploading 10MB data from an instance in US East region to
US West region: average result is 5 seconds.
Optimizing PUT/POST Performance Tests
Oregon Virginia
• Uploading 10MB data from an instance in US East region to
closest CloudFront location: average result is 3.5 seconds.
• How to optimize content download even more? – with
Route 53
Optimizing Content Download – Even More!
Content download
• Latency Based Routing (LBR)
• Run multiple stacks of your application in different AWS
regions around the world.
• Create LBR records for each location and tag the location
with GEO information.
• Route 53 will route end users to the endpoint that provides
the lowest latency.
Optimizing Content Download – Even More!
• LBR Benefits
– Better performance than running instances in single region.
– Improved reliability relative to running in a single region.
– Easier implementation than traditional DNS solutions.
– Much lower prices than traditional DNS solutions.
Optimizing Content Download – Even More!
Optimizing Content Download – Even More!
Optimizing Content Download – Even More!
• Use CloudFront for dynamic content optimization
• Host your origin at multiple AWS locations (or data centers)
– US
– Europe
Optimizing Content Download – Even More!
• Create origin DNS records in Route 53 at each location.
• Route 53 measures the latency between CloudFront and all
configured origins.
• Route 53 resolves origin’s hostname to the closest location.
• Reduce download time.
Optimizing Content Download – Even More!
Optimizing Content Download – Even More!
Optimizing Content Download – Even More!
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
Lower Latency with CloudFront & Route 53
• Optimization equals 55 milliseconds
Optimizing Content Download – Even More!
• CloudFront & Route 53
• Normal interaction
– Users connect to CloudFront
– CloudFront connects to origin
Design For Failure
CloudFront
Region
• What happens if the origin fails to respond to CloudFront?
Design For Failure
CloudFront
Region
• Failures can be detected by Route 53 health checks.
Design For Failure
Region
Health checks
• The traffic shifts to the healthy instances or load balancers.
Design For Failure
Region
CloudFront
Health checks
• Can mix health checks and LBR.
• Can apply the same logic to multi-region deployments.
• Users get connected to the closest region if both regions
are healthy.
• Route 53 detects failures via health checks.
• Users get connected to the healthy region if the closest
region is not passing the health check.
Design For Failure
Region
Design For Failure
Region2
CloudFront
Health checksHealth checks
Region1
• Caching improves performance.
• Caching can also improve availability.
• If your infrastructure is experiencing failure, CloudFront can
server cached content instead of 5xx, 4xx, etc.
Design For Failure
Region
Amazon S3 Custom
OR
Design for Failure
User request A Edge location
Origin
Amazon S3 Custom
OR
Design for Failure
User request A Edge location
Origin
Get image
Amazon S3 Custom
OR
Design for Failure
User request A Edge location
Origin
Get image Get image
Amazon S3 Custom
OR
Design for Failure
User request A Edge location
Origin
Get image Get image
Image
Amazon S3 Custom
OR
Design for Failure
User request A Edge location
Origin
Get image Get image
ImageImage
Amazon S3 Custom
OR
Design for Failure
User request B Edge location
Origin
Amazon S3 Custom
OR
Design for Failure
User request B Edge location
Origin
Get image
Amazon S3 Custom
OR
Design for Failure
User request A Edge location
Origin
Get image Get image
Amazon S3 Custom
OR
Design for Failure
User request A Edge location
Origin
Get image Get image
Amazon S3 Custom
OR
Design for Failure
User request A Edge location
Origin
Get image Get image
Image
• Accelerate all your content with CloudFront.
• Use CloudFront with Route 53 LBR to improve your
performance.
• Design for failure with CloudFront and Route 53.
Summary
Region
Customer Stories
• Low TTLs
Customer Stories
• Query Strings
Customer Stories
• Cookies
• PUT/POST
– “We are excited to use CloudFront's new POST, PUT, PATCH, and DELETE
capabilities to accelerate our RESTful APIs on Amazon EC2. With these
new HTTP methods we can now take advantage of CloudFront’s global
footprint and optimized connections back to our origin servers in AWS.
Routing our customers’ API requests via a CloudFront edge location near
them will help improve their experience by minimizing packet loss and
upload latency. This will help provide a streamlined experience for our
customers.” Ilan Rabinovitch, Tech Lead, Site Reliability Engineering
Customer Stories
Region
• Custom SSL
Customer Stories
Region
• Health Checks and Failover
– “Amazon Route 53’s DNS Failover feature provides high availability
across our multiple AWS regions and gives us the ability to offload
our origins.”
Customer Stories
Region
• AWS Free Usage Tier
• 50 GB CloudFront Data Transfer per Month
• 2,000,000 HTTP/HTTPS Requests per Month
• Learn More: http://aws.amazon.com/free/
• Office Hours with CloudFront Engineers
• July 30th, 2014
• 10 – 11am (PST)
• Register Here: http://aws.amazon.com/cloudfront/
Get Started with CloudFront
Region