Date post: | 28-Jul-2015 |
Category: |
Internet |
Upload: | sergey-chernyshev |
View: | 548 times |
Download: | 1 times |
Extending Applica/ons To The Edge With CDNs
Hooman Behesh/, VP Technology
Webperf meetup NYC – May 12 2015
What is a CDN
• Global network of HTTP caches – Get content closer to users
• Performance benefits • offload
In the past…
Classic CDNs
• Black box service • Making changes sucked! • LiRle to no real-‐/me interfaces – APIs or configura/on management
• Lots of professional services • Decent caching, terrible uncaching
Cacheable
Uncacheable Sta3c Dynamic
How classic CDNs look at your content
Cache-Control: max-age=31536000
Classic CDNs: dynamic content
• Focused on delivery: transport op/miza/on
But some3mes…
I know you’re calling it dynamic, but it’s kind of sta3c some3mes
I know you’re calling it dynamic, but it’s kind of sta3c some3mes
That’s cool, how long is it sta3c for?
That’s cool, how long is it sta3c for? Some3mes a liFle bit,
some3mes a lot
Some3mes a liFle bit, some3mes a lot Neat!
Some3mes a liFle bit, some3mes a lot You look great today!
You look great today! Don’t change the subject! How do I
cache it?
Don’t change the subject! How do I
cache it? Cache it for half an
hour?
Cache it for half an hour? Some3mes it changes
much faster
Some3mes it changes much faster
Cache it for 10 minutes?
Cache it for 10 minutes? Are you just guessing
right now?
Are you just guessing right now?
Just trying to figure out your comfort
level with mistakes!
Expiry guessing!
Terrible Caching
Terrible Caching
Terrible Anxiety
or
(ex: news story)
Things are different today
Today…
• Influenced by clouds • Devops requires programmability and control • CDNs should be part of the applica/on • Real-‐/me integra/on – Changes and APIs
• Unpredictable content expiry • Your content à you need control
What does this all mean?
Control
Visibility
Cacheability
Control
Visibility
Cacheability
Cacheability
• Granularity in caching rules (i.e. cache the way you want) – Customizable cache keys – Ex: geo-‐ip caching – Ex: normalizing user agents
• Cache everything – Expect high cache-‐hit ra/os – No cache-‐control anxiety!
Cacheability
• Granularity in caching rules (i.e. cache the way you want) – Customizable cache keys – Ex: geo-‐ip caching – Ex: normalizing user agents
• Cache everything – Expect high cache-‐hit ra/os – No cache-‐control anxiety!
Cache hit ra3os
Cache Hit Ra3o: Tradi3onal calcula3on
1 -‐ Requests to Origin Total Requests
Origin
Origin
Cache
TCP
Origin
Cache
HTTP
Origin
Cache
Origin
Cache
HTTP
Origin
Cache
HTTP
Origin
Cache
HTTP
Origin
Cache
HOT COLD
Origin
Cache
cache “hit”
Cache Hit Ra3o: Tradi3onal calcula3on
1 -‐ Requests to Origin Total Requests
Isn’t this beFer?
Hits Total Requests
@edge
Isn’t this beFer?
Hits Hits + Misses
@edge
Cache Hit Ra3o
vs. 1 -‐ Requests to Origin Total Requests
Hits
Hits + Misses @edge
Cache Hit Ra3o
vs. 1 -‐ Requests to Origin Total Requests
Hits
Hits + Misses @edge
Offload
Cache Hit Ra3o
vs. 1 -‐ Requests to Origin Total Requests
Hits
Hits + Misses @edge
Offload Performance
Effect on long tail content
Long Tail: Cacheable but seldom fetched
Popular Medium Tail (1hr) Long tail (6hr)
Popular Medium Tail (1hr) Long tail (6hr)
Connect (median)
Popular 14msec
1hr Tail 15msec
6hr Tail 16msec
Popular Medium Tail (1hr) Long tail (6hr)
Connect (median)
Popular 14msec
1hr Tail 15msec
6hr Tail 16msec
Popular Medium Tail (1hr) Long tail (6hr)
Connect (median) Wait (median)
Popular 14msec 19msec
1hr Tail 15msec 26msec
6hr Tail 16msec 32msec
Popular Medium Tail (1hr) Long tail (6hr)
Connect (median) Wait (median)
Popular 14msec 19msec
1hr Tail 15msec 26msec
6hr Tail 16msec 32msec 6,400+ measurements
77,000+ measurements
38,000+ measurements
Popular Medium Tail (1hr) Long tail (6hr)
Isn’t this beFer?
“Dynamic” Content
We’ve been told:
Sta3c = Cacheable
Dynamic = Uncacheable
Cacheable
Uncacheable Sta3c
Dynamic
How our sites used to look
Cacheable
Uncacheable Sta3c Dynamic
How we’re told our sites look
Cacheable
Uncacheable Sta3c
Dynamic
How our sites actually look
Event Driven
Event-‐Driven content:
Sta3c for unpredictable periods of 3me
Three types of content • Sta/c
– Infrequent changes, can cache for a long /me – Cache-‐control headers are good enough – Examples: Images, javascript , css, etc
• Dynamic – Can’t cache at all -‐ must go to origin – DSA and transport is key for performance – Examples: Logins, credit card transac/ons, Ajax, etc
• Event-‐driven – Sta/c, but unpredictably – Cache-‐control headers are not good enough – Examples: That news story, wiki pages, user comments, sports scores, etc
Three types of content • Sta/c
– Infrequent changes, can cache for a long /me – Cache-‐control headers are good enough – Examples: Images, javascript , css, etc
• Dynamic – Can’t cache at all -‐ must go to origin – DSA and transport is key for performance – Examples: Logins, credit card transac/ons, Ajax, etc
• Event-‐driven – Sta/c, but unpredictably – Cache-‐control headers are not good enough – Examples: That news story, wiki pages, user comments, sports scores, etc
Three types of content • Sta/c
– Infrequent changes, can cache for a long /me – Cache-‐control headers are good enough – Examples: Images, javascript , css, etc
• Dynamic – Can’t cache at all -‐ must go to origin – DSA and transport is key for performance – Examples: Logins, credit card transac/ons, Ajax, etc
• Event-‐driven – Sta/c, but unpredictably – Cache-‐control headers are not good enough – Examples: That news story, wiki pages, user comments, sports scores, etc
Caching event-‐driven content
• Not easy – Classic CDNs con/nue to consider it the same as dynamic (Bad caching or panic!)
– Need the right tools to cache • You may not know expiry ahead of /me, but your app knows when things change
Cache normally and then
“uncache” when it changes
Uncaching!
• Purging – Accessible through API – Granular • Dependencies through labels, tags, or surrogate keys
– Must be instantaneous • Ex: news story, part 2
Instant Purge
• Enables caching of event-‐driven content – Can deal with unpredictable invalida/on – As pages update/publish, purge command is sent to CDN
• Instant? – Instant ≠ 12 minutes! – Instant ≠ quick acknowledgement! – <1sec – Determinis/c and predictable
Example: CMS + Purge
WordPress: Before
CDN Node
WordPress: Before
CDN Node
WordPress: Before
CDN Node
WordPress: Before
CDN Node
WordPress: Before
CDN Node
Cache
WordPress: Ajer
CDN Node
WordPress: Ajer
CDN Node
HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!
WordPress: Ajer
CDN Node
HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!
WordPress: Ajer
CDN Node
WordPress: Ajer
CDN Node
WordPress: Ajer
CDN Node PURGE
WordPress: Ajer
CDN Node PURGE
WordPress: Ajer
CDN Node
(Has to be instantaneous!)
PURGE
WordPress: Ajer
CDN Node
HTTP/1.1 200 OKContent-Type: text/htmlContent-Length: 55,666Cache-Control: Totally Long Time!
WordPress: Ajer
CDN Node
Example: customer1.js
Before
CDN Node
Origin http://www.3rdparty.com/customer1.js
(Referer: www.customer1.com)
Before
CDN Node
Origin http://www.3rdparty.com/customer1.js
Before
CDN Node
Origin
HTTP/1.1 200 OKCache-Control: max-age=60Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
http://www.3rdparty.com/customer1.js
Before
CDN Node
Origin
HTTP/1.1 200 OKCache-Control: max-age=60Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
http://www.3rdparty.com/customer1.js
Before
CDN Node
Origin
HTTP/1.1 200 OKCache-Control: max-age=60Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
http://www.3rdparty.com/customer1.js
Before
CDN Node
Origin http://www.3rdparty.com/customer1.js
(Ajer 1 min)
Before
CDN Node
Origin http://www.3rdparty.com/customer1.js VALIDATION
If-Modified-Since: Tue, 12 May 2015 19:51:30 GMT
(Ajer 1 min)
Before
CDN Node
Origin http://www.3rdparty.com/customer1.js
304 Not Modified
(Ajer 1 min)
Before
CDN Node
Origin http://www.3rdparty.com/customer1.js
304 Not Modified
(Ajer 1 min)
Ajer
CDN Node
Origin http://www.3rdparty.com/customer1.js
Ajer
CDN Node
Origin http://www.3rdparty.com/customer1.js
Ajer
CDN Node
Origin
HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
http://www.3rdparty.com/customer1.js
Ajer
CDN Node
Origin
HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
http://www.3rdparty.com/customer1.js
Ajer
CDN Node
Origin
HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
http://www.3rdparty.com/customer1.js
Ajer
CDN Node
Origin
HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Tue, 12 May 2015 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
http://www.3rdparty.com/customer1.js
Ajer
CDN Node
Origin http://www.3rdparty.com/customer1.js
(Ajer 1 min)
Ajer
CDN Node
Origin http://www.3rdparty.com/customer1.js
(Ajer 1 min)
This happens many many 3mes! (many many happy visitors!)
Ajer
CDN Node
Origin
Customer1 changes config
(Ajer 1 min)
Ajer
CDN Node
Origin PURGE customer1.js
Customer1 changes config
(Ajer 1 min)
Ajer
CDN Node
Origin PURGE customer1.js
Customer1 changes config
(Ajer 1 min)
Ajer
CDN Node
Origin http://www.3rdparty.com/customer1.js
(Ajer 1 min)
Ajer
CDN Node
Origin http://www.3rdparty.com/customer1.js
HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Wed, 24 Sep 2014 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
(Ajer 1 min)
Ajer
CDN Node
Origin
HTTP/1.1 200 OKCache-Control: max-age=60, s-maxage=2592000Last-Modified: Wed, 24 Sep 2014 19:51:30 GMTContent-Type: application/javascriptDate: Thu, 25 Sep 2014 12:22:20 GMTServer: ApacheContent-Length: 7835
http://www.3rdparty.com/customer1.js
(Ajer 1 min)
Control
Visibility
Cacheability
Programmability
• Granular Invalida/on API • Configura/on API • Logic at the Edge • All must be instantaneous and real /me
Control at the Edge
• Moving app logic to the edge • Ex: VCL (Varnish Configura/on Language) – Script-‐like configura/on for func/onality at the edge
• Any mechanism offered by CDN to let you do stuff at the edge is good
Logic at the Edge: Examples • Generate content at the edge • HTTP header manipula/on • Origin selec/on • Caching rules • Geo-‐IP rules • Forcing SSL • Serving stale content
– stale-‐while-‐revalidate, stale-‐while-‐error • Etc…
Example: Edge-‐generated Content
JSON Data Center ID
CDN Node
Origin http://www.site.com/which_datacenter.js
JSON Data Center ID
CDN Node
Origin
{ ‘datacenter’ : ‘LAX’ }
http://www.site.com/which_datacenter.js
VCL Snippet
JSONP Geo IP
CDN Node
Origin
myFunction({ “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237”});
http://www.site.com/geo_ip.js?jsonp=myFunction
JSONP Geo IP
CDN Node
Origin
myFunction({ “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237”});
http://www.site.com/geo_ip.js?jsonp=myFunction
JSONP Geo IP
CDN Node
Origin
somethingElse({ “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237”});
http://www.site.com/geo_ip.js?jsonp=somethingElse
Control
Visibility
Cacheability
Real-‐3me Analy3cs
• Stats API • Network stats • HTTP stats (status codes, etc) • Caching stats (hits, misses, errors, etc) • All have to be real-‐/me – Historic data is also a must for long term stats analysis
Logging
• Real /me! • Log streaming – To any logging des/na/on endpoint – Ex: S3, FTP, syslog, etc
Example: Beacon Termina3on at the Edge
Before
CDN Node
Origin
Log Analysis
http://collector.site.com/beacon.img?a=1&b=2&c=3
Before
CDN Node
Origin
Log Analysis
http://collector.site.com/beacon.img?a=1&b=2&c=3
Before
CDN Node
Origin
Log Analysis
HTTP/1.1 200 OKPragma: no-cacheExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35
http://collector.site.com/beacon.img?a=1&b=2&c=3
Before
CDN Node
Origin
Log Analysis
HTTP/1.1 200 OKPragma: no-cacheExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35
http://collector.site.com/beacon.img?a=1&b=2&c=3
Ajer
CDN Node
Origin http://collector.site.com/beacon.img?a=1&b=2&c=3
Ajer
CDN Node
Origin
HTTP/1.1 200 OKPragma: no-cacheExpires: Wed, 19 Apr 2000 11:43:00 GMTCache-Control: no-cache, no-storeLast-Modified: Wed, 21 Jan 2004 19:51:30 GMTContent-Type: image/gifDate: Fri, 20 Jun 2014 12:22:20 GMTServer: ApacheContent-Length: 35
http://collector.site.com/beacon.img?a=1&b=2&c=3
Ajer
CDN Node
Origin
HTTP/1.1 204 No ContentDate: Sat, 21 Jun 2014 23:21:12 GMTServer: Awesome ServerContent-Length: 0
http://collector.site.com/beacon.img?a=1&b=2&c=3
Ajer
CDN Node
Origin
Syslog / S3 / FTP/etc
http://collector.site.com/beacon.img?a=1&b=2&c=3
Summary
• CDNs should be an extension of your applica/ons: – Flexible caching, predictable edge caching, and instant purging to un-‐cache!
– Control at the edge – Real-‐/me visibility and analy/cs
• Full real-‐/me programma/c interface is a must