1
Leif Hedstrom
Content Delivery- in a DevOps World
Chief Architect, Cisco Cloud Services
July 8, 2012
2
Who am I?• One of the drivers to Open Source Traffic Server• Committer for Apache Traffic Server• VP of Apache Traffic Server PMC• ASF member• Chief Architect, Cisco Cloud Services
@zwoop
+zwoop
3
Why DevOps? A short story
4
It’s a disaster!
http://www.flickr.com/photos/historyinanhour/4775644390/
5
Smart people trying to figure it out…
6
Still a disaster, oh the humanity!
http://www.flickr.com/photos/historyinanhour/4775644390/
7
In the mean time …
http://people.apache.org/~weijin/weijin.jpg
8
Small fix, big results
index 30ed932..b74defa 100644--- a/iocore/net/UnixNetVConnection.cc+++ b/iocore/net/UnixNetVConnection.cc@@ -574,6 +574,9 @@ UnixNetVConnection::do_io_close(int alerrno /* = -1 */ ) write.vio.nbytes = 0; write.vio.op = VIO::NONE; + EThread *t = this_ethread();+ bool close_inline = !recursion && nh->mutex->thread_holding == t;+ INK_WRITE_MEMORY_BARRIER; if (alerrno && alerrno != -1) this->lerrno = alerrno;@@ -582,11 +585,8 @@ UnixNetVConnection::do_io_close(int alerrno /* = -1 */ ) else closed = -1; - if (!recursion) {- EThread *t = this_ethread();- if (nh->mutex->thread_holding == t)- close_UnixNetVConnection(this, t);- }+ if (close_inline)+ close_UnixNetVConnection(this, t); }
9
I’m happy!
http://www.flickr.com/photos/mhedstrom/6763754839/
10
DevOps reward (well, maybe … )
11
So what is DevOps?• To me, it’s a collaborative, community driven
way of developing software, services and IT solutions.
• Tear down the borders (and rifts!) between Development and Operations.
• Thrives on healthy Open Source communities.
12
13
14
QPS is no longer a meaningful metric
15
…Other things matters…
16
… and don’t forget features• Handling thousands of concurrent users• Extensible (e.g. plugins, or scripts)• Correctness• HTTP/1.1
• Keep-Alive• Chunking• etc.
17
When all you have is a hammer …
http://www.flickr.com/photos/aai/6936657289/
18
Evaluate the features that matters to you!ATS HAproxy ngin
xSquid Varnish mod_proxy
Worker Threads Y N N N Y Y
Multi-Process N Y Y N Y YEvent-driven Y Y Y Y sometimes YPlugin APIs Y N Y part Y YForward Proxy Y N N Y N YReverse Proxy Y Y Y Y Y YTransp. Proxy Y Y N Y N NLoad Balancer weak Y Y Y Y YPersistent Cache Y N Y Y sorta YESI Y N N Y kinda NICP Y N N Y N NKeep-Alive Y N Y Y Y YSSL Y N Y Y N YPipeline Y N Y Y N Y
19
Forward proxy
20
Transparent (or intercepting) proxy
21
Reverse proxy
22
23
Enemies of all content delivery
Server latency3-Way handshake
Congestion controlDNS lookups
(Page flow)
24
Enemies of all content delivery
Server latency3-Way handshake
Congestion controlDNS lookups
(Page flow)
25
TCP 3-way handshake
26
Enemies of all content delivery
Server latency3-Way handshake
Congestion controlDNS lookups
(Page flow)
27
Congestion control
28
Enemies of all content delivery
Server latency3-Way handshake
Congestion controlDNS lookups
(Page flow)
29
http://news.example.com/http://finance.example.com/http://groups.example.com/
30
http://news.example.com/http://finance.example.com/http://groups.example.com/
vshttp://www.example.com/newshttp://www.example.com/financehttp://www.example.com/groups
31
gzip matters
http://www.flickr.com/photos/marcovdz/4520986339/
32
Reduce your round-trips!
plain gzipTheoretical limit on 20Mbps 78ms 15msLow latency (17ms RTT) 182ms 105msHigh latency (159ms RTT) 1,340ms 840ms
33
Congestion control
34
Real example (200KB uncompressed)
200KB (8 RTs) 40KB (5 RTs)17ms RTT 136ms 85ms159ms RTT 1,272ms 795ms
35
Keep-alive• A primary tool to fight 3-way handshake
latency• Helps against congestion control too
• But not always… Slow starts begins again after some idle time
36
Old Congestion Window Size Algorithm:
IW = min (4*SMSS, max (2*SMSS, 4380 bytes))
(RFC2581)
Typically, 2-4 packets
37
New Linux Initial Congestion Window Size Algorithm:
10
38
SPDY• Addresses many of the problems related to
latency• Not a standard (yet)• SSL requirements, and other quirks, can be a
hurdle
39
40
Some history
41
The Concurrency problem
42
Solution 1: Multi-threading
43
Problems with Multi-threading
44
Solution 2: Event Processing
45
Problems with event processing
46http://www.flickr.com/photos/stuartpilbrow/3345896050/
47
Traffic Server model
48
Four Horseman of Bad Performance• Data copies• Context Switches• Memory allocation• Lock contention
49http://www.flickr.com/photos/simpologist/16734948/
50
http://www.flickr.com/photos/airosan/2232394342/
51
http://trafficserver.apache.org/
Thank you.