12 Nov 0712 Nov 07 CS 622CS 622 11

DHTTP: An Efficient and Cache-DHTTP: An Efficient and Cache-Friendly TransferFriendly Transfer

Protocol for the WebProtocol for the Web

By Michael Rabinovich and Hua WangBy Michael Rabinovich and Hua Wang

Presented by Jerry UseryPresented by Jerry Usery

12 Nov 0712 Nov 07 CS 622CS 622 22

OverviewOverview

• IntroductionIntroduction• DHTTP protocolDHTTP protocol• Implications of DHTTPImplications of DHTTP• Server designServer design• Performance analysisPerformance analysis• Future workFuture work• SummarySummary

12 Nov 0712 Nov 07 CS 622CS 622 33

IntroductionIntroduction

• Two issues addressedTwo issues addressed 1) Violation of end-to-end principle by interception 1) Violation of end-to-end principle by interception caches (Web proxies)caches (Web proxies)

• Impersonation of origin serversImpersonation of origin servers• Web interactions may be disruptedWeb interactions may be disrupted

2) Performance implications of client-initiated TCP 2) Performance implications of client-initiated TCP (Transmission Control Protocol) as transport (Transmission Control Protocol) as transport protocolprotocol

• HTTP generally conceived as file transfer protocolHTTP generally conceived as file transfer protocol• TCP connection overhead, persistence & pipeliningTCP connection overhead, persistence & pipelining

12 Nov 0712 Nov 07 CS 622CS 622 44

IntroductionIntroductionInterception CacheInterception Cache

12 Nov 0712 Nov 07 CS 622CS 622 55

IntroductionIntroductionPerformance ImplicationsPerformance Implications

• TCP connection overhead, persistence & TCP connection overhead, persistence & pipelining penaltiespipelining penalties

• Persistent connectionsPersistent connections• Degradation of throughput and increasing connectionsDegradation of throughput and increasing connections

• Pipelined transmissionsPipelined transmissions• Server must maintain connections, send responses in Server must maintain connections, send responses in orderorder• Head-of-line delays can occur with slow responsesHead-of-line delays can occur with slow responses

12 Nov 0712 Nov 07 CS 622CS 622 66

IntroductionIntroductionMain Ideas of DHTTPMain Ideas of DHTTP

• Two main ideas of DHTTP (Dual-Transport Two main ideas of DHTTP (Dual-Transport HTTP protocol)HTTP protocol)

1) Split Web traffic between UDP (User Datagram 1) Split Web traffic between UDP (User Datagram Protocol) and TCPProtocol) and TCP

• Client typically sends requests via UDPClient typically sends requests via UDP• Server sends response via UDP or TCPServer sends response via UDP or TCP

• Response sizeResponse size• Network conditionsNetwork conditions

2) Server establishes TCP connection to client2) Server establishes TCP connection to client

12 Nov 0712 Nov 07 CS 622CS 622 77

IntroductionIntroductionUDP vs TCPUDP vs TCP

• UDP use for short responsesUDP use for short responses• Reduced number of open server connectionsReduced number of open server connections• Reduced number of TCP connection setupsReduced number of TCP connection setups

• DHTTP benefitsDHTTP benefits• Improves client latencyImproves client latency

• Fewer Web interactions wait for connectionsFewer Web interactions wait for connections• Increases server capacityIncreases server capacity

• Servers manage fewer open connectionsServers manage fewer open connections• Remaining TCP connections reserved for larger objectsRemaining TCP connections reserved for larger objects

• Improved utilization of TCP connectionsImproved utilization of TCP connections• Ordering constraints of pipelining doesn’t existOrdering constraints of pipelining doesn’t exist

12 Nov 0712 Nov 07 CS 622CS 622 88

IntroductionIntroductionDHTTP Server-established TCP connectionsDHTTP Server-established TCP connections

• TCP connection client/server roles reversedTCP connection client/server roles reversed• Some firewall implications (existing Some firewall implications (existing countermeasures suffice)countermeasures suffice)• BenefitsBenefits

• Retains end-to-end Internet principleRetains end-to-end Internet principle• True interception cache IP address useTrue interception cache IP address use• Allows arbitrary deployment of interception cachesAllows arbitrary deployment of interception caches

• Server-initiated TCP reduces message round trips Server-initiated TCP reduces message round trips (even with initial UDP request)(even with initial UDP request)• Message round trips decreases with UDPMessage round trips decreases with UDP• Bottleneck process removed (server-accepted TCP Bottleneck process removed (server-accepted TCP connections)connections)

12 Nov 0712 Nov 07 CS 622CS 622 99

DHTTP ProtocolDHTTP Protocol

• Web clients and servers listen on two portsWeb clients and servers listen on two ports• One UDP, one TCPOne UDP, one TCP

• Servers use well-known ports (or URL-specified)Servers use well-known ports (or URL-specified)• Clients use ephemeral ports (short-lived, per Clients use ephemeral ports (short-lived, per download)download)• Two channels exist between client and serverTwo channels exist between client and server

• UDP used for requests below 1460 bytesUDP used for requests below 1460 bytes• Most HTTP requests metMost HTTP requests met

• Server opens TCP connection or reuses open Server opens TCP connection or reuses open one for larger messagesone for larger messages

• If TCP request sent by client, server may use itIf TCP request sent by client, server may use it

12 Nov 0712 Nov 07 CS 622CS 622 1010

DHTTP ProtocolDHTTP ProtocolMessage ExchangeMessage Exchange

(a) HTTP (b) DHTTP over UDP (c) DHTTP over TCP(a) HTTP (b) DHTTP over UDP (c) DHTTP over TCP

12 Nov 0712 Nov 07 CS 622CS 622 1111

DHTTP ProtocolDHTTP Protocol Message Format DescriptionMessage Format Description

• Response may arrive to client out of orderResponse may arrive to client out of order• Client assigns request ID, then matches itClient assigns request ID, then matches it

• Client advises server of listening port numbersClient advises server of listening port numbers• Channel’s source port number included in IP header Channel’s source port number included in IP header alreadyalready

• UDP request contains client’s TCP port numberUDP request contains client’s TCP port number• TCP request contains client’s UDP port numberTCP request contains client’s UDP port number

• Flag field used for duplicate (resend) requestsFlag field used for duplicate (resend) requests

12 Nov 0712 Nov 07 CS 622CS 622 1212

DHTTP ProtocolDHTTP ProtocolMessage FormatMessage Format

(a) – Request (b) – Response(a) – Request (b) – Response

12 Nov 0712 Nov 07 CS 622CS 622 1313

DHTTP ProtocolDHTTP Protocol

• ReliabilityReliability• DHTTP stipulates that a client may resend UDP requestsDHTTP stipulates that a client may resend UDP requests

• Lessens overheadLessens overhead• Fighting denial-of-service attacks beyond paper’s scopeFighting denial-of-service attacks beyond paper’s scope

• Nonidempotent (e-commerce, etc) requestsNonidempotent (e-commerce, etc) requests• Delegated to TCP channelDelegated to TCP channel

• Congestion controlCongestion control• DHTTP responds to any resent client requests via TCPDHTTP responds to any resent client requests via TCP

• Aids packet loss issuesAids packet loss issues• In congested Internet experiments, only 6% of responses sent In congested Internet experiments, only 6% of responses sent via UDPvia UDP

12 Nov 0712 Nov 07 CS 622CS 622 1414

DHTTP ProtocolDHTTP ProtocolChannel Selection/AlgorithmChannel Selection/Algorithm

• Response sizeResponse size• Network conditionNetwork condition

• Server maintains “fresh requests” and “resent requests” countersServer maintains “fresh requests” and “resent requests” counters• Loss threshold parameter “Loss threshold parameter “LL” (currently 1%)” (currently 1%)• Size threshold parameter “Size threshold parameter “SS” (1460 bytes, default)” (1460 bytes, default)

• AlgorithmAlgorithm1) Choose TCP for all large responses, i.e., whose size exceeds 1) Choose TCP for all large responses, i.e., whose size exceeds SS, as well , as well as for all resent requests.as for all resent requests.

2) If the ratio of resent request counter to fresh request counter exceeds 2) If the ratio of resent request counter to fresh request counter exceeds LL, , enter a “high-loss” mode, else enter a “low-loss” mode.enter a “high-loss” mode, else enter a “low-loss” mode.

3) In the low-loss mode, choose UDP for all small responses, i.e., those 3) In the low-loss mode, choose UDP for all small responses, i.e., those below the size threshold below the size threshold SS..

4) In the high-loss mode, choose TCP for the 1-4) In the high-loss mode, choose TCP for the 1-L L fraction of small fraction of small responses and UDP for the remaining responses and UDP for the remaining LL small responses. small responses.

12 Nov 0712 Nov 07 CS 622CS 622 1515

DHTTP ImplicationsDHTTP Implications• DHTTP and Interception CachesDHTTP and Interception Caches

• DHTTP interception caches intercept UDP DHTTP interception caches intercept UDP requests – TCP requests pass throughrequests – TCP requests pass through• Client is aware it speaks with cacheClient is aware it speaks with cache

• Retains end-to-end principle (even with caching)Retains end-to-end principle (even with caching)

• DHTTP uses UDP channel for short requestsDHTTP uses UDP channel for short requests• Reduces TCP setup costs, connections, Reduces TCP setup costs, connections, response timeresponse time

• DHTTP allows client or server to unilaterally DHTTP allows client or server to unilaterally close TCP connectionclose TCP connection

• Ensures no in-transit data existsEnsures no in-transit data exists

12 Nov 0712 Nov 07 CS 622CS 622 1616

DHTTP ImplicationsDHTTP ImplicationsServer Design DescriptionServer Design Description

• Master processMaster process• Accepts incoming requestsAccepts incoming requests• Three threadsThree threads

• ReadRequest threadReadRequest thread• Reads incoming UDP request, copies into global bufferReads incoming UDP request, copies into global buffer

• PipeRequest threadPipeRequest thread• Pipes global buffer requests to worker processesPipes global buffer requests to worker processes• Global buffer moves requests ASAP so UDP port buffer Global buffer moves requests ASAP so UDP port buffer will not fill up, requests don’t get droppedwill not fill up, requests don’t get dropped

• Maintenance thread checks worker process status Maintenance thread checks worker process status every secondevery second

• If too few idle worker processes, it forks new onesIf too few idle worker processes, it forks new ones• If too many, it kills someIf too many, it kills some

12 Nov 0712 Nov 07 CS 622CS 622 1717

DHTTP ImplicationsDHTTP ImplicationsServer Design DescriptionServer Design Description

• Worker processesWorker processes• Execute individual requests, respond to clientsExecute individual requests, respond to clients

• Reads pipe requests, generates response, chooses Reads pipe requests, generates response, chooses UDP or TCP, sends responseUDP or TCP, sends response

• If TCP connection chosen and one exists to client, it If TCP connection chosen and one exists to client, it reuses itreuses it• One TCP connection per client, possibly many clientsOne TCP connection per client, possibly many clients

• Includes Timeout threadIncludes Timeout thread• Closes TCP connections idle > timeout periodCloses TCP connections idle > timeout period

12 Nov 0712 Nov 07 CS 622CS 622 1818

DHTTP ImplicationsDHTTP ImplicationsServer DesignServer Design

Modified Apache 1.3.6 Web serverModified Apache 1.3.6 Web server

12 Nov 0712 Nov 07 CS 622CS 622 1919

Performance AnalysisPerformance Analysis

• Three-pronged performance studyThree-pronged performance study• Trace-driven simulationTrace-driven simulation

• Measured number/utilization of TCP connections Measured number/utilization of TCP connections experienced by server (HTTP and DHTTP)experienced by server (HTTP and DHTTP)

• Benchmarked Apache HTTP/DHTTP servers with Benchmarked Apache HTTP/DHTTP servers with clients on same LANclients on same LAN

• Compared peak performance and scalabilityCompared peak performance and scalability

• Tested both servers in WAN environment with Tested both servers in WAN environment with congested Internet connectioncongested Internet connection

12 Nov 0712 Nov 07 CS 622CS 622 2020

Performance AnalysisPerformance AnalysisSimulationSimulation

• Used access log from AT&T low-end hosting Used access log from AT&T low-end hosting servicesservices

• Three month durationThree month duration• Contained over 100 million accessesContained over 100 million accesses

• Average response size of 13 KAverage response size of 13 K

• Two threshold values usedTwo threshold values used• 4 K, optimistic value for many small Web responses4 K, optimistic value for many small Web responses• 1460 bytes, conservative value, one Ethernet MTU 1460 bytes, conservative value, one Ethernet MTU (maximum transfer unit)(maximum transfer unit)

12 Nov 0712 Nov 07 CS 622CS 622 2121

Performance AnalysisPerformance AnalysisSimulationSimulation

Number of TCP connections at a server with three connections per clientNumber of TCP connections at a server with three connections per client

12 Nov 0712 Nov 07 CS 622CS 622 2222

Performance AnalysisPerformance AnalysisSimulationSimulation

Connection utilization with three connections per clientConnection utilization with three connections per client

12 Nov 0712 Nov 07 CS 622CS 622 2323

Performance AnalysisPerformance AnalysisSimulationSimulation

Number of TCP connections at a server with one connection per clientNumber of TCP connections at a server with one connection per client

12 Nov 0712 Nov 07 CS 622CS 622 2424

Performance AnalysisPerformance AnalysisSimulationSimulation

Connection utilization with one connection per clientConnection utilization with one connection per client

12 Nov 0712 Nov 07 CS 622CS 622 2525

Performance AnalysisPerformance AnalysisPrototyped Testing ResultsPrototyped Testing Results

Apache performance (bottleneck at server)Apache performance (bottleneck at server)

(a) Throughput with three connections per client(a) Throughput with three connections per client

(b) Latency with three connections per client(b) Latency with three connections per client

12 Nov 0712 Nov 07 CS 622CS 622 2626

Performance AnalysisPerformance AnalysisPrototyped Testing ResultsPrototyped Testing Results

Apache performance (bottleneck at server)Apache performance (bottleneck at server)

(c) Throughput with one connection per client(c) Throughput with one connection per client

(d) Latency with one connection per client(d) Latency with one connection per client

12 Nov 0712 Nov 07 CS 622CS 622 2727

Performance AnalysisPerformance AnalysisPrototyped Testing ResultsPrototyped Testing Results

DHTTP server performance (bottleneck at server)DHTTP server performance (bottleneck at server)

(a) Throughput with three connections per client(a) Throughput with three connections per client

(b) Latency with three connections per client(b) Latency with three connections per client

12 Nov 0712 Nov 07 CS 622CS 622 2828

Performance AnalysisPerformance AnalysisPrototyped Testing ResultsPrototyped Testing Results

DHTTP server performance (bottleneck at server)DHTTP server performance (bottleneck at server)

(c) Throughput with one connection per client(c) Throughput with one connection per client

(d) Latency with one connection per client(d) Latency with one connection per client

12 Nov 0712 Nov 07 CS 622CS 622 2929

Performance AnalysisPerformance AnalysisPerformance ComparisonPerformance Comparison

Comparison of Apache and DHTTP serversComparison of Apache and DHTTP servers

(a) Throughput (b) Latency(a) Throughput (b) Latency

12 Nov 0712 Nov 07 CS 622CS 622 3030

Performance AnalysisPerformance AnalysisPerformance ComparisonPerformance Comparison

Apache and DHTTP server performance under network congestionApache and DHTTP server performance under network congestion

(a) Throughput (b) Latency(a) Throughput (b) Latency

12 Nov 0712 Nov 07 CS 622CS 622 3131

Performance AnalysisPerformance AnalysisPerformance ComparisonPerformance Comparison

Effectiveness of congestion detection in DHTTP serverEffectiveness of congestion detection in DHTTP server

12 Nov 0712 Nov 07 CS 622CS 622 3232

Future WorkFuture Work

• Dividing response among several UDP Dividing response among several UDP packetspackets

• Likely allows higher size thresholdsLikely allows higher size thresholds

• Building native support for nonidempotent Building native support for nonidempotent requestsrequests• Investigation of finer algorithms that track Investigation of finer algorithms that track network conditions at the client and/or subnetnetwork conditions at the client and/or subnet• Dynamic policies for size threshold selectionDynamic policies for size threshold selection

• High-loss vs low-loss environmentsHigh-loss vs low-loss environments

12 Nov 0712 Nov 07 CS 622CS 622 3333

SummarySummary

• IntroductionIntroduction• DHTTP protocolDHTTP protocol• Implications of DHTTPImplications of DHTTP• Server designServer design• Performance analysisPerformance analysis• Future workFuture work• SummarySummary