A Provider-side View ofWeb Search Response Time

YINGYING CHEN, RATUL MAHAJAN,

BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA)

MICROSOFT

Web services are the dominant way to find and access information

Web service latency is critical to service providers as well

Google

Bing

revenue-20%

Latency+2 sec

revenue-4.3%

Latency+0.5 sec

Understanding SRT behavior is challenging

t

300+tS

RT

(m

s)

M T W Th F S Su

peak off-peak

200+t

t

SR

T (

ms)

Our work

Explaining systemic SRT variation

Identify SRT anomalies

Root cause localization

Client- and server-side instrumentation

HTML header

Brand header

BoP scriptsQuery results

Embedded images

query

𝑇 𝑓𝑠 𝑇 𝑓𝑐

𝑇 h𝑒𝑎𝑑

𝑇 𝑏𝑟𝑎𝑛𝑑

𝑇 h𝑖𝑛𝑡𝑐 𝑘1

𝑇 𝑟𝑒𝑠𝐻𝑇𝑀𝐿

𝑇 𝐵𝑂𝑃

𝑇 h𝑖𝑛𝑡𝑐 𝑘2

𝑇 𝑒𝑚𝑏𝑒𝑑

𝑇 𝑟𝑒𝑓

𝑇 𝑠𝑐𝑟𝑖𝑝𝑡

𝑇 𝑠𝑐

𝑇 𝑡𝑐

on-load

Referenced content

Impact Factors of SRT

𝑇 𝑓𝑠

network browser queryserver

𝑇 h𝑒𝑎𝑑𝑇 𝑏𝑟𝑎𝑛𝑑𝑇 h𝑖𝑛𝑡𝑐 𝑘1𝑇 𝑟𝑒𝑠𝐻𝑇𝑀𝐿𝑇 𝐵𝑂𝑃𝑇 h𝑖𝑛𝑡𝑐 𝑘2𝑇 𝑟𝑒𝑓𝑇 𝑠𝑐𝑟𝑖𝑝𝑡𝑇 𝑛𝑒𝑡𝑇 𝑠𝑐𝑇 𝑓𝑐𝑇 𝑒𝑚𝑏𝑒𝑑 𝑇 𝑡𝑐

Primary factors of SRT variation

Apply Analysis of Variance (ANOVA) on the time intervals

ƞ

SRT variance

Variance explained by time interval k

Unexplainedvariance

Primary factors: network characteristics, browser speed, query type Server-side processing time has a relatively small impact

network browser queryserver

𝑇 h𝑒𝑎𝑑𝑇 𝑟𝑒𝑠𝐻𝑇𝑀𝐿𝑇 𝐵𝑂𝑃𝑇 𝑟𝑒𝑓 𝑇 𝑠𝑐𝑟𝑖𝑝𝑡𝑇 𝑛𝑒𝑡 𝑇 𝑠𝑐𝑇 𝑓𝑐 𝑇 𝑡𝑐

Exp

lain

ed

vari

an

ce (

%) 6

0

40

20

0

Variation in network characteristics

RT

T

Explaining network variations

Residential networks send a higher fraction of queries during off-peak hours than peak hours

Residential networks are slower

residential enterprise

RTT

(ms)

25%

1.25t

t

Residential networks are slower

Residential networks send a higher fraction of queries during off-peak hours than peak hours

residential unknownenterprise

Variation in query type

Impact of query on SRT Server processing timeRichness of response page

Measure: number of image

Explaining query type variation

Peak hours Off-peak hours

Browser variations

Two most popular browsers: X(35%), Y(40%) Browser-Y sends a higher fraction of queries during off-peak hours Browser-Y has better performance

Browser-X Browser-Y

Javascript exec time

82%

1.82t

t

Summarizing systemic SRT variation Server: Little impact

Network: Poorer during off-peak hours

Query: Richer during off-peak hours

Browser: Faster during off-peak hours

Detecting anomalous SRT variations

Challenge: interference from systemic variations

Week-over-Week (WoW) approach

+ Seasonality + Noise

Comparison with approaches that do not account for systemic variations

WoW One Gaussian model of

SRT

Change point

detection

False negative 10% 35% 40%

False positive 7% 17% 19%

Conclusions

Understanding SRT is challengingChanges in user demographics lead to systemic

variations in SRT

Debugging SRT is challenging Must factor out systemic variations

Implications

Performance monitoringShould understand performance-equivalent classes

Performance managementShould consider the impact of network, browser, and

query

Performance debugging End-to-end measures are tainted by user behavior

changes

Questions?