Public Deployment of Cooperative Bug Isolation
Ben Liblit, Mayur Naik, Alice Zheng, Alex Aiken, and Michael Jordan
UC Berkeley, Stanford University and University of Wisconsin (pending)
Our Goal: Measure Reality
• We measure bridges, airplanes, cars…– Where is ight data recorder for software?
• Users are a vast, untapped resource– 96,000 new Kazaa users during this workshop– Users know what matters most
• Opportunity for reality-directed debugging– Implicit bug triage for an imperfect world
Bug Isolation Architecture
ProgramSource
Compiler
Sampler
Guesses
ShippingApplication
Pro le& /
€ƒƒ
€‚Statistical
Debugging
Top bugs withlikely causes
Predicates on Program Behavior
• Guess what might be interesting– Branches: Left? Right?– Function returns: Negative? Zero? Positive?– Pairs of variables: Less? Equal? Greater?– Reference counts: Alive? Dead?
• Count how often guesses are true
• Feedback: vector of counts + outcome label
Sampling the Bernoulli Way
• “Next sample” countdown– Geometric distribution
• Split into acyclic regions– Finite threshold weight
Sampling the Bernoulli Way
• “Next sample” countdown– Geometric distribution
• Split into acyclic regions– Finite threshold weight
1
2 1
1
1
2
3
4
Sampling the Bernoulli Way
• “Next sample” countdown– Geometric distribution
• Split into acyclic regions– Finite threshold weight
• Clone acyclic regions– “Fast” & “slow” variants
– Choose at run time
• Result:– Subset of dynamic behavior
– Statistically fair sample
>4?
Multithreaded Programs
• Global next-sample countdown– High contention, small footprint– Want to use registers for performanceThread-local: one countdown per thread
• Global predicate counters– Low contention, large footprintOptimistic atomic increment
Multi-Module Programs
• Forget about global static analysis– Plug-ins, shared libraries– Instrumented & uninstrumented code
• Self-management at compile time– Locally derive identifying object signature– Embed static site information within object le
• Self-management at run time– Report feedback state on normal object unload– Signal handlers walk global object registry
Native Compiler Integration
• Instrumentor must mimic native compiler– You don’t have time to port & annotate by hand
• Our approach: source-to-source, then native
• Hooks for GCC:– Stage wrapping via scripts– Flag management via spec les
ProgramSource
Compiler
Sampler
Guesses
ShippingApplication
CompilerTool Chain
Keeping the User In Control
Public Deployment, To Date
0
200
400
600
800
1000
1200
1400
Evolu
tion
Gaim
GIM
P
Gnum
eric
Nautil
us
Rhythm
box
Rep
orts
Rec
eive
d
GoodErrorCrash
Public Deployment, To Date
0%
20%
40%
60%
80%
100%
Evolu
tion
Gaim
GIM
P
Gnum
eric
Nautil
us
Rhythm
box
GoodErrorCrash
Sneak Peak: Data Exploration
Conclusions
• Public deployment is challenging– Real world code pushes tools to their limits– Large user communities take time to build
• But the results are worth it:
“Thanks to Ben Liblit and the CooperativeBug Isolation Project, this version of
Rhythmbox should be the most stable yet.”
The Cooperative Bug Isolation Projecthttp://www.cs.berkeley.edu/~liblit/sampler/
Join the Cause!