A Delta-Oriented Approach to Support the Safe Reuse of Black-Box Code Rewriters
The 17th International Conference on Software Reuse, May 21-23, 2018
Benjamin Benni, Sébastien Mosser, Naouel Moha, Michel Riveill
�1
753,347 commits
203 open PR, 324 closed
~ 3 (massive) commits per day!2 Context Contribution Validation Conclusion
change
change
changechangechange
change
!3
change
change
changechangechange
change
API change Documentation change
New features Standard may evolve
Driver change Bug fixing
…
!4
Massive codebase
Recurring problems in the changes
From manual to automatic rewritings
!5
… x = kmalloc(sizeof(*a), 0); memset(a, 0, sizeof(*a)) …
Change extract
kzalloc has been added for this very particular reason
Don’t want to reject the whole request for this Automatic check? Automatic rewriting?
!6
… x = kmalloc(sizeof(*a), 0); memset(a, 0, sizeof(*a)) …
Desired mechanism
… x = kzalloc(sizeof(*a), 0); …
This is automatic, and reusable. ie: input-independent and self-
sufficient
!7
… x = kmalloc(sizeof(*a), 0); memset(a, 0, sizeof(*a)) …
Change extract
…kernel developers implemented so-called semantic patch for this.
We have code rewriters to rewrite this…
We want automatic rewritings
!8
Code rewriter definition
P : Program ↦ Program
!9
Black-box Code rewriters
P : Program ↦ Program
We can’t open it and see what is inside, …
!10
Reuse of black-box code rewriters
P : Program ↦Program
We want to reuse those rewriters, and apply them to every incoming
changes!11
1) Does what it is supposed to do when applied with others
(side effect)
2) No syntactic issues between rewriters (interactions, overlaps)
Safe-reuse of black-box code rewriters
!12
http://coccinelle.lip6.fr/Lawal, Muller and al.!13
Rewriters
Base-code exampleSo called “ semantic patches”
!14
Rewriters
Base-code example
!15
Rewriters
Base-code example
!16
Rewriters
Base-code example
!17
Rewriters
Base-code example
!18
Rewriters
Base-code example
!19
Rewriters
Base-code example
!20
Rewriters
Base-code example
!21
Rewriters
Base-code example
Toy example
!22
!23
!24
!25
!26
≡
!27
≡
!28
≡
Initial intent not respected because of interactions
!29
≡
!30
≡
!31
≡
≡z
!32
≡
≡z
!33
≡
≡z
1) We can’t compute every possible sequences and analyse the outcome
2) Today, only the last application is “ensured” => silenced error
!34
1) Does what it is supposed to do when applied with others
(side effect)
2) No syntactic issues between rewriters (interactions, overlaps)
Safe-reuse of black-box code rewriters
!35 Context Contribution Validation Conclusion
If code rewriters are …
P : Program ↦ Program …We can’t do much
!36
P : Program ↦ Program’
Code rewriter 𝝆 is a pair (𝛗, 𝛘)
𝛗 : AST ↦ AST (the rewriter part)
𝛘 : AST x AST ↦ 𝔹 (the checking part - postcondition)
!37
P : Program ↦ Program’
Code rewriter 𝝆 is a pair (𝛗, 𝛘)
AST
Program Program
Reasoning
real-life
!38
P : Program ↦ Program’ 𝝆 = (𝛗, 𝛘)
Iso-functional
apply : AST x P<* ↦ AST
output = 𝛗(𝛗(input)) ; 𝛘 (output, input) holds
!39
P : Program ↦ Program’ 𝝆 = (𝛗, 𝛘)
seq : AST x P<* ↦ AST
output = 𝛗(𝛗(input)) ;
𝛘 (output, input) AND 𝛘 (output, input) holds
apply
Post-conditions check of all rewriters!40
≡
≡z
The seq operator allows us to outline an issue that was
silenced before
!41
P : Program ↦ Program’ 𝝆 = (𝛗, 𝛘) apply seq
P : Program ↦ Program’
We are still considering
What if…
!42
What if…
It exists ⊖, a diff function:
⊖(Program, Program) ↦ Delta<*
P : Program ↦ Program’ 𝝆 = (𝛗, 𝛘) apply seq
!43
P : Program ↦ Program’ 𝝆 = (𝛗, 𝛘) apply seq
⊖(Program, Program) ↦ Delta<*
output = 𝛗(input) Deltas = ⊖(input, output) input ⊕ Deltas = output
!44
P : Program ↦ Program’ 𝝆 = (𝛗, 𝛘) apply seq
output = 𝛗(input) Deltas = ⊖(input, output) input ⊕ Deltas => output
output = 𝛗(input) Deltas = ⊖(input, output) input ⊕ Deltas => output
Deltas ; Deltas
input ⊕ (Deltas ; Deltas) => output
𝛘 (output, input) AND 𝛘 (output, input) holds!45
P : Program ↦ Program’ 𝝆 = (𝛗, 𝛘) apply seq
Isolated applications of rewriters
iso : AST x P* ↦ AST
p , {𝝆1, …, 𝝆n} ↦ pisoDelta
!46
≡
≡z
The iso operator allows us to create a valid output program by ensuring isolations and delaying of rewriters applications and checking all post-
conditions, or outlining incompatible rewriters
!47
True for main languages (diff for Linux or Gumtree for Android)
P : Program ↦ Program’ 𝝆 = (𝛗, 𝛘) apply seq
⊖(Program, Program) ↦ Delta<*
iso
Hypothesis:
!48
Reduce conflict spaceoutput = 𝛗(input)
Deltas = ⊖(input, output) input ⊕ Deltas => output
output = 𝛗(input) Deltas = ⊖(input, output) input ⊕ Deltas => output
conflict? : Delta x Delta ↦ 𝔹
Eg: [setVisibility(method1, private), addStatement(…), …] [setVisibility(method1, public), addAttribute(…), …]
!49
Challenges
ΔΔ
AND ( 𝟀( M, M’ ), 𝟀( M, M’ ) )
conflict? (Δ , Δ)
apply ( Δ;Δ , M )M’
!50
Reduce conflict space
If isolated application failed, because a post condition is violated, start to analyse this very rewriter’s.
Incompatible application?
!51
Our contribution is not techno-specific
!52 Context Contribution Validation Conclusion
R-E-U-S-E
From patches in Coccinnelle Linux-kernel tool…
…to correction of power-consuming anti-patterns in Android app
!53
Correction of power-consuming anti-patterns in Android app
Collaboration with Naouel Moha
!54
Bad practises
More power consumption
Less battery-lifetime
Less Angry Birds!55
• Avoid internal getter/setter call (IGS) • Use static if you can (STA) • Use ArrayMap instead of HashMap (HMU)
Example of rewriters/bad practises
Geoffrey Hecht, Romain Rouvoy, Naouel Moha, and Laurence Duchien. Detecting antipatterns in android apps. In 2nd ACM International Conference on Mobile Software Engineering and Systems, MOBILESoft 2015, Florence, Italy, May 16- 17, 2015, pages 148–149, 2015.
!56
IGS
!57
• More rewriters • Have to work on any Android app
(closed world is over) • Evolving guidelines
A whole new world
!58
Code rewriters are so-called “Spoon processor” Off-the-shelf components
We want to safely reuse all of them
A whole new world
“ What happens when multiple processors have to be applied, ie, when multiple bad-practises
has been detected and must be fixed?”
“…”
!59
!60
!61
!62
≡≡
z
SAME
!63
Isolated application
No conflict!64
Isolated application
No conflict!65
Validation in the Android worldRunnerUp
(https://github.com/jonasoreland/runnerup)
4 rewriters: Lambda, GNP, HMU, IGS
4! = 24 sequences
iso worked fine and some seq don't.!66 Context Contribution Validation Conclusion
Outcome so far
1) Isolated applications of rewriters (iso)
2) Semantic checks (𝛘)
3) Legacy compatible (apply)
4) Conflits “un-silenced” (conflict?)
5) Good/bad sequences finding (seq)
6) Bonus: parallelisation!67 Context Contribution Validation Conclusion
Perspectives and future work
Empirical validation Quantitative validation
Go for another use-case: Git. Rewrite software based on diffs. The final
rewriter is human in case of conflicts.
!68
Thank you for your attention!
Benjamin Benni, Sébastien Mosser, Naouel Moha, Michel Riveill
A Delta-Oriented Approach to Support the Safe Reuse of Black-box Code Rewriters
i3s.unice.fr/~benni [email protected]
!69
Outcome, “summarised”
!70