Virtual Separation of Concerns

Christian Kästner

Photo (cc) Horia Varlan

Feature-Oriented

Product Lines

DatabaseEngine

Printer Firmware

LinuxKernel

7

Variability = Complexity

a unique configuration for every

person on this planet

33 featuresoptional, independent

320 features

more configurations than estimated

atoms in the universe

optional, independent

Correctness?

Maintenance?Comprehension?

static int _rep_queue_filedone(...)DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {

#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return (__db_no_queue_am(dbenv));

#elsedb_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;

#ifdef DIAGNOSTICDB_MSGBUF mb;

#endif...

Excerpt from Oracle’s Berkeley DB

Conditional Compilation

Objections / Criticism

“#ifdef considered harmful”“#ifdef hell”

Designed in the 70th and hardly evolved since

“preprocessor diagnostics are poor”

“is difficult to determine if the code being viewed is actually compiled into the system”

“programming errors are easy to make and difficult to detect”

“incomprehensible source texts”

“maintenance becomes a ‘hit or miss’ process”

“CPP makes maintenance difficult”

“source code rapidly becomes a maze”

Academia: Compositional Approaches

class Stack { void push(Object o) { elementData[size++] = o; } ...}

class Stack { void push(Object o) { elementData[size++] = o; } ...}

refines class Stack { void push(Object o) { Lock l = lock(o); Super.push(o); l.unlock(); } ...}

refines class Stack { void push(Object o) { Lock l = lock(o); Super.push(o); l.unlock(); } ...}

Base / Platform

Feature: Queue

Feature: Diagnostic

aspect Diagnostics { ...}

aspect Diagnostics { ...}

class Stack { void push(Object o) { Lock l = lock(o); elementData[size++] = o; l.unlock(); } ...}

class Stack { void push(Object o) { Lock l = lock(o); elementData[size++] = o; l.unlock(); } ...}

Composition

ModuleComponentsFrameworks, Plug-insFeature-Modules / Mixin Layers / …Aspects / Subjects, Hyper/J, Deltas

A CLOSER LOOK AT PREPROCESSORS

Separation of Concerns

Obfuscationclass Stack {

void push(Object o#ifdef SYNC

, Transaction txn#endif

) {if (o==null

#ifdef SYNC|| txn==null

#endif) return;

#ifdef SYNCLock l=txn.lock(o);

#endifelementData[size++] = o;

#ifdef SYNCl.unlock();

#endiffireStackChanged();

}}

Femto OS

Obfuscation

Error-Pronestatic int _rep_queue_filedone(...)

DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {

#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return

(__db_no_queue_am(dbenv));#else

db_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;

#ifdef DIAGNOSTICDB_MSGBUF mb;

#endif// over 100 lines of add. code

}#endif

#ifdef TABLESclass Table { void insert(Object data, Txn txn) { storage.set(data, txn.getLock()); }}#endifclass Storage {#ifdef WRITE boolean set(…) { ... }#endif}

Advantages• Easy to use

• “annotate and remove”• Already part of many languages / simple tools• Most developers already familiar

• Flexible / Expressive• Language-independent / Uniform

• Low Adoption Barrier• esp. with legacy code

IMPROVING TOOL SUPPORT

Views

[ICSE’08, ViSPLE’08]

View on Feature Eval

View on Configuration “Add and Eval”

[ICSE’08, ICSE‘10, EASE‘11]

FeatureCommander

Controlled

Experiments

Scaling? How Used?

Faster?

3

ERROR DETECTION / PREVENTION

Disciplined

#ifdef WORLDint main() {

printf(“Hello World”);}#endifint main() {#ifdef WORLD

printf(“Hello World”);#endif}

Annotations

Align with code structure

greet.c

printf

msg

VWORLD VBYEprintf

msg msg

main

ε ε

Align with variability model

Disciplined

#ifdef WORLDint main() {

printf(“Hello World”);}#endif

int main() {#ifdef DYNAMIC

if (enabled) {#endif

printf(“Hello World”);#ifdef DYNAIC

}#endif}

int main() {#ifdef WORLD

printf(“Hello World”);#endif}

AnnotationsUndisciplined

int a = #ifdef DYNAMIC

enabled? 3 : 4 +#endif

5;Align with code structure

Align with variability model

static int _rep_queue_filedone(...)DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {

#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return

(__db_no_queue_am(dbenv));#else

db_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;

#ifdef DIAGNOSTICDB_MSGBUF mb;

#endif// over 100 lines of add. code

}#endif

Variability-AwareParserType SystemStatic AnalysisBug FindingTestingModel CheckingTheorem Proving…

Conflicts

References

Pre

sence

Conditio

ns

true

true

WORLD

BYE

Reachability: pc(caller) -> pc(target)Conflicts: ¬(pc(def1) ˄ pc(def2))

true -> true

true -> (WORLD v BYE)

¬ (WORLD ˄ BYE)

Variability Model:

VM -> (true -> true)

VM -> (true -> (WORLD v BYE))

P

WORLD BYE

VM ->¬ (WORLD ˄ BYE)

40BYEWORLD

AST with Variability Information

Extended Lookup Mechanism

greet.c

printf

msg

VWORLD VBYEprintf

msg msg

main

ε ε

…

true -> true

true -> (WORLD v BYE)¬ (WORLD ˄ BYE)

Formalization: CFJ[ASE’08, TOSEM‘11]

Surface Complexity

Handling Complexity

SATProblem

InherentComplexity

Variability-Aware AnalysisType SystemStatic AnalysisBug FindingTestingModel CheckingTheorem Proving…

CIDE

http://fosd.de/cide

open source, fosd.net/cide

Automated Refactorings• Feature Module • Annotationen

class Stack {int[] data;

#ifdef UNDOint backup;void undo() {

/*...*/ }#endif#ifdef TOP

int top() { /*...*/ }#endif

void push(int o) {#ifdef UNDO

backup = top();#endif

/*...*/}int pop() {

/*...*/ }}

class Stack {int[] data;

#ifdef UNDOint backup;void undo() {

/*...*/ }#endif#ifdef TOP

int top() { /*...*/ }#endif

void push(int o) {#ifdef UNDO

backup = top();#endif

/*...*/}int pop() {

/*...*/ }}

class Stack {int[] data;void push(int o) { …

}int pop() {

/*...*/ }}

class Stack {int[] data;void push(int o) { …

}int pop() {

/*...*/ }}refines class Stack {

int top() { /*...*/ }}

refines class Stack {int top() {

/*...*/ }}refines class Stack {

int backup;void undo() {

/*...*/ }void push(int o) {

backup = top();original(v);

}}

refines class Stack {int backup;void undo() {

/*...*/ }void push(int o) {

backup = top();original(v);

}}

Base

Top

Undo

Automated Transformation

[GPCE’09]

THE REAL-WORLD CHALLENGE

Parse and Type check all configurations of the

entire Linux kernel

10,000 features, 6 million lines of C code

TypeChef

AST with Variability Information

Variability-AwareParsing

greet.c

printf

msg

VWORLD VBYEprintf

msg msg

main

ε ε

…

[OOPSLA‘11]

greet.c

printf

msg

VWORLD VBYE+ printf

+ msg + msg

+ main

ε ε

???

Macro expansion needed for parsing Alternative macros

Undisciplined annotations

Parsing Cwithout Preprocessing

Previous Solutions

Disciplined SubsetRequires Code Preparation

Heuristics and Partial AnalysisInaccurate, False Positives

Brute ForceInfeasible Effort

TypeChef

+

* VA

2 43 5

Variability-Aware

Parser

Variability-Aware

Analysis

https://github.com/ckaestne/TypeChef

Variability-Aware

Lexer

(2

*3

)+

4A5¬A

3 +

4A 4¬A˄B +¬A˄B 6¬A

(

(¬A )¬A

4A (¬A

4¬A˄B +¬A˄B 6¬A

4¬A˄B +¬A˄B 6¬A

)¬A

+

64

VB

6

)

true

4

VA

+

3

Variability-Aware Parsers

GNU C + cpp Java + Antenna

X862.6.33.3

Parsing Linux

Variability in Build System (kbuild) and with #ifdef

Variability Model (kconfig)

0 included header files per C file

0 macros per C file

0 conditional macros per C file

0 alternative macros per C file

0 token per C file

0 % conditional

35385901387

340

33549072

X862.6.33.3

0 C files (x86)

0 seconds per file (median)

0 hours total

0 % overhead (undiscipl.)

0

0 syntax errors

76653085

4.1

0X862.6.33.3

X862.6.33.3

Type Checking Linux

Work in Progress

SUMMARY AND PERSPECTIVE

Summary• Problems:

• Separation of Concerns• Obfuscation• Error-proneness

• Retained advantages:• Easy to use• Flexible• Language-independent• Low adoption barrier

• Improvements:• Views• Visual representation• Variability-aware analysis

• Remaining Problems:• Modular Type Checking• Separate Compilation• Black-box Reuse

“Virtual Separation of Concerns”

Perspective

• Preprocessors common in practice, despite criticism • Neglected by researchers• Tool support can address most problems• Currently scaling to Linux

• Interim solution or serious alternative?

Give preprocessors a second chance!

Questions?

http://fosd.de/cide

Picture Credits

© Stuck in Customs (cc by-nc-sa 2.0)

© Horia Varlan (cc by 2.0)