An Open Framework for Certified System Software

Xinyu Feng

Yale University

Why Certified Software?

Ariane 5

Mars Polar Lander

Mars climate orbiter

Something More Relevant …

“A typical modern car contains around 20 built-in microcontrollers. Luxury models can have as many as 80. Such microcontrollers, …, are in constant communication with one another. In extreme cases, a single programming error in one of the control elements can mean life or death.”

-- Fraunhofer Magazine, 2004 (2)

Fig. from:www.howstuffworks.com

Toyota recalled its 160,000 Prius cars in Oct 2005, because of bugs in the software controlling the hybrid gas-electric engine system…

Photo from edmunds.com

How to Guarantee Software Quality?

Hardware

We need a “certified” computing platform!

Buggy? Bootloader + OS + Device DriverCertified Bootloader + OS + Device Driver

Buggy? Runtime Services & LibrariesCertified Runtime Services & Libraries

Buggy? Security InfrastructureCertified Security Infrastructure

Uncertified legacy code becomes second-class citizen!

We need firm control of the lowest-level software! [King et al. S&P'06]

Certified Software: Problem Definition

• Hardware– processors, memory, storage, devices, …

• Software– bootloader, device drivers, OS, runtime, applications,

…

• Need a mathematical proof showing that as long as the hardware works, the software always work according to its specification

specification S

binary code C

formal proof P

OS

timer

KBD

.

.

.

scheduler

spawn, yield, exit, lock, monitors, …

…ctxt ctxt ctxt

.

.

.interrupts

… threads

A Mini-OS

33 MHz

bootloader

1300-line 16bit x86 code,

Bootable!

http://flint.cs.yale.edu/feng/cos

But how to certify the code?

Low-level code: C/Assembly

Certifying the Mini-OS

Many challenges:

Interrupts

Certified the whole system

Many different features

Different abstraction levels

Device drivers / IO

1300 lines of code

Code loading

Concurrency

bootloader

scheduler

timer int. handler

keyboard driver

keyboard int. handler

thread lib: spawn, exit, yield, …

sync. lib: locks and monitors

…

My Contributions

• Specialized program logics– SCAP: stack-based control abstractions– SAGL: modular concurrency verification– CMAP: dynamic thread creation– concurrency with relaxed memory model [ongoing work]

• An open framework for certified systems– OCAP: embedding and interoperation between

different verification systems [TLDI’07]

– interoperability based on semantic models [ongoing work]

[PLDI’06]

[ICFP’05]

[ESOP’07]

Outline of This Talk

• The OCAP Framework

• SAGL: modular concurrency verification

• SCAP: stack-based control abstractions

• Embedding and linking in OCAP

H

RC1

pc1

T1

{(A1, G1)}

Cn

pcn

Tn

{(An, Gn)}

i

. . .

Modules at Different Levels

• All concurrency verification assumes built-in concurrency

• Context switching, scheduler– Too low-level to be certified in

these logics

H

RC1 Cn

. . .

. . .

CS

pc

ctxt

1

ctxt

n

TQ

• Threads & schedulers have never been certified in a single logic!

• Example: how to certify multi-threaded software?

L1

L2

L3

L4

Building Fully Certified Systems

• One logic for all code– Consider all possible

interactions.– Very difficult!

• Reality– Only limited combinations of

features are used.– It’s simpler to use a specialized

logic for each combination.– Interoperability between logics

OCAP

Our Solution

Ln…L1

Mechanized Meta-Logic (CiC)

Modeling of the machine

…C1 Cn

C1C1

Cn

…OS

Cn

…C1 Cn

TCB

The Machine

I1f1:

I2f2:

I3f3:

…

(code heap) C

0

r1

1 2 …

r2 r3 … rn

(data heap) H

(register file) R

(state) S

addu … lw … sw … … j f

(instr. seq.) I

(program) P::=(C,S,pc)

::=(H,R)::={f I}*

pc

Program Specifications

I1f1:

I2f2:

I3f3:

…

(code heap) C

0

r1

1 2 …

r2 r3 … rn

(data heap) H

(register file) R

(state) S

addu … lw … sw … … j f

(instr. seq.) I

(program) P::=(C,S,pc)

::=(H,R)::={f I}*

pc

1

2

3

(spec) ::= {f }*

Invariant-Based Verification

Initial condition: Inv(P0)

P0

c1 P1

c2 P2

c3 … cn Pn

Progress:

if Inv(P), then P’. P c P’.

Preservation:

if Inv(P) and P c P’, then Inv(P’).

Mechanized Meta-Logic (CiC)

OCAP Rules

Ln…

“Domain specific” logics

Modeling of the machine

L1

…C1 Cn

may use different How to link modules?

How to Link Modules

{r1:1, …, rn:n} {P}_{Q}

call f

( _ )t

a

( _ )h

a'

…

…

f:

…

…

{r1:1, …, rn:n} {P}_{Q}

( _ )t

a

( _ )h

a'

How to Link Modules

How to define interpretation?

Encode the invariant enforced in our invariant-based proof methodology.

a should be expressive enough to encode Inv.

An Open FrameworkA set of Hoare-logic rules as the foundational layer

XCAP[Ni&Shao'06] approacha(S) = …

used and generalized in OCAP [TLDI'07]

use “a” as the assertion language

supports first-class code pointers, mutable references, polymorphisms, recursive types, …

Indexed approach

simpler model for weak-ref, partial correctness, concurrency…,than existing work on indexed model [Appel&McAllester'01] [Amed'04] [Appel et al.'07]

ai(S) = … [ongoing work with Cai, Shao & Tan]

OCAP Rules

The OCAP Framework [TLDI'07]

Ln…L1

…C1 Cn

( )L1 ( )LnSoundSound

OCAPSoundness

Mechanized Meta-Logic (CiC)

Modeling of the machine

XCAP

SCAPTAL …SAGL

OCAP

Outline of This Talk

• The OCAP Framework

• SAGL: modular concurrency verification

• SCAP: stack-based control abstractions

• Embedding and linking in OCAP

Ln…

Mechanized Meta-Logic (CiC)

…C1

Modeling of the machine

L1

Cn

SAGLSAGL SCAP

Certifying Concurrent Programs

How to guarantee

non-interference…

H

RC1

pc1

T1

{(A1, G1)}

Cn

pcn

Tn

{(An, Gn)}

i

. . .

in a modular way?

Existing work is not modular/general…

Certifying Concurrent Programs

• Assume-Guarantee (A-G) reasoning [Misra&Chandy’81, Jones’83]

thread modularity, general spec of A&G requires global data invariants

• Concurrent Separation Logic (CSL) [O’Hearn, Brookes 2004]

thread modularity + local reasoning restrictive synchronization pattern

shared resources can be accessed only inside critical regions

• SAGL: extend A-G with local reasoning [ESOP'07]

improved modularity without loss of generality

Assume-Guarantee Reasoning

• Thread T and its environment– Environment: the collection of all other threads except T

• A: assumption about environment’s transition

• G: guarantee to the environment

• a: precondition

A-G Reasoning

To certify each thread:

S, S'. a S A S S' a S'

Non-Interference of threads:

stability of precondition:

G S Nextc(S)

transitions satisfy the guarantee:

i,j . Gi Aj ( i j )

A-G Reasoning

a1a2

a1a2 a1a2

G2

A1A2

G1

a2

a2

a1

a1

Requires global invariants!

SAGL: Overview

Partition of resources: shared & private

p1

a1a2

p2

Threads specs:

(a1, p1), (a2, p2), …

Partition is conceptual:

(p1 p2) (a1 a2)

SAGL: Memory Access

p1

a1a2

p2

Threads have exclusive access to their private resources.

All threads can access shared part.

Needs to guarantee non-interference.

Follows assume/guarantee reasoning.

Partitions can be dynamically adjusted.

p1

a1a2'

p2

p1

a1a2

p2

SAGL – Access Shared Resource

A1

G2

a1 a1

A-G reasoning: a special case where

p1 and p2 are emp.

p1

a1a2

p2'

p1

a1a2

p2

SAGL – Access Private Resource

p1

a1a2

p2

p1

a1a2

p2

p1

a1a2'

p2

SAGL - Redistribution

A1

G2 A1

G2

lock unlock

a1

a1

a1

a1

getNode():

l.acq();

…

l.rel();

-{(ainv , emp)}

Example: List

-{(emp, emp List(x))};

-{(emp, Node(y) List(x))}

-{(List(x), Node(y))}

ainv = free(l) List(x)

free(l) emp

x…

x … y

ainv

SAGL [ESOP’07]

p1

a1a2

p2

Threads have exclusive access to their private resources.

All threads can access shared part.

Needs to guarantee non-interference.

Follows assume/guarantee reasoning.

Partitions can be dynamically adjusted.

A-G reasoning

CSL

OCAP

Outline of this talk

• The OCAP Framework

• SAGL: modular concurrency verification

• SCAP: stack-based control abstractions

• Embedding and linking in OCAP

Ln…

Mechanized Meta-Logic (CiC)

…C1

Modeling of the machine

L1

Cn

SAGL SCAPSCAP

Certifying C & Assembly Code?

How to specify/verify control abstractions?

Stack-based control abstractionscall/return, tail call, exceptions (stack cutting/ stack unwinding),

coroutines/threads context switching

How to formulate the stack invariants?

Problems – call/return

void f(){ void h(){

h(); return;

return; }

}

Stacks are hidden!

f: ...

sw $ra, -4($fp) h:

jal h ;; $ra contains ct

ct: lw $ra, -4($fp) jr $ra

...

jr $ra

fp

stack

??ct

Problems – call/return

void f(){ void h(){

h(); return;

return; }

}

Does f use the right return addr.?

pc

ra

R

void cmp1(int x,jmp_buf env){

if (x == 0)

longjmp(env, 1);

else

return;

}

Problems – setjmp/longjmp

int rev(int x){

if (setjmp(env) == 0){

cmp0(x, env); return 0;

}else{

return 1;

}

}

void cmp0(int x,jmp_buf env){

cmp1(x, env);

}

jmp_buf env = …;

pc

f0

… sp

env

f0 pc

pc

env cannot outlive the stack frame of rev !

…

…

Stack-Based Control Abstractions

A simple system (SCAP) for modular verification of (1) compiled C code & (2) manually-written assembly code

weak-continuation

Tail calls

Exceptions: Stack-cutting

Exceptions: Stack-unwinding

Threads context switching

Coroutines (w. function call)

setjmp/longjmp

Function call/returnPLDI'06

PLDI’06

TLDI’07

YALEU/DCS/TR-1336

• SCAP specifications: (p, g)– p: State Prop– g: State State Prop

Specifications

f:

...

sw $ra, -4($fp)

jal h

ct:

lw $ra, -4($fp)

...

jr $ra

{(p0, g0)}

{(p1, g1)}

g0

g1

g0 S S’ S’.$ra = S.$ra …

• Challenges– f uses the “right” return addr.?

– Hoare triple {p} f {q}?• In different basic blocks!

{$ra = n …}

{$ra = n …}

Program Spec. and Code Pointers

jal f

jal h

jr $ra

jr $ra

g0

g4

p0

p4

g1

p1

g2

p2

g3

p3

…

jr $ra

• Program Specification::=

{f1(p1,g1), …,fn(pn,gn)}

• “safe” to return (jr $ra):– $radom() ($ra)=(p,g)– p holds at the time of return

SCAP : Stack Invariant

p0

g0p1

g1p2

g2p3

g3

g0 S0 S1

S1.$ra (S1.$ra))=(p1, g1) p1 S1

g0 S0 S1 g1 S1 S2

S2.$ra (S2.$ra)=(p2, g2) p2 S2

g0 S0 S1 g1 S1 S2 g2 S2 S3

S3.$ra (S3.$ra)=(p3, g3) p3 S3

jr $ra

Logical control stack

Always safe to return?S0

S1

S2

S3

…

SCAP : Stack Invariant

WFST(n, g0, S0, ) S1. g0 S0 S1

p1,g1.

(S1.$ra)=(p1, g1) p1 S1 WFST(n-1, g1, S1, )

WFST(0, g0, S0, ) S1. g0 S0 S1

Invariant:p S n.WFST(n, g, S, )

p0

g0p1

g1p2

g2p3

g3

jr $ra

Logical control stack

S0

S1

S2

S3

SCAP : Invariant Preservation

• Inv(S): p S n.WFST(n, g, S, )

cS’

p S n.WFST(n,g,S,)

S

p’ S’ n.WFST(n,g’,S’,)

p’,g’

SCAP: call

p0

g0p1

g1

jr $ra

p0

g0jr $ra

p

g

jal f

p S WFST(n, g, S, ) p0 S0 WFST(n+1, g0, S0, )

S S0

n+1

n

…

p S p0 S0

p1

g1

n

…

p S g0 S0 S1 p1 S1

p S g0 S0 S1 g1 S1 S2 g S S2

g0 S0 S1 S0.$ra = S1.$ra

S1S1

S2 S2

SCAP: ret

pgp1

g1

p1

g1jr $ra

p S WFST(n, g S, ) p1 S1 WFST(n-1, g1 S1, )

n

n-1

… n-1

…

p S g S S1

SS1

Generalization: Stack Unwinding/Cutting

g1

p1

jr ra

p

g

+

p1

g1jr ra

p

g

Multi-ret

p1

g1

jr ra

p

g

Tail-call

Example: setjmp/longjmp

int rev(int x){

if (setjmp(env) == 0){

cmp0(x, env); return 0;

}else{

return 1;

}

}

void cmp0(int x,jmp_buf env){

cmp1(x, env);

}

void cmp1(int x,jmp_buf env){

if (x == 0)

longjmp(env, 1);

else

return;

}

jmp_buf env = …;

Applications of SCAP

• malloc/free [Feng et al. PLDI'06, Xiang et al. QSIC'06]

• thread scheduler [Feng et al. TLDI'07]

• garbage collectors [McCreight&Shao PLDI'07]

OCAP

Outline of This Talk

• The OCAP Framework

• SAGL: modular concurrency verification

• SCAP: stack-based control abstractions

• Embedding and linking in OCAP

Ln…

Mechanized Meta-Logic (CiC)

…C1

Modeling of the machine

L1

Cn

SAGL SCAP

Threads and Scheduler

• Thread code C1,…,Cn

– Certified in SAGL• as concurrent code

– Do not know about thread queue

H

RC1

pc1

T1

{(A1, G1)}

Cn

pcn

Tn

{(An, Gn)}

i

. . .

H

RC1 Cn

. . .

. . .

CS

pc

ctxt

1

ctxt

n

TQ

SCAPSAGL +

• Scheduler CS

– Certified in SCAP• as sequential code

– Manages thread queue TQ– Do not touch H

How to Specify Scheduler/yield

jal yieldyield:

.

.

.

jr $ra

yield

gS1

S2|S2||S1|

SAGL SCAP

Embedding of SCAP

((p, g))scap = , S. p S n.WFST(n, g, S, )

┝scap C: ┝ocap C :||If , then

Soundness:

Embedding:

Supports proof reuse.

Embedding of SAGL

H

RC1

pc1

T1

{(A1, G1)}

Cn

pcn

Tn

{(An, Gn)}

i

. . .

H

RC1 Cn

. . .

. . .

CS

pc

ctxt

1

ctxt

n

TQ

(((a,p),A,G))sagl =

, S.

Q={pc1, …, pcn}.

(pci) =((ai,pi),Ai,Gi)

InMem(Q)(p1…pn)(a1…an)

NI((A1,G1), …, (An,Gn))

stable(ai,Ai)

Embedding:

Soundness of SAGL Embedding

┝sagl C: y┝ocap C:||.If , then

Certifying The System

H

RC1 Cn

. . .

. . .

CS

pc

ctxt

1

ctxt

n

TQ

┝sagl Ci:i┝scap Cs:y

┝ocap Cs: y y┝ocap Ci: i

SCAPSAGL

the “link” rule

┝ocap Ci Cs : i y

More Applications of OCAP

TAL [Morrisett'98]

malloc lib. GCs

[TLDI'07] [Lin et al. TASE'07]

Conclusion

• Goal: modular verification of system software– modules use different computation features– modules are at different abstraction levels

• Solution: to certify different modules using different logics– SCAP: stack-based control abstractions– SAGL: modular concurrency verification– CMAP: assume-guarantee with dynamic threads– …– OCAP: interoperation between different verification systems

Conclusion

• Applications

– malloc/free [Feng et al. PLDI'06, Xiang et al. QSIC'06]

– garbage collectors [McCreight&Shao PLDI'07]

– thread scheduler [Feng et al. TLDI'07]

– threads + scheduler [Feng et al. TLDI'07]

– TAL + mem. alloc [Feng et al. TLDI'07]

– TAL + GC [Lin et al. TASE'07]

SCAP

OCAP

Ongoing and Future Work

• Certifying OS– Mini-OS: ongoing work– More realistic OS in the future, e.g. OS for embedded systems

• Concurrency with relaxed memory models – relaxed memory models and STM

• General Hoare-logic (next generation of OCAP)– based on a uniform semantic model of features– support concurrency, partial correctness, frame rules …

• Push verification to high level code– C-like code with inlined assembly– Automated spec. inference and theorem proving

Thank you!

Acknowledgments

Zhong Shao, Hongxu Cai, Rodrigo Ferreira, Yu Guo, Andrew McCreight, Zhaozhong Ni, Gang Tan, Alex Vaynberg, Sen Xiang

Backup Slides

Certifying the Mini-OS

• Modeling of x86 machine– Real mode– PIC, keyboard, hard drive– 8- and 16-bit integers

• Verifying the code– Multi-threaded code

• SAGL with extension of interrupts– Scheduler, thread lib., interrupt handler

• SCAP with extension of interrupts– Interactions with devices (PIC, PIT, keyboard, HDR)

• Finite state machine in the specifications

Statistics

• FIFO scheduler/context switching code– 30 line of MIPS code– 1400 line of proof scripts

• malloc/free: – 42/50 line of MIPS code– 1400/1560 line of proof scripts

Statistics [McCreight et al. PLDI’07]

Statistics – TAL & GC[Lin et al. TASE'07]

SCAP: tail call

p0

g0

jr $ra

p

g

S

n

… j f

p0

g0

jr $ra

S0

n

…

p S WFST(n, g S, ) p0 S0 WFST(n, g0 S0, )

p S p0 S0 p S g0 S0 S1 g S S1

S1S1

CMAP

• A-G reasoning– properly nested structures: P1 || P2

• low-level code– fork/join based structures– dynamic thread environment

• CMAP– unbounded dynamic creation/termination– similar to the support of dynamic mem. alloc.– change A and G to approximate the dynamic

thread env.

…

H1 pc

1

pc

n…

TQ

r31

r0

ct

Scheduler in SCAP

yield:

pick one word (pci) from TQ

swap pci and r31

jr r31

…

jal yield

ct: …

Thread code:

H1 pc

1

pc

n…

TQ

r31

r0

ct

pc

Scheduler in SCAP

yield: (ps, gs)

pick one word (pci) from TQ

swap pci and r31

jr r31

gs

gs (r{r1,…,r30}.[r]=[r]’) Q, Q’.

p’.

Q{[r31]} = Q’{[r31]’}

WFTQ(Q) p’WFTQ(Q’) p’

H1 pc

1

pc

n…

TQ

r31

r0

H1 pc

1

pc

n…

TQ

r31

r0

ct

ct

ps Q.WFTQ(Q) True

Example: A-G reasoning

…

[100] := m;

…

…

[101] := n;

…

100 101

G1: [101] = [101]'

A1: [100] = [100]'

G2: [100] = [100]'

A2: [101] = [101]'

Example: SAGL reasoning

…

[100] := m;

…

…

[101] := n;

…

100 101

-{(emp , 100 _) } -{(emp , 101 _)}

G1: emp

A1: emp

G2: emp

A2: emp

OCAP: Code pointers

codeptr(f,a) (f) = a (Spec) ::= {f a}*

acodeptr

Not well-founded!

Support of first-class code pointers:

OCAP: Code Pointers

A generic specification:

(CdSpec) ::= …

(Assert) a Spec State Prop

(Spec) ::= {f }*

() ,S. …

No interoperation between multiple systems…

acodeptr

a

codeptr

()

OCAP: Foreign Languages

(LangTy) L ::= CiC Terms Type

(CodeSpec) ::= CiC Terms L

Inductive TalType : Type :=

T_int : TalType

T_pair: TalType -> TalType-> TalType

…

Inductive Tal2Type : Type :=

T2_int : TalType

T2_pair: TalType -> TalType-> TalType

…

L ::= TalType | Tal2Type | …

1 ::= T_int | T_pair 1 1 | … TalType

2 ::= T2_int | T2_pair 2 2 | … Tal2Type

OCAP: Specifications

(OCdSpec) ::= <L,( )L,> L.(L Assert)*L

(Spec) ::= {(f1,1), … (fn,n)}

(Assert) a Spec State Prop

(Interp) ( )L L Assert

(LangTy) L ::= CiC Terms Type

(CodeSpec) ::= CiC Terms L

Not well-founded:

( )L

a

OCAP: Specifications

(OCdSpec) ::= <,L,( )L,>

(Spec) ::= {(f1,1), … (fn,n)}

(Assert) a Spec State Prop

(Interp) ( )L L Assert

(LangTy) L ::= CiC Terms Type

(CodeSpec) ::= CiC Terms L

(LangID) ::= n nat

(LangDict) D ::= {1<L1,( )L1>,…, n<Ln,( )Ln>}

LangID (L.L Assert)

( )L

a