From Program Verification to Program Synthesis

Saurabh Srivastava *Sumit Gulwani ♯

Jeffrey S. Foster *

* University of Maryland, College Park♯ Microsoft Research, Redmond

Program Synthesis

• Goal: Automatically generate program

• Important benefits of program synthesis– Reduced programmer burden– Potential to generate novel algorithms

• Objective: Automatic program synthesis, given– Pre/post condition (functional specification)– Some hints about the form of the program

• Ideally: program is automatically proved correct– Synthesized programs can be complicated

scaffold

Proof-theoretic synthesis

Our approach: Synthesize program and proof simultaneously

Benefits:• Pruning invalid proofs prunes invalid programs

– E.g., A program that manipulates a ordered list• Ordered list invariant has to hold, preventing any program

that would violate it

• View synthesis as generalized verification– Reuse existing verification tools and technology

• Tool synthesizes: linear arithmetic programs, sorting, dynamic programming programs

Discrete Line Drawing

(0,0)

(X,Y)

|y-(Y/X)x| ≤ 1/2

Output values

Input

(0,0)

(1,1) (2,1)

(3,2) (4,2)

(5,3)

(6,4) (7,4)

(8,5) (9,5)

(10,6)(11,6)

(12,7)0<Y≤X

Using only linear operations:Bresenham’s line drawing algorithm

Scaffold(1): Functional Specification

• Precondition Fpre

– Formal specification of possible valid inputs– E.g., 0<Y≤X

• Output Postcondition Fpost

– Formal specification that outputs need to meet– E.g., |y-(Y/X)x| ≤ 1/2

•

•

–

–

– E.g., R loop = *;(*)• Acyclic fragment followed by a loop with an acyclic fragment inside

•

–

– E.g., R stack = 1 • Only one extra local variable is available

• Hints about the form of the desired program

• Looping structure R loop

– Acyclic fragments (*), loops (), sequencing (;)– Grammar: L ::= * | (L) | L;L–

•

• Stack R stack

– Maximum number of local variables permitted–

•

Scaffold(2): Resource Constraints

*

*

Scaffold(3): Domains

• Domains for program elements– Expressions Dexp

– Guards Dgrd

– E.g.• linear or quadratic expression over program variables• predicates with array lookups

• Domain for invariants Dprf

– Choice of solver– E.g., linear arithmetic

• Invariant form: ⋀i aix+biy+ci ≥ 0

• Linear arithmetic fixed-point computation tool, i.e., verifier

Outline

Input: Scaffold

Approach: Synthesis conditions

Experiments

Modeling Basic Blocks as Transitions

v := v + 2Y; x++;

Basic Block as input->output transition

inputs: v,x,Youtputs: v’,x’

Sbody : v’ = v + 2Y ⋀ x’ = x + 1;

transition : ⋀i (output vari = fn(input vars))

Task: • Model statements in a

manner that a solver can reason about

• State update and ordering problematic

Sinit

while ( Gloop ) { output (x,y) []Gcond1 → Sbody1

[]Gcond2 → Sbody2 }

expand

0<Y≤X Sinit

while ( Gloop ) { |y-(Y/X)x| ≤ 1/2 []Gcond1 → Sbody1

[]Gcond2 → Sbody2 }

Synthesis Task

Generate values for unknown Sinit/body1/body2, Gloop/cond

Generate proof (invariants)

*;(*)

0<Y≤X Sinit

while ( Gloop ) { |y-(Y/X)x| ≤ 1/2 []Gcond1 → Sbody1

[]Gcond2 → Sbody2 }

Verification condition

(VC)

VC - Safety Constraints

entryoutput

induct

induct:

entry:

output:

0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2

I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’

I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’I

linear

ari

thm

eti

c

verifier

Verification condition

(VC)

Fixed point solutions, i.e., invariants

induct:

entry:

output:

0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2

I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’

I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’

ik, i.e., invariant I

SAT s

olv

er

boolean clauses

Dom

ain

speci

fic

reduce

r

linear arithmetic [PLDI’08]predicate abstraction [PLDI’09]

∃ik: integer SAT instance

Linear arithmetic tricks

i1x+i2y+i3≥0 ⋀ ⋀ ⇒ i1x’+i2y’+i3≥0

Assume a form for I, e.g., i1x+i2y+i3≥0 for I (given the

domains)Gcond2 Sbody2

linear

ari

thm

eti

c

verifier

Verification condition

(VC)

Fixed point solutions, i.e., invariants

induct:

entry:

output:

0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2

I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’

I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’

ik, i.e., invariant I

SAT s

olv

er

boolean clauses

Dom

ain

speci

fic

reduce

r

linear arithmetic [PLDI’08]predicate abstraction [PLDI’09]

∃ik: integer SAT instance

Linear arithmetic tricks

i1x+i2y+i3≥0 ⋀ ⋀ ⇒ i1x’+i2y’+i3≥0

Assume a form for I, e.g., i1x+i2y+i3≥0 for I (given the

domains)Gcond2 Sbody2

linear

ari

thm

eti

c

verifier

Verification condition

(VC)

Unknown statements and guards

induct:

entry:

output:

0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2

I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’

I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’

ik, i.e., invariant I

SAT s

olv

er

boolean clauses

Dom

ain

speci

fic

reduce

r

linear arithmetic [PLDI’08]predicate abstraction [PLDI’09]

∃ik,gk,sk: integer SAT instance

Linear arithmetic tricks

i1x+i2y+i3≥0 ⋀ g1x+g2y+g3≥0 ⋀ x’=s1x+s2y+s3 ⋀ y’=… ⇒ i1x’+i2y’+i3≥0

Assume a form for I, Gcond2,Sbody2, e.g., i1x+i2y+i3≥0 for I (given the

domains)gk and sk

Synthesize statements and

guards!

guards/statements

Statements unknown

Infer I and Sinit,Sbody1,Sbody2

0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2

I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’

I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’

0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2

I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’

I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’

false false

false

false

falsefalse false

falsefalse

Trivial soln: Statements

unknown:

Ask solver

Issue: Well-formedness not ensured•Statements•Conditional Guards

But transitions, i.e., Sinit,Sbody1,Sbody2,

cannot imply false

Remember, • statement unknowns:

transition• ⋀i (output vari = fn(input

vars))• E.g., v’ = v + 2Y ⋀ x’ = x + 1

• Can never imply false

Infer I and Sinit,Sbody1,Sbody2,Gloop,Gcond

Additional Constraints

0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2

I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’

I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’

Well-formedness constraints:•Statement unknowns are not false•Conditional guards form tautologies

Outputs reachable/termination constraints:Exists ranking function r•r bounded from below•r decreases in each iteration

Safety

Termination

Well-formedness

Issue: Termination constraints

Issue: Well-formedness not ensured•Statements•Conditional Guards

Infer I and Sinit,Sbody1,Sbody2,Gloop,Gcond

Additional Constraints

0<Y≤X ⋀ Sinit ⇒ I’ I ⋀ Gloop ⇒ |y-(Y/X)x| ≤ 1/2

I ⋀ Gcond1 ⋀ Sbody1 ⇒ I’

I ⋀ Gcond2 ⋀ Sbody2 ⇒ I’

Solving synthesis conditions does yield valid programs

SynthesisConditions

Safety

Termination

Well-formedness

User input: Program

VCGen

User input: Scaffold

SCGen

Verification Conditions

Verified Program +

Proof

Fixed-pt Solver

Verification(Invariant inference)

Synthesized Program+ Proof

Fixed-pt Solver

Synthesis(Invariant + Program

inference)

Synthesis Conditions

Synthesis Conditions

User Input

linear arithmetic

: one extra variable : *;(*)

: 2|y-(Y/X)x| ≤ 1

: 0<Y≤X

:

:constraint-basedlinear arithmetic+ verifier

Program and Proof

Solution strategy(verifier)

Example: Discrete Line Drawing

FpreFpost

R loop

R stack

Dexp

Program:v := 2Y-X; y := 0; x := 0;while ( x≤X ) { print (x,y) if ( v<0 ) { v := v + 2Y; x++; } else { v := v + 2(Y-X); y++; x++; }}

Invariant:0<Y≤Xv = 2(x+1)Y-(2y+1)X2(Y-X) ≤ v ≤ 2Y

Ranking function:X-x

Dgrd

Dprf

Scaffold

Constraint-based verifier

Outline

Input: Scaffold

Approach: Synthesis conditions

Experiments

Experiments: Linear Arithmetic

• LIA verifier from previous work [PLDI’08]

• Benchmarks– Strassen’s 2x2 matrix multiplication

• R loop = * and R stack = 7 --- for holding the seven intermediate results

– Swapping two integers without a temporary• R loop = * and R stack = 0

– Computing the integral square root• Linear search

– R loop = *;(*) and R stack = 1

• Binary search– R loop = *;(*) and R stack = 2

– Discrete line drawing• R loop = *;(*) and R stack = 1

Experiments: Predicate Abstraction

• Predicate abstraction verifier from previous work [PLDI’09]

• Benchmarks: Sorting– Requires quantified specification and invariants– E.g., All major sorting programs

• Specification: sortedness• Vary resource constraints

– Nested loop, 0 extra variables: bubble sort, insertion sort– Nested loop, 1 extra variable: selection sort– Recursive, 0 extra variables: merge sort– Recursive, 1 extra variable: quick sort

• Benchmarks: Dynamic Programming– Iterative program from its recursive specification– E.g., Fibonacci

• Specification f(0)=1, f(1)=1, ∀k>2: f(k)=f(k-1)+f(k-2)• R loop = *;(*) and R stack = 2

• Generated program maintains sliding window (of size 2) and computes f(n)

1 around 10,000 seconds

8 under 1000 seconds

8 under 10 seconds

Experiments: Synthesis times

Swap

Two

Strass

en's

Sqrt

(linea

r)

Sqrt

(binary)

Bresen

ham's

Bubble So

rt

Insertion So

rt

Selec

tion Sort

Merge S

ort

Quick So

rt

Fibonacc

i

Checkboard LC

S

Matrix

Chain

SS Sh

ortest

Path

All Shorte

st Path

All Shorte

st Path

100

1,000

10,000

100,000

1,000,000

10,000,000

Verification

Synthesis

Discret

e lin

e

0.1

0.01

0.001

seco

nds

(log

scal

e)

Predicate abstraction verifierLIA verifier

Sorti

ng Prg

1

Sorti

ng Prg

2

Sorti

ng Prg

3

Sorti

ng Prg

4

Sorti

ng Prg

5

All pairs

SP (lo

op)

All pairs

SP (b

ody)

Summary: Proof-theoretic synthesis

• Proof-theoretic synthesis– Synthesize programs and proof simultaneously– Treat synthesis as generalized verification

• Principled approach to solving for programs using verification

• User input: scaffold– Functional specification– Space of desired program

• Domains of guards and statements• Resource constraints

• Approach: One-shot synthesis, i.e., no iteration– Constraints, synthesis conditions, encode the desired program

• Safety, termination, and well-formedness

– Solve using existing constraint-based verifiers– Possible to synthesize wide variety of programs