Partial Evaluation:
Types, Binding Times and Optimal Specialisation
Lecture 3: The Types Involved in Partial evaluation
Neil D. Jones
DIKU, University of Copenhagen (prof. emeritus)
I: QUICK REVIEW OF 1. ORDER PARTIAL EVALUATION
Programs are data objects in a first order data domain D, for
example
D = Atom ∪D ×D
A programming language L is a set L-programs together with a
semantic function
[[ ]]L : L-programs→ D ⇀ D
Program meanings are partial functions:
[[p]]L : D ⇀ D
(Omit L if clear from context.) Examples for L = Lisp:
[[(quote ALPHA)]]L = ALPHA
[[(lambda (x) (+ x x))]]L 3 = 6
— 2 —
INTERPRETER, COMPILER, PARTIAL EVALUATOR
An interpreter int (for S written in L) must satisfy:
[[source]]S(d).= [[int]](source.d)
A compiler comp (from S to T, written in L)
[[source]]S(d).= [[[[comp]](source)]]T(d)
A partial evaluator (for L) is a program spec satisfying, for any
program p and data s, d:
[[p]](s.d).= [[[[spec]](p.s)]](d)
— 3 —
TECHNIQUES FOR PARTIAL EVALUATION
I Applying base functions to known data
I unfolding function calls
I creating one or more specialised program points
Example. Ackermann’s function with known n = 2:
a(m,n) = if m=0 then n+1 else
if n=0 then a(m-1,1)
else a(m-1,a(m,n-1))
Specialised program:
a2(n) = if n=0 then 3 else a1(a2(n-1))
a1(n) = if n=0 then 2 else a1(n-1)+1
Less than half as many arithmetic operations as the original:
since all tests on and computations involving m have been re-
moved.
— 4 —
THE FUTAMURA PROJECTIONS
1. A partial evaluator can compile:
targetdef= [[spec]](int.source)
2. A partial evaluator can generate a compiler:
compdef= [[spec]](spec.int)
3. A partial evaluator can generate a compiler generator:
cogendef= [[spec]](spec.spec)
— 5 —
CORRECTNESS OF THE FUTAMURA PROJECTIONS
Simple equational reasoning to verify:
1. [[target]](d).= [[source]]S(d)
2. target.= [[comp]](source)
3. comp.= [[cogen]](int)
(Surprise! It works well on the computer too... )
Practice: tricky it took a year to get right the first time, in
1984.)
— 6 —
II. UNDERBAR TYPES FOR PARTIAL EVALUATION
Isn’t there a type error somewhere?
Self-application f(f) requires f -type A = A→ A (?)
A symbolic version of an operation on values is a corresponding
operation on program texts.
The symbolic composition of programs p, q yields an output
program r.
The meaning of r =
the composition of the meanings of p and q.
Partial evaluation =
the symbolic specialisation of a function
to a known first argument value.
— 7 —
REMAINDER OF THIS TALK
I A notation for the types of symbolic operations. The nota-
tion distinguishes
• the types of values from
• the types of program texts
I Natural definitions of type correctness of a first-order inter-
preter, compiler or partial evaluator.
I State the problem of optimal partial evaluation.
I Show why it’s difficult for typed languages (even first-order).
I Reference a solution by Henning Makholm.
— 8 —
UNDERBAR TYPES FOR SYMBOLIC COMPUTATION
t: type ::= firstorder | type× type | type→type
| typeX
Type firstorder describes values in D.
For each language X and type t, a type constructor
tX
Meaning: the set of X-programs that denote values of type t.
Examples
I Atom ALPHA has type firstorder
I Lisp program (quote ALPHA) has type firstorderLisp
— 9 —
THE MEANING OF TYPE EXPRESSION T IS [[T ]]
[[firstorder]] = D
[[t1→t2]] = [[[t1]] → [[t2]]]
[[t1 × t2]] = {(t1, t2) | t1 ∈ [[t1]], t2 ∈ [[t2]]}
[[ tX
]] = { p ∈ D | [[p]]X∈ [[t]] }
Some type inference rules:
exp1 : t2 →t1, exp2 : t2exp1exp2 : t1
exp : tX
[[exp]]X : t
firstordervalue : firstorder
exp : tX
exp : firstorder
— 10 —
SYMBOLIC COMPOSITION
(α→ γ)(α→ β)× (β → γ)
(α→ γ)(α→ β)× (β → γ)
[[-]] × [[-]]
;
[[-]]
;
-
-
??'
&
$
%
'
&
$
%
'
&
$
%
'
&
$
%
(α→ β) = the set of all programs that
compute a function from α to β.
— 11 —
COMPOSITION OF FINITE TRANSDUCERS
M ; NNM
6
a/c-(q, s)(p, r)
6
r sb/c
'
&
$
%
'
&
$
%-a/b qp
�
-
'
&
$
%
cacba
���
@@@ -----
Point: no intermediate symbol b is ever produced.
— 12 —
COMPOSITION OF PROGRAMS
Consider composition oneto ; squares ; sum where
oneto(n) = [n, n− 1, . . . , 2, 1]
squares[a1, a2, . . . an] = [a21, a
22, . . . , a
2n]
sum[a1, a2, . . . an] = a1 + a2 + . . .+ an.
Straightforward program:
f(n) = sum(squares(oneto(n)))
squares(l) = if l = [] then [] else
cons(head(l)**2, squares(tail(l)))
sum(l) = if l = [] then 0 else
head(l) + sum(tail(l))
oneto(n) = if n = 0 then [] else cons(n, oneto(n-1))
Result of “deforestation”:
g(n) = if n = 0 then 0 else n**2+g(n-1)
— 13 —
PARTIAL EVALUATION
(β → γ)(α× β → γ)× α
(β → γ)(α× β → γ)× α
[[-]] × Identity
fcn-spec
[[-]]
pgm-spec
-
-
??'
&
$
%
'
&
$
%
'
&
$
%
'
&
$
%
— 14 —
A BETTER DEFINITION OF PARTIAL EVALUATION
Type in the diagram:
pgm− spec : (α× β → γ × α)→ (β → γ)
First Curry:
α→(β→γ)→α→β→γThen generalize:
spec : ∀α .∀τ . α→ τ → α→ τ
Usually α must be first order.
Definition. Program spec ∈ D is a partial evaluator if for all
p, s ∈ D,
[[p]] s.= [[[[spec]] p s]]
— 15 —
INTERPRETERS, COMPILERS, ETC. REVISITED
An interpreter int (for S written in L) must satisfy:
[[source]]S.= [[int]]source
A compiler comp (from S to T, written in L)
[[source]]S.= [[[[comp]]source]]T
A partial evaluator (for L) is a program spec satisfying, for any
program p and data s:
[[p]] s.= [[[[spec]] p s]]
— 16 —
TYPE INFERENCE FOR SELF-APPLICATION
The Futamura projections:
[[spec]] int sourcedef= target
[[spec]] spec intdef= compiler
[[spec]] spec specdef= cogen
Do these type-check?
Recall our type inference rules:
exp1 : t2 →t1, exp2 : t2exp1exp2 : t1
exp : tX
[[exp]]X : t
firstordervalue : firstorder
exp : tX
exp : firstorder
— 17 —
TYPES OF INTERPRETERS, ETC.
1. Type of source : τS
2. Type of [[int]] : ∀τ . τS→ τ
3. Type of [[compiler]] : ∀τ . τS→ τ
T
4. Type of [[spec]] : ∀α .∀β . α→ β → α→ β
where α is first order
Remark: Line 3 gives
• the type of the compiling function.
• The type of the compiler text is:
compiler : ∀τ . τS→ τ
T
— 18 —
TYPES DURING COMPILATION
We wish to find the type of
targetdef= [[spec]] int source
Assume program source has type τS
. A deduction:
[[spec]] : ρ→σ→ρ→σ
[[spec]] : τS→τ→τ
S→τ
int : τS→τ
[[spec]] int : τS→τ
source : τS
[[spec]] int source : τ
— 19 —
TYPES DURING COMPILATION
Thus target has type
τ = τL
(as expected).
The deduction uses only the type inference rules and general-
ization of polymorphic variables.
— 20 —
TYPES DURING COMPILER GENERATION: 1
Recall that: compilerdef= [[spec]] spec int where
interpreter int has type ∀τ . τS→τ .
We show: If p has type α→βthen [[spec]] spec p has type α→β
Deduction:
[[spec]] : ρ→σ→ρ→σ
[[spec]] : α→β→α→β →α→β→α→βspec : α→β→α→β
[[spec]] spec : α→β→α→βp : α→β
[[spec]] spec p : α→β
— 21 —
TYPES DURING COMPILER GENERATION: 2
Recall that:
compilerdef= [[spec]] spec int
where interpreter int has type ∀τ . τS→τ .
We just showed: If p has type α→βthen [[spec]] spec p has type α→β
Substituting α = τS, β = τ , we get
compiler = [[spec]] spec int : τS→τ
and so (as desired)
[[compiler]] : τS→τ
— 22 —
TYPES DURING COMPILER GENERATION: 3
We just showed that:
[[compiler]] : τS→τ
Furthermore τ was arbitrary, so
[[compiler]] : ∀τ . τS→τ
By similar reasoning (too big a tree to show!):
[[cogen]] : ∀α∀β . α→β→α→β
— 23 —
III: OPTIMAL PARTIAL EVALUATION
Suppose sint is a self-interpreter and p, p′ are programs such
that
p′ = [[spec]] sint p
Correctness of spec implies
[[p′]] = [[p]]
but p, p′ need not be the same programs.
— 24 —
DEFINITION OF OPTIMAL PARTIAL EVALUATION
Definition Partial evaluator spec is optimal if it removes all
interpretational overhead:
For a natural self-interpreter sint and any program p and input
d,
timep′(d) ≤ timep(d)
Intuitively: spec has removed an entire layer of interpretation.
— 25 —
PARTIAL EVALUATION EXAMPLE
Example. Ackermann’s function with known n = 2:
a(m,n) = if m=0 then n+1 else
if n=0 then a(m-1,1)
else a(m-1,a(m,n-1))
Specialised program:
a2(n) = if n=0 then 3 else a1(a2(n-1))
a1(n) = if n=0 then 2 else a1(n-1)+1
where a1(n) = a(1,n) and a2(n) = a(2,n) are specialised ver-
sions of function a.
— 26 —
TECHNIQUES FOR PARTIAL EVALUATION
I Applying base functions to known data
I unfolding function calls
I creating one or more specialised functions
Specialised Ackermann’s function performs less than half as
many arithmetic operations as the original:
All computations involving m have been removed.
— 27 —
EXAMPLE: PART OF A FIRST-ORDER INTERPRETER
A well-known trick: split the environment into two parallel lists:
ns = (n1,. . . ,nk) names
vs = (v1,. . . ,vk) values
Part of the interpreter text:
eval(exp,ns,vs,pgm) = case exp of
"X" : lookup X ns vs
"e1+e2" : eval(e1,ns,vs,pgm) + eval(e2,ns,vs,pgm)
...
— 28 —
SPECIALISATION OF THE FIRST-ORDER INTERPRETER
Binding times: exp,ns,pgm are static,
while vs is dynamic.
Consequence: all functions in p′ = [[spec]] sint p have form:
evalexp,ns,pgm(vs) = . . .
An annoying problem:
there is only only one argument in each p′ function (!)
This cannot be optimal, i.e., as fast as p !
— 29 —
INHERITED LIMITS DURING SPECIALISATION
This problem: specialised program p′ = [[spec]] sint p inherits
a limit from sint: a specialised function
fa,b(x, y) = . . .
has k′ ≤ k arguments, if sint function f has k arguments.
Thus no function in p′ has more than k arguments(!)
For interpreter function eval, this problem can be solved by
variable splitting, also called arity raising.
Observation: for a fixed p, the interpreter’s variable vs always
has a constant length k.
— 30 —
TECHNICAL SOLUTION
Split evalexp,ns,pgm(vs) = . . . into
evalexp,ns,pgm(v1, . . . , vk) = . . .
By this and similar tricks, a first-order “optimal” spec can be
built.
For the “optimal” spec, p′ = [[spec]] sint p is
identical to p, up to the naming of variables
(and thereby just as fast).
— 31 —
OPTIMALITY IS HARDER FOR TYPED LANGUAGES!
Interpreter example with types (first-order):
eval : Exp -> Names -> Values -> Univ
Univ = Int integer | Pair Univ * Univ | ...
eval exp ns vs = case exp of
"X" : env X
"e1:e2" : Pair (eval e1 ns vs) (eval e2 ns vs)
...
Suppose source program has type [[p]] : N → N .
Then specialised program has a different type:
[[p′]] : Univ→ Univ
Significantly less efficient. With higher-order types: even worse!
— 32 —
A CHALLENGING PROBLEM
To achieve optimal specialisation for a typed programming lan-
guage.
I Stated in 1987
I Unsuccessfully attempted for a number of years
I Solved by Henning Makholm in 1999. Reported in SAIG
2000 (ICFP workshop at Montreal)
— 33 —
MAKHOLM’S SOLUTION
Type of a specialiser:
[[spec]] : Pgm→Data→Pgm
This is correct, but it “doesn’t tell the whole story”
To clarify the problem, extend α→β L to
One version for L-programs: α→βPgm
and another version for data types: αData
αData is a subtype of Data:
the set of encodings of all values of type α
— 34 —
OPTIMALITY REVISITED
The type of a specialiser’s meaning, redone:
[[spec]] :α→β→γ
Pgm→
α
Data→
β→γPgm
Type of a self-interpreter’s meaning:
∀α, β . [[sint]] :α→βPgm
→α
Univ→
β
Univand thus
∀α, β . sint :
α→βPgm →
αUniv →
βUniv
Pgm
Here Univ is a universal data type.
— 35 —
OPTIMALITY AND THE UNIVERSAL DATA TYPE
The optimality criterion: p′ = [[spec]] sint p should be as good
as p.
Alas this is impossible since:
[[p]] : α→β
but
[[p′]] = [[[[spec]] sint p]] :α
Univ→
β
Univ
— 36 —
OPTIMALITY REFORMULATED
A way out: use a self-interpreter with type
[[sintα→β]] :α→βPgm
→ α → β
This can be mechanically obtained from
∀α, β . [[sint]] :α→βPgm
→α
Univ→
β
Univby
[[sintα→β]] p a = decodeβ([[sint]] p encodeα(a))
Optimality reformulated: for any [[p]] : α→β the program
p′ = [[spec]] sintα→β p
is at least as fast as p.
— 37 —
OPTIMALITY ACHIEVED
1. L = a first-order call-by-value language with
2. types unit, integer and sum and product types.
3. The self-interpreter uses a universal type Univ.
4. The self-interpreter has been proven correct (Morten Welin-
der’s phd thesis).
— 38 —
OPTIMALITY ACHIEVED
I Phase 1: specialise using unsophisticated techniques.
The output program uses a universal type Univ.
I Phase 2: Retype output program, using
• Type erasure analysis that uses
• non-standard type inference for
• types that are infinite regular trees.
I Phase 3: an Identity elimination phase, e.g., η-reductions
for product and sum types.
Punch line: It works, and even achieves:
[[spec]] sint sint =α sint
— 39 —
CONCLUSIONS
Contributions:
I A notation for the types of symbolic operations. It distin-
guishes types of values from types of program texts.
I Natural definitions of type correctness of an interpreter or
compiler.
I Makholm: succeeded in solving a long-standing open prob-
lem using the underbar type notation (after some refine-
ment)
More to do:
I Better mathematical understanding of the underbar types.
I How to prove that an interpreter or compiler has the desired
type?
— 40 —