Date post: | 25-Dec-2015 |
Category: |
Documents |
Upload: | russell-stevens |
View: | 222 times |
Download: | 1 times |
Relational Verificationto
SIMD Loop Synthesis
Mark Marron – IMDEA & Microsoft Research
Sumit Gulwani – Microsoft Research
Gilles Barthe, Juan M. Crespo, Cesar Kunz – IMDEA
General SIMD Compilation Compilers struggle to utilize SIMD operations in general purpose code
◦ Text processing, web browser, compiler, etc.◦ Standard library code (C++ STL, .Net BCL) they utilize
Challenges◦ Data structure layouts of composite types◦ Complex data driven control flow◦ Wide ranging code restructuring is often needed
We know most of the needed “tricks” but:◦ Time and implementation effort too large to identify and implement all of them◦ “An Evaluation of Vectorizing Compilers” PACT ‘11
Example (Exists Function)
struct { int tag; int score; } widget;
int exists(widget* vals, int len, int t, int s) { for(int i = 0; i < len; ++i) { int tagok = vals[i].tag == t; int scoreok = vals[i].score > s; int andok = tagok & scoreok; if(andok) return 1; } return 0; }
SIMD Example (Exists Function)… for(; i < (len - 3); i += 4) { m128i blck1 = load_128(vals, i); m128i blck2 = load_128(vals, i + 4);
m128i tagvs = shuffle_i32(blck1, blck2, ORDER(0, 2, 0, 2)); m128i scorevs = shuffle_i32(blck1, blck2, ORDER(1, 3, 1, 3));
m128i cmptag = cmpeq_i32(vectv, tagvs); m128i cmpscore = cmpgt_i32(vecsv, scorevs); m128i cmpr = and_i128(cmptag, cmpscore);
int match = !allzeros(cmpr); if (match) return 1; }…
[ti, si, ti+1, si+1]
[ti+2, si+2, ti+3, si+3]
[ti, ti+1, ti+2, ti+3]
[si, si+1, si+2, si+3]
[ti==t ? 0xF…F : 0x0, …, ti+3==t ? 0xF…F : 0x0]
[si>s ? 0xF…F : 0x0, …, si+3>s ? 0xF…F : 0x0][cmptag0 & cmpscore0, …, cmptag3 & cmpscore3]
(cmpr0!=0 | cmpr1!=0 | cmpr2!=0 | cmpr3!=0)
Performance Impact
4 8 16 32 64 128 256 512 1024 20480.5
1
1.5
2
2.5
Exists Speedup
Array Size
Spee
dup
Overview of Approach Deductive Rewriting of program source to:
◦ Identify high-level structures of interest◦ Rewrite to expose latent parallelism (split, unroll, etc.) and straighten hot-paths
Relational Verification techniques used to:◦ Construct the needed synthesis conditions (for code involving loops!)◦ Produce proof for semantic equivalence of input and result code
Inductive Synthesis of SIMD program fragments to:◦ Identify the best SIMD realizations of the synthesis conditions◦ Produce proofs of correctness wrt. synthesis conditions
Methodology more general than just SIMD Loops!
From Verification to Synthesis Condition Generation
Relational Verification:◦ Prove two programs equivalent under equivalence relations on states
◦ y = x◦ y = x1 + x2 + x3 + x4
◦ y = 5◦ Only a few standard equivalence relations needed in practice
Prove results of two programs are equivalent by showing:◦ If the programs are synchronously executed then at synchronization points the program
states are always equivalent under the relations◦ For our purposes at the start and end of the loop body
Relational Verification
int suml = 0;for(int i = 0; i < len; i+=4){ suml = suml + A[i]; suml = suml + A[i+1]; suml = suml + A[i+2]; suml = suml + A[i+3];}
int sumr = 0;int as0, as1, as2, as3 = 0;for(int i = 0; i < len; i+=4){ as0 = as0 + A[i]; as1 = as1 + A[i+1]; as2 = as2 + A[i+2]; as3 = as3 + A[i+3];}sumr = as0 + as1 + as2 + as3;
Full Loop Invariant: Relational Invariant:
From Verification to Condition Generation
We use “Product Programs” approach◦ “Relational verification using product programs” FM ‘11◦ Rename variables in “left” and “right” programs disjointly◦ Interleave the programs “appropriately”◦ Generates verification conditions on the combined program
Key Idea: ◦ Replace code in “right” program with uninterpreted Function ()◦ Perform Product program construction and VC generation◦ Resulting VC for are needed synthesis pre/post conditions
Relational Synthesis Conditionint suml = 0;for(int i = 0; i < len; i+=4){ suml = suml + A[i]; suml = suml + A[i+1]; suml = suml + A[i+2]; suml = suml + A[i+3];}
int sumr = 0;m128i ac = [0, 0, 0, 0];for(int i = 0; i < len; i+=4){ }sumr = ac.0 + ac.1 + ac.2 + ac.3;
Relational Invariant:
Resulting Synthesis Condition Pre-condition:
◦ ac == [v1, v2, v3, v4]
Post-condtion: ◦ ac == [v1 + A[i], v2 + A[i+1], v3 + A[i+2], v4 + A[i+3]]
Instruction Sequence Search Search space for SIMD instruction sequences is large
◦ Length: frequently need 8 or more instructions◦ Branching: SSE has 200+ instructions
Concrete state space exploration◦ Explore program states instead of instruction sequences◦ Use concrete execution to quickly exclude many candidate instruction sequences
Query SMT solver for a counter example input◦ Eventually either no counter examples or give up
Search for alternative sequences◦ Can generate multiple solutions to find best performance on varying data sizes
Optimize Search Cost model provides upper bound on depth of search
◦ Also used to pick best operation to explore next and to pick shortest path from input to output state
Incrementally expand available instruction set ◦ Start with standard operations (and those seen in input code)◦ Add more specialized operations if desired
Generate multiple initial input-output pairs ◦ One per path in original loop body
Stack machine construction to reduce the branching factor
Cost Model Do not want to compute absolute costs
◦ A very hard problem
Compute relative costs◦ Both programs run on the same data so same cache misses and branch taken/not taken◦ Build simple machine model to encapsulate instruction costs
Cost function a polynomial in terms of loop counts and branch rates◦ Use conservative static estimates for synthesis◦ Can use runtime data for selection in JIT setting
Complete Algorithm
Final SIMD Program
…
for(i I by 4c){
}
…
Restructured Program
…
for(i I by 4c){
}
…
Input Program
…for(i I by c){
}…
Input Program
…for(i I by c){
}…
Restructured Program
…
for(i I by 4c){
}
…
Synthesize
Synth. Cond. Generation
Final SIMD Program
…
for(i I by 4c){
}
…
Cost Ranking Function
Correctness Proof
Body
Synthesis Cond.
Optimistic Vectorize
Restructure Loop
Merge & Cleanup
Cost Score
Simulation Relation (Eq)
CPU Model
SIMD Standard Library Synthesize SIMD implementations of C++ STL and .Net BCL code
Consistent performance improvements◦ Between 2x-4x on large inputs◦ Avoid performance degradation on small inputs
Cost model accurately predicts performance◦ Can pick best implementation based on hardware and input data
Library Function Performance
4 8 16 32 64 128 256 512 1024 20480.5
1
1.5
2
2.5
3
3.5
4
Cyclic Hash
Actual Predicted
4 8 16 32 64 128 256 512 1024 20480.5
1
1.5
2
2.5
3
3.5
4
CountIf
Actual Predicted
String Processing Synthesize standard string functions using PCMPESTRI
◦ Packed Compare Explicit Length Strings, Return Index
Encoded semantics and provided them to synthesizer◦ Synthesized range of common string functions with no other changes◦ Speedup of 3.4x for String.Equals◦ Speedup up to 9.5x for String.IndexOfAny
Impact In Practice 483.Xalan (SPEC CPU)
XML processing framework written in C++
Replaced STL calls with our SIMD implementations
Performance sensitive to input data ◦ Previous work replacing these calls with set structures was +15% to -20% on different data
Synthesized SIMD code produces consistent 2%-5% speedup ◦ Indicates a 1.15x to 1.5x speedup in the STL code which is inline with cost model predictions
Benefits of Approach Proof of correctness from original loop and SIMD version
Separation of correctness and optimization◦ Transform for performant code structure◦ If incorrect proof (or synthesis) will fail later
Approach consistently produces fast SIMD code◦ Robust to details of SIMD instruction set and loop patterns◦ 2x-4x speedups obtained from synthesized SIMD code
Future Work Pointers and object structures
◦ Scatter-Gather support will help ◦ Compact object graphs into arrays (current work)◦ Can we do local data structure transformations?
Apply technique to larger structures and more generally◦ What about loops with small inner-loops (HashTable lookup)?◦ Can we use synthesis as part of general code-gen?
Big Picture Conclusions Big challenges and big benefits using specialized hardware
◦ Both performance and power!
Synthesis complements compilation◦ Small step vs. big step code generation◦ Verification structures synthesis (and eliminates compilation bugs)◦ Can we apply ideas to other compiler actions? Target other hardware?
Idea more general than just compilers or SIMD synthesis◦ Expert provided deductive structure ◦ Inductive synthesis driven by underlying semantics◦ A powerful combination for approaching problems