Protocol verificationA good model is hard to find

John O’Leary

Murali Talupur

Mark TuttleJin Yang

with David James

“Lookin’ for models in all the wrong places.Lookin’ for models in too many faces.

Searching their eyes, looking for tracesof what I'm dreaming of.”

(The tune sung by true Urban Cowboys everywhere)

How to do protocol verification

• Protocol verification in “three easy steps”– Find an interesting protocol – Build a model of the protocol– Verify the protocol

• Building the model can be the hardest step at Intel– See my last two talks at SCL for why [Leonard Tuttle 2006]– Our tools typically assume the existence of a model– Our work often ignores the problem of getting a model

• We need to think hard about where models come from– “Who builds them, who owns them, who maintains them?” -Earlie– Wide-spread technology transfer won’t happen without an answer

The Superman approach• A single verification expert joins a group (late)

– Learns the protocol– Builds the model– Maintains the model– Does the verification (finally)

• This is the wrong approach – It doesn’t scale (very few supermen)– It burns out the supermen (leaving even fewer supermen)– It won’t get much past “technology demonstration”

We need a better approach• We need a cost-effective story for model building:

– Write models our way– It doesn’t cost you any more (much more) than what you do now– And you get significant risk reduction from our analysis

• Even better, accept their way as our way:– Make good use of design artifacts they produce now

• MAS, state transition tables, block diagrams, pipeline diagrams, timing diagrams, message flows

– Make it attractive for them to use what we produce• Elegant models of what is missing in the MAS

Protocol verification• Mission statement:

– Delight engineering organizations by relentlessly reading their documentation and continuously asking questions about their intentions;

– Build operational excellence by driving to increase clarity while ceaselessly pushing to decrease confusion;

– Commit to deliver technology solutions that ensure Intel's leadership in model building, make model-building leadership technology affordable, and fill the pipeline with pipeline models;

– Intituitionalize the author of this mission statement.

Protocol verification• Project goals:

– Verify an unusual cache coherence protocol at Intel– Learn to extract a formal model from a MAS typical of Intel

• Talk outline:– The SIMPL cache coherence protocol and its MAS– A new tool for extracting a model from a MAS– An evaluation of this new tool vs other tools

• A very unusual cache coherence protocol by David James:

Cached copies maintained on a doubly-linked list

• Shared/Invalid/Modifiable cache states on Pointer Lists– Scalable: constant directory size (flexible core tile layouts) – Efficient: optimized pairwise sharing, thread synchronization

SIMPL

S S S S S S

SS

centralized directory distributed directory

Common flows straightforwardExample: read exclusive invalidates shared copies

- S S

S

read invaldata

ack

M I I

M

Efficient: pairwise sharingSupports producer-consumer without going to memory

Mh St

M

modified shared copy

shared copy

producer writes

Mp nSt

M

modified exclusive

copy

missing shared copy

Mh St

M

consumer reads

inval

pair

read

data

Tricky: deadlock avoidanceSuppose cache at the head of list wants to flush its copy

Suppose memory’s request queue is full

S S S

S

head

head doit

ack

- S S

S

S S S

S

head

head

ack’

Piggyback doit request on ack response

Suppose new head’s request queue was also full?

SIMPL

A big protocol: 37 cache states

I F cS nS cSh nSh cSb nSb cSt nSt cM dM nM pM cMh dMh nMh cMp dMp nMp IH IS B R W cSMh cSMb nSMh

nSMb cSM cSMt nSM nSMt dN dNb nNb dNt

A big MAS: 300 pages

SIMPL MAS

A state transition table

• What to do when a core issues a request

• One of seven state transition tables

Preconditions Effects

One state transition

What is this? A cache state

Where does it come from? Another table!

We should be able to use this!

Cache states

We should be able to use this!

What is this? A function call

Where does it come from? Another table!

We should be able to use this!

What is this? Another function call

Where does it come from? Another table!

We should be able to use this!

What is this? Another function call

Where does it come from? A table footnote!

MAS lessons learned• Observations

– Tables depend on tables depend on tables depend on …– Every table likely to change, all must be first class objects– Every table is used in some context, need lexical scoping– Columns have a limited number of interpretations (almost)

• Some hand coding required– Some state entirely missing or vaguely defined

• Communication model, some data structures• If we model it, can it become part of the MAS?

• Some clean up required– Typos, unusual notation, strange formatting artifacts

• Many shorthands used: cPtr, tPtr – x = ECODE really means x = EXCL | x = MOD | x = …

Our solution

• This is the dream:– A front end: extract formal models from design artifacts– A back end: extracts verification artifacts from the formal model

• This is the current state: Murphi input from MAS tables

MAS Transition tables Block diagrams Pipeline diagrams Timing diagrams

Reference models

Hand-coded modeling

Protocol state

machine

FV tool input Murphi, MC/BMC SMT Theorem prover

DV reference models System Verilog SystemC

MAS Transition tables FV tool input

Murphi, MC/BMC

Protocol state

machine

Our solution• Other features

– An internal model of the protocol state machine– A small, expressive collection of table column semantics– A system of rewrites to deal with

• Columns not mapping cleanly into our semantics• Tables requiring clean up before interpretation

State machine model• A modeling language of types, expressions, transitions

– currently looks like murphi, soon to look like unity written in ocaml

• Model fragments built with help of a parser– parse_type “array[1..4] of boolean”

ArrayType (RangeType (IntExpr 1, IntExpr 4), IdType "boolean")– parse_stmt “x := y”

AsgnStmt (IdLval "x", LvalExpr (IdLval "y"))– parse_expr “rec.field = table[index]”

CallExpr ("=",

[LvalExpr (FieldLval (IdLval "rec", "field"));

LvalExpr (IndexLval (IdLval "table", LvalExpr (IdLval "index")))])

Column semantics

• This table could define a– state transition

– function

– enum or record type

• What is the meaning of “cell” in column “hdr”?

• Test: hdr = cell– Cell defines a comparison

• Set: hdr := cell– Cell defines an assignment

• Eval: cell– Cell is a predicate function

– Cell is a side-effecting function

• Null: string– Who knows? Code by hand.

• Enum, FieldName, FieldType, …

• With a small set of canonical semantics, we can give meaning to any table

hdr

cell

Example: the table

• Consider a table of predicates:– Name column is a predicate name– Row column is a junk column– All other columns are tests

Example: the column semantics [ {(Mapping.bindcol ["Name"]) with semantics = Bind [("Q1DO",unit -> boolean); ("Q1GO",unit -> boolean); ("Q0", unit -> boolean); ("Q0AF",unit -> boolean); ("Q0AT",unit -> boolean)] }; Mapping.nullcol ["Row"]; Mapping.testcol ["cqReq"; "code"]; Mapping.testcol ["cPtr"; "level"]; Mapping.testcol ["cPtr"; "aged"]; Mapping.testcol ["tPtr"; "done"]; Mapping.testcol ["tPtr"; "sleep"]; Mapping.testcol ["tPtr"; "code0"]; Mapping.testcol ["tPtr"; "fCode"]; Mapping.testcol ["tPtr"; "tCode"]; Mapping.testcol ["tPtr"; "body"]; Mapping.testcol ["tPtr"; "head"]; Mapping.testcol ["tPtr"; "tail"]; Mapping.testcol ["QC"] ]

Rewrite rules• Rewrite rules enable “tweaking” of

– Column semantics: our semantics may not be exactly right– Table input: tables may require cleaning up before interpretation

• Rewrite rules have two forms:– String based: Rewrite.regexp– Model based: Rewrite.expr, Rewrite.stmt, Rewrite.type, …

• Rewrite rules are the foundation of our generality• Rewrite rules and canonical semantics combine well

– balance need for clean semantics and extensible semantics

String rewrites

Rewrite.regexp regexp pattern

String rewrites

Rewrite.regexp regexp pattern

Rewrite.regexp cPtr cache[ReqPid][ReqAdr]

cPtr.state → cache[ReqPid][ReqAdr].state

String rewrites

Rewrite.regexp regexp pattern

Rewrite.regexp cPtr cache[ReqPid][ReqAdr]

cPtr.state → cache[ReqPid][ReqAdr].state

Rewrite.regexp \(.*\) CPU_\1

Read → CPU_Read

Write → CPU_Write

Lock → CPU_Lock

Model rewrites via matching

Rewrite.expr 2 * x_ x_ + x_

Model rewrites via matching

Rewrite.expr 2 * x_ x_ + x_

x+y = 2*(a+b) → x+y = (a+b) + (a+b)

Model rewrites via matching

Rewrite.expr 2 * x_ x_ + x_

x+y = 2*(a+b) → x+y = (a+b) + (a+b)

x+y = 2*(2*(a+b)) → x+y = ((a+b)+(a+b)) + ((a+b)+(a+b))

Model rewrites via matching

Rewrite.expr 2 * x_ x_ + x_

x+y = 2*(a+b) → x+y = (a+b) + (a+b)

x+y = 2*(2*(a+b)) → x+y = ((a+b)+(a+b)) + ((a+b)+(a+b))

Rewrite.expr x_ = true x_

Rewrite.expr x_ = false not(x_)

Rewrite.expr not(x_ | y_) not(x_) & not(y_)

Model rewrites via matching

Rewrite.expr 2 * x_ x_ + x_

x+y = 2*(a+b) → x+y = (a+b) + (a+b)

x+y = 2*(2*(a+b)) → x+y = ((a+b)+(a+b)) + ((a+b)+(a+b))

Rewrite.expr x_ = true x_

Rewrite.expr x_ = false not(x_)

Rewrite.expr not(x_ | y_) not(x_) & not(y_)

(a=true) | (b=true) = false → not(a | b) → not(a) & not(b)

Model rewrites via matching

Rewrite.expr 2 * x_ x_ + x_

x+y = 2*(a+b) → x+y = (a+b) + (a+b)

x+y = 2*(2*(a+b)) → x+y = ((a+b)+(a+b)) + ((a+b)+(a+b))

Rewrite.expr x_ = true x_

Rewrite.expr x_ = false not(x_)

Rewrite.expr not(x_ | y_) not(x_) & not(y_)

(a=true) | (b=true) = false → not(a | b) → not(a) & not(b)

Rewrite.expr x y

Model rewrites via matching

Rewrite.expr 2 * x_ x_ + x_

x+y = 2*(a+b) → x+y = (a+b) + (a+b)

x+y = 2*(2*(a+b)) → x+y = ((a+b)+(a+b)) + ((a+b)+(a+b))

Rewrite.expr x_ = true x_

Rewrite.expr x_ = false not(x_)

Rewrite.expr not(x_ | y_) not(x_) & not(y_)

(a=true) | (b=true) = false → not(a | b) → not(a) & not(b)

Rewrite.expr x y

array[1..x] of boolean → array[1..y] of boolean

Example: table cleanup

• Clean up column headers:– cqReq. → cqReq, cPtr-> → cPtr, tPtr-> → tPtr

Example: table rewrites

Table.changeHeaders[

Table.rewriteHeader[

Rewrite.regexp "cqReq \\." "cqReq";

Rewrite.regexp "cPtr ->" "cPtr";

Rewrite.regexp "tPtr ->" "tPtr" ]]

table

Example: tweak column semantics

• Test column semantics not quite right:– tPtr.code0 = N should really be isNull(tPtr.code0)– cqReq.code = NN should really be not(isNull(cqReq.code))– T and F should really be true and false

Example: semantics tweaks

Rewrite.expr "x_ = N" "isNull(x_)";

Rewrite.expr "x_ = NN" "not(isNull(x_))";

Rewrite.rhs (Rewrite.expr "T" "true");

Rewrite.rhs (Rewrite.expr "F" "false");

Semantic maps

map = {

name = “action name”;

params = [action parameters];

filename = “file containing table”;

headers = number of header rows;

cleanups = [table rewrites];

uses = [handcoded declarations];

dependson = [semantic maps for other tables];

rewrites = [model rewrite rules];

cols = [column semantic dfns]}

Semantic maps give meanings to tables

Configuration files

let handcoded = "declarations"let cachestates = {map for an enumeration type}let setup = {map for a function}let effects = {map for many functions … dependson = [setup]} let requestaction = {map for state transitions … dependson = [effects]; uses = [handcoded]}

let model = new Environmentlet model = State.doTable model cachestateslet model = Action.doTable model requestactionlet _ = ppMurphi model

Configuration files build models from tables

Tables aren’t everything• We are focused on tables because they are easy• Designers don’t do everything with tables

– Block diagrams, timing diagrams, pipeline diagrams, message flows

• We would like to use these artifacts, too, eventually• Our goal is to use all these sources to build a single model

Other table approaches• Table-based specification is not new

– Parnas tables– Heitmeyer tables– Ptables by Chou, Mannava, and Park for CSI

• Parnas and Heitmeyer focused on controls systems– Flight control system, nuclear reactor control, auto pilots– You have a set of inputs you can use to control a system– You have a set of sensors that monitor a system– For each input, how should you set it based on sensor readings?– Not a natural way to think about cache coherence protocols

• Ptables are the closest to our work– And inspired our work!!

Ptable approach• Ptables are about

– Extracting a textual state transition table from a design document

– Assigning meaning to the table

• Ptables take a table as written and do something with it– No language or GUI for building the table– No rigid philosophy about how protocols should be described

• We buy this approach completely

Ptable approach• Semantic maps require writing a transformation

“cell value” => “murphi fragment”for every cell value in the ptable

• Ptables assemble these fragments into a murphi model

• This approach is universal: there is a map for every table– It could be a lot to write (every test or assignment in the model)– It could be a lot to maintain for a protocol described like SIMPL

• So much is in the supporting functions and not the table– And what if you don’t want to generate murphi any more?

Our approach• We interpret several kinds of tables• We build a real model

– Potential for static analysis of the model– Potential for generating SystemC, Verilog, SMT questions,…

• We have a set of canonical semantics for columns– Potential for static analysis of the maps themselves

• We have a system of rewrite rules– Bridges the gap between

• canonical semantics and arbitrary tables• our modeling language in tables and arbitrary tables

– Provides a smooth trade-off between our way and their way

Our maps are expressive• A ptable semantic map might say

mapping = [

(“cell value1” => “murphi fragment1”);

(“cell value2” => “murphi fragment2”);

(“cell value3” => “murphi fragment3”)]

• Our semantic map would sayMapping.evalcol with semantics = Null [

Rewrite.regexp “cell value1” “murphi fragment1”;

Rewrite.regexp “cell value2” “murphi fragment2”;

Rewrite.regexp “cell value3” “murphi fragment3”]

Our maps can be simple

Ptable map for one SIMPL column:

mapping = [ ("B" => (cState = "B")); ("F" => (cState = "F")); ("I" => (cState = "I")); ("I, cS, nS, cSt, nSt cSb, nSb cSh, nSh cM, dM, nM cMp, dMp, nMp cMh, dMh,

nMh" => (cState = "I") | (cState = "cS") | (cState = "nS") | (cState = "cSt") | (cState = "nSt") | (cState = "cSb") | (cState = "nSb") | (cState = "cSh") | (cState = "nSh") | (cState = "cM") | (cState = "dM") | (cState = "nM") | (cState = "cMp") | (cState = "dMp") | (cState = "nMp") | (cState = "cMh") | (cState = "dMh") | (cState = "nMh")]); ("R" => (cState = "R")); ("W" => (cState = "W")); ("cM, nM" => (cState = "cM") | (cState = "nM")); ("cMh, dMh" => (cState = "cMh") | (cState = "dMh")); ("cMp" => (cState = "cMp")); ("cMp, dMp" => (cState = "cMp") | (cState = "dMp")); ("cMp, nMp cMh, nMh" => (cState = "cMp") | (cState = "nMp") | (cState = "cMh") | (cState = "nMh")); ("cS" => (cState = "cS")); ("cS, cSt cSb, cSh cM, dM cMp, dMp cMh, dMh" => (cState = "cS") | (cState = "cSt") | (cState = "cSb") | (cState = "cSh") | (cState = "cM") | (cState = "dM") | (cState = "cMp") | (cState = "dMp") | (cState = "cMh") | (cState = "dMh")); ("cS, nS" => (cState = "cS") | (cState = "nS")); ("cSb" => (cState = "cSb")); ("cSb, nSb" => (cState = "cSb") | (cState = "nSb")); ("cSh" => (cState = "cSh")); ("cSh, nSh" => (cState = "cSh") | (cState = "nSh")); ("cSt" => (cState = "cSt")); ("cSt, nSt" => (cState = "cSt") | (cState = "nSt")); ("dM" => (cState = "dM")); ("dMp" => (cState = "dMp")); ("dMp, dMh" => (cState = "dMp") | (cState = "dMh")); ("nM" => (cState = "nM")); ("nMp, nMh" => (cState = "nMp") | (cState = "nMh")); ("nNb" => (cState = "nNb")); ("nS" => (cState = "nS")); ("nSb" => (cState = "nSb")); ("nSh" => (cState = "nSh")); ("nSt" => (cState = "nSt"))]

Our maps can be simple

Our map for

the same SIMPL column:Mapping.testcol

Closing words of caution• Our code is very rough

– Constantly changing ideas were simply cobbled together– But we couldn’t have written this talk without writing this code

• We wanted to drive one example to completion– We did only what was needed for this example– We tried always to keep the big picture in mind

• Soon we will be in a good position to do this right.

Conclusions• A method of constructing models from tables

– Take the tables as they appear in the design documents– Impose no language or philosophy on the designers

• Future plans– Complete the SIMPL model (SIMPL itself was just cancelled)

• Rethink current design based on lessons learned– Consider other kinds of tables and design artifacts

• Incorporate Earlie’s list of table-based design styles• Message flow diagrams might be next artifact to consider

– Start work on a new protocol • Maybe from Extended Ring, Larabee, Poulson, …

Back up slides

Intel bites back

• “Don’t tell me all you’re giving me is correctness.”– correctness is not the problem in early design stages– performance, power, size, …

• “You only build models when you’re scared.” –Eugene Earlie

– and after seven coherence protocols, we’re not scared

• Models aren’t complete at any level of abstraction– fill out details where you are uncertain, leave the rest

• Models have value only when connected to RTL– What about documentation, design, understanding, debugging?

• “I’d find your bugs with a little more simulation.”