Tool-support for Invariant-based Tool-support for Invariant-based Specification, Synthesis, and Verification of Specification, Synthesis, and Verification of Synchronization in Concurrent Java Synchronization in Concurrent Java ProgramsPrograms

M.S. Defense

William DengDepartment of Computing and Information Sciences

Kansas State University

http://www.cis.ksu.edu/saves

Goals of the ProjectGoals of the Project

II. Automatic derivation and weaving of synchronization code… multiple language and

synchronization targets (Java, C++, monitors, semaphores, etc.)

… weaving & optimization via abstract interpretation and program specialization techniques

III. Automatic verification of critical safety and liveness properties of woven embedded code

… domain-specific model-checking engines

… built on previous DARPA work –Bandera environment

I. Provide high-level, modular specification of global synchronization aspects

… integrated with UML/RUP

… formal specification via global invariants

… language of composable invariant patterns

… powerful, yet easy to use

IV. Evaluation using Boeing BOLDSTROKE platform and military networking target vehicle electronics (CDA101)

My ContributionsMy Contributions

III. Automatic verification of critical safety and liveness properties of woven embedded code

… Designed and implemented bounded counter version (coarse & fine grain)

I. Provide high-level, modular specification of global synchronization aspects

… Implemented formula & pattern-based specification language

… Designed web pages to guide users in synchronization specification

II. Automatic derivation and weaving of synchronization code… Designed and implemented

translation from specifications to coarse-grain solution

… Implemented translation to Java fine-grain solution

… Implemented language-independent weaving process

… Designed and implemented an approach for specific synchronization

SyncGen Tool ArchitectureSyncGen Tool Architecture

Both coarse-grain and fine-grain solutions are synthesized automatically

.java.java

.java .java

+ Invariant

Fine-grainJavaRepresentationGenerator

IntermediateRepresentationGenerator

PVS

.java+ guarded commands

.java

+

.java

Core code

Synchronizationaspect

SyncGen ContextSyncGen Context

UML Tools

SynchronizationAspectSpecificationTool

IntermediateRepresentationGenerator

Solver/Prover

Course-grainsolution

SynchronizationAspectBack-end

BanderaAnalysis &Transformation

Fine-grainsolution

Specialization EngineBandera

SafetyProperties

LivenessProperties

Code Weaver

OptimizedWoven Code

Invariant &Region tags

Functional CoreCode Templates

(Java, C++, …)

TemplateInstantiation

TraditionalDevelopmentEnvironment

Functional CoreCode

(Java, C++, …)

Finite StateModels

OutlineOutline

Coarse-grain solution generation– Guards

• Formula-based• Pattern-based

– Notification• Formula/Pattern-based

Specific synchronization Bounded Counter Version Open Issues Conclusion

Gyroscope/Rudder SynchronizationGyroscope/Rudder Synchronization

GyroscopeController

Buffer

Produce Consume

Consume Produce

RudderController

RG RR

Gyroscope Value

Resource(RG,1,RR ,1,0)

Empty Buffer SlotResource(RR,1,RG ,1,1)

Exclusion(RG,RR)

Resource(RG,1,RR ,1,0) + Resource(RR,1,RG ,1,1) + Exclusion(RG,RR) Invariant:

Coarse-grain Solution for GuardsCoarse-grain Solution for Guards--- Formula-based Generation--- Formula-based Generation

Resource(RG,1,RR,1,0) + Resource(RR,1,RG,1,1) + Exclusion(RG,RR) Invariant:

Resource(RG,1,RR,1,0) R_in <= G_out

(R_in <= G_out) && (G_in <= R_out + 1) && ((G_in == G_out) || (R_in == R_out))

Desugared Invariant:

Resource(RR,1,RG,1,1)

Exclusion(RG,RR)

G_in <= R_out + 1

(G_in == G_out) || (R_in == R_out)

…producer out

…consumer in

Generating Coarse-grain SolutionGenerating Coarse-grain Solution

G_in++…invariant I holds here

…want I to hold here

Task: generate a condition B that ensures that I holds after counter increment.

Step 1: generate weakest-precondition(G_in++,I)

(R_in <= G_out) && (G_in <= R_out + 1) && ((G_in == G_out) || (R_in == R_out))

(R_in <= G_out) && (G_in+1 <= R_out + 1) && ((G_in+1 == G_out) || (R_in == R_out))

SubstituteG_in+1 forG_in

<await B -> G_in++>

Generating Coarse-grain SolutionGenerating Coarse-grain Solution

Step 2: simplify using decision procedures

1. Convert to disjunctive normal form2. Eliminate disjuncts that are can never be satisfied using decision procedures3. Minimize remaining conjuncts using decision procedures

Example:

(R_in <= G_out) && (G_in+1 <= R_out+1) && (G_in+1 == G_out) || (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out)

1.

(R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out)2.

(G_in+1 <= R_out+1) && (R_in == R_out)3.

Main point: fine a smaller B’ such that B’ && I && A <==>B && I && A.

Generating Coarse-grain SolutionGenerating Coarse-grain Solution

Step 2.1: convert to disjunctive normal form

(R_in <= G_out) && (G_in+1 <= R_out + 1) && ((G_in+1 == G_out) || (R_in == R_out))

(R_in <= G_out) && (G_in+1 <= R_out+1) && (G_in+1 == G_out) || (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out)

Generating Coarse-grain SolutionGenerating Coarse-grain Solution

Step 2.2: eliminate unsatisfiable disjuncts

(R_in <= G_out) && (G_in+1 <= R_out+1) && (G_in+1 == G_out) || (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out)

Intuition: before entrance to region, we know I holds and we know that A holds where A represents some basic properties on counters

A = R_in >= 0 && R_out >= 0 && R_in >= R_out && G_in >= 0 && G_out >= 0 && G_in >= G_out

(R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out)

For each disjunct D_k, ask theorem prover to check not(I && A && D_k). If return is truethen D_k can be eliminated.

Generating Coarse-grain SolutionGenerating Coarse-grain Solution

Step 2.3: minimize remaining disjuncts

(R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out)

Some of conjuncts in the remaining disjuncts are redundant.

For each C_k in D_j, if (I && A && (D_j – C_k) => C_k) then C_k can be removed.

(G_in+1 <= R_out+1) && (R_in == R_out)

Generating Coarse-grain Solution Generating Coarse-grain Solution Pattern Based ApproachPattern Based Approach

Apply the process for each pattern:

For pattern P(R_1,R_2,…,R_n)

Subregion Guard

R_i entry wp(In_i++,I)

R_i exit wp(Out_i++,I)

Generating Coarse-grain Solution Generating Coarse-grain Solution BoundBound

Apply the process for Bound pattern:

Bound(R,n):I = In – Out <= n

Subregion Guard

R entry wp(In++, In – Out <= n) = (In + 1) – Out <= n

R exit wp(Out++, In – Out <= n) = In – (Out + 1) <= n ´ True

Generating Coarse-grain Solution Generating Coarse-grain Solution ExclusionExclusion

Apply the process for Exclusion pattern:

Exclusion(R_1,R_2,…,R_n):I = ÇC 2 Comb(n, n-1) (Æi 2 C (In_i - Out_i == 0))

Subregion Guard

R_i entry wp(In_i++,I) = Æ1 · j · n Æ j i (In_j – Out_j == 0)

R_i exit wp(Out_i++,I) = true

Generating Coarse-grain Solution Generating Coarse-grain Solution ResourceResource

Apply the process for Resource pattern:

Resource(R_p,N_p,R_c,N_c,n):I = In_c*N_c – Out_p*N_p <= n

Subregion Guard

R_p entry wp(In_p++,I) = true

R_p exit wp(Out_p++,I) = true

R_c entry wp(In_c++,I) = In_c*N_c – Out_p*N_p <= n – N_c

R_c exit wp(Out_c++,I) = true

Generating Coarse-grain SolutionGenerating Coarse-grain Solution--- --- Building Guards From Relevant PatternsBuilding Guards From Relevant Patterns

P_1(……) + P_2(……) + ... + P_n(……) Invariant:

Each pattern instance contributes a portion of the guard

R

Counter++ <await B_1 && B_2 && … && B_n -> Counter++>

Intuitively, if R does not appear in P_i, then B_i is true

Why this is correct? wp(S,I_1 &&…&&I_n) = wp(S,I_1) && … wp(S,I_n)

Generating Coarse-grain Solution Generating Coarse-grain Solution Pattern Based ApproachPattern Based Approach

Compared with result generated by formula

G_in++ <await B -> G_in++>

Resource(RG,1,RR ,1,1,0) B_1 = true

Resource(RR,1,RG ,1,1) B_2 = G_in-R_out <=0

Exclusion(RG,RR) B_3 = R_in == R_out

B = B_1 && B_2 && B_3 = G_in – R_out <= 0 && R_in == R_out

Example: the entry of gyroscope region

B’ = (G_in+1 <= R_out+1) && (R_in == R_out)

AssessmentAssessment

In essence, we have performed the formula-based calculation statically to create guard schemas which are then instantiated when a pattern is used

Efficientno need to call DP, convert to DNF, etc.

Easy to optimizefor example: for Bound(R,1), the guard of the entry of R should be in +1 <= out +1 generated by formula; in == out by pattern.

Notification InformationNotification Information

If a counter C_1 increment can cause an await statement A transition from false to true, then there should be a notify/notifyall issue to A.

A: <await B_2 -> C_2++>

B: {P}C_1++{Q}

Observation: we can calculate Q as the strong postcondition: sp(C_1++,P); P can be I&&G (G is the guard of B).

Thus if Q implies B_2, we should issue notify/notifyall.

Notification Information Example: Notification Information Example: Readers/Writers ProblemReaders/Writers Problem

<await In_w - Out_w == 0 -> In_r ++>

Reader Entry

Reader Exit

{ In_w - Out_w == 0 && In_r - Out_r >= 1 } <Out_r++> { In_w - Out_w == 0 && In_r - Out_r >= 0 }

Writer Entry

<await In_w - Out_w == 0 && In_r - Out_r == 0 -> In_w ++>

Notify reader entry?

Writer Exit

{ In_w - Out_w == 1 && In_r - Out_r == 0 } <Out_w++> { In_w - Out_w == 0 && In_r - Out_r == 0 }Notify writer entry

Notifyall reader enter

Notify writer entry

Notification InformationNotification Information

P_1 Not (P_2 implies P_1)

If Q_2 implies P_1Then S_2 needs to notify S_1

S_2

P_2

Q_1 Q_2

S_1

Q_2 = sp(S_2, P_2)

Generalize sp tospn(S_2,P_2)

If exists n s.t. spn(S_2,P_2) implies P_1Then S_2 should notify/notifyall S_1.

We want to decide the notification information from S_2 to S_1

Notifyall vs. NotifyNotifyall vs. Notify

1. Resource(R_p,3,R_c,1,0):one enables three.

2. Resource(R_p,3,R_c,2,0):two enables three.

3. Exclusion(R_1,…,R_n):when (In_i – Out_i == 0)…

Three cases for Notifyall:

Cases 1 and 2 are normal:

Case 3 is sort of burst:

Three cases for Notify:1. Writer exit to writer entry in

Readers/writers:one enables one.

2. Resource(R_p,1,R_c,3,0):three enables one.

3. Group(R1,1,…,Rn,Nn):Ni enables one for R1…

Formalize notifyall (from S_2 to S_1):

Notification Information Notification Information Generation for BoundGeneration for BoundP_1 = In_B–Out_B<=n-1

Not (P_2 implies P_1)

Q_2 implies P_1Out_B++

P_2 = In_B–Out_B<=n

Q_1 = In_B–Out_B<=n Q_2 = In_B–Out_B<=n-1

In_B++

Entry Exit

…belongs to the case 1 of notify (one out let one enter). Thus a notify from the exit to the entry.

Notification Information for Notification Information for Exclusion PatternExclusion Pattern

Exclusion(R_1,R_2,…,R_n):

Subregion Notification Information

R_i entry None

R_i exit Notifyall R_j entry where i != j

Notification Information for Notification Information for Resource PatternResource Pattern

For pattern Resource(R_p,N_p,R_c,N_c,n):

Subregion Notification Information

R_p entry None

R_p exit Notifyall case 1: N_p/N_c >= 2; case 2: if 2>N_p/N_c>1. The exit of R_p to the entry of R_c. Notify case 1: if N_p/N_c =1; case 2, if N_p/N_c < 1.

R_c entry None

R_c exit None

Notification Information for Notification Information for Other Patterns-- SummaryOther Patterns-- Summary

Exclusion(R1,R2,..,Rn)Notifyall case 3: from the exit of Ri to the entry of Rj (i != j)

Resource(R_p,N_p,R_c,N_c,N)Notifyall case 1: N_p/N_c >= 2; case 2: if 2>N_p/N_c>1. The

exit of R_p to the entry of R_c. Notify case 1: if N_p/N_c =1; case 2, if N_p/N_c < 1.

Barrier(R1,R2)Notify case 1, the entry of R1 notify the exit of R2. Same for R2.

Relay(R1,R2) Notify case 1, the entry R1 notify the exit of R2.

Group(R1,N1,..,Rn,Nn)Notifyall case 3: if Nj>1, the entry of Ri notifyall the exit of Rj. Notify case 3, if Nj=1.

Notification Information for Notification Information for Gyroscope/RudderGyroscope/Rudder

Resource(RG,1,RR ,1,1,0) The exit of R_G notify theentry of R_R

Resource(RR,1,RG ,1,1)

Exclusion(RG,RR) The exit of R_R notifyall theentry of R_G; The exit of R_Gnotifyall the entry of R_R.

The exit of R_R notify theentry of R_G

Overall effect The exit of R_R notify theentry of R_G; The exit of R_Gnotify the entry of R_R.

Specific SynchronizationSpecific Synchronization

R_B1

R_B2

R_B3

R_C1

R_C2

R_C3

R_C4

R_C5

Barber Thread Customer Thread

fetch

inform

leave

B1 B2 C1 C2

Specific groups:0: B1 and C1;1: B2 and C2.

Problem: monitor can not

differentiate B1 and B2.

1. Strength monitor to keep track of the relation?2. Use object (multiple copies) instead of static method

(one copy) to keep the relation.

Relation: threads -> specific group

Solution:

Implementation of Specific Implementation of Specific SynchronizationSynchronization

Main (init)

Component Threads SynManager Cluster Objects

initializes initializes

call calls

Component Threads Cluster Classescall

Current Implementation:

Proposed approach: use a central controller, SynManager

Implementation of Specific Implementation of Specific Synchronization Cont.Synchronization Cont.

Forming

Cluster synchronization transition:

code

1. builds an array of cluster names;2. specifies synchronization (anonymous, forming, specific,

or dissolving) for each cluster;3. defines actors (the types of threads);4. specifies specific group;5. provides the maximum number of specific groups.

User has to provide in main

Specific

Inform

Anonymous Fetch

Dissolving leave

0 1

B1 C1 B2 C2

Specific group -> cluster instances

Threads -> specific group index

ClusterInstances

Bounded Counter VersionBounded Counter Version

Problem: unbounded In/Out counters…partial state space check

Bounded counter solution to check state space exhaustively.

Pattern Bounded Counter

Bound B = In - Out

Exclusion E_i = In_i – Out_i

Resource R_pc = In_c*N_c - Out_p*N_p

Example: Readers/Writers Example: Readers/Writers ProblemProblem

CLUSTER: RWSTATE SPACE VARIABLES: E1_0, E1_1, B1;LOCAL VARIABLES: ;REGION: ReaderENTER: <AWAIT E1_1 == 0 --> E1_0++;>NOTIFY: ;NOTIFYALL: ;EXIT: <E1_0--;>NOTIFY: ;NOTIFYALL: Writer_in;

REGION: WriterENTER: <AWAIT E1_0 == 0 && B1 == 0 --> E1_1++;B1++;>NOTIFY: ;NOTIFYALL: ;EXIT: <E1_1--;B1--;>NOTIFY: Writer_in;NOTIFYALL: Reader_in;

Open IssuesOpen Issues

Exception handlingDeadlock checking of SyncGen

outputProblem: nested or overlapping regions. Suggested solution: for nested or overlapping regions check B_1 && B_2… && B_n != false.

Scheduling issues: prioritiesResponses: balking and timeout…

ConclusionConclusion

Successfully implemented Mizuno’s approach

Developed the bounded counter version solution for model checking

Proposed an implementation for specific synchronization

To Do ListTo Do List

Web-pages, manual, and tutorialTutorial lecture for CIS720Clean up code for release

Questions?Comments?