Verification Case Studies with ObjectCheck
Fei Xie
(Joint work with James C. Browne, Robert P. Kurshan, and Vladimir Levin)
Presentation at Microsoft Research, April 10, 2003
2
Presentation Outline
• Overview and Architecture of ObjectCheck •• Modeling and Verification of a TinyOS Modeling and Verification of a TinyOS
RunRun--time Image with ObjectChecktime Image with ObjectCheck•• More Case StudiesMore Case Studies•• Summary and Future WorkSummary and Future Work• Work Built on ObjectCheck
3
ObjectCheck Overview
• An integrated development, validation, and mode-checking environment for software system designs;– System designs are specified in xUML;– xUML is an executable subset of UML;
• Developed in conjunction with – FormalCheck (Robert P. Kurshan, et. al.);– SDLCheck (Vladimir Levin and Husnu Yenigun);– Goal: Software/hardware co-design and co-verification;
• Sub-goal: Model checking of software system designs in xUML.
4
Executable UML (xUML)
• Has well-defined Execution Semantics;
• Utilizes UML Action Semantics recently adopted by OMG;
• Can be compiled to procedural codes;
• Tools provided by:– Project Technologies;– Kennedy Carter;– Hyperformix (SES);– …
5
Architecture and Workflow of ObjectCheck
Property Specification Interface xUML IDE Error Visualizer
xUML-to-S/R Translator
COSPAN Model Checker
S/R ModelS/R Query
Error Report
Error Track
Designer
xUML ModelProperty
Error Report Generator
6
Presentation Outline
• Overview and Architecture of ObjectCheck •• Modeling and Verification of a TinyOS Modeling and Verification of a TinyOS
RunRun--time Image with ObjectChecktime Image with ObjectCheck•• More Case StudiesMore Case Studies•• Summary and Future Work Summary and Future Work • Work Built on ObjectCheck
7
Case Study on TinyOS
• A run-time image of TinyOS, a component-based operating system for networked sensors;
• xUML models for TinyOS components have been constructed from their C source codes to enable:– Model-driven development on design level;– Software/hardware co-design and co-verification;
• Two properties are to be checked:– Repeated transmission on physical network;– No consecutive 0’s in any sequence-number sequence.
8
More on TinyOS
• An OS for networked sensors that have– Limited computation ability and energy supply;– High concurrency requirements;– Diversities in designs and usages;– High reliability requirement;
• Is component-based– Run-time images are specialized for different sensors;– Only necessary components are selected and composed.
• More information -- http://webs.cs.berkeley.edu/tos
9
Step-by-Step DemonstrationDesigner
Property Specification Interface xUML IDE Error Visualizer
Property xUML Model Error Report
xUML-to-S/R Translator Error Report Generator
S/R Query S/R Model Error Track
COSPAN Model Checker
10
11
12
13
14
15
16
17
18
Step-by-Step DemonstrationDesigner
Property Specification Interface
Property
xUML IDE Error Visualizer
xUML Model Error Report
xUML-to-S/R Translator Error Report Generator
S/R Query S/R Model Error Track
COSPAN Model Checker
19
20
Step-by-Step Demonstration
Property Specification Interface
Property
xUML-to-S/R Translator
Designer
xUML IDE Error Visualizer
xUML Model Error Report
Error Report Generator
S/R Query S/R Model Error Track
COSPAN Model Checker
21
22
23
24
Step-by-Step Demonstration
Property Specification Interface
Property
xUML-to-S/R Translator
Designer
xUML IDE Error Visualizer
xUML Model Error Report
Error Report Generator
S/R Query S/R Model Error Track
COSPAN Model Checker
25
26
27
28
29
30
31
32
Step-by-Step Demonstration
Property Specification Interface
Property
xUML-to-S/R Translator
Designer
xUML IDE Error Visualizer
xUML Model Error Report
Error Report Generator
S/R Query S/R Model Error Track
COSPAN Model Checker
33
34
35
Step-by-Step Demonstration
Property Specification Interface Error Visualizer
Property
xUML-to-S/R Translator
Designer
xUML IDE
xUML Model Error Report
Error Report Generator
S/R Query S/R Model Error Track
COSPAN Model Checker
36
37
38
39
40
41
42
43
Statistics from Model Checking “Repeated Output” Property
• Four model checking runs with different combinations of reduction algorithms – Start at the same time on the same host;– The host is a SUN with 8 CPUs and 2GB memory.
1384S80MOnOff
1379S102MOnOn17438S596MOffOn
19370S596MOffOffTime UsageMemory UsageSMCSPOR
Conclusion: SMC helps and SPOR doesn’t.
44
Presentation Outline
• Overview and Architecture of ObjectCheck •• Modeling and Verification of a TinyOS Modeling and Verification of a TinyOS
RunRun--time Image with ObjectChecktime Image with ObjectCheck•• More Case StudiesMore Case Studies•• Summary and Future WorkSummary and Future Work• Work Built on ObjectCheck
45
Model Checking of NASA robot Controller
• A typical control-intensive embedded system;• Conducted by Natasha Shyrigina using ObjectCheck;
– 37 properties were checked.– 22 properties were successfully checked.– 6 bugs were found.
consists of
defines position of
specifies
is a part of
R15
GlobalRepresentation
. number_of_joints
. type of search
. abort_var….
R1
R11
Follows to thespecified
R2
R5
R4
Kinematics
InverseKinematics
OSCAR Library
R7
R6
Trajectory (TJ)* TJ_ID. position. final_position. EE_ID(R3)
TrajectoryPoint (TP)*TP_ID. TJ_ID (R4). point
R3
R9
R10
has
can havedifferent
can have anumber of
Is A
is a component of
requires
OSCAR Libraries* OS_ID. TC_ID (R10). status* link-ID
R8
ForwardKinematics
Arm (A)* Arm_ID. arm_status
is configured by
End-Effector (EF)* EE_ID. Current position. Limit. status. end_position. ee_reference
JointConfiguration (JC)*JC_ID. Joint_ID(R7). direction. trial_angle. type. NoinSet. TS_counter. Status
Joint (J)* Joint ID. current_angle. limit_Max.. limit_Min. EE_ID (R2). reference. Arm_ID (R1) . DT_ID (R11). JointStatus
is interfaced with
TrialConfiguration (TC)*TC_ID. DT_ID(R8). configuration. validSolutions. SS_ID(R13). status. LockConfiguraion. TP_ID(R5). JC_ID(R6)
Is evaluated byDecision Tree
Joint motion iscontrolled byDecision Tree
R13Is a componentof Search Space
R14
Checker (Ch)* Ch_ID. counter. recovery_status. abort_var
is checked by
R16
R15
Is A
R19
Performance Criterion (PC)
* PC ID . average value . status . FC_ID (R16) . scale . PC_name .TS_ID (R15)
Kinetic energy distortion Criterion
* PC ID (R19) *Criteria name .criterion value
System Compliance Criterion
* PC ID (R19) *Criteria name .criterion value
Inertia Criterion * PC ID (R19) *Criteria name .criterion value
Geometric Criterion* PC ID (R19) *Criteria name .criterion value
Constraint Criterion * PC ID (R19) * Criteria name . Error . criterion value
R17
Performance Monitoring
Decision Making
DecisionTree (DT)*DT_ID . optimal_solution . status R12
R13
R18
has
has
has
creates
Search Space (SS) * SS_ID . SS_size . DT ID (R12)
FactorialSearchSpace (FSS)
* FSS_ID . SS_ID (R17) . condition
SimpleSearchSpace(SSS)* SSS_ID . SS_ID (R18) . condition
Fused Criterion (FC)* FC ID . PI . TS_ID (R14)
Consists of a number of trial configurations
Evaluates a trial configuration
R11Controls motion of each joint
R8
Is used for evaluation of a trial configuration
NASA Robot Controller
Properties to be Model checked
48
Model Checking of Online Ticket Sale System
• A typical commercial transaction system;• Presented at FASE 2002;• Focus: Integrated state space reduction.
49
An Online Ticket Sale System (Class Diagram)
50
An Online Ticket Sale System (A State Model)
51
Some Verification Statistics of Online Ticket Sale System
• Verification of a liveness property– After an agent is assigned to a customer,
eventually the agent will be released.• Statistics related to state space reductions
1450.3S74.0MOnOn6668.3S17.3MOffOn44736.S113.73MOnOff
-Out of MemoryOffOffTime UsageMemory UsageSMCSPOR
52
Presentation Outline
• Overview and Architecture of ObjectCheck •• Modeling and Verification of a TinyOS Modeling and Verification of a TinyOS
RunRun--time Image with ObjectChecktime Image with ObjectCheck•• More Case StudiesMore Case Studies•• Summary and Future WorkSummary and Future Work• Work Built on ObjectCheck
53
Summary and Future Work
• ObjectCheck– Integrates industrial software development environments
and model checkers with research tools;– Provides comprehensive automation for development,
validation, and model checking of xUML models;– Has enabled verification of non-trivial software system
designs modeled in xUML.
• Future work is focused on– State space reduction capability of ObjectCheck;– Hardware/software co-design and co-verification.
54
Related Work
• Most closely related work– UML Model Checking toolset from University of
Michigan;– vUML tool from Åbo Akademi University;
• There is also related work on model checking of statecharts with different semantics.
55
Additional Information
• http: //www.cs.utexas.edu/users/ObjectCheck• Selected publications:
– Fei Xie and James C. Browne. Verified Systems by Composition from Verified Components. Submitted for review.
– Fei Xie, James C. Browne, and Robert P. Kurshan. Translation-based Compositional Reasoning for Software Systems. Submitted for review.
– Fei Xie and James C. Browne. Integrated State Space Reduction for Model Checking Executable Object-oriented Software System Designs. In Proc. of FASE 2002.
– Fei Xie, Vladimir Levin, and James C. Browne. ObjectCheck: A Model Checking Tool for Executable Object-oriented Software System Designs. In Proc. of FASE 2002.
– Fei Xie, Vladimir Levin, and James C. Browne. Model Checking for an Executable Subset of UML. In Proc. of ASE, 2001.
56
Presentation Outline
• Overview and Architecture of ObjectCheck •• Modeling and Verification of a TinyOS Modeling and Verification of a TinyOS
RunRun--time Image with ObjectChecktime Image with ObjectCheck•• More Case StudiesMore Case Studies•• Summary and Future WorkSummary and Future Work• Work Built on ObjectCheck
57
Work Built on ObjectCheck
• An integrated state space reduction framework;• Integration of model checking into component-
based development of software.
58
Integrated State Space Reduction Framework
xUML-to-S/R Translation
S/R Model S/R Level Query
Model Checking with COSPAN
Success Report / Error TrackBasic Model Checking Process
Symbolic VerificationLocalization Reduction
Partial Order Reduction
xUML Model xUML Level Query
ReducedxUML Model
Reduced xUML Level Query
User-Driven State Space Reduction
Verification Task
Verification Subtasks
DecompositionAbstractionSymmetry Reduction
59
Reduction Steps for Checking P0
Customers, DispatcherAgents, Ticket Sever
Step 1: Symmetry Reduction
Step 2: Decomposition
Step 3: Symmetry Reduction
Step 4: Decomposition
Step 5: Case SplittingStep 6: Symmetry Reduction
Customers, DispatcherAgents, Ticket Sever
P1
Customers Dispatcher Agents, Ticket ServerP21 , P22 P31 , P32
P33 , P23
Agents, Ticket ServerP41 , P42 P43 , P44
Ticket ServerAgentsP41 , P42 P43 , P44
P5
Ticket ServerP6AgentP41 , P42 P43 , P44
P0
60
Evaluation of User-driven State Space Reduction
• Directly model checking P0 on OTSS – Two customer instances and two agent instances;– SPOR and SMC are both applied.– Memory usage: 152.79M– Time usage: 16273.7S
• Memory and time usages for discharging subtasks at the leaf nodes of the reduction tree.
0.63S0.04S0.01S0.04S0.01S1.81S0.02STime
0.35M0.29M0.28M0.29M0.28M0.95M0.30MMemory
P6P44P43P42P41P22P21
61
Integration of Model Checking into Component-based Development
• Temporal properties of a component are – Established with assumptions on the environment of the
component; – Verified under these assumptions and then packaged with
the component.• Selecting a component for reuse considers not only
its functionality but also its temporal properties. • Properties of a composed component are verified
by reusing verified properties of its sub-components and applying compositional reasoning.
62
Sensor Component
63
Properties of Sensor ComponentProperties:
Repeatedly (Output);After (Output) Never (Output) UntilAfter (OP_Ack);After (Done) Eventually (Done_Ack);Never (Done_Ack) UntilAfter (Done);After (Done_Ack) Never (Done_Ack) UntilAfter(Done);
Assumptions:After (Output) Eventually (OP_Ack);Never (OP_Ack) UntilAfter (Output);After (OP_Ack) Never (OP_Ack) UntilAfter (Output);After (Done) Never (Done) UntilAfter (Done_Ack);Repeatedly (C_Intr);After (C_Intr) Never (C_Intr + A_Intr + S_Schd) UntilAfter (C_Ret);After (ADC.Pending) Eventually (A_Intr);After (A_Intr) Never (C_Intr + A_Intr + S_Schd) UntilAfter (A_Ret);After (STQ.Empty = FALSE) Eventually (S_Schd);After (S_Schd) Never (C_Intr + A_Intr + S_Schd) UntilAfter (S_Ret);