© 2014 Carnegie Mellon University
Optimizing Robotic Team
Performance with
Probabilistic Model Checking
Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Sagar Chaki, Joseph Giampapa, David Kyle (presenting), John Lehoczky October 22nd, 2014
Report Documentation Page Form ApprovedOMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering andmaintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, ArlingtonVA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if itdoes not display a currently valid OMB control number.
1. REPORT DATE 22 OCT 2014
2. REPORT TYPE N/A
3. DATES COVERED
4. TITLE AND SUBTITLE Optimizing Robotic Team Performance with Probabilistic Model Checking
5a. CONTRACT NUMBER
5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) Kyle /David S.
5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) Software Engineering Institute Carnegie Mellon University Pittsburgh,PA 15213
8. PERFORMING ORGANIZATIONREPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited.
13. SUPPLEMENTARY NOTES
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT
SAR
18. NUMBEROF PAGES
19
19a. NAME OFRESPONSIBLE PERSON
a. REPORT unclassified
b. ABSTRACT unclassified
c. THIS PAGE unclassified
Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18
2
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Copyright 2014 Carnegie Mellon University This material is based upon work funded and supported by the Department of Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States Department of Defense. NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL IS FURNISHED ON AN “AS-IS” BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. This material has been approved for public release and unlimited distribution except as restricted below. This material may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at [email protected]. DM-0001796
3
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Model Checking
Pentium floating point bug (1995): inspired Intel to model check chips
Now being applied to software, as well
System of Interest (e.g., Code or Chip Design)
Model
Properties
of Interest
Model
Checker (typically based on
boolean-satisfiability)
Proof of Correctness
Counterexample
4
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Probabilistic Model Checking
Model Checking is purely boolean; a property is true or false.
For some systems, we want probabilities
System of Interest (e.g., Code or Chip Design)
Model (e.g., Markov chain)
Properties
of Interest
Probabilistic
Model
Checker
Analysis Output Probability of each Property
Many kinds exist;
we use Discrete Time
Markov Chains (DTMC)
5
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
DTMCs and Multi-Agent Robotic Systems
• Benefits:
1. Performance vs physics-based simulation
2. Exact results. Given a model, probabilities are calculated exactly
• Essential problems:
1. Modelling physical systems is difficult
– Can’t just extract from a design or program code; must observe system to model it
– Physical systems are continuous. Probabilistic Model Checking relies on discrete states
– Given an imperfect model based on finite observations, how does that impact predictions?
2. Robots interact. Modelling an entire system of multiple robots is hard.
6
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Our Contributions
1. Model robots individually:
1. observe and measure individual behavior
2. discretize observations in time and space, create Markov models
3. compose these models into a Markov model of the whole system
2. Use known statistical error on the measurements made of the individual robots to produce estimates of error of the outputs of model checking the whole system.
7
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
S2 S1
Scenario
For our experiments, and as an illustration, we imagine a mine sweeping scenario. Objective: find a mine/IED in a constrained space (i.e., a drainage culvert under a road).
M B
8
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Discrete Time Markov Chains
DTMCs have:
• A set of states, each representing a discrete point in time
• Transitions between states, with probabilities associated. Probabilities are based only on the current state.
Searching
Waiting
Found, Returning Not found, Returning
Not found, Returned Found, Returned
0.8
0.3 0.7
0.2
0.5 0.4
Lost
0.5 0.6
9
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Modal DTMCs
We contribute a variant:
• States can also have “mode change” transitions.
• Mode changes can represent interaction between robots, or the environment.
• In our paper, we show how to convert to a basic DTMC for use with existing model checkers.
Searching
Waiting
Found, Returning Not found, Returning
Not found, Returned Found, Returned
GO
MINE TIMEOUT
0.5 0.4
Lost
0.5 0.6
10
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Discrete Time and Space
We represent robot position with states representing different grid positions, and different times.
R2, C1, T1
Waiting
R2, C2, T2 R1, C2, T2
R1, C3, T3 R2, C3, T3
R1
R3
R2
C1 C2 C3 C4 C5 C6 C7 C8
R2, C4, T4 R2, C2, T4 …
R2, C5, T5
…
R2, C1, T5
Found, and Returned
MINE
GO
REPORTED
0.5 0.5
1 1
1 1
1
1
1
11
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Composing Modal DTMCs
• Modal DTMCs allow us to model individual robots, then easily compose them together.
• To create an individual model:
1. Run the robots individually, with pre-planned mode changes
2. Observe the robot’s behavior
3. Create a Modal DTMC with transition probabilities based on observation, and mode changes as pre-planned
• Then, collect the individual modal DTMCs into a whole-system modal DTMC, and convert it to a non-modal DTMC
• Details of this construction, and correctness proof, are in the paper.
12
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Error Estimation
• Probabilistic Model Checking itself has no error; given a model, it finds exact probabilities.
• However, modeling a robotic system will certainly not be perfect.
• Many kinds of error might appear causing a model to not reflect reality. We looked at handling one: the statistical errors due to observing the individual robots only a finite number of times.
• To examine this specific kind of error, we assume:
• That the system can be fully described by a DTMC
• That we have figured out the states of that DTMC
• But the transition probabilities are observed over the course of finite trials
13
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Dirichlet-based Distribution of DTMCs
To analyze error, we create a random distribution of DTMCs.
For each transition, we use the counts of the times that transition was observed to describe the Dirichlet distribution of transition probabilities for that state, which includes a variance which shrinks with more observations.
State 0
State 1 State 2 State 3
12 3 8
14
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Model Checking a distribution of DTMCs
We randomly generate a large number of DTMCs from the distribution we created. We model check each DTMC, which gives us probabilities for our properties of interest. This gives us a mean and standard deviation for those outputs.
State 0
State 1 State 2 State 3
12 3
8
State 0
State 1 State 2 State 3
0.52 0.13
0.35 State 0
State 1 State 2 State 3
0.52 0.13
0.38 State 0
State 1 State 2 State 3
0.52 0.13
0.32 State 0
State 1 State 2 State 3
0.52 0.13
0.35
15
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Experiment
• Used the simulator V-REP, with Kilobot models based on observations of real Kilobots
• Simulated Kilobots individually, used observations to create models for various team configurations (using Modal DTMCs), and predicted outcomes, using our Dirichlet sampling technique. Our metrics:
• Probability base learns of mine (SUCCESS)
• Expected number of bots that return to base (RETURNED)
• Simulated those teams in V-REP, and compared those outcomes to predicted outcomes.
16
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Experiment Results – SUCCESS metric
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
3-2-1 4-6-1 4-6-2 5-6-2 5-6-7 6-1-7 6-5-7 7-3-5 7-3-6 7-6-1
95-%ile
5-%ile
Mean
Observed
'1111 r ~ , ; ; 0 ~ .. u • • ~~ • l ~
~ ~
u u
n ~ ~
~ ~ ~
~~ • I .,
0 •
• I I I I
...... ..... I I • I I I I
-- Software Engineering Institute Carnegie l\ lellon UniYt~rsity
17
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
0
0.5
1
1.5
2
2.5
3-2-1 4-6-1 4-6-2 5-6-2 5-6-7 6-1-7 6-5-7 7-3-5 7-3-6 7-6-1
95-%ile
5-%ile
Mean
Observed
Experiment Results – RETURNED metric
--
l • • ' :
' • ' i I I I
• ' :
! I I
1 I I
• • ~
Software Engineering Institute Carnegie l\ lellon UniYt~rsity
•
I I
• ' • • I
•
• I
18
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Questions?
19
SIMPAR 2014, Bergamo, Italy
David Kyle, 0ct. 22, 2014
© 2014 Carnegie Mellon University
Contact Information
Presenter
David Kyle
CPS/ULS initiative, DART project
Telephone: +1 412-268-9636
Email: [email protected]
Software Engineering Institute
4500 Fifth Avenue
Pittsburgh, PA 15213-2612
USA
Web
www.sei.cmu.edu
www.sei.cmu.edu/contact.cfm
SEI Email: [email protected]
SEI Phone: +1 412-268-5800
SEI Fax: +1 412-268-6257