Summer of InnovationHybrid Systems Group

Stanley Bak

Inspiration + Potential

Automatic Ground Collision Avoidance System (Auto-GCAS)

F-16 Heads-up Display

Horizon Line

Altitude (Feet)

Inspiration + Potential

(Video)

Auto-GCAS

Automatic Ground Collision Avoidance System

F-16 AircraftGood News:

Very well-studied system!

F-16 AircraftGood News:

Very well-studied system!

Bad News:Differential Equations ~15 variables (dimensions)NonlinearDiscrete SwitchesLook-up Tables

F-16 AircraftGood News:

Very well-studied system!

Bad News:Differential Equations ~15 variables (dimensions)NonlinearDiscrete SwitchesLook-up Tables

Good news:Lots of meaningful research problems!

Summer Work

Chris Elliott – Model Problems for Hybrid Systems Verification

Xin Chen – Reachability of Aircraft Models

Hao Ren – Property Composition with ODE Models

Dung Tran – Scalability Improvements for Reachability

SoI Hybrid SystemsModel Problems

Chris Elliott03 Aug 2017

Agenda

• Group Description• Group Objectives• Group Accomplishments• Lessons Learned• Recommended Future Directions

Task Services Group 2

Group Accomplishments

• Team Design Meeting on Leveraging Formal Analysis for UxAS V&V• Identified Multiple Candidate Topics:

• Closed Loop Heterogeneous System Model (Distributed Mission-Task-Autopilot-DubinPhysics)

• Hi Fidelity Nonlinear Platforms (F16) in .m ode45 Format• *Consensus Dynamics for Distributed Squads of UxAS agents in .m ode45 Format

• Provides hooks to Task FSM with Multiple Vehicle Dependency on Communications

• Ode45 non-stiff differential equation solver (medium order method) format chosen, conducive to Analytical Environment for Reachability Tools (HYST)

Task Services Group 3

Hybrid System UxAS Distributed Squad

• UxAS Distributed Tasking invokes local agent optimization• Optimization computational time a function of many parameters (e.g. Area

Search, Task Options, number of agents in squad, etc)• Optimization complete Vehicle Actions Commanded

• Final waypoints returned from RouteAggregatorService• Task publishes a complete TaskImplementationResponse message

• Hypothesis: leverage Reachability analysis to establish communication requirements (agent time connectivity that may be time variant)

Hybrid System UxAS Distributed Squad

-2 -1.5 -1 -0.5 0 0.5 1 1.5 2

E [nm]

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

N [n

m]

UxAS Connectivity

1

2

3

4

5

0 10 20 30 40 50

t [sec]

0

10

20

30

40

50

60

70

80

90

100

Cos

t

RouteAggregatorService

-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5

E [nm]

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

N [n

m]

UxAS Connectivity

1

2

3

4

5

6

0 10 20 30 40 50

t [sec]

0

10

20

30

40

50

60

70

80

90

100

Cos

t

RouteAggregatorService

Hybrid System UxAS Distributed Squad

-5 0 5

E [nm]

-5

-4

-3

-2

-1

0

1

2

3

4

5

N [n

m]

UxAS Connectivity

1 2 3 4 5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27 28

29 30

31 32 33 34 35 36 37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59 60

61 62

63 64

0 50 100 150 200 250

t [sec]

0

10

20

30

40

50

60

70

80

90

100

Cos

t

RouteAggregatorService

Hybrid System UxAS Distributed Squad

Hybrid System UxAS Distributed Squad

-0.5 0 0.5 1 1.5 2 2.5 3 3.5

E [nm]

-1

0

1

2

3

4

5

N [n

m]

UxAS Connectivity

1

2

3

4

5

6 7

8

0 10 20 30 40 50

t [sec]

0

10

20

30

40

50

60

70

80

90

100

Cos

t

RouteAggregatorService

Summary

• Consensus times a function of graph adjacency• More complex for time variable (switching graphs)• Construct established for compatibility to Reachability Tools

• Future Work: leverage Hyst to set comm consensus requirements for more complex time varying squad dynamics

Lessons Learned

• “AUTONOMY IS HARD. BUILD MORE MODELING AND SIMULATION, AND TEST TO V&V”

• SOI Goal: show this is not a tractable solution using novel techniques.

1) We find the Killer Defect in UxAS and document the process to detect it.2) *We reduce M&S time required w/ Formal Analysis3) *We reduce Flight Testing Time w/ Formal Analysis4) *We improve UxAS (a REQUIREMENTS description, Better Code, etc).5) We fail miserably and reach no consensus

Backup Material

Elliott, Christopher M. "Distributed multi-agent systems-A literature survey and inquisitive discussion." Aerospace Conference, 2014 IEEE. IEEE, 2014.

Das, Abhijit, and Frank L. Lewis. "Cooperative adaptive control for synchronization of second‐order systems with unknown nonlinearities." International Journal of Robust and Nonlinear Control 21.13 (2011): 1509-1524.

Beard, Randal W., and Vahram Stepanyan. "Information consensus in distributed multiple vehicle coordinated control." Decision and Control, 2003. Proceedings. 42nd IEEE Conference on. Vol. 2. IEEE, 2003.

TaskServiceBaseentry: TaskServiceOn = 1

Processing Time for RouteAggregatorService - AGGREGATOR ROLE (Consider Random Fail on Receipt)

FinalRoutes : Upon reception of a TaskImplementationRequest, a Task is informed of the option that was selected by theassignment service. At this point, a Task must create the final set of waypoints that include both enroute and on-task

waypoints from the specified vehicle location. The Task is required to create the enroute waypoints since a routerefinement is possible, taking advantage of the concrete prior position of the selected vehicle. The Task remains in theFinalRoutes state until the route request is fulfilled by the RouteAggregatorService

.

FinalRoutes

Variable Delay a function ofAREA Search ParametersMATRIX OPTIMIZATION here Discussion w/ DerekROUTE COLLECTOR

OptionSelected : When the final waypoints are returned from the RouteAggregatorService, the Task publishes a completeTaskImplementationResponse message. A Task will remain in this state until an EntityState message includes this Task IDin its AssociatedTaskList. If during this state, a subsequent UniqueAutomationRequest is made, the Task returns to theSensorRequest state and immediately attempts to fulfill the requirements of the new UniqueAutomationRequest. This

behavior implies that a Task can only be part of a single AutomationRequest and subsequent requests always overrideprevious requests.

OptionSelected

Simple Delay (Consider Closed Loop onTask Execution)

Active: If the Task is in the OptionSelected state and an EntityState message is received which includes the Task ID inthe AssociatedTaskList, then the Task switches to the Active state and is allowed to publish new waypoints and sensorcommands at will. A Task remains in the Active state until a subsequent EntityState message does not list the Task ID inits AssociatedTaskList. At which point, a transition to Completed is made. Note that a Task can reliquish control indirectlyby sending the vehicle to a waypoint not tagged with its own ID. Likewise, it can maintain control indefinitely by ensuringthat the vehicle only ever go to a waypoint that includes its ID. If a UniqueAutomationRequest message that includes thisTask ID is received in the Active state, it transitions to the Completed state.

Active

TaskImplementationRequest (2ms) (Consider Random Fail on Receipt)

OptionsPublished : When routes are returned to the Task, it will utilize all route and sensor information to identify andpublish the applicable TaskOptions. The determination of TaskOptions is key to overall mission performance and vehiclebehavior. It is from this list of options that the assignment will select in order to perform this particular Task. After

publication of the options, a Task waits in the OptionsPublished state until the TaskImplementationRequest message is

received, whereupon it switches to FinalRoutes.

OptionsPublished

Simple Delay (Consider Random Fail onTaskInit)

Init: This is the state that all Tasks start in and remain until all internal initialization is complete. For example, a Task may need to load complex terrain or weather data upon creation and will require some (possibly significant) start-up time. When a Task has completed its internal initialization, it must report transition from this state via the TaskInitialized message.

guarantee "all tasks remain in init until all internal initialization is complete, transitioning to idle after. When a Task has completed its internal initialization, it must report transition from this state via the TaskInitialized message."

Init

Processing Time for RouteAggregatorService - COLLECTOR ROLE (Consider Random Fail on Receipt)

OptionRoutes : After the SensorManagerService has replied with the appropriate sensor calculations, the Task can request

waypoints from the RouteAggregatorService that carry out the on-Task goals. For example, an AreaSearchTask can request routesfrom key surveillance positions that ensure sensor coverage of the entire area. The Task remains in the OptionRoutes state untilthe RouteAggregatorService replies.

guarantee "[Not originally specified: if in the OPTION_ROUTES state and an expected RouteResponse_in is received send anevent(TaskPlanOptions_out)]"

guarantee "The Task remains in the OptionRoutes state until the RouteAggregatorService replies. When routes are returned [IMPLIED: from the route aggregator service the service will be in theOPTIONS_PUBLISHED state] [ASSUMPTION: the replies are assumed to be RoutePlanResponse_in]"

OptionRoutes

Processing Time for SensorManagerService (Consider Random Fail on Receipt)

SensorRequest : When a Task is notified of its inclusion (by noting the presence of its ID in the Tasks list of an

UniqueAutomationRequest message), it can request calculations that pertain to the sensors onboard the vehicles that arealso included in the UniqueAutomationRequest message. While waiting for a response from the SensorManagerService, aTask is in the SensorRequest state and will remain so until the response from the SensorManagerService is received.

guarantee "[Not originally specified: in the SENSOR_REQUEST state, if the expected SensorFootprintResponse_in isreceived send a RouteRequest_out]"

guarantee "[IMPLIED: in the SensorRequest state] After the SensorManagerService has replied with the appropriatesensor calculations [IMPLIED: the service will then be in OptionRoutes state]" :

SensorRequest

UniqueAutomationRequest (2ms) (Consider Random Fail on Receipt)

Idle: This represents the state of a Task after initialization, but before any requests have been made that include the Task.

UniqueAutomationRequest messages trigger a transition from this state into the SensorRequest state.

guarantee "[Not originally specified: in the idle state, once an automation request is recieved that is present in the set of expected task IDs, send a SensorFootprintRequests_out]"

guarantee "UniqueAutomationRequest messages trigger a transition from the idle state into the SensorRequest state. When a Task is notified of its inclusion (by noting the presence of its ID in the Tasks list of an UniqueAutomationRequest message), it can request calculations that pertain to the sensors onboard the vehicles  Â

Idle

[RouteRequest_out]

[TaskImplementationResponse_out]

[TaskPlanOptions_out]

[TaskComplete_out]

[SensorFootprintRequests_out][init_complete && TaskInitialized_out]

[SensorFootprintRequests_out]1

[RouteRequest_out]

[EntityState_in && ...entity_state_task_id_present]

2

Aircraft Modeling and Verification

Xin Chen

Formal Model of an Aircraft

Image Credit: Flight Dynamics in Wikipedia

Formal Model of an Aircraft

• Defined by a hybrid automaton.

• 12 continuous state variables:

• Feedback control in U,V,W triggered by time steps.

force orientation angularvelocities

position

Property Verification

Hardness

• Continuous dynamics defined by large non-polynomial ODEs.

• Control program contains non-polynomial expressions, saturation functions and lookup tables.

Our Solution

• Consecutively adding control inputs.

• Linearizing the “key” variables.

• Better strategy to find proper overapproximation errors.

• More sophisticated flowpipe merging and splitting.

Result of a Simple Controller

Compositional ReasoningHao Ren

RELIQE Framework for Compositional Reasoning:Can reason about Cyber or Physical Components

Hybrid Group

CNTRL THROTTarget.val

Actuator_Input Actual.val

Vehicle model:Target_Speed.val

Actual_Speed.val

Above vehicle components involve time-dependent properties, over bounded horizon/history/memory/states

Time-dependent Property Composition: Illustration and Approach

Hybrid Group

We derive strongest system contract 𝑃𝑃𝑠𝑠𝑠𝑠𝑠𝑠 from component contracts (𝐴𝐴𝑖𝑖 ⇒ 𝐺𝐺𝑖𝑖),∀𝑖𝑖 ∈ {1, … ,𝑁𝑁} and connectivity 𝑥𝑥𝑝𝑝 = 𝑥𝑥𝑞𝑞 ,∀ 𝑝𝑝, 𝑞𝑞 ∈ 𝐶𝐶 using QE:

𝑃𝑃𝑠𝑠𝑠𝑠𝑠𝑠 = ∃ 𝑥𝑥1 …∃𝑥𝑥𝑚𝑚 �𝑖𝑖=1

𝑁𝑁

𝐴𝐴𝑖𝑖 ⇒ 𝐺𝐺𝑖𝑖 ∧ �𝑝𝑝,𝑞𝑞 ∈𝐶𝐶

𝑥𝑥𝑝𝑝 = 𝑥𝑥𝑞𝑞

The above only works for time-independent or “memoryless” properties.

Time-dependent case simple illustration: Consider cascade of two copies of same component, with property:𝑂𝑂𝑂𝑂𝑂𝑂𝑝𝑝𝑂𝑂𝑂𝑂 𝐶𝐶𝐶𝐶𝐶𝐶𝑝𝑝𝑖𝑖 > 𝒑𝒑𝒑𝒑𝒑𝒑 𝐼𝐼𝐼𝐼𝑝𝑝𝑂𝑂𝑂𝑂 𝐶𝐶𝐶𝐶𝐶𝐶𝑝𝑝𝑖𝑖 , 𝑖𝑖 = 1,2;

Cascade connectivity of two components:𝑂𝑂𝑂𝑂𝑂𝑂𝑝𝑝𝑂𝑂𝑂𝑂 𝐶𝐶𝐶𝐶𝐶𝐶𝑝𝑝1 = 𝐼𝐼𝐼𝐼𝑝𝑝𝑂𝑂𝑂𝑂 𝐶𝐶𝐶𝐶𝐶𝐶𝑝𝑝2

Then one may infer:𝑂𝑂𝑂𝑂𝑂𝑂𝑝𝑝𝑂𝑂𝑂𝑂 𝐶𝐶𝐶𝐶𝐶𝐶𝑝𝑝2 > 𝒑𝒑𝒑𝒑𝒑𝒑 𝒑𝒑𝒑𝒑𝒑𝒑 𝐼𝐼𝐼𝐼𝑝𝑝𝑂𝑂𝑂𝑂 𝐶𝐶𝐶𝐶𝐶𝐶𝑝𝑝1 .

But this involves a new variable, namely, doubly time-shifted input of the first component.

Extension to composition of time-dependent properties involves two steps:• Pre-QE Step: Determine system order, i.e., max time-shifts needed in property composition• QE Step: Compose component properties and their time-shifted versions, equaling system order in number

+1

Actuator_Input

Actual.val

+0u

+0

+0e

e_int

e_dot

+0

+0

+1

+1

Target. val

+0

+0

Target_Speed. val

+0

Actual_Speed. val

+0

u = P*e + D*e_dot + I*e_int;

Actual.val = prev(Actual.val, 0.0) + 0.1*Actuator_Input

given: u = P*e + D*e_dot + I*e_intOne-shift: pre_u = P*pre_e + D*pre_e_dot + I*pre_e_int

Determine System order:

Create shifted component property expressionsEqualing system order in number:

Determining System Order

• Create variable-dependency tree• Each “pre” adds 1 to variable order• Two types of paths:

• Output-2-Input path• Output-2-Loop path

• Path order = order sum along the path• System order = Max(path order)

Hybrid Group

RELIQE: Extension for Time-dependent Properties

CNTRL THROTTarget.val

Actuator_Input Actual.val

Initial first step constraint:Actual_Speed = Target_Speed / 𝟓𝟓𝟓𝟓General constraint:Actual_Speed = 𝟎𝟎.𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗 Actual_Speed + 𝟎𝟎.𝟎𝟎𝟎𝟎 Target_Speed − Actual_Speed + 𝟎𝟎.𝟎𝟎𝟎𝟎𝟗𝟗𝟗𝟗𝟗𝟗𝟗𝟗 Target_Speed

Vehicle model:

Hybrid Group

Manual result:

COMPOSE result:

Target_Speed.valActual_Speed.val

Time-dependent extension also requires:• Generation of initial conditions on outputs, equaling system order in number, by substituting initial conditions on states• Use of induction to prove postulated system contracts from inferred strongest system contracts (since contracts are recursive)Extended RELIQE works for Temporal-Logic with bounded horizon/memory, e.g., Next (X), Last (L), Past (P) operators

Verification of large scale linear systems with one million dimensions

Dung Tran, Stanley Bak and Taylor T. Johnson03 Aug 2017

Large scale linear systems

Linear systems �̇�𝑥 = 𝐴𝐴𝑥𝑥 , 𝑥𝑥 0 = 𝑥𝑥0 ∈ 𝑆𝑆0 𝐴𝐴 ∈ 𝑅𝑅𝑛𝑛,𝑛𝑛 ≥ 1000000Problem: Reachability analysis using over/under-approximation does not work due

to computation cost. Hylaa approach: Simulation-based reachability analysis

Hybrid Group 2

Reachability analysis with Hylaa

High level idea Star Set

𝑥𝑥0 = 𝑐𝑐 + ∑𝑖𝑖=1𝑛𝑛 𝛼𝛼𝑖𝑖𝑣𝑣𝑖𝑖 , 𝐿𝐿𝛼𝛼 ≤ 𝑃𝑃

𝑥𝑥 𝑡𝑡 = 𝑒𝑒𝐴𝐴𝐴𝐴𝑥𝑥0 = 𝑒𝑒𝐴𝐴𝐴𝐴𝑐𝑐 + �𝑖𝑖=1

𝑛𝑛

𝛼𝛼𝑖𝑖(𝑒𝑒𝐴𝐴𝐴𝐴𝑣𝑣𝑖𝑖)

= ̅𝑐𝑐 𝑡𝑡 + ∑𝑖𝑖=1𝑛𝑛 𝛼𝛼𝑖𝑖 �𝑣𝑣𝑖𝑖(𝑡𝑡)

Hybrid Group 3

�̇�𝑥 = 𝐴𝐴𝑥𝑥, 𝑥𝑥 0 = 𝑥𝑥0 ∈ 𝑆𝑆0 ⟹ 𝑥𝑥 𝑡𝑡 = 𝑒𝑒𝐴𝐴𝐴𝐴𝑥𝑥0

Need (n+1) simulations

Reachability analysis with Hylaa

Reachability analysis with ode solver (Python- odeint)

Hybrid Group 4

System dimension One-step 1000-steps Reachability analysis time

100,000 0.204 s 204 s ≈ 204 × 105 𝑠𝑠≈ 231 𝑑𝑑𝑑𝑑𝑑𝑑𝑠𝑠

1,000,000 3.5 s 3500 s ≈ 3500 × 106≈ 111 𝑑𝑑𝑒𝑒𝑑𝑑𝑦𝑦𝑠𝑠

Computing 𝒆𝒆𝑨𝑨𝑨𝑨𝒗𝒗, is there a better way?

Krylov subspace approximation for 𝒆𝒆𝑨𝑨𝑨𝑨𝒗𝒗

Objective: Approximate 𝒆𝒆𝑨𝑨𝑨𝑨𝒗𝒗, v is an specific vector

𝒆𝒆𝑨𝑨𝑨𝑨𝒗𝒗 ≈ 𝜷𝜷𝑽𝑽𝒎𝒎𝒆𝒆𝑯𝑯𝒎𝒎𝑨𝑨𝒆𝒆𝟏𝟏 𝛽𝛽 = 𝑣𝑣 2 𝑉𝑉𝑚𝑚 = [𝑣𝑣1 𝑣𝑣2 … 𝑣𝑣𝑚𝑚] orthonormal basic of Krylov subspace 𝐻𝐻𝑚𝑚 is (𝑚𝑚 × 𝑚𝑚) upper Heisenberg matrix 𝑒𝑒1 = 1 0 . . . 0 𝑇𝑇, a basic vector of 𝑅𝑅𝑛𝑛

Hybrid Group 5

(𝑉𝑉𝑚𝑚,𝐻𝐻𝑚𝑚) is derived from Arnoldi algorithm

Arnoldi Algorithm

Krylov subspace approximation for 𝒆𝒆𝑨𝑨𝑨𝑨𝒗𝒗

Krylov subspace method (Python - Krypy packet) vs. Ode solver (Python odeint)

Hybrid Group 6

The fact is, time for running Arnoldi algorithm dominates the time consumption.

Krylov subspace approximation for 𝒆𝒆𝑨𝑨𝑨𝑨𝒗𝒗

Simulation/Reachability analysis time with Krylov subspace method (for one random example)

Hybrid Group 7

System dimension first-step 1000-other steps Reachability analysis time

100,000 0.051 s 0.002 x 1000 s ≈ 2.051 × 105 𝑠𝑠≈ 2.4 𝑑𝑑𝑑𝑑𝑑𝑑𝑠𝑠

1,000,000 1.069 s 0.095 x 1000 s ≈ 3500 × 106≈ 3 𝑑𝑑𝑒𝑒𝑑𝑑𝑦𝑦𝑠𝑠

Big improvement. Can we improve more?

Odeint

231 days

111 years

vs.

Krylov subspace approximation with GPU

Implement the Arnoldi algorithmusing GPU. Modify Arnoldi algorithm to make It runnable with a set of vectors in Parallel.

Hybrid Group 8

Summary and on-going works

Exploit Krylov subspace method for reachability analysis. Integrate the method into Hylaa.

On-going works Test/compare the performance of different methods in practical systems.

Hybrid Group 9