Real Time Reachability Analysis for Marine...

IN DEGREE PROJECT INFORMATION AND COMMUNICATION TECHNOLOGY,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2018

Real Time Reachability Analysis for Marine Vessels

SUDAKSHIN GANESAN

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

i

Abstract

Safety verification of continuous dynamical systems require the computa-tion of the reachable set. The reachable set comprises those states the systemcan reach at a specific point in time. The present work aims to compute thisreachable set for the marine vessel, in the presence of uncertainties in thedynamic modeling of the system and in the presence of external disturbancesin the form of wind, waves and currents. The reachable set can then be usedto check if the vessel collides with an obstacle. The dynamic model used isthat of a nonlinear maneuvering model for the marine vessel. The dynamicson the azipod actuators are also considered.

Several methods are considered to solve the reachability problem for themarine vessel. The first method considered is that of the Hamilton JacobiReachability analysis, where a dynamic game between the control input andthe disturbance input is played. This results in a dynamic programmingproblem known as the Hamilton Jacobi Bellman Isaacs (HJBI) equation. Itis solved using the Level-Set method, but it suffers from the curse of dimen-sionality. The other method considered is the use of set-theoretic approach,where an over-approximation of the reachable set is computed, in the contextof safety verification. But on the downside, large sets of admissible controlyields highly over-approximated reachable sets, which cannot be used

In order to overcome the disadvantages posed by the first two methods,emphasizing on the real-time computation, a third method is developed, wherea supervised classification algorithm is used to compute the reachable setboundary. The dataset required for the classification algorithm is computedby solving a 2 Point Boundary Value Optimal Control Problem for the ma-rine vessel. The features for classification algorithm can be extended, so as toinclude the uncertainties and disturbances in the system. The computationtime is greatly reduced and the accuracy of the method is comparable to theexact reachable set computation.Keywords: Reachable set, Safety Verification, Marine dynamics, MachineLearning, Optimal control.

ii

Sammanfattning

Säkerhetsverifiering av kontinuerliga dynamiska system kräver beräkningav mängden av tillstånd som kan nås vid en specifik tidpunkt, givet dess ini-tialtillstånd. Detta arbete fokuserar påatt bestämma denna mängd av nåbaratillstånd för ett marint fartyg under modellosäkerheter och externa störningari form av vind, vågor och strömmar. Den nåbara mängden av tillstånd an-vänds sedan för att kontrollera om fartyget riskerar att kollidera med hinder.Den dynamiska modell som används i våra studier är en icke-linjär modelldär även dynamiken hos azipod-ställdonen betraktas.

Arbetet studerar flera metoder för att lösa problemet: en klassisk Hamilton-Jacobi nåbarhetsanalys, en mängd-teoretisk teknik, samt en ny metod baseradpåmaskininlärning. Numeriska simuleringsstudier bekräftar att den föreslagnamaskininlärningsmetoden är snabbare än de tvåalternativen.

Keywords: Säkerhetsverifiering, nåbarhetsanalys, fartyg dynamisk

iii

Acknowledgements

This thesis project was carried out at ABB Corporate Research Center, at Västerås,Sweden, at the Control, Optimization and Analytics group. It was supervised bythe Division of Automatic Control at the Royal Institute of Technology (KTH) inStockholm, Sweden.

I would like to thank my supervisors Hamid Reza Feyzmahdavian and WinstonGarcia Gabin at ABB for giving me the freedom to explore the problem statementand the solution methods, which gave me a flavor for research in an industry andalso for their continuous support throughout the project. I would like to thank mySupervisor and Examiner Mikael Johansson at KTH for giving me the opportunityto carry out this interesting project and also importantly for steering the researchin an interesting direction.

At the end I would like to express my biggest gratitude to my loved ones, myfamily and friends, who have supported me throughout the entire process. I will begrateful forever for your love.

Sudakshin Ganesan,Stockholm, 15th August 2018

Contents

i

Contents iv

List of Figures vi

List of Acronyms ix

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Outline of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Literature Survey 5

3 Problem Formulation 93.1 Nonlinear Maneuvering Model of the Marine Vessel . . . . . . . . . . 93.2 The Reachability Problem . . . . . . . . . . . . . . . . . . . . . . . . 11

3.2.1 Hamilton Jacobi Reachability Analysis . . . . . . . . . . . . . 123.2.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2.3 Over-approximate Set Theoretic Reachable set computation . 15

3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Classification based Reachability Analysis 194.1 Optimal Control Problem . . . . . . . . . . . . . . . . . . . . . . . . 20

4.1.1 Optimal Control Solution . . . . . . . . . . . . . . . . . . . . 214.2 Support Vector Machine for boundary classification . . . . . . . . . . 234.3 Implementation in ACADO . . . . . . . . . . . . . . . . . . . . . . . 24

4.3.1 Dataset Creation . . . . . . . . . . . . . . . . . . . . . . . . . 264.3.2 Including Uncertainties and Disturbances . . . . . . . . . . . 27

4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

iv

CONTENTS v

5 Simulation Results 295.1 Maneuvering reachable set for different initial conditions . . . . . . . 295.2 Maneuvering reachable set for uncertain Mass matrix . . . . . . . . . 305.3 Maneuvering reachable set in the presence of disturbances . . . . . . 31

6 Conclusion and Future Work 336.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

6.2.1 Incorporating the geometry of the vessel . . . . . . . . . . . . 346.2.2 Optimal control solution for obstacle avoidance . . . . . . . . 346.2.3 Lifting the assumptions made for computing the Reachable

set and improving the accuracy of the method . . . . . . . . 35

Bibliography 37

List of Figures

1.1 Verification of the vessel’s safety using the Forward reachable set. . . . . 3

2.1 Exponential increase in complexity as the number of dimensions in thestate space increases [1]. . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.1 Motion in 6 degrees of freedom . . . . . . . . . . . . . . . . . . . . . . . 103.2 Illustration of the level set method, which encodes the target set and

the reachable set using value functions. . . . . . . . . . . . . . . . . . . 123.3 Forward Reachable set for the velocity states in the surge and the sway

direction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.4 Illustration of the overaproximation of reachable set used for safety ver-

ification [2]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.5 Illustration of the steps involved in approximate reachable set calculation

[2]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.6 Reachable set for the marine vessel with 3 DOF maneuvering model, for

3 min into the future, admissible control being 1/10th the original value. 17

4.1 Simplified 2D illustration of a cost limited reachable set for a giventhreshold cost of J0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.2 Illustration of the multiple shooting algorithm [3] . . . . . . . . . . . . 214.3 Illustration of separating hyperplane & support vectors . . . . . . . . . 24

5.1 The Reachable set for the marine vessel with initial surge velocity, u= 1 m/s. The boundary between the reachable and non reachable isclassified using the SVM algorithm. . . . . . . . . . . . . . . . . . . . . . 30

5.2 The plots show the reachable set computed for varying initial surgevelocities ranging from 1-5 m/s. The final plot shows the classificationboundary with different initial conditions. . . . . . . . . . . . . . . . . 31

5.3 The plot shows the reachable sets for the marine vessel in the presenceof uncertainties. The uncertainty is present in the inertia matrix, whereit can vary 25% of the Mass matrix. . . . . . . . . . . . . . . . . . . . . 31

vi

List of Figures vii

5.4 The plot shows the reachable set in the presence of disturbance. The dis-turbance in the direction of Surge, Sway and Yaw directions is assumedbe within 1/10th the maximum actuator force. . . . . . . . . . . . . . . 32

List of Acronyms

DOF Degree of FreedomGPS Global Positioning SystemDP Dynamic PositioningHJBI Hamilton Jacobi Bellman IsaacsPDE Partial Differential Equation2PBVP 2 Point Boundary Value ProblemCORA COntinous Reachability AnalyzerNED North-East-DownCoG Center of GravitySVM Support Vector MachineSQP Sequential Quadrartic ProgrammingODE Ordinary Differential EquationsZOH Zero Order HoldNLP Nonlinear ProgrammingACADO Automatic Control And Dynamic OptimizationPCA Principle Component Analysis

ix

Chapter 1

Introduction

1.1 Background

The once conservative maritime industry is now moving towards digitalization andautonomy. The advantages provided by increased autonomy in the maritime indus-try allows for lower fuel costs and also improves crew, passenger and cargo safety.The change is largely brought about by the scrutiny of regulatory and environ-mental bodies for the shipping industry to be more efficient, modern and mostimportantly safe. In order to bring about autonomy, the first step is to providesituational awareness to the captain, where he can perceive what is around the ves-sel and know about how his ship can maneuverer in the particular sea state. Thenext step in automating marine vessels would be to develop algorithms that act asdecision support systems. Since the evolution of autonomy is a continual process, itis important to create algorithms that suggests the captain of what could be done.The final goal is to develop algorithms that can make decisions on its own, butwhich can be overridden by the captains commands.

Whatever the level of autonomy in the marine vessels, the most important fea-ture that cannot be compromised upon is the guarantee of safety for the vessels.The present work tries to solve the part of the puzzle, wherein the captain un-derstands his ship’s maneuverability better, thereby enabling him to take a safecourse.

Many real world systems evolve according to complex nonlinear dynamics. Thebehavior of such systems is hard to predict and can be non-intuitive. Many ofthese systems are safety critical, which means verification plays an important rolein checking the specification properties of the system. Verification of safety criticalsystems is challenging due to multiple reasons. The challenge stems from the factthat, the systems considered are generally nonlinear in nature, and also is high-dimensional. Further for a safety critical system, all the system behaviors mustbe accounted for, in the presence of disturbances and uncertainties in modeling.Since there exists infinitely many possible trajectories that the system can evolve

1

2 CHAPTER 1. INTRODUCTION

with a given initial control and disturbance space, simulation based approaches arerendered useless. Thus formal verification methods are needed for guaranteeingperformance and safety properties of systems.

Reachability analysis of marine vessels online, is a recent problem, where onetries to compute online the reachable set in the state space of the marine vessel insome amount of time in the future, given the input constraints due to the ship’sactuator system, and also given some bounds on the external disturbance. Theexternal disturbance arises from various factors including, wind, wave and currentforces, which cannot be measured accurately. The problem is nontrivial since thecomputation of the reachable set is difficult. The complexity for computation stemsfrom:

• Complex nonlinear dynamics involving Coriolis force and nonlinear dampingfor the vessel.• Maneuvering model is 3 DOF, which means 6 nonlinear ordinary differential

equations are needed to characterize the behavior, which contributes to the curseof dimensionality.

1.2 Applications

The applications for reachability analysis for the marine vessels, extend the levelof autonomy of vessels in general. The primary application of reachability for themarine vessel is for the captain to understand where the ship can maneuver. Thisoffers a type of internal perception, for the captain to know about his ship’s abilityto maneuver. Also, when the ship is in the open sea, there are quite long periods oftime, the ship does not receive GPS coordinates, and the control systems requiringGPS data get affected. Reachable set could then be used as dead-reckoning for thecontrollers (or for manual control).

The next example extends to safety verification, as mentioned earlier. Thereachable set entails all the possible state-space configurations, the dynamics canevolve in. This can be checked against an obstacle, to verify if the system issafe (Figure 1.1). Since, the algorithms developed in this thesis also provide thecontrol signal along with the reachable set, a collision-avoidance algorithm can beformulated as a natural extension.

The area of reinforcement learning and the synthesis of learning based con-trollers, try to explore the state-space, to optimize a reward function to obtain acontrol signal. A learned control system may be guaranteed safe during deploy-ment, but it need not be safe while learning. Consider the example of a quadcopterlearning to fly on a desired height. It needs to explore the state-space, and theremight exist a search, where the quadcopter might reach an unsafe set of states(hitting the ground or the ceiling). So, reachability analysis can be used to ensuresafety in this learning process.

1.3. OBJECTIVE 3

Figure 1.1: Verification of the vessel’s safety using the Forward reachable set.

As a final example, reachability analysis can be used to initiate parking ofmarine vessels. In the present day situation, the captain uses a ad-hoc methodfrom experience, to understand when the ship needs to de-accelerate for parking. Inmany cases, this results in a not so smooth velocity profile for the ship. BackwardReachability analysis, answers the question of where should the captain initiateparking, if he needs to land in the parking spot at the present time.

1.3 Objective

The research objective is to compute online, the reachable set for the nonlinear shipmaneuvering model, in the presence of disturbance in the form of waves, winds andcurrents and also taking into account uncertainties in the maneuvering model. Thequestions that are answered by the thesis are,

1. Given the azipod actuator constraints of the ship, where could the ship be?(in a certain amount of time in the future).

2. Given the bounds for maximum allowable disturbance and uncertainty in the

4 CHAPTER 1. INTRODUCTION

vessel’s model, where could the ship be? (in a certain amount of time in thefuture.)

Since reachability for a marine vessel is a new problem, several methods wereused to solve it. Each method reuires a trade-off between the generality of thedynamical system considered and the computation time. The most applicable so-lution, to our problem lays an emphasis on ’real-time computation’, which involvesmachine learning for the most approximate solution for the reachability problem.

1.4 Outline of the thesis

This thesis is organized in four chapters as follows. The first chapter motivatesthe reachability problem for marine vessels. Chapter 2 explains the state of theart in marine dynamics and how the disturbances & uncertainties are handledin the literature and also how reachable sets are computed. Chapter 3 explainsnonlinear dynamic model used for maneuvering of the marine vessel. It also sum-marizes the preliminary results obtained using the Hamilton Jacobi based methodand set-theoretic based methods. Chapter 4 talks about the major contribution ofthe thesis, which talks about the Classification based computation. Chapter 5 de-tails the simulation results obtained for the marine vessel, in different experiments.Finally, in Chapter 6, conclusions and future work are discussed.

Chapter 2

Literature Survey

Model based control for steering, dynamic positioning of ships has become state-of-the art since Linear Quadratic Regulators came to be used in the ship industry.The exact model derived using the first principles, are given by a set of compli-cated differential equations describing 6 degrees-of-freedom (6 DOF) motion. Forslow speed applications, such as Dynamic Positioning (DP), mooring and slow speedreference tracking, a simpler model can be considered, that is linear in the kineticpart. DP models and its simplifications are detailed in [4]. On the other hand, inhigh speed applications such as automatic course control, high speed position track-ing, and path following, the nonlinear effects from the Coriolis and the dampingterms become dominant. This model is developed in [5], where complete modeling,identification, and control design for maneuvering a ship along a desired path ispresented. The identification is performed on a scaled down ship called CyberShipII. This thesis uses the model proposed in [5] for the computation of the reachableset.

Although Reachability analysis is a well-studied field in the control and the com-puter aided verification community, the problem is solved exactly only for a classof problems. Reachability analysis is studied extensively in computer software andin discrete state spaces, where properties can be formulated by the use of LinearTemporal Logic and other advanced formalisms that checks whether the runs ofa transition system, or words of a finite automaton satisfy a set of properties [6].This logic can specify the dangerous states the discrete state space needs to avoid,thereby verifying the safety properties of the system. While the reachability prob-lem can be solved exactly and efficiently for discrete state spaces, the computationfor continuous state spaces with more dimensions generally become increasinglydifficult.

In [7], the Hamilton Jacobi method is developed, where it is shown that the solu-tion can be obtained by computing the zero-level subset of the solution of Hamilton-Jacobi-Bellman-Isaacs Equations (HJBE). Unfortunately, while this methods canfind accurate approximations to the reachable set for systems with complicated

5

6 CHAPTER 2. LITERATURE SURVEY

nonlinear dynamics, their computational cost scales exponentially with the sys-tem’s state space dimension 2.1. The major advantage of using HJ reachabilitystems from the fact that, the solutions are accurate without approximations, whichmeans it can be used for guaranteed safety analysis.

Figure 2.1: Exponential increase in complexity as the number of dimensions in thestate space increases [1].

Recently there have been several advances in HJ reachability theory and ap-plications. In [8], a new technique is developed that decomposes the dynamics ofnonlinear systems into subsystems which are coupled through common states, con-trols and disturbances, to tackle the curse of dimensionality. The reachable sets arecomputed for individual subsystems and the reachable set for the complete systemis reconstructed without incurring additional approximation errors. Computationsdone using this techniques become degrees of magnitudes faster. In the case involv-ing adversarial disturbances, their technique provides slighly conservative reachablesets. The authors in [9] try to approximate solutions of the HJ PDE by implement-ing and analyzing learning-based algorithms to approximate the solution of certaintypes of HJ PDEs using neural networks. The HJ reachability is applied to motionplanning for a vehicle in [10], using an algorithm called FasTrack, which providesa safety controller for the vehicle along with a guaranteed tracking error bound.The tracking bound is computed offline using the exact HJ reachability computa-tion.The algorithm works by alternating between the target set being reach goal

7

set and the reach avoid set in the case of obstacles. In [11], a general frameworkis developed for ensuring safety while learning, implemented in quadcopters, whichreduce the conservatism online as the system learns more information about theenvironment.

The intractability posed by the HJ reachability analysis calls for other approx-imate methods that rely on over approximate computation of the reachable sets,that check if the over approximate forward reachable set intersects a set of undesiredstates. This work uses the CORA toolbox [2] , which accepts continuous, nonlinearsystems for computing the over approximate reachable set. It can also determinethe reachable set for systems with parametric uncertainty. The sets are representedby zonotopes, and the error incurred due to linearization is used for enlarging thereachable set. The downside of this approach is that there is a trade-off betweenthe accuracy of the reachable set, the nonlinearity in the system dynamics and thesize of the admissible control input set. So, the over approximation of the reach-able set becomes impractical, if both the nonlinearity and the admissible controlset is large. Other tools such as SpaceEX [12], Flow∗ [13], can also can representnon-convex sets and are especially promising for reachability analysis of nonlinearmodels. The ellipsoidal toolbox [14] on the other hand approximates all the sets asa representation of ellipsoids, can be used to compute the reachable set of linear,continuous and hybrid systems.

The work done in [15], forms the basis for the main results derived in this the-sis. In this work, supervised machine learning techniques are used to accuratelypredict cost-limited reachable sets of dynamical systems in real-time. A 2 PointBoundary Value Problem (2PBVP) is solved over the state space grid to createthe dataset for the classification algorithms. The 2PBVP solver employs Cheby-shev Pseudospectral Methods and sequential quadratic programming. Two machinelearning algorithms, Locally-weighed regression and Support Vector Machines, areused to demonstrate the method for query based classification of the reachable set.The classical dubin’s car is used as a case study and the computation time requiredis 4 orders of magnitude less than the traditional exact approach. This techniqueis used, by the same authors to solve the real-time problem of kinodynamic motionplanning for fixed-wing UAV navigating through a forest in [16]. The frameworkrelies on a lookup table that stores precomputed optimal solutions to the 2PBVP.

Chapter 3

Problem Formulation

In this chapter, the details of the nonlinear maneuvering dynamical model for thevessel is developed. The model also incorporates the dynamics and the constraintsposed by the azipod actuators of the vessel. Further, the reachability problem isposed and tackled using the Hamilton Jacobi Reachability method and by Set-theoretic computation schemes. The advantages and drawbacks of both the meth-ods are evaluated.

3.1 Nonlinear Maneuvering Model of the Marine Vessel

When the ship maneuvers in the open sea, the ship can have motion in 6 degreesof freedom (DOFs). For the purposes of this this thesis, it is interesting to justconsider the motion in the horizontal plane, characterized by Surge (Longitudinalforward motion), Sway (Sideways motion) and Yaw (rotation about the verticalaxis) describing the heading of the vehicle. The earth-fixed reference frame {e}is approximated by the North-East-Down (NED) frame. Let another coordinateframe {b} be attached to the center of gravity of the vessel. The discussion followsfrom the model posed by Fossen [5]

The kinematic relationship between the earth-fixed position vector η = [x, y, ψ]′

and the body-fixed velocity vector ν = [u, v, r]′ is given by,

η = R(ψ)ν (3.1)

where,

R(ψ) =

cos(ψ) −sin(ψ) 0sin(ψ) cos(ψ) 0

0 0 1

Deriving the equations for the rigid body dynamics, using Newton’s second law,

we arrive at,MRB v + CRB(v)v = τ (3.2)

9

10 CHAPTER 3. PROBLEM FORMULATION

Figure 3.1: Motion in 6 degrees of freedom

whereMRB is the rigid-body system inertia matrix, and CRB is the correspond-ing Coriolis matrix, and τ = [X,Y,N ]′ is a generalized vector of Forces (X,Y) andMoment N. This is provided by the superposition of the actuator forces and mo-ments. The external actuator forces are also represented in the body-fixed referenceframe. Here, MRB and CRB take the form,

MRB

m 0 00 m 00 0 Iz

;CRB(v)

0 0 −mv0 0 mumv −mu 0

The forces encountered by the ship’s hull due to its interaction with the watercan be predominantly characterized by the hydrodynamic added mass, radiation-induced potential damping and restoring forces. Hydrodynamic potential theoryprograms can be used to compute the added mass and damping matrices by in-

3.2. THE REACHABILITY PROBLEM 11

tegrating the pressure of the fluid over the wetted surface of the hull or it can beidentified during maneuvering experiments. For 3 DOF maneuvering, the restoringforces (hydrostatics due to the Archimedes principle) can be neglected. Here, theadded mass (MA) and the corresponding Coriolis matrix (CA) take the form,

MA =

−Xu 0 00 −Yv 00 0 −Nr

;CA(v) =

0 0 Yvv0 0 −Xuu−Yvv Xuu 0

The damping forces and moments can be represented by linear terms [Xu, Yv, Nr].For a constant surge speed, only these linear terms come into picture. But sincewe are seeking a globally valid model, a nonlinear representation called the secondorder modulus model where the coefficients are based on experimental data andcurve fitting. Hence the total damping matrix can be collected into D(v) as,

D(v) =

−Xu −X|u|u|u| 0 00 −Yv − Y|v|v|v| − Y|r|v|r| −Y|v|r|v| − Y|r|r|r|0 −N|v|v|v| −N|r|v|r| −Nr −N|v|r|v| −N|r|r|r|

The final dynamics of the marine vessel can be written down as below, using su-perposition,

(MRB +MA)v + (CRB(v) + CA(v) +D(v))v = τ + w(t) (3.3)

where w(t) represents a time-varying disturbance that is caused due wind, wavesand ocean currents.

The actuators can be modeled using a first order system that models the timedelay of the mechanical azipods to achieve a given force (X,Y) and moment (N) .

Ti = uiTs− TiTs, where i = X,Y,N (3.4)

Hence the system is modeled using nine states x = (x, y, ψ, u, v, r, Tx, Ty, Tr) andthree control inputs u = (ux, uy, ur). The concise notation for states and inputsbeing, x = (η, ν, τ) and u respectively.

3.2 The Reachability Problem

In the reachability problem, we try to determine the set of states the system canreach, under the influence of a set of control inputs & disturbances and a setuncertain parameters. This work concentrates on three different approaches forcomputing the reachable set for the vessel.

• Hamilton Jacobi Reachability Analysis


• Set-Theoretic Reachable set computation.

• Classification based Reachability Algorithm

The next sections, in this chapter, detail the first two methods, its application toship’s reachability analysis and its drawbacks. The next chapter details the mainresults of the thesis for computing the reachable set for the marine vessel, using thethird method, where supervised machine learning is used to compute the reachableset and how the drawbacks posed by both Hamilton Jacobi analysis and the set-theoretic approaches are overcome.

3.2.1 Hamilton Jacobi Reachability AnalysisHamilton Jacobi reachability analysis falls under the umbrella of optimal controlproblems and differential games, which are theoretical tools that can be used forverification of safety-critical systems. The reachable set of a dynamical system canbe computed with respect to a target set, which describes the set of final conditionsunder consideration. The final goal could be a desirable set of states (goal set) oran unsafe set of states (avoid set). The goal of Hamilton Jacobi (HJ) reachabilityis to compute different flavours of reachability namely, Backwards Reachable setand the Forward reachable set. Let us setup the mathematical theory behind HJreachability according to [7].

Figure 3.2: Illustration of the level set method, which encodes the target set andthe reachable set using value functions.


Let x ∈ Rn be the system state, which evolves according to,

x = f(x(t), u(t), w(t)), u(t) ∈ U,w(t) ∈W (3.5)

where, u(t) is the control input and w(t) is the disturbance acting on the system.The sets W and U represent the control and state constraints respectively. Intu-itively, a Backward Reachable set represents the set of states x ∈ Rn from which thesystem can be driven into some set <0 ⊆ Rn at the end of time horizon t. Here, thecontrol input tries to steer the system towards the goal, whereas the disturbancetries to steer away from the goal. Hence, we want to compute the set,

<(t) = {x : ∃w(·) ∈W, ∀u(·) ∈ U, x(0) ∈ <0} (3.6)

Formally, let Jt(x, u(·), w(·)), be the cost accumulated, after a finite time horizon.If we consider Jt(·) as the signed distance between the system state and the targetregion at the terminal state of the system, we can determine if the trajectory reachedthe target if the signed distance is negative. The target set, <0 can be encoded asthe zero sub level set of a bounded and Lipschitz continuous function v(x), suchthat,

<0 = {x ∈ Rn|v(x) ≤ 0} (3.7)

The reachable set can be encoded, using the value function as,

<(t) = {x ∈ Rn|V (t, x) ≤ 0} (3.8)

If we define the cost as,

Jt(x, u(·), w(·)) = v(x(0)) (3.9)

then the system reaches the target set under the control u(t) and disturbance w(t)only if, Jt(x, u(·), w(·)) ≤ 0. The idea of the level-set method is illustrated in theFigure 3.2.

Since the control input wants to drive the system to the target, it wants tominimize the cost in 3.9, whereas disturbance wants to maximize ( or steer awayfrom the target set) the cost. Hence, there is a dynamic game played between thecontrol input and the disturbance input, resulting in,

maxw

minu

Jt(x, u(·), w(·)) (3.10)

Solving 3.10, using the principle of dynamic programming, we land up in theHamilton-Jacobi-Isaacs equation,

∂V (t, x)∂t

+H(t, x, λ) = 0, V (0, x) = v(x) (3.11)

H(t, x, λ) = maxw

minu

λ.f(x, u, w) (3.12)


The interpretation of <(t) is that, if x(t) ∈ <(t), then the control input can drivethe system to the target set at time 0, irrespective of the disturbance. Here, if thetarget state is desirable, then we are interested in verifying if there exists a controlinput that can steer the system to the target despite the worst case disturbance. Butif the target set <0 is an obstacle (avoid set), we need to minimize over disturbanceand maximize over the control input.

3.2.2 ImplementationThe implementation of Hamilton Jacobi Reachability analysis in Level-Set toolbox[17]in Matlab, using the ship’s dynamics, was carried out. Since the computation timegrows exponentially with the number of states [7], A simpler system, taking intoaccount only the surge and Sway velocity states, was implemented in the Level-Settoolbox. Referring to the surge and sway equations (equation 3.3), it can be seenthat the control (u(t)) and the disturbance inputs (w(t)) enter the system affinely.The optimal control input and the optimal disturbance input is required to computethe Hamiltonian required to solve the PDE (equation 3.11). If the control inputsare constrained (as in our case), such that u(t) ∈ [umin, umax], w(t) ∈ [wmin, wmax]this results in a bang-bang control. The inputs to the toolbox are the functionsrepresenting,

• Dynamics equations in ODE

x = f(x(t), u(t), w(t)) = F (x(t)) + u(t) + w(t) (3.13)

• Optimal control input

u∗(t) = argmaxu{λ.f(x(t), u(t), w(t))} = argmaxu{λ.u} ={umax, if λ > 0umin, if λ ≤ 0

(3.14)

• Optimal disturbance input

w∗(t) = argminw{λ.f(x, u, w)} = argminw{λ.w} ={wmin, if λ > 0wmax, if λ ≤ 0

(3.15)

Figure 3.3, shows the forward reachable set, where the target set is a goal set, hence


the control tries to maximize the Hamiltonian and the disturbance tries to minimizethe Hamiltonian.The reachable set is computed for 1 s forward in time, but the computation for

Figure 3.3: Forward Reachable set for the velocity states in the surge and the swaydirection.

solving the HJ equation takes 77s. When including the third state, the computa-tion takes minutes, and if all the six states, including the kinematic and dynamicrelationships, the program becomes intractable, and the Level-set toolbox throwsan error statement that, "Dimension > 5 is dangerously large". Hence, althoughthis method, for computing the reachable set provides lots of advantages, suchas, providing theoretical guarantees, ability to handle sets of arbitrary shape andproviding the optimal control inputs for reaching the desired state (or avoiding,based on the application), Hamilton Jacobi reachability suffers from the curse ofdimensionality, and proves to be incompatible for the application at hand.

3.2.3 Over-approximate Set Theoretic Reachable setcomputation

The exact reachable set computation, as seen in the previous section, is time con-suming and proves intractable as the number of dimensions in the state spaceincreases. Hence, an over-approximation of the reachable set can be employed,where the accuracy of the reachable set is compromised for faster computationtime. Over-approximation of the reachable set is useful, in the context of safetyverification. Clearly, if the over-approximated set of reachable states does not in-tersect the unsafe set, it means the original dynamical system is safe as well (Figure3.4.


Figure 3.4: Illustration of the overaproximation of reachable set used for safetyverification [2].

This thesis uses the Continuous Reachability Analyzer-CORA [2], which is aformal verification tool of cyber-physical systems. All the sets (in Rn) are repre-sented, by a Zonotope representation, which are parametrized by a center c ∈ Rnand generators gi ∈ Rn, defined as,

Figure 3.5: Illustration of the steps involved in approximate reachable set calcula-tion [2].

Z = {c+p∑i=1

βigi|βi ∈ [−1, 1], c ∈ Rn, gi ∈ Rn} (3.16)

Zonotopes are a compact way of representing sets in high dimensions and, opera-tions required for reachability analysis, such as, linear maps,M⊗Z := {Mz|z ∈ Z}and Minkowski addition Za ⊕ Zb := {za + zb|za ∈ Za, zb ∈ Zb} can be computedefficiently [2]. For a linear system, characterized by,

x = Ax+Bu x(0) ∈ X0 ⊂ Rn, u(t) ∈ U ⊂ Rm (3.17)

The idea is to first compute the the set of all solutions, from the initial setof states X0, which involves the computation of the matrix exponential eAt butassuming no uncertainty in the control input. The next step is compute a convexhull which encompasses the solution and the initial state. The final step is to enlargethe computed convex hull to account for the uncertain input (Figure 3.5).

3.3. SUMMARY 17

Since, the marine vessel is a nonlinear system, the above algorithm can beapplied only if the nonlinear system is linearized. In CORA, the system is linearizedin the center of the present state zonotope and the control set zonotope. Thelinearization error is accounted for, by including it in as an uncertain input, therebyenlarging the approximate reachable set. The Figure 3.6, shows the reachable setcomputed for the marine vessel using the kinematic and dynamical relationships(equation 3.1, 3.3).

Figure 3.6: Reachable set for the marine vessel with 3 DOF maneuvering model,for 3 min into the future, admissible control being 1/10th the original value.

The assumed constraints on the control inputs is one-tenth the original value forthe real vessel. If the admissible control set is enlarged than this value, the uncer-tainty in the system explodes, thereby rendering an impractical over-approximation.Since the nonlinearity in the dynamics of the marine vessel is huge, the admissiblecontrol input set cannot be high, because both contribute to the error in approx-imation of the reachable set. The computation of the forward reachable set for 3min into the future, requires 5 s of computation time. In the Figure 3.6 , it can beseen that the solution obtained by this method is clearly the over-approximation ofthe true trajectories (black lines representing random trajectories).

3.3 Summary

In this chapter, the nonlinear maneuvering model of the ship is introduced, alongwith the actuator dynamics and constraints. The nonlinear model is used to com-pute the reachable set of the marine vessel, given the actuator limits and the limits


on the maximal allowable disturbance. The first proposed method using the theoryof Hamilton Jacobi reachability analysis provides exact calculation of the reachableset, by solving the HJI PDE, using the level set method. But on the downside, thecomputation is O(en), where n is the number of states for the system. Hence, itproves intractable for the ship dynamics, which has as many as 6 states (withoutconsidering the actuator dynamics). In the other method involving, set theoreticcomputations, we try to linearize the nonlinear model and to enlarge the evolvedreachable set with this linearization error. This method results in impractical reach-able sets, as the linearization error grows, or if the admissible control range is large.

The results obtained in this chapter, using both the Hamilton Jacobi analysis,and the set theory based computation methods, motivated the author to use asupervised Machine learning based tequenique to best approximate the reachableset. This forms the content of the next chapter.

Chapter 4

Classification based ReachabilityAnalysis

This chapter focuses on the implementation of the classification based algorithm,used for computing the reachable set for the marine vessel. The first step in the

Figure 4.1: Simplified 2D illustration of a cost limited reachable set for a giventhreshold cost of J0

algorithm is to sample the state space randomly and combine two points which actas the start point and the goal point. The next step is to compute a time optimalcontrol strategy which minimizes the final time, where the trajectory is constrainedto start and finish at particular points in state-space. If the time required to reachthe goal point, is less than the threshold final time as required by the user, we

19

20 CHAPTER 4. CLASSIFICATION BASED REACHABILITY ANALYSIS

label the start-goal pair to be reachable, with the threshold final time (Figure 5.4).This is iterated for different start-goal pairs and each pair is labeled to be eitherreachable or non-reachable. Support Vector Machines is used for separating theboundary between the reachable and non-reachable points. Given a new querystart-goal pair, the trained Support Vector Machine can classify the query pointas either reachable or non-reachable. This work is based on the theory formulatedin [15] and builds on it for the computation of reachable sets in the presence ofuncertainties and disturbances.

The organization of this chapter is as follows. Section 4.1 develops the optimalcontrol formulation. In Section 4.1.1, we solve the optimal control problem usingMultiple Shooting method and Sequential Quadratic Programming (SQP).In Sec-tion 4.2 it is shown how the developed dataset can be used to learn the nonlinearboundary separating the reachable and the non-reachable points in space. Later,in Section 4.3.1 and Section 4.3 talks about creation of the dataset and how it isimplemented with the ACADO Toolbox [18] respectively Finally, in Section 4.3.2an extension is presented to include uncertainties and disturbances into the solutionframework.

4.1 Optimal Control Problem

Consider the nonlinear model of the ship, including the actuator dynamics (equation3.3,3.4) represented by x = f(x(t), u(t)), where x ∈ which evolves over a timehorizon [t0, tf ].

Let us define a cost,

J = φ(x(t), tf )︸︷︷︸Mayer term

+∫ tf

t0

L(x(τ), u(τ), τ)dτ︸︷︷︸Lagrange Term

(4.1)

and the corresponding cost-limitted reachable set, for the arbitrary start state xa as[19]

R(xa, U, J0) = {xb ∈ X‖∃u ∈ U & ∃t′ ∈ [t0, tf ] (4.2)s.t. x(t′) = xb & J∗ ≤ J0} (4.3)

where x(t) and u(t) represents the state vector and control vector respectively, Xrepresents the state constraints and U represents the admissible control inputs. J∗is the optimal cost for going from start (xa) to the goal (xb) state. Equation 4.2represents the set of all goal states, which obey the state constraints, such that theoptimal control responsible for arriving at the goal state obeys control constraintsand that the cost for arriving at the goal state is less than the threshold cost J0.Since, we are interested in the minimum time optimal control for the vessel, thecost J = tf needs to be minimized. Specifically the time optimal solution J∗ is

4.1. OPTIMAL CONTROL PROBLEM 21

given by:

J∗ =minimizeu

∫ tf

t0

dt

subject to u(t) ∈ U (actuator Constraints)x(t) ∈ X (state Constraints)x(t) = f(x, u) (Ship Dynamics + Actuator Dynamics)x(0) = xa, x(tf ) = xb

(4.4)Equation 4.18 is referred to as 2 Point Boundary Value Problem (2PBVP), trajec-tory optimization problem or the steering problem.

4.1.1 Optimal Control SolutionThe first step in solving the optimal control problem is transcription. Since theoptimization presented in 4.18 is continuous, it has infinitely many decision vari-ables. Thus the problem 4.18 needs to be discretized. Multiple shooting methodis used for discretization. The discretization leads to a Nonlinear program whichis solved using a Sequential Programming Method. Following is the details of themultiple shooting algorithm and Sequential Quadratic Programming for solving theproposed optimal control problem.

Figure 4.2: Illustration of the multiple shooting algorithm [3]

4.1.1.1 Multiple Shooting

Multiple shooting works by breaking up a trajectory into some number of segments,and guessing the solution in these grid points Figure 4.2. The solution approachis analogous to shooting a cannon at a target. The algorithm first simulates thetrajectory of the cannon and tries to minimize the defect between the shot, final


cannon state and the target state. The simulation is usually broken down into mul-tiple segments (Multiple Shooting) and it is stringed together by the optimizationalgorithm.

The continuous control input is discretized using a polynomial in the generalcase or using Zero Order Hold (ZOH) in the simple case in the fixed time grid0 = t0 < t1 < ... < tN = T . The discrete control represented by ZOH, sayu(t, q) = qk for t = [tk, tk+1]. The ODE, present in the optimal control problem issimulated separately on each iteration interval [tk, tk+1], using the ship dynamics,starting with artificial initial values si:

xk(t, sk, qk) = f(xk(t, sk, qk, qk)), t ∈ [tk, tk + 1] (4.5)

x(tk, sk, qk) = sk (4.6)

Similarly, we numerically compute the integrals,

lk(sk, qk) =∫ tk+1

tk

L(xk(tk, sk, qk), qk)dt =∫ tk+1

tk

dt (4.7)

The final issue of stringing together of the trajectories is left to the NLP solver asconstraints. Also, we choose the time grid on which the inequality path constraintsare checked (commonly the same grid as the one used for control discretization).So we arrive at the final NLP formulation,

minimizes,q

φ(sN ) +N−1∑i=0

li(si, qi)

subject to x0 = s0 (x0 = xa) (initial value)xi(ti+1, si, qi) = si+1 ≤ 0 i = 0, ...N − 1 (Stitching the trajectories)h(si, qi) ≤ 0 i = 0, ...N − 1 (path constraints)r(sN ) = 0 (xN = xb) (terminal constraints)

(4.8)

4.1.1.2 Sequential Quadratic Programming

The nonlinear program is solved by linearizing the cost function and the constraints.This yields a quadratic programming subproblem. The SQP problem requires thecost function, and the constraints to be twice continously differentiable, since aHessian needs to be computed at each time step. The nonlinear program (Equation4.8) obtained through multiple shooting can be summarized as,

minimizex

h(x)

subject to g(x) ≤ 0r(x) = 0

(4.9)

4.2. SUPPORT VECTOR MACHINE FOR BOUNDARY CLASSIFICATION 23

The Lagrangian of the problem is defined as,

L(x, λ, σ) = h(x) + λT g(x) + σT r(x) (4.10)

where, λ and σ are the lagrange multipliers. A guess point xk is chosen and asearch direction dk is chosen, which is the solution of the quadratic programmingsubproblem,

minimized

h(xk) + ∆h(xk)T d+ 12d

T∆2xxL(xk, λk, σk)d (4.11)

subject to g(xk) + ∆g(xk)T d ≤ 0 (4.12)r(xk) + ∆r(xk)T d = 0 (4.13)

4.2 Support Vector Machine for boundary classification

The idea behind this approach is to approximate the reachability boundary usinga nonlinear binary supervised classification algorithm [20]. It tries to separate thetraining (start, goal) pairs as being reachable or non-reachable sets. The trainednonlinear classifier can then be used to predict whether a new pair of (start, goal)pair is reachable or not. For the application of computing the reachable set for amarine vessel, in this thesis, a binary Support Vector Machine algorithm with anonlinear kernel function is employed. SVM seeks to design a function h(wTx+ b)affinely dependent on the training dataset x with the weights w and a bias termb, that correctly maps a vector x to a label y ∈ {−1,+1}, which is the separatinghyperplane between the two classes. In our example, -1 means that the given (start,goal) pair is not reachable within the defined cost threshold, whereas +1 means thepair is reachable. The parameters are designed to minimize the misclassificationthat occur in the training set, without over fitting the data. This optimizationproblem can be formulated as: finding b and w, that minimize ‖w‖, such that forall the data points (xi, yi),

yih(xi) ≥ 1 (4.14)

The support vectors are the xi on the boundary, those for which yih(xi) = 1.The resulting problem is a quadratic programing problem, and the optimal solution(w, b) enables the classification of a new test data point z as,

class(z) = sign(zw + b) (4.15)

It is often computationally simpler to solve the dual of the above problem bytaking the positive Lagrange multipliers αj multiplied to each constraint and sub-


Figure 4.3: Illustration of separating hyperplane & support vectors

tracting from the objective function to get,

maximizeα

m∑i=1

αi −12

m∑i,j=1

yiyjαiαj〈 xi, xj〉

subject to αi ≥ 0, i = 1, ...,mm∑i=1

αiyi = 0

(4.16)

In the above case, it is assumed that the data is perfectly separable, but in realityl1-regularization is performed which introduces slack variables to 4.16. Also sincewe are dealing with a nonlinear boundary classification problem, there is no simplehyperplane separating the two types of data. A kernel function creates linearlyseparable data in the higher dimension. Hence a kernel function (Gaussian in thiscase), k(x, z) = e‖x−z‖

2 replaces the dot product in the above set of equations.

4.3 Implementation in ACADO

The previous sections introduced the general theory behind the solution of theoptimal control problem. In this section, we will adapt the theory to solve the time-optimal control of the ship, including the actuator dynamics. Since the SQP methodintroduced in Section 4.1.1.2, requires the constraints to be twice differentiable, andthe ship dynamics (Equation ), pertaining to the velocity states have absolute valuefunctions, an approximation needs to be carried out.

4.3. IMPLEMENTATION IN ACADO 25

Approximation 1: The cubic polynomial fit for the ν|ν| term in the velocitystates, i.e.

ν|ν| = a0ν3 + b0ν

2 + c0ν + d ν = [u, v, r]′ (4.17)

Approximation 2: The cross coupling terms, |v|r, |r|v in the differential equationspertaining to sway and yaw rate dynamics are assumed to be small.

The cubic approximations are carried out in the state space domains of the indi-vidual velocity states, providing a really good fit to the actual dynamics. The aboveapproximations work quite well in converting an ill-posed optimization problem toa problem that can be solved robustly. Moreover, the cross-coupling terms in thesway and yaw rate dynamics can be assumed to be zero while considering the caseof maneuvering, as neither sway nor yaw rate is high during maneuvering in theopen sea.

In this thesis, the ACADO Toolkit [18], which is a software environment andalgorithm collection for automatic control and dynamic optimization has been usedto solve the optimal control problem:

J∗ =minimizeu

T

subject to u(t) ∈ U (actuator Constraints)x(t) ∈ X (state Constraints)x(t) = f(x, u) (Ship Dynamics + Actuator Dynamics)x(0) = xa, x(tf ) = xb

(4.18)Here, the differential equation constraint corresponds to the a kinematics and

the dynamics of the ship (Equation [3.3]), as well as the dynamics of the actua-tor(Equation [3.4]) (modeled by the first order system in each DOF). U correspondsto how the ship’s azipods are constrained by the maximum and minimum thrustthat can be produced in all the 3 DOF. The states corresponding to heading andthe velocity states are also constrained by a maximum value, when the ship ismaneuvering in the open sea.

Problem 4.12 can be cast in ACADO [18], which tries to compute the time-optimal control policy for the ship starting from xa to reach xb. ACADO code iswritten so that Problem 4.18 is transcribed by the Multiple Shooting technique. Thecontrol vector is discretized using 40 discretization points. The maximum final timethreshold is 200 s, 0 ≤ T ≤ 200. So the discretization interval is 5 s. For the aboveposed problem with box input constraints, the exact Hessian can be computed,which can give results with higher accuracy, albeit being computationally slow.Since we are dealing with a nonlinear optimization problem, the sufficient conditionsof optimality can only guarantee the convergence to a local minima. The exactHessian works in our favour, where the solution almost always converges to theglobal minima.


4.3.1 Dataset CreationThe dataset that needs to be trained by the supervised binary classifier requires twolabels of either being reachable or not. The dataset is created by changing differentvalues in the optimization problem and solving it to gather the information of itbeing reachable or not.

The important details of how the initial and the final values of the differentstates are constrained, according to the condition of maneuvering in the open sea,are outlined as follows. The initial values of states x, y, ψ can without loss ofgenerality be chosen as zero, since the acquired reachable set can be translated toany x, y and be rotated with the current heading ψ to obtain the reachable setat any x, y, ψ. Among the velocity states (u, v, r), during maneuvering, it can beassumed for the sake of minimizing the complexity, only the surge velocity changes(from zero to maximum allowed surge speed u = umax, with linear interpolation inbetween), and the sway velocity and yaw rate remains close to zero, since, duringmaneuvering, the ship is not expected to turn unexpectedly. The initial statefor the thruster in the surge direction is constrained by the steady-state thrustervalue, which has been responsible for producing the initial surge velocity (Line 7 inAlgorithm 1). The final states of (x, y) take the form of a sampled grid in space infront of the marine vessel. All the other states are not constrained, and are left totake any value.

To summarize, the problem 4.18 needs to be solved multiple times looped overdifferent initial surge velocities (interpolated between u = 0 to u = umax), and alsoover the grid of final spacial states of (x, y). Finally in the online phase, the presentstate information of the ship is received (Line 17 in Algorithm 1), and the SVMpredicts the reachable set boundary for the given query surge velocity. In order toplot the boundary, various final state points are plotted. This boundary representsthe reachable set, if the ship was at poisition η = [0, 0, 0]′. So the position statesneeds to be translated and rotated with the current position information (Line19,20 in Algorithm 1). Finally, the rotated and translated reachable set is plotted.

4.3. IMPLEMENTATION IN ACADO 27

Algorithm 1: Classification based Reachable set Computation1 OFFLINE2 input: umax, Tthreshold

3 Interpolate: uvector = [0, umax]4 Interpolate: xfinal = 2-D Mesh in (x,y);5 while xb ∈ xfinal && uinit ∈ uvector do6 Thrust0 = Solve steady state surge equation with

[u, v, r] = 0 & u = uinit;7 Topt = Solve Problem 4.18 with

xa = [0, 0, 0, uinit, 0, 0, Thrust0, 0, 0] & xb(1 : 2) = xfinal;8 if Topt ≤ Tthreshold then9 label=1;

10 else11 label=-1;12 end13 dataset= [xfinal;uinit; label];14 end15 SVMModel= Train(dataset);16 ONLINE17 xnow = xi, ynow = yi, ψnow = ψi, unow = ui;18 [xreach, yreach] = predict boundary(SVMModel(uinit));19 xsolution = xnow + xreachcos(ψnow) + yreachsin(ψnow);20 ysolution = ynow − xreachsin(ψnow) + yreachcos(ψnow);21 Plot(xsolution, ysolution);

4.3.2 Including Uncertainties and DisturbancesThe uncertainties in the system dynamics and external disturbances due to thesea condition alter the reachable set R. The idea of including uncertainties is asfollows. The inertia matrix could be uncertain, due to the loading conditions onthe vessel. Hence the inertia matrix can vary from [M −∆M ,M + ∆M ], where ∆M

corresponds to the uncertainty introduced due to the different loading conditions ofthe marine vessel. The algorithm presented in Section 4.3 can be carried out withvarying inertia matrix (linearly interpolated from M − ∆M to M + ∆M ). Let itproduce m such varying Inertia matrices. We obtain m such reachable sets <1...<m.If the target set is a goal set (defined in ...) the overall reachable set correspondsto <uncertain = <1 ∩ <2.. ∩ <m. The overall reachable set is an intersection of theindividual sets, since, for all the possible uncertainties in the marine vessel, thereexists a control that can steer the ship to the goal set <uncertain. Also, for thecomputation of the avoid set or the unsafe set (defined in ...), the reachable set


becomes <uncertain = <1 ∪<2..∪<m. The overall reachable set is the union of theindividual sets, since, despite all the possible control actions of the marine vessel,there exists an uncertainty that can drive the vessel to the unsafe set <uncertain.

A similar approach can be adopted to include disturbance, wherein the distur-bance can act in all the 3 DOF in either the positive direction or the negativedirection.

−dmaxu ≤ wu(t) ≤ dmaxu (4.19)−dmaxv ≤ wv(t) ≤ dmaxv (4.20)−dmaxr ≤ wr(t) ≤ dmaxr (4.21)

where, w(t) = [wu(t), wv(t), wr(t)]′ represents the disturbance vector and themaximum values are represented by dmax = [dmaxu , dmaxv , dmaxr ]′. Since, the dis-turbance shrinks or enlarges the reachable set, either the maximum value or theminimum value (maximum in the negative direction) produces the most deviationfrom the normal reachable set(< is monotonous with respect to disturbance w(t)).Hence there are eight possible combinations of possible disturbance values (eitherdmax or −dmax). Similar to the approach used for uncertainties, the goal reachableset is <disturbance = <1∩<2..∩<8 and the avoid set is <disturbance = <1∪<2..∪<8.

4.4 Summary

In this chapter, the reachable set is computed in real time, in three major steps.The first phase is the creation of dataset, wherein a time-optimal control problemis solved using various features such as varying initial surge velocity, variation inthe inertia matrix, and varying disturbances in the 3 DOF, and if the optimal timeis less than the threshold time required, it is labeled as +1 and if not, it is labeledas -1. The required reachable set (goal or avoid set) in the presence of disturbancesand uncertainties can be calculated either using the intersection or the union of thedifferent sets. The next phase of the algorithm is learning a nonlinear binary SVMclassifier using the created training dataset to separate the boundary of points instate-space, where the ship can reach in a certain threshold time. The final stageis the prediction of the reachable set, using the state information and as the inputto the trained SVM model, in real time.

Chapter 5

Simulation Results

In this chapter, the results for the classification based reachability analysis has beendetailed. The model used for the simulations is briefed in Chapter 3. The methodused for the computations is detailed in Chapter 4.

5.1 Maneuvering reachable set for different initialconditions

The plot 5.1 shows the reachable set for the marine vessel computed using theclassification based method. The state space is sampled, with 50mX50m boxes.The initial surge velocity is taken to be 1 m/s. The computation assumes, that thereachable set is computed for 200 s in the future. The red points are those with thelabel of feasibility for the optimal control problem, while the blue points are notfeasible within the given time frame.

The plot 5.2 shows how the reachable set is different for different initial veloci-ties. This can be used to compute the reachable set online, while the ship is moving,where the surge velocity is fed back from the sensors. The final plot in Figure 5.2shows the classification boundary with different initial conditions.

29

30 CHAPTER 5. SIMULATION RESULTS

Figure 5.1: The Reachable set for the marine vessel with initial surge velocity, u =1 m/s. The boundary between the reachable and non reachable is classified usingthe SVM algorithm.

5.2 Maneuvering reachable set for uncertain Mass matrix

Figure 5.3 shows the reachable sets in the presence of uncertainty in the Massmatrix. The reachable set in the presence of uncertainties can be handled usingthe theory formulated in Section 4.3.2. If the target set is a goal set, the outerreachable set is the solution which contains the set of all states the vessel can reachgiven the worst uncertainty. On the other hand, if the expected reach set is anavoid set, the solution is that of the smallest set, using the intersection of the threesets. The final result is intuitive, since when the Mass matrix is higher, for thesame propulsion, the ship has a smaller reachable set. Hence the sets are nestedinto one other.

5.3. MANEUVERING REACHABLE SET IN THE PRESENCE OFDISTURBANCES 31

Figure 5.2: The plots show the reachable set computed for varying initial surgevelocities ranging from 1-5 m/s. The final plot shows the classification boundarywith different initial conditions.

Figure 5.3: The plot shows the reachable sets for the marine vessel in the presenceof uncertainties. The uncertainty is present in the inertia matrix, where it can vary25% of the Mass matrix.

5.3 Maneuvering reachable set in the presence ofdisturbances

The figure 5.4 shows the different reachable sets in the 8 worst possible combinationsin the disturbance vector. The possibilities arise from the fact that there can be

32 CHAPTER 5. SIMULATION RESULTS

two directions in space (max or min) in all the 3 DOF. In the experiment the initialsurge velocity is 1 m/s. The disturbance is bounded by a maximum of 1/10th themaximum actuator force.

Figure 5.4: The plot shows the reachable set in the presence of disturbance. Thedisturbance in the direction of Surge, Sway and Yaw directions is assumed be within1/10th the maximum actuator force.

Chapter 6

Conclusion and Future Work

In this chapter, the results for the thesis are summarized and a possible list of thedirections in which the research can be extended is discussed.

6.1 Conclusions

The results of this thesis provides a way of transitioning up the levels of autonomyfor marine vessels. This thesis in particular tries to solve the problem of reachabilityfor the marine vessel. The reachability analysis for the marine vessel, provides theset of all states the vessel can be in in some future time, given the dynamics of thevessel, the actuator constraints and the maximum and minimum bounds in whichthe disturbances can lie. The reachable set can be used for obstacle avoidancealgorithms and as a basic application be used for dead-reckoning, where the captainof the ship knows where the ship could be in future time, if the position sensors donot give accurate results.

The thesis starts with computing the reachable set for the vessel, using theHamilton Jacobi reachability, which provides many advantages such as, providingthe exact solution, used for any complex nonlinear dynamical system, integratingthe control and the disturbance inputs seamlessly and providing set solutions ofarbitrary complex shapes. The Level-set toolbox is used to solve the PDE thatarises while solving the Hamilton Jacobi method. But the major disadvantageis that the method having need to solve a partial differential equation, becomesintractable, as the dimensions of the dynamical system becomes more than 5.

In order to overcome this curse of dimensionality, another method using set the-oretical calculations was considered. This method provides an over-approximationof the exact reachable set. In this thesis, the CORA toolbox was used to computethe reachable set. The method involves the linearization of the dynamical system,in order to evolve the solution of the reachable set over time. The obtained reach-able reachable set is then enlarged to accommodate the errors obtained during thelinearization and to accommodate the admissible control set. The issue encoun-

33

34 CHAPTER 6. CONCLUSION AND FUTURE WORK

tered for the marine vessel, was that the dynamical system is highly nonlinear,thereby rendering the over-approximation useless. But the computation providesreasonable results, if the the admissible control range is 1/10th the actual range.

The classification based approach tries to overcome the challenges posed by thefirst two methods. Although the emphasis is on real-time computation, this methodprovides really good approximation to the exact reachable set. The dataset is cre-ated by solving many optimal control problems and is given to a classifier to clas-sify the reachable boundary.The advantages for the method is that it can computethe reachable set in real-time, and can tackle the problems posed by the previousmethods. The thesis also takes into account the actuator dynamics to reduce theapproximations posed on the dynamics. This work also shows how disturbancesand uncertainity can be brought into this framework. On the downside, the num-ber of problems to be solved is huge, but it does not prove to be a hindrance forreal time analysis, since it needs to be solved only off line. Also, the dynamics isapproximated using a third order polynomial, wherein an absolute value function isinvolved, since the optimal control problem needs to by continuously differentiableto provide robust results.

6.2 Future Work

6.2.1 Incorporating the geometry of the vessel

The modeling of the marine vessel, assumes that the mass of the ship is concentratedon its Center of Gravity (CoG). Thus, the computed Reachable set is exact, onlyfor the Center of Gravity (CoG). The consequence of such a reachable set can beunderstood from an example. Consider the case of obstacle avoidance of the ship.It does not mean that the bow of the ship does not collide with the obstacle, ifthe CoG does not crash with the obstacle. Thus the geometry of the ship needsto be integrated into the modeling. This can be accomplished by having multiplepoints of interest (Bow and aft, along with CoG), on which the dynamics are arecomputed.

6.2.2 Optimal control solution for obstacle avoidance

In the real time reachability analysis algorithm developed for this thesis, the actualsolution trajectories are not used, but only the information whether the solutionexists or not (feasibility). A natural extension would be to use these computedtrajectories, by storing them in a database, and a real time algorithm chooses oneof the trajectories for motion planning, which is the time optimal path for reachingpoint B from A. The path can be broken up, so that the resulting smaller path isobstacle free and a new path is generated once point B is reached. The HamiltonJacobi reachability is used for real-time motion planning in [10], where both theoptimal control and error bound (reachable set) is used for motion planning.

6.2. FUTURE WORK 35

6.2.3 Lifting the assumptions made for computing theReachable set and improving the accuracy of the method

• The algorithm developed now, assumes the fact that the ship is maneuvering,and thus the initial conditions for the sway velocity and yaw rate is assumed tobe zero. But in reality, the ship could have small yaw rate and sway velocity,which can be incorporated into creating the dataset.

• The actual force constraints for the actuators are not of the box shape, butis characterized by ellipsoids, which need to be used as constraints for theoptimal control problem.

• The assumptions made on the cross-coupling nonlinear terms need to be lifted,in order to provide accurate results.

• The accuracy of the SVM model can be increased, by using more featuresother than the state vector. The Principle Component Analysis (PCA) couldbe used, to determine which feature influences the accuracy of the predictionmodel the most.

Bibliography

[1] M. Chen and C. Tomlin, “High Dimensional Reachability Analysis : Address-ing the Curse of Dimensionality in Formal Verification,” 2017.

[2] M. Althoff, “CORA 2016 manual,” 2016. [Online]. Available: https://tumcps.github.io/CORA/

[3] M. Diehl, “Numerical Optimal Control (preliminary and incomplete draft),”vol. 2017, 2017.

[4] T. I. Fossen, Handbook of Marine Craft Hydrodynamics and Motion Control,2011.

[5] R. Skjetne, Ã. Smogeli, and T. I. Fossen, “Cams 2004,” pp. 65–70, 2004.

[6] C. Baier and J.-P. Katoen, Principles of Model Checking, 2008, vol. 26202649.

[7] S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin, “Hamilton-Jacobi Reacha-bility: A Brief Overview and Recent Advances,” 2017.

[8] M. Chen, S. L. Herbert, M. Vashishtha, S. Bansal, and C. J. Tomlin, “De-composition of Reachable Sets and Tubes for a Class of Nonlinear Systems,”2018.

[9] V. R. Royo, D. Fridovich-Keil, S. Herbert, and C. J. Tomlin, “Classification-based Approximate Reachability with Guarantees Applied to Safe TrajectoryTracking,” 2018.

[10] S. L. Herbert, M. Chen, S. Han, S. Bansal, J. F. Fisac, and C. J. Tomlin, “FaS-Track: a Modular Framework for Fast and Guaranteed Safe Motion Planning.”

[11] J. F. Fisac, A. K. Akametalu, M. N. Zeilinger, S. Kaynama, J. Gillula, andC. J. Tomlin, “A General Safety Framework for Learning-Based Control inUncertain Robotic Systems,” 2017.

[12] G. Frehse, C. Le Guernic, A. Donzé, S. Cotton, R. Ray, O. Lebeltel, R. Ripado,A. Girard, T. Dang, and O. Maler, “SpaceEx: Scalable verification of hybridsystems,” in Lecture Notes in Computer Science (including subseries LectureNotes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2011.

37

38 BIBLIOGRAPHY

[13] X. Chen, E. Ábrahám, and S. Sankaranarayanan, “Flow*: An analyzer for non-linear hybrid systems,” Lecture Notes in Computer Science (including subseriesLecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics),vol. 8044 LNCS, pp. 258–263, 2013.

[14] A. A. Kurzhanskiy and P. Varaiya, “Ellipsoidal techniques for reachabilityanalysis of discrete-time linear systems,” IEEE Transactions on AutomaticControl, 2007.

[15] R. E. Allen, A. A. Clark, J. A. Starek, and M. Pavone, “A Machine LearningApproach for Real-Time Reachability Analysis.”

[16] R. Allen and M. Pavone, “Toward a real-time framework for solving the kino-dynamic motion planning problem,” in Proceedings - IEEE International Con-ference on Robotics and Automation, 2015.

[17] I. Mitchell and J. Templeton, “A Toolbox of Hamilton-Jacobi Solvers for Anal-ysis of Nondeterministic Continuous and Hybrid Systems,” Hybrid Systems:Computation and Control, pp. 480–494, 2005.

[18] B. Houska, H. J. Ferreau, and M. Diehl, “ACADO toolkit-An open-sourceframework for automatic control and dynamic optimization,” Optimal ControlApplications and Methods, vol. 32, no. 3, pp. 298–312, 2011.

[19] S. M. LaValle, “Planning algorithms,” Planning Algorithms, vol. 9780521862,pp. 1–826, 2006.

[20] T. Hastie, R. Tibshirani, and J. Friedman, “The Elements of Statistical Learn-ing,” The Mathematical Intelligencer, vol. 27, no. 2, 2001.

www.kth.seTRITA-EECS-EX-2018:738

Date post:	22-Apr-2020
Category:	Documents
Upload:	others
View:	6 times
Download:	0 times

Real Time Reachability Analysis for Marine...

Documents