Real-time monitoring and delay management of a transport information system

UNIVERSITY OF L’AQUILA

MASTER DEGREE THESIS

“Real-time monitoring and delay management of atransport information system”

Master Degree in Computer Science

Candidate:

Lorenzo Sfarra

Supervisor:

Dott. Ivano Malavolta

Co-supervisor:

Dott. Mattia D’Emidio

Academic Year 2013/2014

Contents

Introduction 9

1 Requirements and Specifications 10

1.1 BusOnAir: an overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.2 BusOnAir: development plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.2.1 BusOnAirClient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1.2.2 BusOnAirVehicle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.2.3 BusOnAirServer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.3 Stops, bus lines and timetable management . . . . . . . . . . . . . . . . . . . . . . 13

1.4 Shortest-path calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.5 Delay management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.6 Real-time tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2 Models and Algorithms 16

2.1 EAP - Earliest Arrival Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2 Time Expanded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.2 Time Expanded Simplified - Model . . . . . . . . . . . . . . . . . . . . . . . 19

2.2.3 Fully Time Expanded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3 Delays in Time Expanded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.4 EAP in Time-Expanded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.4.1 EAP in Fully Time-Expanded . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.5 Bicriteria Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3 Server module changes 24

3.1 From Simplified Acyclic Time Expanded to Fully Time Expanded . . . . . . . . . . 24

3.2 Fully Time Expanded: the model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2.1 Multiple days support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.3 Delay management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.4 Performance Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2

Chapter 0 Master Degree Thesis - Academic Year 2013-2014 Contents

4 REST APIs 28

4.1 APIs: CREATE/UPDATE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.1.1 stations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.1.2 routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.3 runs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.1.4 stops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.1.5 checkpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.1.6 REAL TIME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.2 LOAD IN RAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.3 APIs: RETRIEVE DATA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.3.1 routes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.3.2 stops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

4.3.3 runs and checkpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

4.3.4 stations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

4.3.5 research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.4 USING APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.4.1 Direct Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.4.2 Users clients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4.4.3 Tracker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5 PERFORMANCE ASSESSMENT AND IMPROVEMENTS 43

5.1 Introduction to results and used terms . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.2 Neo4j . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.3 Previous heuristic results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.4 System and performance analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.4.1 Hardware level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.4.2 Neo4j configuration level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.4.3 Code level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

5.5 Current heuristic results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.6 Update operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

5.6.1 CheckPoint passed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

5.6.2 Adding a CheckPoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.6.3 Restore a Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.6.4 Update position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.7 Conclusion and future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.8 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3


A APIs related code 55

A.1 UDP Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

A.2 Command line scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

A.2.1 Checkpoint info . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

A.2.2 Checkpoint passed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

A.2.3 Runs for a given route and relative time . . . . . . . . . . . . . . . . . . . . 59

A.2.4 Update position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

A.2.5 Run info . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

A.2.6 Running buses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

A.2.7 Stop info . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

B Algorithms 64

B.1 Shortest Paths in DAGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

B.2 FT-ShortestPath . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Bibliography 67

4

Introduction

Background

One of the new century’s challenge is to create different ways to provide better and smarter

solutions related to the services to the city’s population. In fact, the concept of ”city” is becoming

more and more central for the economic growth. A UN forecast released in 2008 reports that

70 percent of all humans will live in urban areas by 2050, even though cities occupy only about

3 percent of Earth’s land surface. This urban growth can be very positive for economic growth

and social development, and even for environmental issues[1]. This growing number of people

concentrated in a single limited space leads to different problems that can be addressed only by

providing better services. The way in which they have to be provided, of course, is different from

case to case, but administrators share a certain number of common targets to be reached[2].

One of the most important is the improvement of the services related to the public transport: less

private vehicles, less traffic, less environmental impact. Succeeding in this challenge means to

improve the economical system of a State: a 2006 study reports that an efficient public transport

system can extremely increase the revenues of a State: the estimation of the loss related to the

traffic congestion is between 1 percent and 3 percent of GDP. So, how to address this problem?

One of the solution is, of course, investing in the infrastructures. A parallel solution is to provide an

adeguate service to the user, that will make his life easier using ITS, Intelligent Transport System.

ITS: Intelligent Transportation System

A European Union directive[3] defines ITS as follows:

Intelligent Transport Systems (ITS) are advanced applications which without embodying intelligence

as such aim to provide innovative services relating to different modes of transport and traffic

management and enable various users to be better informed and make safer, more coordinated and

’smarter’ use of transport networks

—DIRECTIVE 2010/40/EU

Another interesting definition is directly taken from the Atlanta Regional Commission[5]:

5


ITS are the application of a combination of technologies to the existing transportation system to

save time, lives and money. The goal is two fold: safer, quicker travel with enhanced mobility and

optimizing existing transportation infrastructure

—ATLANTA REGIONAL COMMISSION

It is evident from these definitions that ITS have to put the user at the center, giving him a

better and safer service with all the possibile information in a clear and simple way.

The following list describes just few applications regarding ITS:

• Variable speed limits: used to dynamically change the speed limit based on different fac-

tors like road congestion;

• Dynamic traffic light sequence: a paper shows that such a system could emulate the

judgment of a traffic police officer on duty[4];

• Variable parking fee: adjusting the parking fee per hour depending on different factors like

how busy the street is;

• Incident and emergency management;

• Floating car: low-cost methods (GPS, . . . ) for obtaining travel time and speed data less

expensive than sensors or cameras, with more coverage and works in all weather conditions.

• . . . .

The task of the administrators is to understand the real needs of their area, and consequently

to create and improve ITS related to them.

The BusOnAir project

The BusOnAir project is born to address all the problems related to the public (or private)

transports on the road. The thesis was focused on L’Aquila, and the data kindly provided by

A.M.A., Azienda Mobilita Aquilana. The complete description of what BusOnAir is and the detailed

background is described in the previous thesis about this subject, called ”BusOnAir - Un sistema

informativo per trasporti pubblici: modelli, algoritmi e tecnologie” and it will not be repeated here,

except for, of course, necessary information needed as a background to describe what was done

in the current thesis.

The services to be provided can be categorised in three parts:

• Services at the bus stop: the scenario is that a user goes to a bus station S and can

look into a panel where he can find the timetable with real-time information like delays, for

example, or information on how to reach another station, which lines passes through S, at

which time, and how many minutes he has to wait to take the bus he needs;

6


Figure 1: Bus stop services scenario.

• Services on board: Tool to present different kind of data to the passengers: the route, the

current position on a map, the destination, chance to buy a bus ticket. Services should be

automatic, the driver should not be involved in any of them;

• Web services: A catchy interface to interact with the user request and that gives useful

information in a clear and simple way. One of the most important features should be the

research of a path between two bus stations or between two points. All of the services

should also be supported by mobile platforms (iPad, Android, . . . ) adapting the content and

style to screen size.

Additionally, BusOnAir can be seen even as a service for a transport agency: real time moni-

toring of their vehicles, emergency handling, counting the number of passengers on a given bus,

. . . .

The technology impact should not be difficult: on the buses will be enough a box with a GPS/G-

PRS/GSM tracker that will send real time data to the server. All the rest can be easily embedded

in a single server.

Benefits

The benefits of this service are, in part, already listed above:

• an expected overall improvement thanks to the fact that the agency can have a real time

perspective on the buses status;

• The bus stop services will decrement the waiting time: the passengers will be completely

informed about the exact position and time when to take the bus;

• The mobile app increases the available data to the passengers, even using notifications;

7


• The mobile app, with a smartphone with GPS enabled, can guide the user to the exact point

he is looking for;

• The passenger can buy the ticket at anytime, with the online services or through SMS;

• Real time informations on board;

Part of the benefits are present even for agencies:

• real time overview of all the buses (delays, problems, . . . );

• easy to know how many passengers use a bus at a given time, and make statistics thanks

to these info;

• A user-friendly web and mobile interface available for their users;

• The trip will be safer, because there is no need anymore to talk to the driver!

8

Chapter 0 Contents

Thesis Statement

BusOnAir has the clear goal to be a complete system that has to provide a series of benefits

to the passengers first, but even to transport agencies.

The first version, developed during the previous thesis, has created a very solid base to what will

be the final product.

The work done in this thesis is focused to obtain the following results:

• Extension of the model: the previous version supports only one traffic day, while the new

version will support seven days;

• Multi-days research: a consequence of the previous point is that it will be possible to re-

search for a path lasting two or more days;

• Real-time support: it will be possible, for example, to follow a running bus moving along a

route on the map;

• Delay management: in case of a delay, the overall system should react modifying the infor-

mations on the model and provide correct data to the user accordingly;

• Performance improvements: techniques to improve the overall performance.

All of these points will be analysed in the next chapters.

9

Chapter 1

Requirements and Specifications

This chapter will analyse the system’s specifications and requirements. The problems and

the technologies introduced in the previous chapter will help to understand what BusOnAir is and

which services it has to provide.

1.1 BusOnAir: an overview

BusOnAir is a project composed by different modules:

• busOnAirServer: a complete web application to hold all the buses timetables, handle data

received dynamically from the running buses, calcutate shortest paths from a Start point to

an End point, generically provide different kinds of APIs to query, create or update elements;

• busOnAirClient: composed by a website and a mobile app. It should provide different

services to the user, such as:

– consult buses lines, drawing the path on a map;

– perform a research from a point Start to a point End at a given time T ;

– follow the running buses in real time on the map, thanks to an animation based on the

GPS position sent by the real bus during the lifetime of a run;

Additionally, using features like the geolocalization provided by a smartphone, the app

should have additional features:

– Using the current position of the user, get the name of the closest bus stop S and print

the list of all the lines passing through S. If the user clicks over a specific line, give all

the details about the given line and draw its path on the map;

– Alarm: notify to the user when the bus he has to take, according to the selected

itinerary, is coming to his bus stop;

10

Chapter 1 1.2. BusOnAir: development plan

– interactive experience: the user can mark a bus as taken and he will be notified when

he has to stop, walk to or wait in another stop. If a delay occurs, the app should behave

accordingly;

– preferred routes: it should be possible to save the preferred routes on the phone mem-

ory.

• busOnAirVehicle: send real-time data about the current position. Every bus should send,

thanks to this module, its GPS position once at every given time-interval.

Figure 1.1: Presentation tier - client.

1.2 BusOnAir: development plan

1.2.1 BusOnAirClient

Composed by a web site and a mobile app, is the real interface that allows the users to inter-

act with BusOnAirServer. Using HTML5/CSS/JavaScript and web technologies, and having as a

requirement the fact that the mobile app has to support different platforms (Android, iOS, Black-

Berry, . . . ...), a proposed solution is the use of PhoneGap, a free and open source framework

that allows you to create mobile apps using standardized web APIs. Even if this is not mandatory,

the web site and the mobile app can share even the same interface, using:

• HTML5: for the contents, in this case the device and the screen size does not matter. When

supported and allowed by the user, the HTML5 Geolocalization API can be used, too;

• CSS3: presentation and animations. This, of course, will change according to the device it

has to serve;

• JQueryMobile: presentation, animations: a HTML5-based user interface system designed

to make responsive web sites and apps that are accessible on all smartphones, tablets and

larger screens.

11

Chapter 1 1.2. BusOnAir: development plan

Even if the heavy work will be performed server-side (BusOnAirServer), these specifications

require a medium-high level smartphone, that at least has to support HTML5, CSS3. Geolocaliza-

tion will be highly recommended, but not mandatory: the user has to be able to pick his position,

by tapping or typing the address.

The maps will be served and provided by OpenStreetMap and Google Maps.

The mobile app is not part of the previous thesis and is not part of the current one. The web

interface is covered in the Chapter 4.

1.2.2 BusOnAirVehicle

This module is needed to regularly send the current position from a running vehicle to the

server. The initial idea, inherited by the first release of BusOnAir, was to develop a simple mo-

bile application to perform this task, or to use already existent solutions for software tracking like

OpenGTS (http://opengts.sourceforge.net).

Instead, later in this thesis will be possible to check how to replace this idea with a GSM/G-

PRS/GPS tracker, provided by Smart Alliance, that will be in charge of this task through a GPRS

connection. Due to some limitations of this tracker, it will not be possible to set specific URL format

to send data, and it is not possible to establish an HTTP connection, too. So, it will be necessary

to develop a simple UDP server (section A.1) that will receive the data from the tracker and that

will forward the message to the BusOnAirServer, translated according to the specific API.

This configuration and use of the tracker is covered on Chapter 4.

1.2.3 BusOnAirServer

The previous thesis and the current one are almost completely focused on this module. The

BusOnAirServer module has to provide mainly the following services:

• Store a timetable and all of its components: buses lines, routes, stations, bus stops, . . . ...

• Provide a simple and solid API to retrieve the informations;

• Provide a solid API to update informations, statically (for example, importing week by week

the new timetables) or dynamically (for example, updating the position of a bus in real time);

• Provide an API to perform a research to find the shortest path between two points.

All the previous points have to be optimised in terms of execution time. This modules deserves

a complete chapter to be completely analysed: chapter 3 will cover it in details.

12

Chapter 1 1.3. Stops, bus lines and timetable management

1.3 Stops, bus lines and timetable management

The BusOnAirServer module has to keep the following info:

• the existent bus stops in the territory;

• buses lines of the public transport network;

• a timetable linking together a bus line that passes through a given bus stop at a certain time;

These info should be easy to retrieve in order to:

• research stops inside a certain distance;

• research of the buses that will pass through a certain bus stop;

• visualisation of lines and routes belonging to a certain bus line;

• visualisation of the exact time where a given bus will pass at a certain stop.

To solve timetable problems and formalise them, the focus of the study is centered on two

models: Time Expanded and Time Dependent. The two models are already studied and com-

pared in the scientific literature[8][9][10][11]. The first version of this thesis studied which of the

previous models was better to describe the whole system in all its complexity, and the answer is

that, even if Time-Dependent is faster in terms of shortest-path researches, the Time-Expanded

model is much more adaptable and extendible to reflect the more realistic constraints and require-

ments, such as the delay management.

1.4 Shortest-path calculation

The system has to be able to calculate the shortest path between two points, Start and End,

expressed as two tuples (latitude, longitude), at a given time t.

The response to a query formulated like above includes:

1. a pedonal path to reach the first bus stop from Start ;

2. a bus route until a given stop;

3. if the bus stop is not the last one:

• a pedonal path to reach the next bus stop where to take the second bus;

• the bus route until a given stop;

• go back to point 3.

4. a pedonal path to reach the destination point End .

13

Chapter 1 1.5. Delay management

The studies in the literature[8][10] in this field of research considered the shortest path between

two bus stops, but not the possible scenario in which the input is composed by two geographical

points, not coincident with two bus stations. The previous thesis faced this problem, finding not

only the shortest path between two bus stations but even calculating the closest bus stops with

respect to two geographical points and optimising even the walking paths.

The main problems in literature about the previous services to provide to the users are:

• EAP: Earliest Arrival Problem: shortest path with the minimal arrival time;

• MNTP: Minimum Number of Transfers Problem: shortest path with the minimum number

of bus transfers;

• Bi-Criteria Optimization Problems (Pareto-optimimal problems): calculate the shortest

path with two optimisation criteria.

The proposed solutions are based on the Dijkstra’s algorithm implemented with a Fibonacci

heap, that solves the problem with a computational complexity O(|E|+ |V |log|V ).

1.5 Delay management

The system has to take into consideration the delay of the buses, intended as the difference

between the real time in which a bus crosses a bus stop and the predicted time according to the

timetable.

The system should be able to:

• Receive the buses info and store the delay of each bus;

• Propagate the delay to all the next bus stops, updating the corresponding arrival time.

Another important aspect is that, due to a consistent delay, the context of a research could

change. For example, suppose that, according to the timetable, the users are allowed to take the

bus line M from the bus stop S at a certain time t1 because, taking the bus line L, they are able

to reach the bus stop S at a certain time t0 < t1. Now, introducing a delay d to the bus line L

could lead to the situation in which the bus L reaches the bus stop S at a time t2 = t0 + d such

that t2 > t1. It is clear that in such a situation, the users will lose the bus M. In this case, if a

user performs a research while the delay is already accumulated, the system should change the

response to the research accordingly. The consequences of this aspect are that there could be:

• loss of one or more bus transfers possibilities: if the path was calculated to perform a

bus transfer from the bus A to the bus B in the bus stop S, but A has a delay that will cause

the transit on the bus stop S after B, as explained in the previous example;

• introduction of one or more bus transfers possibilities: imagine that in a given stop S

will transit three buses: A, B, C in this time order. Reaching S with B a user has only the

14

Chapter 1 1.6. Real-time tracking

chance to perform a bus transfer with C. But if A has a delay such that it will reach S after

B, then the time order changes to B, A, C or B, C, A, and so there will be two possible bus

transfers for the users: with A or with C.

1.6 Real-time tracking

The real-time tracking has to:

• retrieve the geographical position;

• calculate the bus progress over its path.

The system should be able to translate a geographical position expressed in terms of (latitude,

longitude) to a data structure coherent with the model adopted in the code: this is mandatory to

calculate different information such as the last stop passed by a bus and the delay, calculated as

the difference between the real time in which a bus crosses a bus stop and the predicted time

according to the timetable.

15

Chapter 2

Models and Algorithms

As already seen in the Introduction, an ITS has the goal to provide a quicker travel with en-

hanced mobility and optimizing existing transportation infrastructure, using different kinds of infor-

mation.

Based on them, the system has to reply to a user’s query with a response that takes into account

all the possible static and real-time data, according to the user’s needs. Basically, there exist

different problems and models related to this research area, and in the first part of this chapter

will analyse the following two categories of problems:

• EAP: Earliest Arrival Problem. The task is to find the best connection between the departure

station s and the arrival station t, assuring the earliest possible arrival at t from a specific

departure time at s;

• MNTP: Minimum Number of Transfers Problem. In this case, a query consists of a departure

station s and an arrival station t, and the task is to find an itinerary that minimises the number

of train transfers, in terms of number of times that a passenger has to move from one bus to

another one.

It’s even possible to consider a combination of the two problems above to have a multicriteria

approach to solve more complete problems.

The second part of the chapter will provide an overview of the models needed to represent all the

information relative to a public transport system, with the focus on the timetable formalisation.

The two most common approaches are Time Expanded and Time Dependent which transform

the initial network into a weighted digraph, in which it will be possible to perform a path search

executing classical algorithms such as Dijkstra.

The Time Dependent approach is not explained in this thesis and not used anywhere in the

project: please refer to the previous thesis[12] or other documents available in the References[8][9]

if you need to learn more about it.

The suggested model should be optimum in terms of query execution’s time.

16

Chapter 2 2.1. EAP - Earliest Arrival Problem

This chapter is largely inspired by the previous thesis on this topic[12] and from the ”Efficient

Models for Timetable Information in Public Transportation Systems”[8] paper.

2.1 EAP - Earliest Arrival Problem

Public transport problems are linked to the concept of timetable, defined thanks a set of el-

ements: bus stops, bus, departure and arrival time, traffic days, . . . , formally introduced in the

subsection 2.2.1. There exist different important problems on timetable information. One of them

is EAP: Earliest Arrival Problem.

The input is composed by:

• s: departure station;

• t: arrival station;

• t0: departure time.

.

The departure time includes the departure day as explained in the previous section.

Connections are valid if they depart at least at the given departure time t0, and the optimization

criterion is to minimize the difference between the arrival time and the given departure time.

The most common approaches to solve EAP are based on Time Expanded and Time Dependent.

As already reported before, this thesis does not cover the Time Dependent model.

2.2 Time Expanded

2.2.1 Introduction

One of the approach to formalise and model the kind of data described above is Time Ex-

panded [8][9][10][11].

Time Expanded creates a digraph in which every node corresponds to a specific time event

like the departure from a station, and edges between them represent either elementary connec-

tions between the two events (i.e., served by a train that does not stop in-between) or waiting

within a station. Edges will have a fixed cost, assigned in the way described in the following para-

graphs.

The term timetable defines a list of transition times of every train from a specific location, most of

the times intermediate stations, from the route that starts from a departure station s to an arrival

station t.

Considering the scenario of the buses, a timetable contains data for: bus stops, busses, connect-

ing stops, departure and arrival times of buses in the bus stops, and traffic days. More formally[8],

17

Chapter 2 2.2. Time Expanded

we are given a set of buses Z, a set of bus stops B and a set of elementary connections C, whose

elements c are 5-tuples of the form c = (Z, S1, S2, td, ta):

• bus Z leaves station S1 at time td;

• the immediately next stop of bus Z is station S2 at time ta.

If x denotes a tuple’s field, then the notation x(c) specifies the value of x in the elementary

connection c.

The departure and arrival times td(c) and ta(c) of an elementary connection c ∈ C within a day

are integers in the interval [0, 1439] representing time in minutes after midnight. Given two time

values t and t′ with t ≤ t′ , the cycle difference ∆(t, t′) is the smallest nonnegative integer l such

that l = t′ − t(mod 1440):

• ∆(t, t′) = l

• l = t′ − t(mod 1440)

The length length(c) of an elementary connection c is defined as ∆(td(c), ta(c)) and represents

the time in minutes between td(c) and ta(c). A timetable is valid for a number of N traffic days. At

a station S ∈ B is possible to transfer from one bus to another. Such a transfer is only possible

if the time between the arrival and the departure at that station S is larger than or equal to a

given, station-specific, minimum transfer time, denoted by transfer(S). Let P = (c1, . . . , ck) be

a sequence of elementary connections together with departure times depi(P ) and arrival times

arri(P ) for each elementary connection ci, 1 ≤ i ≤ k . We assume that the times depi(P ) and

arri(P ) include data regarding also the departure/arrival day by counting time in minutes from the

first day of the timetable. Such a time t is of the form t = a × 1440 + b, where a ∈ [0, N − 1] and

b ∈ [0, 1439]. Hence, the actual time within a day is t(mod1440) and the actual day is t/1440 . Such

a sequence P is called a consistent connection from station A = S1(c1) to station B = S2(ck) if it

fulfills some consistency conditions:

• the departure station of ci+1 is the arrival station of ci;

• the time values depi(P ) and arri(P ) correspond to the time values td and ta.

More formally, P is a consistent connection if the following conditions are satisfied:

• ci is valid on day depi(P )/1440

• S2(ci) = S1(ci+1)

• depi(P ) = td(ci)(mod1440)

• arri(P ) = length(ci) + depi(P )

• depi+1(P )− arri(P ) ≥

0, if Z(ci+1) = Z(ci)

transfer(S2(c1)), otherwise

18

Chapter 2 2.2. Time Expanded

2.2.2 Time Expanded Simplified - Model

This subsection describes the simplified Time expanded model, based on the time-expanded

digraph. The time-expanded digraph is constructed as follows[8]: there is a node for every time

event (departure or arrival) at a station and there are two types of edges. For every elementary

connection (Z, S1, S2, td, ta) in the timetable, there is a connection edge (train edge) in the graph

connecting a departure node belonging to station S1 and associated with time td ,to an arrival

node belonging to station S2 and associated with time ta.

For each station S, all nodes belonging to S are ordered according to their time values. Let

v1, . . . , vk be the nodes of S in that order. Then, there is a set of stay edges (vi, vi+1), 1 ≤ i ≤

k?1, and (vk, v1), connecting the time events within a station and representing waiting within that

station. The cost of an edge (p, q) is ∆(tp, tq), where tp and tq are the time values associated with

nodes p and q, respectively.

So, the terminal points of the connection edges (train edges) generates the nodes set of the

graph. The set E of the arcs is divided in two subgroups:

• Ec composed by the connection arcs between stations (train edges). For each elementary

connection c ∈ C exists an arc e ∈ Ec that connects together the nodes corresponding to

the departure event and the arrival event of that connection. Formally, an arc e = (u, v) ∈

Ec ⇐⇒ ∃c ∈ Cs.t.:

– u is the node corresponding to the departure event defined by Z(c), s1(c) and td(c)

– v is the node corresponding to the arrival event defined by Z(c), s2(c) and ta(c)

• Et composed by transfer arcs within a station (stay edges). For each station s ∈ B let V (s)

be the set of all the nodes such that station(v) = s. Consider the nodes in V (s) ordered in

this way: u ≤ v ⇐⇒ time(u) ≤ time(v), for u, v ∈ V (s). So Et contains all the connection

arcs between time ordered nodes. There is the addition of a connection arc between the last

and the first node of V (s) to guarantee movements during the night and in the next days.

This thesis does not cover in details the Realistic Time Expanded Model, that mostly repre-

sents an extension of the model reported in this paragraph. Anyway, it’s going to introduce some

concepts needed during the current thesis.

Extending a Time Expanded model, it possible to create a Realistic Time Expanded digraph as

follows[9]:

• keep for each station a copy of all departure and arrival nodes, called transfer nodes;

• introduce the stay edges between the transfer nodes;

• For every arrival node create an edge to the first transfer node with time value greater than

or equal to the time of the arrival node plus the minimum time needed to change trains at

19

Chapter 2 2.3. Delays in Time Expanded

Figure 2.1: Train transfers in Time Expanded within a station.

the corresponding station. Additionally, if it departs from the corresponding station, add a

second edge from the arrival node to the departure node of the same train.

2.2.3 Fully Time Expanded

The Time Expanded graph can be extended to completely cover the whole timetable, main-

taining a day-by-day node’s copy. The stay edges between the last and first node of a station,

introduced in the previous paragraph, are now removed: whenever there is an overnight edge in

the i-th copy, the edge is redirected to the corresponding node in the i+ 1th copy, if i < N ; in the

N -th copy, overnight edges are deleted. In contrast to the previous Time Expanded models, in

this case each node has, as an attribute, not only the time of the day td in the interval [0, 1439],

but also the absolute time in the timetable: for the i-th day, t = td + i× 1440.

A Fully Time Expanded graph has two important properties:

• it does not contains cycles, so it’s a DAG: Directed Acyclic Graph;

• each path u− v always has the same temporal value (time(v)− time(u) is constant)[12].

2.3 Delays in Time Expanded

We do not live in a perfect world, and very often a train or a bus is not in time.

This fact introduces an additional requirement to take into consideration, because a user has to

be informed of a given delay if he is following a running bus, and he has to receive a correct reply

20

Chapter 2 2.3. Delays in Time Expanded

Figure 2.2: Delay in time expanded. ∆ = 8 minutes

to his research that takes into consideration the fact the a running bus is not passing through a

given station in the expected time.

So it is not enough to increase the travel time, but it is necessary to update edges within stations

as well. So there are three necessary steps[13] to perform:

• Step 1: Increase Connection Weight. Increase only the weight of the identified delayed

connection edge. Any other connection edge is unchanged;

• Step 2: Update Station Edges: For all subsequent stops in stations, update both transfer-

departure and arrival-transfer edges. For the first one, just increase its weight from 0 to ∆,

for the second one decrease each arrival-transfer edge by ∆;

• Step 3: Validate Station Edges: it can happen that the edges weight are not more valid

(the edge weight is smaller than the transfer time for this station). If this is the case, the

target node has to be changed to the next reachable transfer node.

It can happen that, because of a train’s delay, new connections are possible. For example, the

user can miss a run with the given delay, but he can take the next run easily.

This requires to add a new arrival-departure edge to the departure event of the delayed train[13].

Indicate the delayed train as β, that reaches a given station with a ∆ minutes delay.

1. Delete the arrival-transfer edge (aβ , tβ), the arrival-departure edge (aβ , dbeta), and insert a

new arrival-transfer (aβ , y) for this train with weight ∆.

2. Insert a new edge (y, tβ) from the transfer node y of the delayed train (first possible depar-

ture available for a passengers transfer) to the original transfer node tβ with zero length.

21

Chapter 2 2.4. EAP in Time-Expanded

2.4 EAP in Time-Expanded

The following theorem states that an appropriate shortest-path computation on Time Ex-

panded graph solves EAP:

THEOREM 1. A shortest path in the time-expanded digraph from the first departure node s at the

departure station A with departure time later than or equal to the given start time t0 to one of the

arrival nodes of the destination station B constitutes a solution to the simplified version of EAP in

the time-expanded model. The actual path can be found by Dijkstra’s algorithm in time

O(me + nelogne), where ne and me are the number of nodes and edges, respectively, of the

time-expanded digraph.

—THEOREM 1 ABOUT TIME EXPANDED AND EAP

This theorem is demonstrated by a previous master degree thesis[14], and reported in the

previous thesis about BusOnAir[12].

2.4.1 EAP in Fully Time-Expanded

The focus of this thesis goes to the Fully Time Expanded model, because is the model adopted

and implemented. The two properties described in the last lines of the Fully Time Expanded

above suggest that the shortest path can be extremely optimized: single source shortest paths in

DAGs can be calculated in linear time visiting nodes on topological order (section B.1).

Because of each path u−v always has the same temporal value, it’s enough a visit of the graph to

individuate a path u−v. This will grant that this path is the shortest. EAP can be solved executing

a graph’s visit and finding the first node reachable in the destination’s station.

The arcs weight is a natural number expressing the minutes from midnight plus the number of

days in minutes, too. The implementation provides a priority queue based on these values.

A performance improvement can be achieved blocking the algorithm’s execution when a node

belonging to the arrival station is reached. In fact, the Dijkstra’s algorithm processes the nodes in

a distance’s order from the source node; all the nodes reachable from the source node are added

to the priority queue based on their temporal value (absolute time). So, because of this temporal

visit, there is a topological visit (no arc goes behind in time), too.

2.5 Bicriteria Problem

Now consider a bicriteria optimisation problem, with the criteria:

• EA: Earliest Arrival

• MNT: Minimum Number of Transfers

According to the previous thesis, based on the experimental studies on the Pareto optimality

on Time Expanded graphs[15], on average, there are few Pareto optimal paths relative to multi

22

Chapter 2 2.5. Bicriteria Problem

criteria optimisation problems in the context of timetables.

Having a departure station A, an arrival station B and a departure time t, the interest is on the

following three variants of the problem:

• calculation of all the Pareto optimal solutions, that consists on finding the set of all the

possible not dominated paths;

• minimise the first criteria limiting the second one inside a certain limit: having the two

criteria (X, Y), the algorithm has to calculate the optimal solution that minimises X keeping

Y below a certain value. In the case of the criteria (EA, MNT), this problem is considered

as the Earliest Arrival problem with Bounded number of Transfers (EABT): find a connection

between A and B such that the arrival time to B is minimum and the number of vehicle’s

transfers does not exceed a certain value k ;

• first lexicographic solution: between all the solution minimising X, choose the one having

the minimum Y. More formally, (a, b) < (a′, b′) ⇐⇒ (a < a′) or (a = a′ and b < b′).

The most frequent criteria are EA and MNT, but there are, of course, other criteria that can be

considered:

• LD: Least Duration, path that minimises the travel’s time;

• Cheapest Connections: path that minimises the tickets cost;

• Safest Connection: path the minimise the risk to lose a bus transfer because of a delay.

23

Chapter 3

Server module changes

This chapter will focus on the server module.

There will be an introduction of the concepts and models reported in the server and the new

features introduced by this thesis analysing, mostly, what happens in the core of the project when

it has to respond to some request made through an API call.

3.1 From Simplified Acyclic Time Expanded to Fully Time Ex-

panded

The previous version of BusOnAir was based on a SATE(Simplified Acyclic Time Expanded)

model and had the support for one traffic day only. This solution had some limitations:

• It was not possible to distinguish between traffic days. In fact, there was only one day,

meaning that all the days were equivalent. This limitation can describe a not realistic sce-

nario, because there are good chances, in a public transport system, that on certain days

(Saturday, Sunday, or Christmas) the timetable is completely different;

• It was not possible to find a path traveling during the night;

By adopting the Fully Time Expanded model, already described in the previous chapters, the

server eliminates the limitations just described, making the scenario much more realistic.

3.2 Fully Time Expanded: the model

The Fully Time Expanded model reflects the same elements as the SATE ones. The previous

thesis analyses the following elements in details[12]:

• Station: bus station. It holds a list of different Stops in temporal order;

24

Chapter 3 3.2. Fully Time Expanded: the model

• Route: route of the Time Expanded model; represents a path between a departure station

and an arrival station;

• Stop: a bus stop, that belongs to a Run. The most important attributes to list here are:

– staticTime: the time in which the bus belonging to a given run should reach this stop,

according to the timetable;

– time: the time in which the bus belonging to a given run reaches this stop. It can be

different from the staticTime, introducing a delay if time > staticTime.

• Run: contains a series of Stops, and describes the route of a running bus along its Stops at

a specific time;

• CheckPoint: useful to monitor a Run along its route. Each Stop has its CheckPoint, but

sometimes there is the necessity to have additional points between two stops to better con-

trol the status of a running bus. These points are named CheckPoints.

From this point of view, the proposed Fully Time Expanded model is exactly the same as the

Simplified Acyclic Time Expanded proposed in the previous version of BusOnAir.

3.2.1 Multiple days support

The model introduced in this thesis supports different days, without the ”one-day only” limita-

tion of the previous version: the time was expressed by a natural integer in the interval [0, 1439],

the minutes from midnight.

With the current version, the time is expressed by an absolute time described by the correspond-

ing day, expressed in minutes, too, and the minutes from midnight. An example is, considering

0, . . . , 6 as Monday, . . . , Sunday, and considering Wednesday as the corresponding day, the time

is expressed by the following equation: time = 2 × 1440 + n, where n is a natural integer in the

interval [0, 1439], the minutes from the midnight of Wednesday.

As already reported in the previous sections, the SATE model implemented in the previous

version of BusOnAir is acyclic (this is what the A stands for): Let V (s) be the set of all the

nodes v such that station(v) = s. Consider V (s) with a temporal order, such that, for two nodes

u, v ∈ V (s), time(u) ≤ time(v). A consequences of this was that there were no arcs crossing the

midnight, and no arcs linking the last node and the first node in temporal order.

In the current version, time(u), for each node u, is an absolute value, expanded to consider the

combination of the day’s value and the minutes from midnight calculated as seen before. In this

case, without introducing any cycle and thanks to the property that states that time(u) considers

even the day’s attribute, it is possible to link different days and to have trips planned to cross the

midnight.

25

Chapter 3 3.3. Delay management

The prototype proposed in this thesis supports seven days, but it is possible to expand the

number of days without modifying the code.

3.3 Delay management

A complete and detailed description of the delay management was already presented in the

dedicated section 2.3 : the additional details here are provided to understand how to implement

the theoretical approach already seen.

When the running bus passes a checkpoint, together with the list of operations to execute to

respond to the correspondent API call, let’s focus on the (simplified) code responsible to update

the status of the Run:

Listing 3.1: CheckPoint Passed

1 public void checkPointPass(CheckPoint lastCP , int time) {

2 // transform the time in minutes

3 int timeInMinutes = (int) Math.round(time / 60.0);

4 // delay calculation

5 int ritardo = timeInMinutes - lastCP.getTimeInMinutes ();

6 // end of the Run

7 if (lastCP.getNextCheckPoint () == null) {

8 // restore the Run , explained later

9 restore ();

10 } else {

11 // add the current Run as Running

12 Runs.getRuns ().addRunningBus(this);

13 // update the last checkpoint visited

14 setLastCheckPoint(lastCP);

15 // update the last update time

16 setLastUpdateTime(time);

17 // Update coordinates

18 setLatitude(lastCP.getLatitude ());

19 setLongitude(lastCP.getLongitude ());

20

21 // update all the other stops to reflect the delay

22 Stop nextStop = lastCP.getTowards ();

23 while (nextStop != null) {

24 // Propagating the delay to the next stops

25 nextStop.setTime(nextStop.getTime () + ritardo);

26

Chapter 3 3.4. Performance Improvements

26 nextStop.updateStopPosition ();

27 nextStop = nextStop.getNextInRun ();

28 }

29 }

Almost all the code is self-explaining.

Line 7 checks if the passed CheckPoint is the last one of this Run. If this is the case, calling the

restore() method of a Run object, the code sets back all the passed Stops time to the staticTime

value and resets the lastUpdateTime value, too.

Line 12 adds the Run to the running buses, because the system maintains a list of them that can

be retrieved requesting a particular API ( 4.3).

3.4 Performance Improvements

Part of the work of this thesis is based on improving the overall performance of the system,

responding to different queries. The chapter 5 covers these aspects with all the needed details.

27

Chapter 4

REST APIs

In this chapter there will be a detailed overview of the old and new APIs.

Some of the old APIs are changed, and/or their response it’s changed, to reflect mostly the real

time requirement and to support different traffic days instead of only one after the adoption of the

Fully Time Expanded model.

In the appendix, section A.2, there are a lot of command line examples using the APIs.

4.1 APIs: CREATE/UPDATE

This section contains the APIs to create nodes and relationships related to the model. The

create APIs should be used by administrators only. These APIs should never be read by users

and protected from a potentially dangerous improper use.

4.1.1 stations

Create different stations from a JSON file

/stations/bulkimport

Request method POST

QueryParam input : JSON string to be translated in a boa.server.domain.json.Stations object

Returns JSON object {message: <string>, status: <int>}

Usage curl -H ”Content-Type: application/json” -X POST -d @stations.json

$SERVER URL/stations/bulkimport

28

Chapter 4 4.1. APIs: CREATE/UPDATE

Create or update a station from a JSON file

/stations/createorupdate

Request method POST

QueryParam input : JSON string to be translated in a boa.server.domain.json.Station object


Usage curl -H ”Content-Type: application/json” -X POST -d @station.json

$SERVER URL/stations/createorupdate

4.1.2 routes

Create different routes from a JSON file

/routes/bulkimport

Request method POST

QueryParam input : JSON string to be translated in a boa.server.domain.json.Routes object


Usage curl -H ”Content-Type: application/json” -X POST -d @routes.json

$SERVER URL/routes/bulkimport

Create or update a route from a JSON file

/routes/createorupdate

Request method POST

QueryParam input : JSON string to be translated in a boa.server.domain.json.Route object


Usage curl -H ”Content-Type: application/json” -X POST -d @route.json

$SERVER URL/routes/createorupdate

4.1.3 runs

Create different runs from a JSON file

/runs/bulkimport

Request method POST

QueryParam input : JSON string to be translated in a boa.server.domain.json.Runs object


Usage curl -H ”Content-Type: application/json” -X POST -d runs.json

$SERVER URL/runs/bulkimport

Create or update a run from a JSON file

/runs/createorupdate

Request method POST

QueryParam input : JSON string to be translated in a boa.server.domain.json.Run object


Usage curl -H ”Content-Type: application/json” -X POST -d @run.json

$SERVER URL/runs/createorupdate

29


4.1.4 stops

Create different stops from a JSON file

/stops/bulkimport

Request method POST

QueryParam input : JSON string to be translated in a boa.server.domain.json.Stops object


Usage curl -H ”Content-Type: application/json” -X POST -d stops.json

$SERVER URL/stops/bulkimport

Create or update a stop from a JSON file

/stops/createorupdate

Request method POST

QueryParam input : JSON string to be translated in a boa.server.domain.json.Stop object


Usage curl -H ”Content-Type: application/json” -X POST -d @run.json

$SERVER URL/stops/createorupdate

4.1.5 checkpoints

Create all the checkpoints for a given Run

/runs/{id}/createallcheckpoints

Request method GET

PathParam id : a Run id


Usage curl $SERVER URL/runs/202/createallcheckpoints

Create or update a checkpoint from a JSON file

/runs/{id}/checkpoints/createorupdate

Request method POST


QueryParam input : JSON string to be translated in a boa.server.domain.json.Checkpoint object


Usage curl -H ”Content-Type: application/json” -X POST -d @checkpoint.json

$SERVER URL/runs/202/checkpoints/createorupdate

Create all the checkpoints for all the runs

/checkpoints/bulkcreation

Request method GET


Usage curl $SERVER URL/checkpoints/bulkcreation

30


Create or update a checkpoint from a JSON file

/runs/{id}/checkpoints/createorupdate

Request method POST


QueryParam input : JSON string to be translated in a boa.server.domain.json.Checkpoint object


Usage curl -H ”Content-Type: application/json” -X POST -d @checkpoint.json

$SERVER URL/runs/202/checkpoints/createorupdate

4.1.6 REAL TIME

[REAL-TIME] : Add a checkpoint to a given run

/runs/{id}/rt/addcheckpoint

Request method GET


QueryParam time minutes from midnight of the first day (day× 1440 + minutes)

QueryParam lat checkpoint latitude

QueryParam lon checkpoint longitude


Usage curl $SERVER URL/runs/202/rt/addcheckpoint?time=480&lat=42.123&lon=13.44

[REAL-TIME] : Restore the stops of a given run to their static time.

/runs/{id}/rt/restore

Request method GET



Usage curl $SERVER URL/runs/11/rt/restore

[REAL-TIME] : Restore all the runs.

/runs/restoreall

Request method GET


Usage curl $SERVER URL/runs/restoreall

31

Chapter 4 4.2. LOAD IN RAM

[REAL-TIME] : Set the last visited CheckPoint in the Run

/runs/{id}/rt/checkpointpass

Request method GET



QueryParam checkpointid checkpoint ID


Usage curl $SERVER URL/runs/13/rt/checkpointpass?checkpointid=543&time=450

[REAL-TIME] : Updates Run position, calculate and set the last visited CheckPoint

/runs/{id}/rt/updateposition

Request method GET



QueryParam lat checkpoint latitude

QueryParam lon checkpoint longitude


Usage curl $SERVER URL/runs/77/rt/updateposition?time=480&lat=42.123&lon=13.432

4.2 LOAD IN RAM

Load all the nodes and relationships in memory

/init/loadinram

Request method GET

Returns JSON object {nodesNumber: <int>, relsNumber: <int>, time: <int>}

Usage curl $SERVER URL/init/loadinram

4.3 APIs: RETRIEVE DATA

4.3.1 routes

Retrieve a Route.

/routes/{id}

Request method GET


Returns JSON object {”from”: ”/stations/<int>”, ”id”: ”<int>”, ”towards”: ”/stations/<int>”,

”url”: ”/routes/<int>”, ”line”: <string>}

Usage curl $SERVER URL/routes/10

32

Chapter 4 4.3. APIs: RETRIEVE DATA

ciao.

/routes/getall

Request method GET

QueryParam objects TRUE or FALSE

Returns JSON object {”routelist”: [”/routes/<int>”, . . . , ”/routes/<int>”]} OR

{”routesObjectsList”: [$JSON ROUTE, . . . , $JSON ROUTE]}

Usage curl $SERVER URL/routes/10/getallruns

Get all Runs related to this Route.

/routes/{id}/getallruns

Request method GET

PathParam id : a Route id


Returns JSON object {”runslist”: [/runs/<int>, . . . , /runs/<int>]} OR {”runsObjectsList”:

[$JSON RUN, . . . , $JSON RUN]}

Usage curl $SERVER URL/routes/10/getallruns?objects=TRUE

Get all the Stations touched by this Route.

/routes/{id}/getallstations

Request method GET



Returns JSON object {”stationslist”: [/stations/<int>, . . . , /stations/<int>]} OR

{”stationsObjectsList”: [$JSON STATION, . . . , $JSON STATION]}

Usage curl $SERVER URL/routes/10/getallstations?objects=TRUE

Retrieve the first available Run for this Route.

/routes/{id}/getfirstavailablerun

Request method GET


Returns JSON object {”id”: <int>, ”route”: <int>, ”firstStop”: <int>, ”firstCheckPoint”:

<int>, ”firstCheckPointTime”: <int>}

Usage curl $SERVER URL/routes/10/getfirstavailablerun

33


4.3.2 stops

Retrieve a Stop.

/stops/{id}

Request method GET

PathParam id : a Stop id

Returns JSON object {”id”: ”<int>”, ”nextInRun”: ”/stops/<int>”, ”prevInRun”:

”/stops/<int>”, ”run”: ”/runs/<int>”, ”staticTime”: ”<int>”, ”station”:

”/stations/<int>”, ”stationName”: ”<string>”, ”time”: ”<int>”, ”url”: ”/stops/<int>”}

Usage curl $SERVER URL/stops/5

Get all the Stops.

/stops/getall

Request method GET


Returns JSON object {”stoplist”: [/stops/<int>, . . . , /stops/<int>]} OR {”stopsObjectsList”:

[$JSON STOP, . . . , $JSON STOP]}

Usage curl $SERVER URL/stops/getall?objects=TRUE

Get the next Stop in this station, in chronological order.

/stops/{id}/getnextinstation

Request method GET


Returns JSON object $JSON STOP

Usage curl $SERVER URL/stops/10/getnextinstation

Get the previous Stop in this station, in chronological order.

/stops/{id}/getprevinstation

Request method GET



Usage curl $SERVER URL/stops/10/getprevinstation

Stop’s time property. Different from staticTime in case of a bus delay or the bus comes earlier.

/stops/{id}/gettime

Request method GET



Returns JSON object {”time:” /<int>}

Usage curl $SERVER URL/stops/10/gettime

34


4.3.3 runs and checkpoints

Get all Stops in this Run.

/runs/{id}/getallstops

Request method GET





Usage curl $SERVER URL/runs/10/getallstops?objects=TRUE

Get all the Runs.

/runs/getall

Request method GET


Returns JSON object {”runlist”: [/runs/<int>, . . . , /runs/<int>]} OR {”runsObjectsList”:


Usage curl $SERVER URL/runs/getall?objects=TRUE

[REAL-TIME] Get all the Runs currently active.

/runs/getallrunningbuses

Request method GET


Returns JSON object {”runlist”: [/runs/<int>, . . . , /runs/<int>]} OR {”runsObjectsList”:


Usage curl $SERVER URL/runs/getall?objects=TRUE

Retrieve a Run.

/runs/{id}

Request method GET


Returns JSON object $JSON RUN

Usage curl $SERVER URL/runs/5

35


Retrieve a CheckPoint that belongs to a Run.

/runs/{id}/checkpoints/{idcp}

Request method GET


PathParam idcp: a CheckPoint id

Returns JSON object {”dt”: ”<int>”, ”from”: ”/stops/<int>”, ”id”: ”<int>”, ”latLon”: ”lat”:

”<float>”, ”lon”: ”<float>”, ”next”: ”/runs/<int>/checkpoints/<int>”, ”prev”: ””, ”to-

wards”: ”/stops/<int>”, ”url”: ”/runs/<int>/checkpoints/<int>”}

Usage curl $SERVER URL/runs/202/checkpoints/55677

[REAL-TIME] Returns the expected time of passage at the CheckPoint.

/runs/{id}/checkpoints/{idcp}/gettime

Request method GET


PathParam idcp: a CheckPoint id

Returns JSON object {”time”: <int>}

Usage curl $SERVER URL/runs/202/checkpoints/55677/gettime

Get all the CheckPoints of a given Run.

/runs/{id}/checkpoints/getall

Request method GET


Returns JSON object {”checkpointlist”: [/runs/<int>/checkpoints/<int>, . . . ,

/runs/<int>/checkpoints/<int>]}

Usage curl $SERVER URL/runs/202/checkpoints/getall

[REAL-TIME] Get the last visited CheckPoint in a Run.

/runs/{id}/rt/getlastgpscheckpoint

Request method GET


Returns JSON object $JSON CHECKPOINT

Usage curl $SERVER URL/runs/202/rt/getlastgpscheckpoint

[REAL-TIME] Get the last visited Stop in a Run.

/runs/{id}/rt/getlastgpsstop

Request method GET



Usage curl $SERVER URL/runs/202/rt/getlastgpsstop

36


[REAL-TIME] Get the last sent GPS position in a Run.

/runs/{id}/rt/getlastgpsposition

Request method GET


Returns JSON object {”latLon”:{”lat”: ”<float>”, ”lon”: ”<float>”}, ”time”: ”<int>”}

Usage curl $SERVER URL/runs/202/rt/getlastgpsposition

[REAL-TIME] Calculate the last Stop passed in a Run.

/runs/{id}/rt/calculatelaststop

Request method GET



Usage curl $SERVER URL/runs/202/rt/calculatelaststop

[REAL-TIME] Calculate the last CheckPoint passed in a Run.

/runs/{id}/rt/calculatelastcheckpoint

Request method GET


Returns JSON object $JSON CHECKPOINT

Usage curl $SERVER URL/runs/202/rt/calculatelastcheckpoint

4.3.4 stations

Get all the Stations.

/stations/getall

Request method GET


Returns JSON object {”stationlist”: [/stations/<int>, . . . , /stations/<int>]} OR


Usage curl $SERVER URL/stations/getall?objects=TRUE

Retrieve a Station.

/stations/{id}

Request method GET

PathParam id : a Station id

Returns JSON object {”id”:”<int>”,”latLon”: ”lat”:”<float>”, ”lon”: ”<float>”, ”name”:

”<string>”, ”url”: ”/stations/<int>”}

Usage curl $SERVER URL/stations/15

37


Get all the Stops in this Station.

/stations/{id}/getallstops

Request method GET

PathParam id Station id




Usage curl $SERVER URL/stations/15/getallstops?objects=TRUE

Get all the Routes passing through this Station.

/stations/{id}/getallroutes

Request method GET



Returns JSON object {”routelist”: [/routes/<int>, . . . , /routes/<int>]} OR

{”routesObjectsList”: [$JSON ROUTE, . . . , $JSON ROUTE]}

Usage curl $SERVER URL/stations/15/getallroutes?objects=TRUE

[REAL-TIME] Get the first Stop in the Station from specified time.

/stations/{id}/getfirststopfrom

Request method GET




Usage curl $SERVER URL/stations/15/getfirststopfrom

Get the closest stations from a given point within a given distance.

/stations/getneareststations

Request method GET

QueryParam lat latitude

QueryParam lon longitude

QueryParam range range, maximum distance


Returns JSON object {”stationlist”: [/stations/<int>, . . . , /stations/<int>]} OR


Usage curl $SERVER URL/stations/getneareststation?lat=42.44&lon=13.25&range=50

38

Chapter 4 4.4. USING APIs

4.3.5 research

Get the closest stations from a given point within a given distance.

/directions/getdirections?lat1=lat1&lon1=lon1&lat2=lat2&lon2=lon2&departureday=depday

&departuretime=deptime&minchangetime=minchtime&criterion=crit

Request method GET

QueryParam lat1 departure latitude

QueryParam lon1 departure longitude

QueryParam lat2 arrival latitude

QueryParam lon2 arrival longitude

QueryParam departureday departure day. 0 is Monday, . . . , 6 is Sunday

QueryParam departuretime departure time in minutes from midnight

QueryParam minchangetime Minimum time (in minutes) allowed for a bus change. Not yet imple-

mented.

QueryParam criterion 2nd optimisation criterion (1st is arrival time). Allowed: DURATION, MIN-

CHANGES, MINWALK

Returns JSON object {”directionslist”: [”routes”: [”arrStation”:”<str>”, ”arrTime”: ”<str>”,

”deptStation”: ”<str>”, ”deptTime”:”<str>”, ”latLon”: [”lat”:<float>,”lon”:<float>,

”lat”:<float>, ”lon”:<float>, . . . ], ”numOfStops”: <int>,”routeId”:”<int>”], ”walks”

:[”distance”: <int>, ”latLon”: [”lat”:<float>, ”lon”: <float>], . . . ]]}

Usagecurl $SERVER URL/directions/getdirections?lat1=42.2&lon1=13.1&lat2=42.8&

lon2=13.7&departureday=1&departuretime=400&criterion=DURATION

4.4 USING APIs

4.4.1 Direct Requests

It is obviously possible to perform requests pointing directly the APIs listed in the previous sections,

where usage examples are provided, too. The usage examples use curl, a command line, multi protocol tool

to transfer data from or to a server, useful to try GET and POST requests.

4.4.2 Users clients

First web release

BusOnAir provides a intuitive user interface to obtain all the info without, of course, the need to know

what there is behind the scenes.

The previous version of BusOnAir already had a user interface built with jade, a template language, to show

different aspects like, for example, a map using OpenStreetMap and Google Maps with their APIs to provide

the visualisations of the departure time of buses, walk start points point, stops.

The map is interactive (you can click on two points to perform a research from the first to the second one,

click on a stop marker to see a popup with the info about that stop, . . . ). The user can check the bus lines

and their routes, too, and perform a research using the trip planner, that supports only one traffic day (no

results if there are no more buses at this time for that route, or if the trip plan should continue during the night

39


and the day after).

No delay support.

Figure 4.1: Web user interface, version 1.

Second web release

The second release, done in this thesis, introduces some additional features that were not present in the

previous release. The most important of them are:

• Delay support: if the bus has passed a CheckPoint with a certain delay, it will be clearly shown;

• Real Time: You can see the bus moving along his path according to the current bus GPS position, for

all the route’s length;

• Research on multiple days: with this feature, there will be results even if there are no buses today

anymore, starting from tomorrow. Or, for example, it will be possible to start your trip now, and finish it

tomorrow after the night. The days limit is 2 at the moment, but it can be easily set to any value.

Figure 4.2: Web user interface, version 2. A bus is moving between two stations.

40


In parallel with this release, the work in this thesis started the porting of the current features to a new

template. The raw template was provided in HTML/CSS format, to be ported to jade.

All the Google Maps utils and technology are completely replaced by leaflet, a JavaScript library for mobile-

friendly interactive maps, so all the javascript logic had to be changed accordingly.

The porting is almost completed, excepted for the real-time bus movement along his path (the javascript to

perform the animation is ready, anyway).

Figure 4.3: Web user interface, future version.

Mobile app

The project BusOnAir has as a requirement a mobile app that should be supported by the largest number

of mobile devices (Android, iOS, BlackBerry, Windows Phone, . . . ). The app should be able to let the user

visualise the bus lines and their path on a map, real time updates on a bus, and to research for the shortest

path relative to his needs. Another requirement is that, having the change to geolocalize the user through

his GPS or other geolocalization technologies, the app should support the user through the activities offered

by the app.

Having already implemented the web interface using HTML5/CSS/JavaScript technologies, a possible so-

lution is to adopt PhoneGap, a free and open source framework that allows to create mobile apps using

standardized web APIs, reusing a good part of the already developed code.

4.4.3 Tracker

Smart Alliance has provided a GSM/GPS/GPRS tracker that was used to move and act like a real bus to

test the APIs. The tracker itself cannot be set to establish an HTTP connection, and cannot format a request

according to a given API. It is necessary to develop a UDP Server UDPServer (section A.1) to receive data

from the tracker, and this server will forward the messages to the BusOnAir server.

41


Figure 4.4: Web user interface, future version.

In this section there will be a brief list of commands to set up the tracker to send data.

The configuration is done sending SMS to the SIM card number inside the tracker.

Having a password 123456, the commands are the following:

1 begin +123456 // settings initialisation

2 t030s005n +123456 // report the Geo -info at 30s intervals for 5 times

3 apn +123456+ +$APN_here // Set the APN , it depends on the SIM carrier

4 adminip +123456+ +$UDPServer_IP_here+ +port // Set the receiving server

and the port

According to the previous configuration, the tracker will send the following message to UDPServer :

1 {lat: X.X long: Y.Y speed: Z dd/MM/yy hh:mm bat:F signal:F imei:xyz}

UDPServer will be in charge to translate this message into the following request to the BusOnAir server:

http://busonair.public.ip/runs/id/rt/updateposition?time=xx&lat=X.X&long=Y.Y

42

Chapter 5

PERFORMANCE ASSESSMENT

AND IMPROVEMENTS

5.1 Introduction to results and used terms

This section will present a series of results related to a trip research in different system statuses. These

results will be useful to understand how the proposed solution, described in the next paragraphs, dramati-

cally improves the whole system performances. After the choice of three different couples start/end points,

expressed by their latitude and longitude, the tests show the different system’s reactions based on three

system’s states:

1. The server is just started, nothing is loaded in the RAM and the cache is empty. The correspondent

table voice is State 1;

2. The same query was already done once, so the DB nodes and relationships related to this research

are already loaded in the cache. The correspondent table voice is State 2;

3. The same query, for the third time, when all the needed data is loaded and cached. The correspondent

table voice is State 3.

The next tests results will show that there is a huge difference in terms of execution time between the

beginning and the end of the research for the above three states. It will be evident that the whole system’s

performance will be dramatically improved after a period of time T in which, depending on the number of

queries and on the differences between the researched coordinates, the RAM and the system’s cache will

be filled with nodes and relationships. After a series of presented results, there will be a description of the

possible solutions to improve the overall performance and what has been implemented.

These tests were performed with the curl utility, specifying the kind of information to show as a parameter.

In our case, the voices of interest, reported in the results table, are:

• Name lookup: time, in seconds, it took from the start until the name resolving was completed;

• Connect: time, in seconds, it took from the start until the TCP connect to the server was completed;

• Pre-transfer: time, in seconds, it took from the start until the file transfer was just about to begin;

43

Chapter 5 5.2. Neo4j

• Start-transfer: time, in seconds, it took for all redirection steps include name lookup, connect, pre-

transfer and transfer before the final transaction was started.

• Total: sum of all the previous elements.

As explained in details in the previous chapters regarding the research algorithms and techniques, a

research query receives as input the latitude and longitude of the starting point, and the latitude and longitude

of the destination, together with the starting time. Using the Neo4j Spatial library that adds the ability to

perform spatial operations on the data like searching for data within a given distance from a specific point,

you find all the possible Stations where you can start from. Going on with the execution of the FT-shortestPath

algorithm (section B.2), you have to consider all the transfers from a bus line to another one, and, in case,

the walking time between the the two bus lines.

5.2 Neo4j

The database used in the BusOnAir project is Neo4j, an open source project entirely written in Java. It

is a total transactional database, memorising all the data in a directory. Within an open transaction it will

be possible to create nodes and setting attributes on them, and link nodes with relationships that can be

directional or not.

Neo4j admits indices to memorise nodes thanks to a given label, and accessing to the nodes iterator with a

certain label.

Because of his tree structure, the exploration of these data structures results faster with respect to an ER,

because the research of nodes related to other nodes is a primitive operation without different steps to take,

like the SQL JOINs, on different tables.

5.3 Previous heuristic results

The first research is made from a starting point (42.3415, 13.34876) to the arrival point (42.37445,

13.28216), a distantce of about 11km as the crow flies, at 7AM. The points are expressed by latitude and

longitude.

Request method GET

Request URL /directions?lat1=42.34151&lon1=13.34876&lat2=42.37445&lon2=13.28216&time=420

Name lookup Connect Pre-transfer Start-transfer Total

State 1 0.001 0.001 0.002 5.124 5.128

State 2 0.001 0.001 0.001 0.456 0.459

State 3 0.001 0.001 0.002 0.125 0.129

Table 5.1: ”Roio Colle” to ”Cese di Preturo”: 11km as the crow flies

The difference between the research in State 1 and the State 3 is about 5 seconds.

44

Chapter 5 5.4. System and performance analysis

The second research is made from a starting point (42.34453, 13.4011) to the arrival point (42.33706,


longitude.

Request method GET



State 1 0.001 0.001 0.001 6.151 6.154

State 2 0.001 0.001 0.001 1.111 1.114

State 3 0.001 0.001 0.001 0.177 0.180

Table 5.2: ”Terminal” to ”Monticchio”: 8km as the crow flies

The difference between the research in State 1 and the State 3 is about 6 seconds.

The third research is made from a starting point (42.34286, 13.346246) to the arrival point (42.35866,


longitude.

Request method GET



State 1 0.001 0.001 0.001 4.678 4.681

State 2 0.001 0.001 0.001 0.550 0.554

State 3 0.001 0.001 0.001 0.170 0.173

Table 5.3: ”Aquilone” to ”Paganica”: 13km as the crow flies

The difference between the research in State 1 and the State 3 is about 3.5 seconds.

5.4 System and performance analysis

An important point emphasized by the previous section is that, having the DB nodes and relationships

loaded in RAM, dramatically improves the performances. One of the crucial targets of the current thesis was

to provide a solution to decrement the overall time research.

The rest of the section will explore different ways to improve the performances, at three different levels:

1. Hardware level

2. Neo4j configuration level

3. Code level

The graph database used for BusOnAir is Neo4j: Neo4j loads all the data lazily, meaning it loads them

into memory at the first access. Nodes and relationships are cached using LRU (Least Recently Used)

caches, so having an appropriate hardware and cache strategy is a key requirement for good performances.

45


5.4.1 Hardware level

From the hardware point of view, the first step that should be taken is to use a fast persistence media.

A typical memory drive today has an average seek time of 5ms, but just using a solid state drive (SSD) that

has an an average seek time of <100microseconds will make the overall scenario about 50 times faster.

The second step is to increase the RAM. How to calculate how much RAM we need? According to the

Configuration & Performance chapter of the official documentation[6], it is possible to manually calculate the

size needed to contain nodes and relationships. A rough formula to calculate the needed memory:

• nodes size = nodes number × 9 bytes

• relationships size = relationships number × 33 bytes

The previous version of BusOnAir was using the Time Expanded Simplified Acyclic (TESA), supporting

only a single day. The previous equation for the TESA model in the DB was:

• nodes size = 15451× 9 bytes = 139059 bytes = 135.8K = 0.13M

• relationships size = 106108× 33 bytes = 3419.50K = 3.4M

So, the needed size to contain the Neo4j DB with the TESA model, for the data referring to the L’Aquila

bus transport service, is about 3.5M.

The work done in these thesis extends the time table to 7 days. So, roughly, for the Fully Time Expanded

model, the equation changes like the following way:

• nodes size = 53948× 9 bytes = 485532 bytes = 474.15K = 0.46M

• relationships size = 328940× 33 bytes = 10600.60K = 10.35M

In this case, the needed space is increased to roughly 10.8M. Additionally, on a standard mechanical

drive you can handle graphs with a few tens of millions of primitives with 2GB of RAM. The number grows

up to hundreds of millions of primitives with 4-8GB of RAM, billions with 16-32GB. In the same way as many

other cases, having a great hardware basis is not enough.

5.4.2 Neo4j configuration level

Neo4j can be configured in order to get optimum performance thanks to some parameters regarding the

two main components that can be configured: Neo4j caches and the JVM that Neo4j runs in. A very impor-

tant point to obtain a good performance is to be sure that the JVM doesn’t spend too much time performing

garbage collection. Neo4j itself provides the chance to enable logging from the garbage collector, in order to

monitor how the heap reacts to the given operations. After weeks of tests in the BusOnAir scenario with a

4GB RAM server, 500M of heap size has revealed to be an excellent value.

This value for the JVM tuning is done editing the neo4j-wrapper.conf configuration file:

1 # Initial Java Heap Size (in MB)

2 #wrapper.java.initmemory =3

3 # Maximum Java Heap Size (in MB)

4 wrapper.java.maxmemory =500

46


What about cache? Neo4j has two different types: file buffer cache, whose purpose is to improve both

read and write performance, and object cache . The first one caches the data in the same format as it

is stored on the durable storage device, while the second one caches the nodes, relationships and their

properties in an ad hoc format to guarantee an high traversal speeds and transactional writes. For the

object cache configuration, is important to choose the right cache type in order to not incur in frequent I/O

operations. The default configuration proposes the cache type = soft setting, that is defined in the official

documentation as follows[7]:

[cache type = soft] Provides optimal utilization of the available memory. Suitable for high

performance traversal. May run into GC issues under high load if the frequently accessed parts of

the graph does not fit in the cache.

—”CACHES IN NEO4J”, NEO4J DOCUMENTATION

Analysing the BusOnAir project, from the Hardware Level paragraph above, the DB is relatively small

and could be completely contained in memory, thus the Garbage Collector should not be invoked even when

high load operations are executed.

This is why, starting from the first release of BusOnAir in which the DB was smaller than the current one, the

cache type was changed to strong, defined as follows[7]:

[cache type = strong] will hold on to all data that gets loaded to never release it again. Provides

good performance if your graph is small enough to fit in memory.

—”CACHES IN NEO4J”, NEO4J DOCUMENTATION

5.4.3 Code level

From the code point of view there are different actions you can take to modify the system performance.

Most of them should be evaluated based on the underlying system and its properties. First of all, according

to what is already explained in the System and performance analysis section, Neo4j loads all the data lazily,

meaning it loads them into memory at first access. The proposed and implemented solution in this thesis is

based on the idea that loading all the nodes and relationships when the system is started, through a new

API introduced only for this purpose and that should be protected and used only by the administrator, can

increase the overall performance.

Thus the idea is to load the database in memory when the system starts, because the database’s size is

small enough to be completely contained in RAM and in the heap, too.

A GET request to /init/loadinram?relationships={true} (that it is automatically requested by an AJAX call

from the more browser friendly /init page that shows some statistics about the operation) will start the visit

of the graph thanks to a tooling.GlobalGraphOperations object.

The most relevant part of the code is the following:

Listing 5.1: Load in RAM

1 for (Node node : graphOperation.getAllNodes ()) {

2 // [....]

3 // Just with the iteration , node is loaded. Load relationships , too?

47

Chapter 5 5.5. Current heuristic results

4 if (loadRelationships) {

5 relationshipsNumber += IteratorUtil.count(node

6 .getRelationships ());

7 }

8 }

It is enough to iterate through the nodes to have them loaded in memory. Even for relationships, it is

enough to access them once.

Another performance-related improvement from the previous version is the introduction of a method to control

the number of pending operations per transaction. This problem was not relevant when the first release of

BusOnAir was released, because of a very small database size. Now, with large amount of data to update

or create like in the data importing procedure, this problem has to be faced to avoid an OutOfMemoryError

because of memory constraints.

One of the possible solutions is to use the Cypher Query Language, a declarative graph query lan-

guage, integrating queries in the REST scenario: Cypher provides the global USING PERIODIC COMMIT

query hint for updating queries in a controlled way.

Another solution, the one implemented during this thesis, is to create a configuration file (as a Java inter-

face to contain constant values), and express specific configuration variables, like TRANSACTIONS LIMIT:

Listing 5.2: TRANSACTIONS LIMIT example

1 Transaction tx = DbConnection.getDb ().beginTx ();

2 // [..]

3 if (( pendingOperations > 0) && (pendingOperations % BoaConfig.

TRANSACTIONS_LIMIT == 0)) {

4 tx.success ();

5 tx.finish ();

6 tx = DbConnection.getDb ().beginTx ();

7 }

8 // [..]

The code shows how to check if the number of pending operations per transaction exceeds the limit. If

this is the case, the current transaction will be committed and replaced with a newly opened transaction.

5.5 Current heuristic results

This section lists the results of the current system after that nodes and relationships are loaded in mem-

ory using the technique explained in the previous section. Additionally, the results will be compared to the

ones discussed at the beginning of this chapter.

The first research is made from a starting point (42.3415, 13.34876) to the arrival point (42.37445,


longitude.

The research in State 1 now is more than 1.5 seconds faster than before. The research in State 2 and

State 3 is almost unchanged.

48

Chapter 5 5.6. Update operations

Request method GET


Name lookup Connect Pre-transfer Start-transfer Total Previous

State 1 0.001 0.001 0.002 3.565 3.569 5.128

State 2 0.001 0.001 0.001 0.447 0.450 0.459

State 3 0.001 0.001 0.002 0.120 0.124 0.124

Table 5.4: ”Roio Colle” to ”Cese di Preturo”: 11km as the crow flies

The second research is made from a starting point (42.34453, 13.4011) to the arrival point (42.33706,


longitude.

Request method GET



State 1 0.001 0.001 0.001 3.283 3.286 6.154

State 2 0.001 0.001 0.001 0.428 0.431 1.114

State 3 0.001 0.001 0.001 0.177 0.180 0.180

Table 5.5: ”Terminal” to ”Monticchio”: 8km as the crow flies

The research in State 1 now is almost 3 seconds faster than before. The research in State 2 now is

more than 0.5 seconds faster than before. The research in State 3 is unchanged.

The third research is made from a starting point (42.34286, 13.346246) to the arrival point (42.35866,


longitude.

Request method GET



State 1 0.001 0.001 0.001 3.293 3.296 4.681

State 2 0.001 0.001 0.001 0.347 0.550 0.554

State 3 0.001 0.001 0.001 0.167 0.170 0.173

Table 5.6: ”Aquilone” to ”Paganica”: 13km as the crow flies

The research in State 1 now is almost 1.5 seconds faster than before. The research in State 2 and State 3

is almost unchanged.

5.6 Update operations

This section evaluates the difference in terms of response time related to create and update operations:

49

Chapter 5 5.6. Update operations

• State 1 is the response time without all the nodes and relationships loaded in RAM;

• State 2 is the response time with all the nodes and relationships loaded in RAM.

The time is expressed in seconds.

5.6.1 CheckPoint passed

Called when a Run passes a CheckPoint at a given time.

In this case we have to distinguish between two different cases. In fact the response time is directly depen-

dent on the given time passed as an argument: if its value does not introduce a delay, then the system just

updates the last passed checkpoint and it returns.

But if the time value introduces a delay, then the system has to modify all the successive bus stops in this

Run to reflect this delay, updating the time attribute that will affect all the other system’s features, such as a

research that includes one or more of the stops affected by this delay.

Without delay

The following table analyses the case of a bus that passes the checkpoint on time.

Request method GET

Request URL /runs/92/rt/checkpointpass?checkpointid=54747&time=33420

State 1 1.779

State 2 0.910

State 3 0.117

With delay

The following table analyses the case of a bus that does not pass the checkpoint on time, introducing a

delay.

Request method GET

Request URL /runs/92/rt/checkpointpass?checkpointid=54747&time=33600

State 1 2.313

State 2 1.713

State 3 0.117

50

Chapter 5 5.7. Conclusion and future work

5.6.2 Adding a CheckPoint

Add a CheckPoint to a Run, based on latitude, longitude and a time.

Request method GET

Request URL /runs/92/rt/addcheckpoint?lat=42.232&lon=13.324&time=33420

State 1 1.794

State 2 0.879

State 3 0.120

5.6.3 Restore a Run

Restore a Run, resetting all the attributes to the initial, static values.

Request method GET

Request URL /runs/92/rt/restore

State 1 1.565

State 2 0.785

State 3 0.112

5.6.4 Update position

Update the position of a Run with latitude, longitude and the time when the Run sent this coordinates.

Request method GET

Request URL /runs/92/rt/updateposition?time=5432&lat=42.33033&lon=13.48218

State 1 1.717

State 2 1.165

State 3 0.115

5.7 Conclusion and future work

The results listed above show that in State 2 there is a huge improvement thanks to the fact that nodes

and relationships are loaded into the memory.

Why it still takes some seconds to perform the first query? Because during the research operation the

algorithm creates a series of TransientStop objects, from the class TransientStop that extends Stop, that

holds all the transient attributes needed during the execution of the algorithm: examples of these attributes

are the walk distance from a Stop, departure time and so on.

These kind of information cannot be loaded statically when the system is initialised, because the contained

data is completely dynamic and depends on different dynamic factors, like current latitude and longitude and

the time in which the research has to be executed.

This is not a big problem anyway, because after a small window of time in which queries are done for the

first time, the cache will be filled and the time of the research operation will be in the order of milliseconds,

as the previous section has shown (State 3 values).

51


Even if these results are already good enough for a graph of this size, the target of a future work on BusOnAir

can be the improvement of the response time thanks to the use of different models and algorithms recently

introduced in the literature related to this field of research.

For example, the Dynamic Timetable Model (DynTM)[16] is a new graph-based model that reduces the

number of changes needed in the graph as a consequence of a timetable modification, producing a smaller

number of nodes and arcs compared to the Time Expanded models explained in this thesis, and therefore,

a smaller query time.

Another category of models to solve such problems, proposed in the literature, is the the category of models

representing the timetable as an array, instead of graph. The most successful of this category are Connection

Scan Algorithm (CSA)[16][18] and Round-bAsed Public Transit Optimized Router (RAPTOR)[16][17]. CSA

exploits the acyclic nature of some timetables to solve the earliest arrival problem, and all the elementary

connections of a timetable are stored in a single array which is scanned only once for each query.

In RAPTOR the timetable is stored as a set of arrays of trips and routes which are used by a dynamic

programming algorithm to solve the multi-criteria problem.

52

Conclusion

The result of the work done during this thesis is a real-time system that is able to react to delays, extend-

ing its support to a whole week, incrementing the overall system performance.

This result is the sum of the following elements:

• Extension of the model: from SATE, Simplified Acyclic Time Expanded to Fully Time Expanded : the

previous version supports only one traffic day, while the new version supports seven days (but there is

no limitation on the number of the days that this model can support);

• Multi-days research: a consequence of the previous point is that it is now possible to research for a

path lasting two or more days. In the previous version a trip starts and ends at the same day. With

the new version is potentially possible to have seven-days trips, even if the design choice is to limit

the research up to two days: this parameter can be changed by the transport system administrator

according to the needs of the area he has to serve;

• Real-time support: it is now possible to obtain data from a GPS/GPRS/GPS tracker from a running

bus. These data will be used to interactively show to the user the bus moving along a route;

• Delay management: a consequence of the real-time capability is that, in case of a delay, the overall

system should react modifying the informations on the model and provide correct data about a given

Run, updating the time in which the bus will reach the route’s bus stops;

• Performance improvements: optimizing the Neo4J settings techniques and loading all the nodes and

relationships into the main memory, the system had a huge overall performance’s improvement;

• Introduction of new APIs to reflect the points above.

Future Work

The future version of BusOnAir should study different solutions to improve what is already done and to

add additional features:

• Improve performance: the current model is already good enough for the size of the graph, but it is

possible to migrate to new, more efficient and more recent models briefly described in section 5.7.

• APIs security: this is a very important point that has to be carefully fronted. There are some APIs that

has to be called only by administrators, because they are going to completely modify the data on the

model. Requests made by the GPS tracker present on a given running bus to update the bus position

have to be protected, too, because if a malicious user will capture the requests, he can make fake

53


requests causing an unpredictable system’s behavior on the wrong data. So an authentication/autho-

rization mechanism is required, to assure that these APIs are protected. Another element introduced

in this thesis is the fact that the GPS tracker will not directly make a HTTP request to the BusOnAir

server, as designed by the previous thesis. In fact, the tracker can only send data to an UDP Server

(subsection 4.4.3, section A.1) that will translate this data into a request to the BusOnAir server ac-

cording to the APIs. Of course this increases the complexity of the system that should adopt another

layer of security in the communications between the bus and the UDPServer, and then between the

UDPServer itself and the BusOnAir Server.

Additionally, future work on this thesis has to be done to:

• completely port the new user interface, according to what is already done in this thesis, described in

section 4.4.2.

• develop a mobile app that should implement all the characteristics described in this thesis. A brief

overview of the mobile app is provided in section 4.4.2.

54

Appendix A

APIs related code

A.1 UDP Server

The code of the UDP Server that receives the GPS position by the tracker.

Based on the SocketServer chapter of the official python documentation.

Listing A.1: UDP Server that receives GPS position by the tracker

1 import SocketServer

2 from datetime import datetime

3 import urllib

4 import urllib2

5

6 class MyUDPHandler(SocketServer.BaseRequestHandler):

7 """

8 self.request consists of a pair of data and client socket , and since

9 there is no connection the client address must be given explicitly

10 when sending data back via sendto ().

11 """

12

13 BOA_SERVER_IP = "http ://X.X.X.X"

14 BOA_SERVER_PORT = 8000

15

16 def handle(self):

17 data = self.request [0]. strip ()

18 socket = self.request [1]

19 args = self.tracker_data_to_json_format(data)

20 run = args.pop(’run’)

21 get_params = urllib.urlencode(args)

22 url = "%s:%d/runs/%d/rt/updateposition ?%s" %(self.BOA_SERVER_IP ,

23 self.BOA_SERVER_PORT , run , get_params)

55

Chapter 1 A.1. UDP Server

24 #print "{} wrote :". format(self.client_address [0]),

25 #print data

26 socket.sendto(data.upper (), self.client_address)

27 req = urllib2.Request(url)

28 response = urllib2.urlopen(req)

29 response_data = response.read()

30 print response_data

31 socket.sendto(response_data , self.client_address)

32

33 def tracker_data_to_json_format(self , data):

34 """

35 Transform s, the data sent by the GPS tracker , to a JSON format.

36 We only need lat and lon (long in the tracker data).

37 """

38 args = {}

39 b = data.replace(":", "").split(" ")

40 i = 1

41 while i < len(b):

42 current_item = b[(i - 1)]

43 if current_item == "lat":

44 args[’lat’] = b[i]

45 elif current_item == "long":

46 args[’lon’] = b[i]

47 elif current_item == "imei":

48 args[’run’] = self.run_from_imei(current_item)

49 i += 1

50 # Add time information

51 now = datetime.now()

52 # TODO add the day value , in minute

53 mins = int((now - now.replace(hour=0, minute=0, second=0, microsecond

=0)).total_seconds () / 60)

54 args[’time’] = mins

55 return args

56

57 def run_from_imei(self , imei):

58 """

59 Return the RUN associated to the given IMEI.

60 """

61 #TODO

62 return 5

63

64 if __name__ == "__main__":

56

Chapter 1 A.2. Command line scripts

65 HOST , PORT = "localhost", 9999

66 #HOST , PORT = "UDPServer_IP_Here", 9999

67 server = SocketServer.UDPServer ((HOST , PORT), MyUDPHandler)

68 server.serve_forever ()

A.2 Command line scripts

This section will list a series of command line scripts, written with Bash and Python, to perform different

tasks querying the APIs.

57


A.2.1 Checkpoint info

Listing A.2: checkpoint info

1 #!/ usr/bin/env python

2 import sys

3 import json

4 import urllib2

5

6 if len(sys.argv) < 3:

7 print "[HELP] 2 params needed :\n\n\t- runID\n\tcheckpointID\n"

8 sys.exit (1)

9

10 URL_CP="http :// localhost :8000/ runs/" + sys.argv [1] + "/checkpoints/" + sys.

argv [2]

11 URL_CP_TIME="http :// localhost :8000/ runs/" + sys.argv [1] + "/checkpoints/" +

sys.argv [2] + "/gettime"

12 jsonData = urllib2.urlopen(URL_CP)

13

14 data = json.load(jsonData)

15 latLon = data[’latLon ’]

16 stop = data[’from’]

17 print "\nCheckpoint LAT ,LNG: " + latLon[’lat’] + "," + latLon[’lon’]

18 print

19

20 jsonData = urllib2.urlopen(URL_CP_TIME)


22 print "\nCheckpoint time: " + data[’time’]

23

24 # Take the station name related to the stop ... related to the checkpoint

25 URL_STOP="http :// localhost :8000" + stop

26 jsonData = urllib2.urlopen(URL_STOP)


28 print "Station: " + data[’stationName ’]

29 print

A.2.2 Checkpoint passed

Listing A.3: checkpoint passed

1 #!/ bin/bash

2 if [ $# -lt 3 ]; then

3 echo "[HELP] 3 params needed:"

4 echo

58


5 echo " - runID"

6 echo " - checkpointID"

7 echo " - time"

8 exit;

9 fi

10

11 URL="http :// localhost :8000/ runs/$1/rt/checkpointpass?checkpointid=$2&time=

$3"

12 URL_CP="http :// localhost :8000/ runs/$1/checkpoints/$2"

13

14 curl $URL

15 echo -e "\n\nCheckpoint INFO:\n"

16 curl $URL_CP

17 echo -e "\n"

A.2.3 Runs for a given route and relative time

Listing A.4: Runs for a given route and relative time

1 #!/usr/bin/env python

2 # Print all the runs for the given route and relative start time

3

4 import sys

5 import json

6 import urllib2

7


9 print "[HELP] 1 param needed :\n\n\t- routeID\n"

10 sys.exit (1)

11

12 # WHICH ROUTE WE COVER?

13 URL_RUNS="http :// localhost :8000/ routes/" + sys.argv [1] + "/getallruns"

14 jsonData = json.load(urllib2.urlopen(URL_RUNS))

15 runs = jsonData[’runlist ’]

16 print "There are %d runs.\n\n" %(len(runs))

17

18 for run in runs:

19 URL_RUN="http :// localhost :8000" + run

20 jsonData = json.load(urllib2.urlopen(URL_RUN))

21 stop = jsonData[’firstStop ’]

22 URL_STARTTIME="http :// localhost :8000" + stop

23 jsonData = json.load(urllib2.urlopen(URL_STARTTIME))

59


24 print "%s: static: %s, time: %s\n" %(run , jsonData[’staticTime ’],

jsonData[’time’])

25 \end{listlisting}

26

27 \subsection{Current position}

28

29 \begin{lstlisting }[ caption=Position]

30 #!/bin/bash

31

32 if [ $# -lt 1 ]; then

33 echo "[HELP] 1 param needed:"

34 echo

35 echo " - runID"

36 exit;

37 fi

38

39 URL_CP="http :// localhost :8000/ runs/$1/rt/getlastgpscheckpoint"

40 URL_STOP="http :// localhost :8000/ runs/$1/rt/getlastgpsstop"

41 URL_GPS="http :// localhost :8000/ runs/$1/rt/getlastgpsposition"

42

43 echo

44

45 echo -e "LAST STOP\n"

46 curl $URL_STOP

47

48 echo -e "\n\nLAST CHECKPOINT\n"

49 curl $URL_CP

50

51 echo -e "\n\nLAST GPS POSITION\n"

52 curl $URL_GPS

53

54 echo -e "\n\n"

A.2.4 Update position

Listing A.5: Update Position

1 #!/ bin/bash

2

3 if [ $# -lt 4 ]; then

4 echo "[HELP] 4 params needed:"

5 echo

6 echo " - runID"

60


7 echo " - lat"

8 echo " - lon"

9 echo " - time"

10 exit;

11 fi

12

13 URL="http :// localhost :8000/ runs/$1/rt/updateposition?lat=$2&lon=$3&time=$4

14 curl $URL

15 echo

A.2.5 Run info

Listing A.6: Run info


2

3 import sys

4 import json

5 import urllib2

6


8 print "[HELP] 1 param needed :\n\n\t- runID\n"

9 print "\t[-restore ]\n"

10 sys.exit (1)

11

12 # WHICH ROUTE WE COVER?

13 URL_RUN="http :// localhost :8000/ runs/" + sys.argv [1]

14 jsonData = json.load(urllib2.urlopen(URL_RUN))

15 print "\nROUTE: %s\n" %( jsonData[’route’]. split("/")[-1])

16

17 URL_RESTORE="http :// localhost :8000/ runs/rt/" + sys.argv [1] + "/restore"

18 if len(sys.argv) > 2:

19 # Restore run

20 urllib2.urlopen(URL_RESTORE)

21

22 URL_CHECKPOINTS="http :// localhost :8000/ runs/" + sys.argv [1] + "/checkpoints

/getall"

23

24 # Get the checkpoints

25 jsonData = json.load(urllib2.urlopen(URL_CHECKPOINTS))

26 checkpoints_time = {}

27

28 print "\nRUN STARTS :\n"

61


29

30 for cp in jsonData[’checkpointlist ’]:

31 cp_num = cp.split("/")[-1]

32 URL_CP_TIME="http :// localhost :8000/ runs/" + sys.argv [1] + "/checkpoints/"

+ cp_num + "/gettime"

33 cpTimeJSON = json.load(urllib2.urlopen(URL_CP_TIME))

34 checkpoints_time[cp_num] = cpTimeJSON[’time’]

35

36

37 for k in sorted(checkpoints_time):

38 print " - " + k + ": " + checkpoints_time[k]

39

40 print "\nRUN ENDS."

A.2.6 Running buses

Listing A.7: Running buses

1 #!/ bin/bash

2

3 echo "RUNNING BUSES:"

4 echo

5 curl http:// localhost :8000/ runs/getallrunningbuses

6 echo

7 echo

A.2.7 Stop info

Listing A.8: Stop info


2

3 import sys

4 import json

5 import urllib2

6


8 print "[HELP] 1 param needed :\n\n\t- stopId"

9 sys.exit (1)

10

11 URL_CP="http :// localhost :8000/ stops/" + sys.argv [1]

12 jsonData = urllib2.urlopen(URL_CP)

13


62


15 print "\nSTATION: " + data[’station ’]

16 print "\nStatic time: " + data[’staticTime ’] + " ( * 60 = " + str(int(data[

’staticTime ’]) * 60) + ")"

17 print "\nRun: " + data[’run’]

18 print

63

Appendix B

Algorithms

B.1 Shortest Paths in DAGs

A Directed Acyclic Graph or DAG is a particular directed graph that does not have direct cycles: for each

vertex, we cannot come back to it traversing the arcs of the graph.

A directed graph can be said acyclic if an in-depth visit does not present arcs that go in the reverse sense.

DEF. Each DAG admits at least one topological order. The topological order can be calculated in linear

time.

On every DAG is possible to create a topological order disposing in order all the nodes such that there

will be no reverse arc. In other terms, if it exists an arc from A to B, then necessarily A will be present before

B in the order. The topological order of a DAG always exists, but it does not mean that is necessarily unique.

An algorithm to create a topological order could consist on inserting all the sinks at the end of the order, then

remove the sinks from the DAG and repeat until the DAG is empty.

An efficient algorithm (in O(n+ n) with n number of nodes, m number of arcs) consists in performing a visit

of the graph inserting in the order a node only when all of it neighbours have been visited, too. Single-source

shortest paths calculation on DAGS can be efficiently solved using the following algorithm[12]:

Input: DAG G = (V,E)

arcs weight p(e) : e ∈ E

source vertex s ∈ V

Output: For all the vertices u that can be reached by s, d(u) is the distance from s to u, prev(u) is the

predecessor of u in the shortest path from s to u

Listing B.1: DAG-Shortest-Paths

1 for all u ∈ V do

2 d(u)←∞

3 prev(u)← null

4 end for

5 Linearize G

6 for all u ∈ V , in linearised order do

64

Chapter 2 B.2. FT-ShortestPath

7 for all edges (u, v) ∈ E do

8 update(u, v)

9 end for

10 end for

B.2 FT-ShortestPath

Input: Fully Time Expanded Graph: G = (V,E)

arcs weight p(e) : e ∈ E

source vertex s ∈ V

Stack toVisit

Output: For all the vertices u reachable from s, assigns to d(u) the distance between s and u, and to prev(u)

the predecessor of u in the shortest path from s to u.

Listing B.2: FT-Shortest-Paths

1 G′(V ′(s)E′(s))← LoadSubGraph(G, s)

2 for all u ∈ V ′(s) do

3 d(v)←∞

4 prev(v)← null

5 end for

6 d(s)← 0

7 enqueue s onto toVisit

8 while toVisit is not empty do

9 u← dequeue from toVisit

10 for all e = (u, v) ∈ E′(s) do

11 if d(u) + p(e) < d(v) then

12 prev(v)← u

13 d(v)← d(u) + p(e)

14 end if

15 remove e

16 if v does not have incoming arcs in E′(s) then

17 enqueue v onto toVisit

18 end if

19 end for

20 end while

65

Bibliography

[1] Brian Handwerk, Half of Humanity Will Live in Cities by Year’s End, National Geographic News, 2008.

[2] Trasporti intelligenti, IBM Institute for Business Value.

[3] DIRECTIVE 2010/40/EU OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL, http:

//eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:207:0001:0013:EN:PDF, 23 12

2011.

[4] Khalid A.S. Al-Khateeb, Jaiz A.Y. Johari and Wajdi F. Al-Khateeb, Dynamic Traffic Light Sequence,

Science Publications, Journal of Computer Science, 2008.

[5] WHAT IS ITS, http://www.atlantaregional.com/transportation/roads--highways/

intelligent-transportation-system.

[6] Chapter 22. Configuration & Performance, http://docs.neo4j.org/chunked/milestone/

configuration.html, Neo4j Official Documentation.

[7] Chapter 22.6. Caches in Neo4j, http://docs.neo4j.org/chunked/stable/configuration-caches.

html, Neo4j Official Documentation.

[8] Evangelia Pyrga, Frank Schulz, Dorothea Wagner, Christos D. Zaroliagis, Efficient models for timetable

information in public transportation systems, ACM Journal of Experimental Algorithmics 12, 2007.

[9] Matthias Muller-Hannemann, Frank Schulz, Dorothea Wagner, Christos D. Zaroliagis, Timetable Infor-

mation: Models and Algorithms, ATMOS 2004: 67-9.

[10] Frank Schulz, Timetable information and shortest paths, PhD Thesis, Universitat Karlsruhe (TH),

Fakultat Informatik, 2005.

[11] Mathias Schnee, Fully Realistic Multi-Criteria Timetable Information Systems, PhD Thesis, Technical

University of Darmstadt, 2009.

[12] Michele Rastelli, BusOnAir - Un sistema informativo per trasporti pubblici: modelli, algoritmi e tecnolo-

gie, Master degree thesis, 2011.

[13] Daniel Delling, Kalliopi Giannakopoulou, Dorothea Wagner, and Christos Zaroliagis, Timetable Informa-

tion Updating in Case of Delays: Modeling Issues, http://www.diei.univaq.it/arrival/abstracts/

Delling.pdf, 2008.

[14] Domenico Stranieri, Modelli ed algoritmi per il calcolo di percorsi minimi in presenza di ritardi in ambito

ferroviario, Master degree thesis, 2011.

[15] Matthias Muller-Hannemann, Karsten Weihe, Pareto shortest paths is often feasible in practice Pro-

ceedings of the 5th International Workshop on Algorithm Engineering, 2001.

66

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:207:0001:0013:EN:PDF

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:207:0001:0013:EN:PDF

http://www.atlantaregional.com/transportation/roads--highways/intelligent-transportation-system

http://www.atlantaregional.com/transportation/roads--highways/intelligent-transportation-system

http://docs.neo4j.org/chunked/milestone/configuration.html

http://docs.neo4j.org/chunked/milestone/configuration.html

http://docs.neo4j.org/chunked/stable/configuration-caches.html

http://docs.neo4j.org/chunked/stable/configuration-caches.html

http://www.diei.univaq.it/arrival/abstracts/Delling.pdf

http://www.diei.univaq.it/arrival/abstracts/Delling.pdf

Chapter 2 Bibliography

[16] Alessio Cionini, Gianlorenzo D?Angelo, Mattia D?Emidio, Daniele Frigioni, Kalliopi Giannakopoulou,

Andreas Paraskevopoulos, and Christos Zaroliagis, Engineering Graph-Based Models for Dy-

namic Timetable Information Systems, http://drops.dagstuhl.de/opus/volltexte/2014/4752/

pdf/p046-05-cionini.pdf.

[17] Daniel Delling, Thomas Pajor, and Renato F. Werneck, Round-Based Public Transit Routing, pages

130?140. SIAM, 2012.

[18] Julian Dibbelt, Thomas Pajor, Ben Strasser, and Dorothea Wagner. Intriguingly simple and fast transit

routing, In 12th Symp. Exp. Alg. (SEA), volume 7933 of LNCS, pages 43?54. Springer, 2013.

67

http://drops.dagstuhl.de/opus/volltexte/2014/4752/pdf/p046-05-cionini.pdf

http://drops.dagstuhl.de/opus/volltexte/2014/4752/pdf/p046-05-cionini.pdf

Acknowledgements

Thanks to my family that supports me in everything I do, even when I take arguable decisions.

Thanks to my friends, seriously a second family.

Thanks to Mr. Flammini and Mr. Bermond because, with their help during these two years, they gave me the

opportunity to see them as great professors and, more important, as great persons, too.

Thanks to Mr. Malavolta and Mr. D’Emidio for their support during the thesis.

And, because you are reading this...thank to you.

Date post:	18-Jul-2015
Category:	Software
Upload:	lorenzo-sfarra
View:	206 times
Download:	2 times

Real-time monitoring and delay management of a transport information system

Software