D4.3 Final results of the tests with lessons learnt, conclusions and recommendations Grant agreement no.: 325075 Pilot type A co-funded by the European Union under the Competitiveness and Innovation Programme ICT Policy Support Programme DG Communications Networks, Content and Technology Version number: 1.0 Main author: NavCert Dissemination level: PU Lead contractor: Due date: 31.12.2014 Delivery date: 17.12.2014 Delivery date updated document
Transcript
D4.3 Final results of the tests with lessons learnt, conclusions and
recommendations
Grant agreement no.: 325075
Pilot type A co-funded by the European Union under the Competitiveness and Innovation Programme
ICT Policy Support Programme
DG Communications Networks, Content and Technology
Version number: 1.0
Main author: NavCert
Dissemination level: PU
Lead contractor:
Due date: 31.12.2014
Delivery date: 17.12.2014
Delivery date updated document
Page left intentionally blank
CONTROL SHEET
Version history
Version Date Main author Summary of changes
0.1 2014-02-18 Monika Hentschinski template
0.2 2014-09-08 to
2014-11-13 Monika Stapelfeld
including input pilot sites
0.3 2014-11-17 Martin Grzebellus internal review
0.4 2014-11-20 to
2014-11-25 Ana Paul peer review
0.5 2014-11-20 to
2014-11-25 Kate Yeadon language review
0.6 2014-11-25 to
2014-12-02 Martin Grzebellus, Monika
Stapelfeld integrating updates
0.7 2014-12-02 to
2014-12-05 pilot sites review, flavouring
1.0 21-12-2014 Andy Rooke Authorised
Name Date
Prepared Martin Grzebellus, Monika Stapelfeld 2014-11-17
Reviewed Ana Paul, Kate Yeadon 2014-11-25
Authorized Andy Rooke 21-12-2014
Circulation
Recipient Date of submission
Project partners 21-12-2014
European Commission 21-12-2014
D4.3 Final results
17/12/2014 4 v 1.0
TABLE OF CONTENTS
Control sheet ......................................................................................... 3
Table of contents .................................................................................. 4
The evaluation was performed according to the agreed test specification and methodology
(Deliverable 4.1). The definition of all KPIs in detail is also documented in this deliverable.
The figure below shows the KPIs tested by the pilot sites. The major KPIs were measured by
almost all of the pilot sites. In the lower part of the graph some deviation between planning
and realisation can be seen. Compared to HeERO1 the test phases were shorter and there
was less time to re-plan or re-adjust between the two test phases.
D4.3 Final results
17/12/2014 21 v 1.0
Figure 1: Measurement of Planned KPIs
Tested as planned
Planned, not tested
Not planned, not tested
Tested additionally
4.1 Validation Procedure
The validation considered the following criteria:
LU BE BG ES DK TR
KPI_001a planned but not evaluatedplanned but not evaluatedevaluated as plannedevaluated as plannednot plannedplanned but not evaluated
KPI_001b evaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_002a planned but not evaluatedevaluated as plannedevaluated as plannedplanned but not evaluatedplanned but not evaluatedevaluated as planned
KPI_002b evaluated as plannedevaluatedevaluated as plannedevaluated as plannedevaluatedplanned but not evaluated
KPI_003 evaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_004 evaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_005 evaluated as plannedplanned but not evaluatedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_006 evaluated as plannedplanned but not evaluatedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_007a evaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_007b not plannedplanned but not evaluatedevaluated as plannedevaluated as plannedevaluatednot planned
KPI_008 not plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluatednot planned
KPI_009 evaluatednot plannedevaluated as plannedevaluated as plannedevaluatednot planned
KPI_010 not plannednot plannedevaluated as plannedevaluated as plannednot plannednot planned
KPI_011 not plannednot plannedevaluated as plannedevaluated as plannednot plannednot planned
KPI_012 not plannednot plannedevaluated as plannedevaluated as plannednot plannednot planned
KPI_013 evaluated as plannednot plannedevaluated as plannedevaluated as plannedevaluated as plannedevaluated as planned
KPI_014 not plannednot plannedevaluated as plannedevaluated as plannednot plannednot planned
KPI_015 planned but not evaluatednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_016 planned but not evaluatednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_017 not plannedplanned but not evaluatedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_018 not plannednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_019 not plannednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_020 not plannednot plannedplanned but not evaluatedevaluated as plannednot plannednot planned
KPI_021 not plannedplanned but not evaluatedevaluated as plannedplanned but not evaluatedevaluatednot planned
KPI_022 not plannedplanned but not evaluatedevaluated as plannedplanned but not evaluatedevaluatednot planned
KPI_023 not plannednot plannedplanned but not evaluatedevaluated as plannednot plannednot planned
KPI_024 not plannedplanned but not evaluatedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_025 not plannedplanned but not evaluatedevaluated as plannedplanned but not evaluatednot plannednot planned
KPI_026 not plannednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_027 not plannednot plannedplanned but not evaluatedplanned but not evaluatednot plannednot planned
KPI_028 a planned but not evaluatedplanned but not evaluatedplanned but not evaluatedevaluated as plannedplanned but not evaluatednot planned
KPI_028 b planned but not evaluatednot plannedevaluated as plannedevaluated as plannedplanned but not evaluatednot planned
KPI_028 c not plannednot plannednot plannedevaluated as plannednot plannednot planned
KPI_29 not plannedplanned but not evaluatedplanned but not evaluatedevaluated as plannednot plannednot planned
KPI_30 planned but not evaluatednot plannednot plannednot plannednot plannednot planned
KPI_31 planned but not evaluatednot plannednot plannednot plannednot plannednot planned
D4.3 Final results
17/12/2014 22 v 1.0
- Time series diagrams of the values of relevant KPIs
- Fundamental KPI statistical description for every time series (mean, median,
variance, standard deviation, skew, kurtosis and histogram with normal probability)
- Discussion
The procedures for the creation of the KPI time series diagrams, and the fundamental KPI
statistical description, are described in [2], and conducted in accordance to [1] and [3].
Statistical parameter Definition Comments
Time series diagram of KPI - Graphical representation of time series values v measurement time stamps
Mean x=
1
n∑i= 1
n
xi A numerical measure of the central location of the data values
Median The value at the middle when the data is sorted in ascending order
Variance s
2=
1
n− 1∑i= 1
n
( xi− x )2
A numerical measure of data values dispersion around the mean
Standard deviation σ=
s
√n
An observation variable proportional to the square root of its variance
Skewness γ1=μ3
μ2
3 /2 A measure of the symmetry of the data distribution
Kurtosis γ2=μ4
μ2
2− 3
A measure of the peakedness of the data distribution
Histogram with normal probability diagram
- A graphical representation of the frequency distribution of a KPI value
Table 1: Statistical Parameters Definition
4.2 Timings
Due to the fact that some of the defined KPIs are based on timing issues, timers were
defined already in HeERO 1 to allow a consistent measurement of KPIs in all pilot sites.
These are summarised in Figure 2 (provided by the Czech Pilot Site during HeERO 1)
overleaf.
D4.3 Final results
17/12/2014 23 v 1.0
Figure 2: Time Intervals during eCall
4.3 Results for KPIs
In the following figures the values for the most important KPIs are shown as well as the test
cases from the HeERO1 pilot sites As the project time of HeERO1 was one year longer than
HeERO2, HeERO1 pilot sites had more time between the test phases and there was more
time to readjust the equipment.
As most of the pilot sites provided different test cases there are more columns than pilot sites
within the graphics. The test cases per pilot sites differ, either in time (phase 1 and phase 2
of testing),) or in the different PSAPs or IVSs used.
D4.3 Final results
17/12/2014 24 v 1.0
Figure 3: Results for KPI5: MSD Presentation Time
The test cases from Romania, Greece (HeERO1) and from Turkey are notable in that they
are about 10 seconds longer than the results from the other pilot sites. If these three pilot
sites are excluded the mean value of the Duration until MSD is Presented in PSAP is 17.5
with a standard deviation of 7.9 seconds.
Figure 4: Results for KPI7: Voice Channel Blocking time
The voice channel blocking time does not differ as much as the MSD presentation time. The
minimal values are reached by the Bulgarian and Dutch test sites.
D4.3 Final results
17/12/2014 25 v 1.0
Figure 5: single test cases for KPI7: Voice Channel Blocking time
Five seconds seems to be a plausible optimal minimum to be reached for a full scale
deployment across Europe.
At the moment a mean value of 9.2 seconds with a standard deviation of 2.8 seconds is
reached, giving much room for improvement.
Figure 6: Results for KPI8: time for Call Establishment
The time for call establishment differs greatly between the individual implementations. There
is unexpectedly no differentiation between eCalls with long numbers and with TS12 call set
up though in the dormant implementation the registration in the network takes place prior to
the call set up phase.
D4.3 Final results
17/12/2014 26 v 1.0
4.4 Results of Questionnaires
Questionnaires were used to complement the quantitative analysis by a qualitative
assessment of special aspects. These questionnaires are designed to help to identify issues,
concerns and areas for improvement for stakeholders. These questionnaires have been
developed for two areas: Powered Two-Wheeled (P2W) and Dangerous Goods transport.
The evaluation of the questionnaires was done by the pilot site’s representative for the
specific scopes: Spain for P2W and Luxembourg for Dangerous Goods Vehicles.
It was planned to distribute the questionnaires in all pilot sites but this failed in most pilot
sites (except Bulgaria) because of the short time allowed and the fact that stakeholders were
missing who were willing to accept responsibility for this.
Spain received more than 600 answers to their questionnaire and created an excellent report
which is attached in the Annex of this document (eCall for P2W - User Acceptance Report).
The conclusions of this report are summarised in the following chapter.
The results of the Dangerous Goods questionnaire are summarised by the Luxembourg pilot
site in the subsequent chapter.
4.4.1 Powered Two-Wheeled
The user acceptance study conducted by RACC draws some interesting conclusions:
- On the sample of surveyed motorcycle drivers: one in three respondents has suffered an
accident that required the emergency services to intervene. The results don’t provide a
statistically relevant significance between the experience or age of the motorists and the
accidents. More than half of the respondents who has experienced an accident didn’t alert
the emergency services. Not surprisingly the average arrival time of the emergency services
at the accident site is significantly higher in inter-urban areas than urban areas. However, in
both cases it was clear that the arrival time at the accident site could be clearly improved, for
example with the help of an automatic alert to the closest PSAP.
- On the level of acceptance of the eCall service for motorcycles: a large majority of
respondents motorists would like to have eCall in their motorcycles, and in slightly smaller
proportions they would be willing to change their helmet to have the full functionality of the
system, although they consider that the estimated increase in price compared to the price of
D4.3 Final results
17/12/2014 27 v 1.0
a conventional helmet is rather expensive. On the transition to a future eCall service for
motorcycles, users are receptive to paying for aftermarket devices to use eCall on their
motorcycles manufactured before the system becomes mandatory.
- On the expectations of users: surveyed motorists expect a high functionality from the eCall
service for motorcycles, and require some features that, conceptually, are outside the scope
of the eCall service. In this case, these features (sending of personal data and medical
history of the driver and passenger, etc.) might be implemented as value added services by
private service providers.
From the results of the first laboratory tests, as well as the expectations generated among
potential users, we can conclude that, although there are still many unresolved issues
(technical, standardisation, conceptual, privacy-related, ergonomics-related, costs, etc.), this
service will reduce fatalities, especially in interurban areas by providing a faster and more
efficient management of motorcycle accidents.
D4.3 Final results
17/12/2014 28 v 1.0
4.4.2 Dangerous Goods Vehicles
The HeERO2 project has extended its evaluation of eCall support to dangerous goods
transports.
An important question is: How can the PSAPs get information about which dangerous goods
are loaded into a vehicle that has just reported an accident via the eCall service?
The standard eCall message transfers a Minimum Set of Data (MSD), containing
“emergency relevant” information. To embed information about the loads of commercial
vehicles, the data part of the Optional Additional Data can be used in two ways:
It could contain all the relevant data that needs to be transferred to the emergency
services
It could contain a reference to an external source with the relevant data e.g. a web
service providing detailed information about loaded dangerous goods or a link to a pdf
with the transport documentation.
To get a feeling how the users – in this case the logistic companies – think about the different
solutions for providing dangerous goods information with eCall we created a questionnaire
that asked the users for their opinion on which option is more relevant for them.
In addition to the general questions the questionnaire had three main sections:
1. Which information do you think it is important for the emergency services to know
when your truck is involved in an incident?
( 0 = ‘not important at all’ and 5 = ‘very important’)
0 1 2 3 4 5
What type of dangerous goods are loaded (UN-number)
The danger code of the goods
What quantity of dangerous goods are loaded
The current status of the dangerous good e.g. temperature
What damage has occurred to the containers
What damage has occurred to the truck
Who is the sender
Who is the receiver
What type of truck is involved
What speed was the truck going before the accident
How the DGs should be handled after the accident
2. Which information would you consider important to provide to the emergency services
in that your truck is involved in an incident:
( 0 = ‘not important at all’ and 5 = ‘very important’)
D4.3 Final results
17/12/2014 29 v 1.0
0 1 2 3 4 5
What type of dangerous goods are loaded (UN-number)
The danger code of the goods
What quantity of dangerous goods are loaded
The current status of the dangerous good e.g. temperature
What damage has occurred to the containers
What damage has occurred to the truck
Who is the sender
Who is the receiver
What type of truck is involved
What speed was the truck going before the accident
3. In case that your truck is involved in an incident which way would you think is a
practical way to provide the information to the emergency services?:
( 0 = ‘not practical at all´ and 5 = ‘very practical’)
0 1 2 3 4 5
The information can be stored in a device in the truck and transferred to the emergency services in case of an accident
The information is stored in an electronic transport document e.g. a pdf file. In case of an accident, this document is provided to the emergency services.
The dangerous goods information is stored in your own database In case of an accident you would grant access to the needed information to the emergency service
The dangerous goods information is stored in the database of a centralised secured service. In case of an accident this service provides the needed information to the emergency service
The questionnaire was send to all HeERO2 partners for distribution to their logistic
companies and to 72 members of the Logistic Cluster Luxembourg. Unfortunately the
feedback was very low. We only received 3 answers from Luxembourg and 3 answers from
Bulgaria. However the results are still very interesting as they show a trend:
In order to maintain the anonymity of the persons who provided their feedback, we will
examine the average value of the answers. The responders were 3 large, 2 medium and 1
small logistic company from Bulgaria and from Luxembourg.
1. Which information do you think it is important for the emergency services to know
when your truck is involved in an incident?
( 0 = ‘not important at all’ and 5 = ‘very important’)
What type of dangerous goods are loaded (UN-number) 5.0
The danger code of the goods 4.2
What quantity of dangerous goods are loaded 4.0
The current status of the dangerous good e.g. temperature 3.3
D4.3 Final results
17/12/2014 30 v 1.0
What damage has occurred to the containers 4.3
What damage has occurred to the truck 3.8
Who is the sender 3.0
Who is the receiver 2.3
What type of truck is involved 2.2
What speed was the truck going before the accident 1.5
How the DGs should be handled after the accident 4.8
2. Which information would you consider important to provide to the emergency services
in that your truck is involved in an incident:
( 0 = ‘not important at all’ and 5 = ‘very important’)
What type of dangerous goods are loaded (UN-number) 5.0
The danger code of the goods 4.2
What quantity of dangerous goods are loaded 4.0
The current status of the dangerous good e.g. temperature 3.5
What damage has occurred to the containers 4.3
What damage has occurred to the truck 3.8
Who is the sender 3.0
Who is the receiver 2.3
What type of truck is involved 2.3
What speed was the truck going before the accident 1.3
3. In case that your truck is involved in an incident which way would you think is a
practical way to provide the information to the emergency services?:
( 0 = ‘not practical at all´ and 5 = ‘very practical’)
The information can be stored in a device in the truck and transferred to the emergency services in case of an accident
2.3
The information is stored in an electronic transport document e.g. a pdf file. In case of an accident, this document is provided to the emergency services.
1.8
The dangerous goods information is stored in your own database In case of an accident you would grant access to the needed information to the emergency service
0.5
The dangerous goods information is stored in the database of a centralised secured service. In case of an accident this service provides the needed information to the emergency service
3.7
All responders agreed that it is very important that the emergency services know about and
receive information about the type of the dangerous goods (the UN-number) and the danger
code. Nearly all the responders think that the quantity of the dangerous goods loaded is
important or very important information to have and that it is important to know about the
damage of the container, while the damage to the truck does not seem to be important at all.
The responders also think that it is very important that the emergency services receive
information about the handling of the dangerous goods. The information about the sender
D4.3 Final results
17/12/2014 31 v 1.0
and receiver of the dangerous goods seems to be less important. Not surprising nearly
nobody wants to see that the speed of the truck is provided to the emergency service.
The most interesting result concerned the question how the information about the dangerous
goods loaded should be provided:
Most of the responders see a central database as the best way for providing this information,
while responders from the large logistic companies say that a central database is not useful
for privacy reasons.
Nearly all responders agree that a link to an electronic transport document or access to a
database of the logistic company is not useful.
The storage of the dangerous goods in the on-board units is seen much differentiated:
3 responders coming from the bigger companies say it is very useful, the other 3 say it is
absolutely not useful.
Conclusion:
Even with the low number of responders the questionnaire shows some interesting results:
There is a common agreement about which information is needed by the emergency
services and should be provided to them: the type of the dangerous goods, the quantity and
if possible the damage of the container. The other information seems to be not as relevant.
Regarding the manner the information is provided to the emergency service, a central
database that secures full privacy of the data could be useful and could be recommended.
Storing the dangerous goods information in the IVS in the truck seems only is feasible for
large logistic companies that can afford the higher costs of the devices and the training of the
drivers.
4.5 Recommendations on Performance Requirements
The KPIs in WP4 were developed to measure the performance of the components and
processes. Within the HeERO projects the aim was to get values that measured “best
practise” and not to reach target values. Based on the results of the KPIs, in HeERO 1
recommendations for target values were developed and then updated. The KPIs were
defined only for those critical values which were measured during test drives on roads.
Many of these KPIs measure the critical spans of time, which are influenced by the
respective implementation of the IVS and/ or PSAP suppliers. Other KPIs measure the
D4.3 Final results
17/12/2014 32 v 1.0
accuracy of the GNSS or the correctness of the heading information. This has to be
complemented by thresholds that are only measurable in a laboratory such as antenna gain,
acceleration or voice quality.
The following list provides a short definition of some of the KPIs together with the thresholds.
KPI05 describes the time duration from the initiation (automatically or manually) of an
eCall to the presentation of the MSD content in the PSAP.
KPI07 represents the time the transmission of MSD blocks the voice channel. The
time the voice channel is blocked can be defined as a time between a successful call
setup (“connected” is reported by the network) and the opening of voice
communication in both directions after the MSD has been transmitted successfully or
the MSD transmission has been abandoned (after time out) and the voice
communication has been opened on both sides in both directions.
KPI07a evaluates the duration of the voice channel blocking if an automatic
retransmission of the MSD is initiated by the IVS on request by the PSAP.
KPI08 refers to the observed time difference between the time of the eCall initiation
(automatic and manual) and the time of the eCall reception at PSAP.
[seconds] mean median std. dev. minimum maximum
KPI05 12.9 13.0 3.3 8.6 21.0
KPI07 9.2 9.4 2.8 3.0 13.0
KPI07a 8.7 9.3 3.8 1.9 17.0
KPI08 7.2 6.8 4.0 2.4 17.0
Table 2: recommended values for KPIs 05 to 08
For the following KPIs the thresholds are specified directly.
KPI05, the time for MSD presentation shall be around 8 seconds based on
KPI07 plus average call establishment time of 3.5 to 4 seconds for a normal
call, as such about 3 seconds for emergency set up
KPI07, the voice blocking time should be as specified by the ETSI TS 126 269
around 4 seconds
KPI09, the accuracy of position shall reach the typical GNSS accuracy of 3 to
5m. In addition there shall be requirements for signal strength and how much
loss of signal is allowed.
For KPI13 the test criterion shall be to measure the travel direction not the
direction of the vehicle to be able to identify the right lane of the highway with
physical separation
D4.3 Final results
17/12/2014 33 v 1.0
4.6 General Recommendations
The following recommendations are provided:
The heading information is given to the PSAP as required by EN 16072. The
manufacturers have to be made aware of these requirements.
When retransmitting the MSD the IVS should update the MSD content
according to EN16072.
For type approval a performance requirement for audio should be defined and
evaluated by the technical services.
Voice channel blocking time and MSD transmission times have to be as short
as possible and be in line with the defined thresholds for these KPIs. Vehicle
manufacturers shall implement best practise to minimize voice channel
blocking time.
In order to roll-out the use of filtering instances, procedures, guidelines,
criteria and rules are to be set up to provide the long numbers of PSAPs to
filtering instances. It is however recommended to do steps forward in setting
up this certification framework together with the definition of the conformity
assessment for PSAPs
Cross border communication between adjacent member states is an important
requirement for successful deployment of eCall. Today, emergency services
between different member states in Europe are not interconnected for data
exchange. A further study should evaluate data integration concepts in more
detail
Looking to the next generation of emergency calls, there the use of in-band
modem technology is to be replaced.
4.7 Conclusions
The intention of having HeERO pilot sites has been to evaluate if the requested performance
of the eCall service can lead to a deployment of the approved eCall standards in the existing
public mobile networks and within the existing 112 system. This means that the testing has
had a strong focus on the eCall standards and on capturing the key performance indicators,
the KPIs. Other issues, such as the response time of the rescue services and ambulances,
the use of EUCARIS and the use of the VIN in the operational rescue chain, as well as non-
operational issues, such as legal liability, privacy issues, periodic time inspections, change of
a car ownership, etc were not included in the work of the HeERO pilot sites.
The outcome of the tests performed tests reported in this document confirm that pan-
European eCall works according to expectations however there is still room for
improvement, both in European standards and in its implementation by the suppliers.
D4.3 Final results
17/12/2014 34 v 1.0
One still open issue is the lack of commitment from some MNOPs to implement the
eCall flag.
D4.3 Final results
17/12/2014 35 v 1.0
5 Results of Pilot Sites
5.1 Belgium
5.1.1 General
The Belgian pilot site planned to install a filtering instance at Touring, an eCall-PSAP service
centre at Astrid and to drive around with 6 IVS from NXP. Also the eCall-flag would be
deployed by Mobistar in part of the country.
The architectural discussion was held between Astrid and Touring for interconnection of the
2 servers. Following picture shows the way the different servers were interconnected and
where/how the data was stored.
Figure 7: server architecture (BE)
5.1.1.1 Filtering instance at Touring
The filtering instance was installed on a test server of Touring. The server SW was bought
from Oecon. Also an in-band modem was bought to receive the eCalls. The EN16102
interface was provided to Astrid. The necessary access through firewalls and other security
mechanisms was provided. The picture below shows a snapshot of the graphical user
interface of the Oecon server.
GSM/PSTN
PABXModem
PABX
PCbrowser
Filtercentrale
Push X
ML
Web of XML pull
CIC/PSAP DatacenterAstrid
D4.3 Final results
17/12/2014 36 v 1.0
Figure 8: graphical user interface of eCall server (BE)
Although hardware was rather old (test-server) the server performed well and almost all
necessary tests could be executed. Only the re-transmit function was missing at the filtering-
instance side. This could be bought as an extension to the server-SW, but additional and
unplanned budget would be needed.
5.1.1.2 PSAP at Astrid
Astrid subcontracted the development of a test server to a small company. The SW was
installed on a test server of Astrid. It was properly receiving the MSDs. See below a snapshot
of the GUI.
The test-PSAP was setting up connection through the firewalls towards the test server at
Touring, using XML to extract all MSDs from the server and storing them in a local database
for access by the GUI. This concept functions quite well. It is however to be noted that, for a
real roll-out, an agreement should be made between PSAPs and filtering instance on filtered
transfers of the MSDs. As the implementation of these filtering rules is probably not really
difficult from technical point-of-view, this was not further worked out. Moreover, even if MSDs
would be sent unfiltered to the PSAP, this should not jeopardise the operation of the
services, nor impact on the operation of the operators at both the filtering instance and the
PSAP.
D4.3 Final results
17/12/2014 37 v 1.0
Figure 9: snapshot of the GUI on PSAP (BE)
5.1.1.3 IVS
NXP, partner in the project, was to provide 6 IVS. For this they co-operated with S1NN. The
IVS were delivered (see picture right), but first tests revealed several technical issues. These
were solved throughout the first part of the project, besides of one important issue: the audio-
quality was so bad that the operator at the filtering instance could not understand the person
speaking in the IVS.
Figure 10: IVS by NXP/S1NN (BE)
D4.3 Final results
17/12/2014 38 v 1.0
The cause of the technical issue was a configuration mismatch in the SW provided by NXP.
During the project, NXP transferred its activities related to HeERO2 to Telit. Although, S1NN
remained supporting the project, nor NXP, nor Telit solved the audio issues.
Figure 11: IVS by Geneva Microsystems (BE)
During the project, Geneva Microsystems was contacted to supply IVS with audio
functionality. Geneva Microsystems also provided 6 IVS. Tests with these IVS performed
well, but also here, a few technical issues needed to be resolved. Most of these issues were
solved, but unfortunately in the last SW-version, a new SW-bug slipped in which resulted in
non-functional GPS-reception. One of these technical issues was low reception and
performance in the 900 MHz band of the GSM network. To solve this, Geneva Microsystems
needed to execute a re-design. Geneva Microsystems started a new product design, but this
was unfortunately not available during the course of the project.
Summarized: when first tests were executed, none of both IVS performed good enough to be
distributed to the 6 assigned test drivers. This resulted in execution of only a limited number
of tests.
5.1.1.4 eCall flag by Mobistar
Mobistar rolled out the eCall flag in part of the Belgian country, the blue area on the picture
aside, covering about 7000 km². Mobistar also provided 12 SIM-cards for insertion in the IVS.
In the routing of the switching network, a special filter was installed for those 12 SIM-cards
that, when dialling 112 issuing an emergency call with eCall flag, the call was re-routed to the
Touring Filtering instance. The eCall-flag detection and handling functioned well. However,
some restrictions detected due to choices made:
The re-routing for these 12 SIMs was only available in the Mobistar network and not
on the 2 other networks (Belgacom and Base). At one occasion, it happened that the
IVS could not detect the Mobistar network (probably caused by a weak signal at a
D4.3 Final results
17/12/2014 39 v 1.0
specific location), so the IVS registered in another network. When the emergency call
was signalled, the call was routed to a real PSAP. As this is the fall back later on, this
is not a problem for roll-out.
It was not possible to do cross border tests with 112 eCall flag, as the areas where
the eCall flag was rolled out in Belgium and neighbouring countries (Luxemburg)
were not next to each other.
5.1.1.5 Test coordination at Testronic labs
At Testronic labs, IVS were available to execute tests. Also access to the filtering service
was provided. Access to the test-PSAP was not provided, but this link showed to give the
least problems.
5.1.2 Recommended KPIs
In the beginning of the project, it was estimated that timestamps to measure KPÏs would be
available. Unfortunately, this was not the case.
The table hereunder shows the different KPI’s we have measured.
KPI Description KPI Description
01b Number of manually initiated eCalls 06 Success rate of established voice
transmissions
02a Success rate of completed eCalls
using 112
07a Duration of voice channel blocking
02b Success rate of completed eCalls
using long number
08 Time for call establishment
03 Success rate of received MSDs 28a Number of cross-border tests
04 Success rate of correct MSDs 28b Number of interoperability tests
Table 3: measured KPIs (BE)
The other KPI’s haven’t been measured because of lack of means or because it was not
possible (e.g. KPI_01a: number of automatically initiated eCalls).
The number of eCall tests was about 532 and for the two IVS 375. There were 443 without
eCall flag and 89 with eCall flag (respectively 286 and 89 with the two IVS). Despite this
large amount of data, it was not possible to evaluate statistically more than 10 tests.
D4.3 Final results
17/12/2014 40 v 1.0
5.1.3 Evaluation results
5.1.3.1 Time series
For the time series, we have hereunder evaluated two KPI’s.: The duration of voice channel
blocking (KPI_007a) and the time for call establishment (KPI_008).
Figure 12 : time series KPI 7 and 8 (BE)
5.1.3.2 Statistical evaluation
For the KPI’s analysed we have the following results:
KPI NXP Geneva Microsystems
1b 323 52
2a 75% 79%
2b 75% 79%
3 73% 73%
4 73% 73%
7a 13 13
8 24 24
Table 4 : Results KPIs (BE)
The results for KPI’s nr 3, 4, 7a and 8 are the same for the two IVS because it was not
possible to distinguish them in our server. Furthermore, the results for KPI’s 3 and 4 are the
same because all the received MSDs were correct.
D4.3 Final results
17/12/2014 41 v 1.0
We have seen that for KPI’s 7a and 8, we had time series.
Specifically, the time before we have the audio connection (KPI nr 8) is above 24 s. It is too
high and not acceptable for road safety.
KPI_007a KPI_008
Mean 13 s 24 s
Median 14 s 25 s
Minimum 6 s 19 s
Maximum 20 s 31 s
Range 14 s 12 s
Standard deviation 5 s 4 s
Skewness -0.1464 0.02912
Kurtosis -1.4111 -1.5810
Table 5 : statistics KPIs 7 and 8 (BE)
Figure 13: Distribution KPI 7a (BE)
D4.3 Final results
17/12/2014 42 v 1.0
Figure 14: Distribution KPI 8 (BE)
For these two distributions, they aren’t similar to Gauss curves. The standard deviation is of
course high.
5.1.3.3 Discussion of results
In general, besides the technical open issues, which have not been solved due to resourcing
and priorities at the providers for the IVS and the system, the results of the tests are quite
good. However, still a few items to be noted:
The use of In-band modem technology is not state-of-the-art. Between pushbutton
and audio connection, timings of 20 seconds were often measured, mostly consumed
to transfer the MSD in the audio band. A silence of 20 seconds in case of
emergencies is not acceptable.
During the tests, retransmits could not be tested. As the combination of filtering
instances and PSAPs with retransmits poses several questions, it is a pity that this
could not be further worked out. However, all considerations have been captured in a
document and have been sent to the WP6-leader for further integration into
recommendations.
5.1.4 Interoperability tests
Cross border testing with eCall flag is not straight forward. For this, it is advised to set up a
consortium of different countries with MNOs who are capable and willing to roll out the eCall
D4.3 Final results
17/12/2014 43 v 1.0
flag in the different countries with adjacent geographical areas. Instead of Cross border
testing interoperability tests were executed in following tests:
Members of the LUX pilot site visited Belgium and executed tests of their IVS using the long
number of the filtering instance. These tests gave similar results as former tests, of course
noting the shortcomings which were known.
5.1.5 Recommendations
The use of in-band modem technology is to be replaced by out-of-band technologies
or other performant architectures. By enabling filtering instances to connect to
PSAPs, the market for private eCall will become attractive, thus replacing the public
eCall and related in-band technologies. DEL 6.7 will elaborate in more detailed on the
strategy behind filtering instances.
In order to roll-out the use of filtering instances, procedures, guidelines, criteria and
rules are to be set up to provide the long numbers of PSAPs to filtering instances. It
was expected to make steps forward and create (draft) certification guidelines for
certifying filtering instances to connect to PSAPs in Belgium. Due to the complex
situation of both the Belgian political playfield as well as the uncertain and shifting
approach for introducing eCall in the European commission, the certification
guidelines have not been drafted, yet. It is however recommended to do steps
forward in setting up this certification framework.
Per today, cross border testing across different countries was not feasible.
Emergency services between different countries in Europe are not interconnected in
a uniform way, even outside the scope of eCall. Also MNOs between different
countries are often in competing positions, making cross border tests not a
straightforward test case. To do steps forward, in cross border and cross country
eCall, an integrated project should be set up on European level.
5.1.6 Conclusions
Although there are technical issues and many of the tests could not be executed as planned,
it is not to be expected that technical issues will be the roadblock for rolling out eCall in
Belgium.
5.2 Bulgaria
5.2.1 General
The information within this chapter is based on the test results including the second
realization stage of the Bulgarian pilot - after eCall Flag implementation, Sofia’ PSAP SW
application upgrade for integration with eCall test environment, connection to national VIN
database or EUCARIS. During the test phase the incoming calls were treated as test calls
with further processing only in the test environment, incl. simulation of emergency call centre,
D4.3 Final results
17/12/2014 44 v 1.0
or were answered automatically without intervention. The test client was used to display
MSD data on its screen (incl. GIS location) and the voice channel was activated.
This chapter outlines the Bulgarian results including results of the second evaluation phase
of HeERO2.
5.2.2 Recommended KPIs
The recommended KPIs were successfully evaluated during the testing periods in both
phases of the Bulgarian pilot.
5.2.3 Evaluation results
5.2.3.1 Evaluation process
In order for all possible KPIs to be evaluated IVS and PSAP logs were made. Some KPI
parameters are based mainly on IVS logs (assuming that the IVS events are synchronized
with PSAP events with time difference <0.5 seconds) and only checked for compliance with
the PSAP logs afterwards. This is done for double check of the test results.
In order to evaluate KPIs IVS devices from two manufacturers were used – TUS’ IVS and
ICOM’s IVS.
For the KPI logging and evaluation some IVS devices were configured in special testing
mode making automatic eCalls in predefined periods of time making more tests while driving.
These tests produced logs used to measure most of the KPIs that don’t require operator to
be logged on from PSAP side. Some of the automatically performed eCall tests were made
from stationary devices (no heading information available), but most were conducted from
inside moving vehicles travelling across the country. These tests were performed in a test
scenario where the car was equipped with a device operating in automatic mode and an
interactive voice response system answering from PSAP side. A pre-recorded message was
played to the driver after MSD was sent in order to confirm the established voice connection.
These tests were performed in the manner showed in Figure 15.
IVS power ONIVS initialization
procedureUpdate the MSD &
Call 112
Send MSDDriver hears
automatic IVR response
IVS write log file Wait for timeout
Figure 15: testing process (BG)
D4.3 Final results
17/12/2014 45 v 1.0
For the KPIs involving actual voice connection to 112 operators the eCalls were made
manually including initialisation, MSD transfer, talking to test operator, confirming voice
connection and integrity of received data.
In order to be able to evaluate as many KPIs as possible the IVS devices record different
events and parameters. The following table shows different types of IVS events and
parameters and the logging capabilities of the manufacturers:
Event Short Description
IVS
Manufacturer
TUS ICOM
Dial Start IVS starts dialling the PSAP. X X
Ring Start Call has reached PSAP. X X
Call Established
Connection is established. Due to the PUSH mode
voice connection is open, but is used by the In-
Band modem.
X X
IVS sends ‘START’
signal
IVS attempts to sync with PSAP and asks for MSD
transfer. X X
IVS starts sending
MSD MSD transfer has started X X
IVS stops sending
MSD
MSD transfer has ended. At this point voice
communication is available between operator and
occupant in the car.
X X
Call end Operator or occupant hangs up. X X
Parameter
MSD successfully
sent
‘Yes’ or ‘No’ based on PSAP feedback to IVS.
PSAP feedback includes info about MSD’s
successful transfer and decoding.
X X
eCall flag Shows whether None /Auto/Manual flag is used. X X
GSM level Signal strength of GSM network. X X
Satellites in view Number of usable satellites. X X
Heading Heading in degrees according to North. X X
D4.3 Final results
17/12/2014 46 v 1.0
GDOP Shows the geometric dilution of precision. X X
GPS coordinates Accident GPS coordinates. X X
Speed Speed over ground. X X
Time between
successful positioning
fixes
Measures the time between two GPS position
fixes with accuracy of 1 second. X X
Number of
passengers
Implemented in a later version as a device
configuration parameter. X X
* True GPS coordinates used for Recent Vehicle
Location n-1 and n-2. X X
Table 6: Events and parameters recorded by different IVS devices (BG)
The PSAP is capable of logging all internal events during an eCall. The events are separated
in two categories – ‘info’ and ‘debug’. It also logs and archives all information regarding the
MSD message: its binary representation and its decoded content.
The following KPIs were measured mainly with IVS logs and then only checked for
compliance with the PSAP log: KPI_001a, KPI_001b, KPI_002a, KPI_002b, KPI_003,
KPI_006, KPI_007a, KPI_009, KPI_010, KPI_011, KPI_012 and KPI_013.
The following KPIs were measured by cross examining IVS and PSAP logs: KPI_004,
KPI_005, KPI_006, KPI_008, KPI_021, KPI_022, KPI_023, KPI_024, KPI_025 and KPI_028b
(using logs from foreign PSAPs).
5.2.3.2 Evaluation results
KPI TUS IVS/
Test PSAP Sofia
ICOM IVS/
Test PSAP Sofia
Unit Remarks and notes about
method of testing
KPI_001a 1 609 45 - Logging in IVS
KPI_001b 1 074 147 - Logging in IVS
KPI_002a 93 98 % Logging in IVS and PSAP (if
eCall flag is available)
Note: eCall flag is available in
Mtel network
KPI_002b 85 96 % Logging in PSAP (short
D4.3 Final results
17/12/2014 47 v 1.0
number is used during the
initial tests)
KPI_003 93 85 % Logging in IVS and PSAP
KPI_004 100 100 % Logging in IVS and PSAP
KPI_005 11 11 s Logging in IVS and PSAP
KPI_006 98 100 % Logging in IVS and PSAP
KPI_007a 5 3 s Logging in IVS
KPI_007b 5 3 s Logging in IVS
KPI_008 4 5 s Logging in IVS and PSAP
MNO measures once - 2.5
sec + 112 to PSAP time
KPI_009 acceptable 1.5 m Logging in IVS
KPI_010 10 >12 - Logging in IVS
KPI_011 1.37 <1.5 - Logging in IVS
KPI_012 1 1 s Logging in IVS
KPI_013 100 100 % Logging in IVS
KPI_014 100 100 % Logging in IVS and PSAP
Note: Test VIN data base is
used.
KPI_015 N/A N/A % Logging in PSAP
(if Bulgaria becomes a
member of eCall EUCARIS
agreement or after connection
with Traffic Police vehicle
registers DB)
Note: The connection to
EUCARIS is not available.
KPI_016 N/A N/A s Logging in PSAP
(if Bulgaria becomes a
member of eCall EUCARIS
agreement or after connection
D4.3 Final results
17/12/2014 48 v 1.0
with Traffic Police vehicle
registers DB)
Note: The connection to
EUCARIS is not available.
KPI_017 N/A N/A % Logging in PSAP
Note: The value is not
available as no real PSAP is
used but only "test PSAP".
KPI_018 N/A N/A s Logging in PSAP
Note: The value is not
available as no real PSAP is
used but only "test PSAP".
KPI_019 N/A N/A s No
KPI_020 N/A N/A % No
KPI_021 92 10 - Logging in PSAP
Note: Measure the cases
when an operator accepts the
eCall in test environment in
PSAP Sofia
KPI_022 100 100 % Logging in PSAP
KPI_023 <3 <3 s Logging in IVS
MNO measures once - 2,5
sec.
KPI_024 <1 s Logging in IVS
KPI_025 2.5 2.5 s Logging in PSAP
Note: The source of
information is COCON report.
KPI_026 N/A N/A s Logging in PSAP
Note: The value is not
available as no real PSAP is
used but only "test PSAP".
KPI_027 N/A N/A s No
D4.3 Final results
17/12/2014 49 v 1.0
KPI_028 a N/A N/A - Logging in PSAP
KPI_028 b 104 8 - Logging in IVS and PSAP
KPI_028 c N/A N/A - No
KPI_29 N/A N/A s Logging in PSAP
Note: The value is not
available as no real PSAP is
used but only "test PSAP".
Table 7: summary of results (BG)
5.2.3.3 Comments
KPI_001a, KPI_002a
The first phase of the Bulgarian pilot included testing before the implementation of the eCall
flag. Instead the short number 107 was used as a substitute to the eCall flag. This is why
tests performed with 107 are actually referenced and measured for both KPI_001a
(automatically initiated eCalls) and KPI_002a (Success rate of completed eCalls using long
number).
KPI_003
A link-layer ACK received by the IVS is counted as a successful delivery of the MSD to the
PSAP. The low value comes only from the automatic tests using ‘long’ number (107). The
success rate with the later manual tests using eCall flag is 100%.
KPI_007a, KPI_007b
This time is measured as the time elapsed between two events: IVS sends ‘START’ signal
and IVS stops sending MSD.
KPI_008
This time is measured as the time elapsed between an eCall is initiated (the GSM modem
starts dialling) until the PSAP picks up the call.
KPI_009
The number of available satellites is available and measured in all test sessions. However
this does not give the accuracy in meters, rather than a reference for the accuracy of the
D4.3 Final results
17/12/2014 50 v 1.0
positioning. Having in mind that the average number of usable satellites is 10, which gives
accuracy well below 100m, KPI_009 is marked as ‘acceptable’.
KPI_012
The IVS uses fixed positioning rate of 1 s.
KPI_013
Heading information uses the true direction of travel as delivered by the GPS module.
Contrary to the standard, heading is delivered using true geographical north instead of
magnetic north.
KPI_014
The decoding of the VIN number was done with a local VIN database configured for testing
purposes, but architecturally the same as a real-life solution.
KPI_021, KPI_022
The IVS logs voice calls, including call-backs. The value in KPI_022 is based on a verbal
connection between operator and driver.
KPI_024
The routing in the national 112 network is done entirely inside a software environment which
leads to very low latency times. The exact time cannot be measured, but with tests it was
confirmed that it is well below 1 second.
5.2.3.4 Time series
5.2.3.4.1 TUS IVS/ Test PSAP Sofia
D4.3 Final results
17/12/2014 51 v 1.0
Figure 16: KPI_007 time series TUS IVS (BG)
As it can be seen in Figure 16 the values for duration of voice block are very similar. In rare
cases there are extreme values, but analysis of the logs showed that these are cases where
caused by poor GSM signal strength forcing several retransmissions of the MSD.
5.2.3.4.2 ICOM IVS/ Test PSAP Sofia
Figure 17: KPI_007 time series ICOM IVS (BG)
D4.3 Final results
17/12/2014 52 v 1.0
Figure 18: KPI_008 time series ICOM IVS (BG)
5.2.3.5 Statistical evaluation
Table 8 shows different statistical parameters of KPI_007 and KPI_008.
KPI_007 KPI_008
TUS IVS ICOM IVS ICOM IVS
Minimum 4.4 2.0 1.0
Maximum 9.6 13.0 18.0
Mean 4.7 3.2 5.6
Median 4.4 3.0 5.0
Variance 2.3 2.5
Standard deviation 0.56 1.5 1.6
Skewness 3.351 5.1 4.05
Kurtosis 29.88 31.49
Table 8: statistical parameter (BG)
D4.3 Final results
17/12/2014 53 v 1.0
5.2.3.5.1 TUS/ Test PSAP Sofia
Figure 19: distribution of KPI_007 for TUS (BG)
In Figure 19 the distribution of KPI 7 shows, that the most data are between 4 and 6
seconds. Less but notable amount is between 6 and 8 seconds, which are still very
acceptable values.
5.2.3.5.2 ICOM/ Test PSAP Sofia
Figure 20: distribution of KPI_007 for ICOM (BG)
D4.3 Final results
17/12/2014 54 v 1.0
Figure 21: distribution of KPI_008 for ICOM (BG)
5.2.4 Interoperability tests
This chapter documents the results of the interoperability tests between TUS IVS, ICOM IVS
and PSAPs in Greece, Croatia and Romania.
PSAP KPI 2b KPI 3 KPI 4 KPI 6 KPI 7a
IVS
TUS
IVS
ICOM
IVS
TUS
IVS
ICOM
IVS
TUS
IVS
ICOM
IVS
TUS
IVS
ICOM
IVS
TUS
IVS
ICOM
GR 100 - 100 - 100 - 100 - 11 -
HR 100 100 100 100 100 100 100 50 7 4
RO 100 100 100 100 100 100 100 100 N/A N/A
Table 9: KPI 2b, 3, 4, 6 and 7a with foreign PSAPs (BG)
KPI_007a is not available due to lack of IVS log data and unavailability of the Romanian
PSAP for further testing after IVS logging upgrade.
5.2.5 Recommendations
The following recommendations are given:
The heading information is given to the PSAP as provided by the GPS receiver in the
IVS. This creates inaccuracies when the vehicle is not moving during eCall initiation.
D4.3 Final results
17/12/2014 55 v 1.0
Thus the IVS manufacturers should provide a better direction information by
implementing their own calculation algorithms, e.g. based on previous GPS positions.
When retransmitting the MSD not all IVS devices update the MSD content. As
updating the fields gives new information to the PSAP operator thus leading to better
judgement of the incident environment it is recommended that all IVS devices update
the content of the re-sent MSDs with the latest GPS and other available information.
5.2.6 Conclusion
The IVS devices used in all test sessions are designed and developed by two independent
manufacturers (TUS and ICOM), but perform in a very similar way and the KPIs measured
with them differ only in acceptable ranges. This means that in addition to the successful
interoperability tests during the project, PSAP implementation should not have problem
communicating with different brands of IVS equipment though both the IVS devices and the
PSAP are still in a pilot implementation stage.
The eCall flag is already implemented by one Bulgarian MNO – Mobiltel. The other two
national MNOs are already conducting tests in test environments.
5.3 Denmark
5.3.1 General
This chapter outlines the Danish results of the first and second phases of testing within
HeERO2.
The results reflected here, are solely derived from vehicles equipped with IVS driving
throughout Denmark and calling the Danish HeERO2 PSAP Test Environment. The figure
below shows the locations in Denmark where the calls have been made from.
D4.3 Final results
17/12/2014 56 v 1.0
Figure 22: Location of pilot test calls, as indicated in the IVS-unit (DK)
It shows all attempted call, where a satellite position within the frame of the map was
registered by the IVS-units. Please not, that the land mass between Zealand and Bornholm
belongs to Sweden. Please also note the four dots in the Baltic Sea, which are errors.
5.3.2 Recommended KPIs
The recommended KPI’s 1a and 2a couldn’t be tested during the project. The test-design did
not allow for testing automatically initiated calls, and none of the Danish Mobile Network
Operators supports the eCall flag yet.
5.3.3 Evaluation results
5.3.3.1 Evaluation process
Basis of the evaluation are the datasets derived in field testing with a number of eCall-
equipped vehicles driving around Denmark on other business.
D4.3 Final results
17/12/2014 57 v 1.0
Figure 23 – An IVS-unit installed in a test vehicle
5.3.3.1.1 Phase one testing
Phase one testing was conducted in June and July 2014 with drivers calling to a setup with
an Intelligent Voice Response (answering machine).
Phase one evaluation is based on: Log from IVS, Log from PSAP (eCall-router), and log from
drivers (manually entered information in a log book).
5.3.3.1.2 Phase two testing
Phase two testing was conducted Monday 22 September 2014 from 9 am to 3 pm, and
Tuesday 23 September 2014 from 9 am to 3 pm with a dedicated PSAP operator standing by
at the HeERO2 eCall test equipment.
Phase two evaluation is based on: Log from IVS, Log from PSAP (eCall-router), log from
drivers (manually entered information in a log book), log from PSAP operator (manually
entered information in a log book).
5.3.3.1.3 Time deltas
It was not possible to calibrate the timestamps in the IVS-logs with the timestamps in the
PSAP-log. In some of the IVS-logs time and date was not always set correct (showing time
since boot, instead of actual time).
At the PSAP, the clock loses seconds every day.
Therefore, some of the time deltas in this report differ from the definitions. Please note, that
the resulting time deltas have been validated in lab-testing with external clocks (stop
watches).
Please see below for calculation of KPI5, KPI 7 and KPI 8
D4.3 Final results
17/12/2014 58 v 1.0
5.3.3.1.4 IVS-units tested
Nine IVS-units participated in the test. All units were retrofit devices. For the purpose of ease
of use for the test drivers, the units were put in small cases and all the test driver had to do,
was add power from the vehicle.
Five units from Fujitsu-TEN with special profile SIM cards installed (could only call
to/be called from a limited set of telephone numbers: 112 and the long-number used
for testing). These units were part of both phase one and phase two
Two units from Fujitsu-TEN with normal Danish SIM-cards installed. These units were
part of both phase one and phase two
Two units from GMV with Spanish SIM-cards installed. These units were only part of
phase two
Figure 24: The IVS-units used for testing (DK)
5.3.3.1.5 KPI 5
Instead of calculating T2-PSAP – T0-IVS, we simply add up the results from finding KPI 7
and KPI 8.
5.3.3.1.6 KPI 7a
Fujitsu-TEN
Instead of calculating T2-PSAP – T1-IVS, we calculate T2-PSAP – T1-PSAP and add the
time indicated in the IVS-logs for the establishment of a communication channel
(EVT0_IND_VOICE_CALL_START), until the modem synchronization commences (first
“EVT0_IND_ECALL_SYNC_DETECTED” after “EVT0_IND_VOICE_CALL_START”).
GMV
D4.3 Final results
17/12/2014 59 v 1.0
Instead of calculating T2-PSAP – T1-IVS, we calculate T2-PSAP – T1-PSAP.
Please note further
T2-PSAP is the timestamp for when the MSD can be presented at the PSAP. In the test-
setup, and in the live environments, where will be an additional delay due to an obligatory
voice response telling callers in the test setup, that they have “reached the eCall test
environment, please hold.”.
5.3.3.1.7 KPI 7b
KPI7b are calculated from the eCall routers data as the difference in time between the two
following events:
event:dtmf_detected signal=5 call-ref=INTERNAL
event:call_voice_connection_established
5.3.3.1.8 KPI 8
Fujitsu-TEN
Instead of calculating T0-PSAP – T0-IVS, this KPI have been derived directly from
timestamps in the Fujitsu-TEN IVS log, as the time from initiating (T0-IVS) to the IVS has
established communication channel (EVT0_IND_VOICE_CALL_START).
GMV
Instead of calculating T0-PSAP – T0-IVS, this KPI have been derived directly from
timestamps in the GMV IVS log, as the time from initiating (T0-IVS) to the IVS has
established communication channel (MSD_Start).
5.3.3.1.9 KPI 9 and KPI 13
The evaluation of „Accuracy of position“ and „Success rate of heading information“ are based
on the opinion of trained PSAP-operators only as „Acceptable“ or „not acceptable“.
To calculate the KPI 9, the formula given in D4.1 has been used (even though the Danish
Pilot site, do find the formula misleading): “Accuracy of position”=”Acceptable accuracy”/”not-
acceptable accuracy”
5.3.3.1.10 KPI 15 and 16
As Denmark is not part of EUCARIS, these two KPI’s cannot be tested.
5.3.3.1.11 KPI 17, 18 19, 20, 26 and 27.
As the Danish eCall implementation is based on a principle of „minimum work routine
changes“, there are no changes in the communication protocols between PSAP and TMC
and Rescue forces. Therefore, these KPI’s have not been evaluated.
D4.3 Final results
17/12/2014 60 v 1.0
5.3.3.1.12 KPI 10
Only the GMV-logs had information about number of satellites.
5.3.3.1.13 Dormant testing (KPI 32)
All IVS-units had the capability of letting the GSM-radio is dormant (not registering on the
network, until a call is made). All tests were performed with this setting.
5.3.3.2 Evaluation results for phase one
KPI Name of KPI Fujitsu-TEN
7 units Unit
KPI_001a Number of automatically initiated eCalls N/A -
KPI_001b Number of manually initiated eCalls 185 -
KPI_002a Success rate of completed eCalls using 112 N/A %
KPI_002b Success rate of completed eCalls using long number 94% %
KPI_003 Success rate of received MSDs 99% %
KPI_004 Success rate of correct MSDs 100% %
KPI_005 Duration until MSD is presented in PSAP 13.6 s
KPI_006 Success rate of established voice transmissions 95% %
KPI_007a Duration of voice channel blocking 9.9 s
KPI_007b Duration of voice channel blocking: automatic retransmission of MSD Phase 2 s
KPI_008 Time for call establishment 3.7 s
KPI_009 Accuracy of position Phase 2 %
KPI_010 Number of usable satellites N/A -
KPI_011 Geometric dilution of precision N/A -
KPI_012 Time between successful positioning fixes N/A s
KPI_013 Success rate of heading information Phase 2 %
KPI_014 Success rate of VIN decoding without EUCARIS N/A %
KPI_015 Success rate of VIN decoding with EUCARIS N/A %
KPI_016 Time for VIN decoding with EUCARIS N/A s
KPI_017 Dispatch time of incident data to rescue forces N/A %
KPI_018 Mean time to activate rescue forces N/A s
KPI_019 Dispatch time of incident data to TMC N/A s
KPI_020 Success rate of presented incident data in TMC N/A %
KPI_021 Number of successful call-backs Phase 2 -
KPI_022 Success rate of call-backs Phase 2 %
D4.3 Final results
17/12/2014 61 v 1.0
KPI_023 GSM network latency N/A s
KPI_024 112 national network latency N/A s
KPI_025 112 operator reaction time N/A s
KPI_026 Time for acknowledgement of emergency services N/A s
KPI_027 Total response time N/A s
KPI_028 a Number of cross-border tests N/A -
KPI_028 b Number of interoperability tests TBD -
KPI_028 c Number of cross regional tests N/A -
KPI_29 Dispatch time of Intermediate PSAP N/A s
KPI_30 Number of calls flagged as dangerous good N/A -
KPI_31 Number of successful access of dangerous goods information N/A -
KPI_32 Number of Dormant SIM card tests 185
Table 10 – Results derived from phase one pilot test (DK)
Please note, that KPI’s noted N/A are not measured (see above). KPI’s noted Phase 2 was
derived from phase two.
Figure 25 – geographical distribution of success rate of Calls made during phase one pilot test, with the satellite position registered in the IVS-units (DK)
D4.3 Final results
17/12/2014 62 v 1.0
The few red dots in Figure 28 indicate that either the call was not successful, or that the MSD
was not received at the PSAP.
5.3.3.3 Comments on results from phase one
The majority of failed calls occurred at the beginning of the test-period. There is
therefore a risk that these calls may have failed due to the drivers handling of the
equipment.
Some timestamps in both IVS-units and at eCall-router have been reconstructed. We
believe these reconstructions have been conducted so that they represent correct
timing. The cause for the need for reconstructions are:
o If an IVS-unit calls shortly after it has been booted, it has not yet received
timing information from the GSM-network.
o The clock in the eCall-router is missing approximately a minute a day.
Some of the logs in the IVS-units were not readable. For these calls, data have been
reconstructed by extrapolating data from eCall-router-logs with the drivers manually
entered log-info. Missing data have been omitted from the statistical calculations.
For the majority of the MSDs received at the PSAP, the flag “Position cannot be
trusted” was set, even though the position given, was pretty accurate.
5.3.3.4 Time series from phase one
The figure to the right shows the time series of the KPIs 5 and 7. Note that we do not have
data for KPI 7, for the event causing the maximum for KPI 5. Gaps in the curve indicates a
successful event, with lack of data in the IVS-log
D4.3 Final results
17/12/2014 63 v 1.0
5.3.3.5 Statistical evaluation from phase one
Table 11: Statistics of manual initiated calls for successful calls during first
phase (DK)
The statistics (Table 11) and the time series
(Figure 26), shows, that KPI 5 and KPI 7
normally are pretty close to a mean time of
13,5 and 10.0 seconds, with only a few calls
(approximately 10) causing relatively long
waiting time for the driver.
KPI5 KPI7
Minimum 10.1 7.1
Maximum 27.9 24.9
Mean 13.5 9.9
Median 13.5 10.0
Std dev 2.4 2.2
Skewness 2.1 2.5
Figure 26: time series for successful during first phase. (DK)
D4.3 Final results
17/12/2014 64 v 1.0
5.3.3.6 Evaluation results phase two
KPI Name of KPI
Fujitsu-TEN DORMANT SIM-CARD
5 units
Fujitsu-TEN Normal
SIM-CARD 1 unit
GMV Roaming
SIM-CARD
2 units Unit
KPI_001a Number of automatically initiated eCalls N/A N/A N/A -
KPI_001b Number of manually initiated eCalls 43 9 21 -
KPI_002a Success rate of completed eCalls using 112 N/A N/A N/A %
KPI_002b Success rate of completed eCalls using long number
93% 67%1 95%
%
KPI_003 Success rate of received MSDs 100% 100% 95%
%
KPI_004 Success rate of correct MSDs 100% 100% 100%
%
KPI_005 Duration until MSD is presented in PSAP 13.1 11.8 18.7
s
KPI_006 Success rate of established voice transmissions 93% 67% 100%
%
KPI_007a Duration of voice channel blocking 9.6 8.5 11.6
s
KPI_007b Duration of voice channel blocking: automatic retransmission of MSD
9.1 8.9 8.8 s
KPI_008 Time for call establishment 3.6 5.2 7.1
s
KPI_009 Accuracy of position 77% 20% 13%
%
KPI_010 Number of usable satellites N/A N/A 9.1 -
KPI_011 Geometric dilution of precision N/A N/A N/A -
KPI_012 Time between successful positioning fixes N/A N/A N/A s
KPI_013 Success rate of heading information 69% 0%2 6%
3 %
KPI_014 Success rate of VIN decoding without EUCARIS N/A N/A N/A %
KPI_015 Success rate of VIN decoding with EUCARIS N/A N/A N/A %
KPI_016 Time for VIN decoding with EUCARIS N/A N/A N/A s
KPI_017 Dispatch time of incident data to rescue forces N/A N/A N/A %
KPI_018 Mean time to activate rescue forces N/A N/A N/A s
KPI_019 Dispatch time of incident data to TMC N/A N/A N/A s
KPI_020 Success rate of presented incident data in TMC N/A N/A N/A %
KPI_021 Number of successful call-backs 0 5 14 -
KPI_022 Success rate of call-backs 0%4 100% 87% %
KPI_023 GSM network latency N/A N/A N/A s
1 This poor result may be caused by one faulty IVS or that the vehicle was driving in areas with risk of
poor network-coverage (or a combination of the two). 2 This KPI is based on one single IVS. There could be a problem with this single device, and not to the
fact, that it was equipped with a normal SIM card. 3 The GMV-devices are prototypes (as are the Fujitsu devices). The poor result is based on the fact
that the GMV-device sends data on the cars current heading, and not the heading the car was driving, prior to the call (accident). 4 Call back was tried 35 times. Please see comments below for explanation for this poor result.
D4.3 Final results
17/12/2014 65 v 1.0
KPI_024 112 national network latency N/A N/A N/A s
KPI_025 112 operator reaction time N/A N/A N/A s
KPI_026 Time for acknowledgement of emergency services N/A N/A N/A s
KPI_027 Total response time N/A N/A N/A s
KPI_028 a Number of cross-border tests N/A N/A N/A -
KPI_028 b Number of interoperability tests TBD TBD TBD -
KPI_028 c Number of cross regional tests N/A N/A N/A -
KPI_29 Dispatch time of Intermediate PSAP N/A N/A N/A s
KPI_30 Number of calls flagged as dangerous good N/A N/A N/A -
KPI_31 Number of successful access of dangerous goods information N/A N/A N/A -
KPI_32 Number of Dormant SIM card tests 43 9 21 -
Table 12 – Results derived from phase two pilot test (DK)
Figure 27 – geographical distribution of calls made during phase two, with the satellite position registered in the IVS-units (DK)
In Figure 27 the red dots that indicate that either the call was not successful, or that the MSD
was not received at the PSAP. Please also note the green dot in the Baltic Sea (and the
three red dots), where no vehicle was during testing.
5.3.3.6.1 Comments on results from phase two
In lab-testing 14 May 2014, all Fujitsu-TEN equipment was tested for call-back, and it
worked. BUT, when conducting the phase two „call back did NOT work for Fujitsu-
D4.3 Final results
17/12/2014 66 v 1.0
TEN equipment with profiled SIM-card (to simulate the Dormant SIM-card
requirements). So even though call back for Fujitsu-TEN with the special profiled SIM-
cards was tried 35 times, there was no success.
One of the Fujitsu-TEN units started phase 2 with calling 112, instead of the long
number as it was configured to call with. This gave some additional information from a
real life operator, who had difficulties hearing what the driver was saying.
It took at least one reboot of the IVS for it to get back to its proper configuration, and
we no longer harassed real PSAP-operation.
GMV-units were equipped with Spanish SIM-cards. As we only tested with long-
number, the roaming may have caused problems, which would not happen, if the
roaming call could have used 112 instead. With long-number there will be some data
traffic from the Danish MNO to the Spanish MNO. With 112 (or TS12 call) the call will
be handled exclusively and prioritized within Denmark. This may have inflicted
negatively on the values for KPI_002b, KPI_003, KPI_005, KPI_007a and KPI_008.
It happened at least one time with Fujitsu-TEN equipment that the Satellite position in
the MSD was from the last call (made 15-20 minutes earlier). This was only detected
due to the manual communications between test-operator and test-driver.
Some of the test-drivers put equipment on the back-seat of the car, and they had to
turn around when talking, for the operator to properly hear them.
Some calls did not have data package with the call, but it was retrieved when a
resend of MSD was requested
Notes from interview with test-operator:
Audio quality is a real issue. Static occurs for all equipment, making it difficult for the
operator to communicate with driver.
The visual implementation on screen is perfect.
I’m pretty impressed with the accuracy of the system, this will make a difference.
Resend of MSD only takes five seconds, this is really nice, as it increases my trust in
the position
I’m a little nervous with the call-button. It would be nice, if you had a plastic guard
over the button that drivers have to switch open, before pushing for manual 112-call.
5.3.3.7 Time series from phase two
The four figures below show the time series of the KPIs 5 and 7. Gaps in the curve indicate a
successful event, with lack of data in the IVS-log.
D4.3 Final results
17/12/2014 67 v 1.0
Figure 28: Phase two calls (DK)
Figure 29: Phase two calls with FJT M2M SIM card (DK)
D4.3 Final results
17/12/2014 68 v 1.0
Figure 30: Phase two calls with FJT normal SIM card (DK)
Figure 31: Phase two calls with FJT normal SIM card (DK)
D4.3 Final results
17/12/2014 69 v 1.0
Figure 32: Phase two calls with GMV normal SIM card (DK)
5.3.3.8 Statistical evaluation from phase two
KPI5
(all calls)
KPI7
(all calls)
KPI5
(FJT
M2M)
KPI7
(FJT
M2M)
KPI5
(FJT
normal)
KPI7
(FJT
normal)
KPI5
(GMV)
KPI7
(GMV)
Minimum 9.9 6.9 9.9 6.9 13.3 7.0 15.9 9.9
Maximum 31.3 24.3 19.0 14.1 20.8 11.8 31.3 24.3
Mean 15.2 10.1 13.1 9.6 16.6 8.5 18.7 11.6
Median 14.4 10.0 13.4 10.0 15.8 8.0 18.0 10.0
Std dev 3.8 2.5 2.2 1.6 3.8 1.9 3.4 3.2
Skewness 1.4 3.0 0.7 0.1 1.0 1.1 2.9 3.3
Figure 33: statistical data phase 2 (DK)
The GMV shows poorer results than FJT. This may be due to the fact, that the GMV-units
were equipped with Roaming SIM-Cards and that they had to use a long-number for calling
instead of 112.
D4.3 Final results
17/12/2014 70 v 1.0
5.3.3.9 Discussion of results
Figure 34: geographical distribution of failed calls for phase 1 and 2 (DK)
Figure 34 shows where the IVS-units had registered to be, while attempting a call that either
failed, or where the MSD was not received at the PSAP.
The data presented above are based on non-complete datasets, as not all log-files from IVS
units were readable or accessible. This gives raise to interesting results (see footnote 2,
page 64).
The equipment used for the pilot-test were prototypes (among the first), and some of them,
arrived very late to the pilot site, because the vendor had to give the IVS-unit the capability of
letting the GSM-radio be dormant (not registering on the network, until a call is made).
The problems with obtaining, configuring and using special profiled SIM-Cards , underlines a
recommendation to put all “Dormant”-functionality on the IVS, and not in the SIM-card profile.
5.3.4 Recommendations
IVS must be in compliance with the eCall standards regarding interpretation and
understanding of what are in the MSD. We noticed, that the GMV-units sends the
D4.3 Final results
17/12/2014 71 v 1.0
“Heading”-direction the vehicle has post-crash, instead of sending the heading the
vehicle was driving pre-crash.
Focus on audio quality when installing a retrofitted IVS in vehicles. Maybe this must
be further standardised.
5.3.5 Conclusions
The system works. Even with prototypes (and all the problems that follow) our operators in
the pilot-test could recognize the value of eCall.
But we need to have better retrofit able IVS-units in the market.
5.4 Luxembourg
5.4.1 General
This chapter outlines the Luxembourg results of the evaluation phase of HeERO 2. In
addition to phase 1, KPIs 9 and 13 were evaluated.
5.4.2 Recommended KPIs
The recommended KPI 2a couldn’t be tested during the project as the eCall Flag was not
updated into POST Luxembourg’s MSCs until the end of August. Unfortunately the first tests
after that showed that the Bit setting (6 and 7) for the routing was not added in the current
SW version. A SW version implementing a workaround could only be used in a closed test
environment beginning of October. This test environment is in fact a shielded metal box, with
a size of 80cm x 80cm x 30cm that contains the antenna of a base station. This box is
needed to avoid any interference of new feature or functionalities with the commercial
network. Therefore only a couple of tests were possible and putting an IVS into this box and
pressing the eCall button. The tests showed that the updated version works well. The final
SW version including the complete eCall routing support is planned for implementation into
POST Luxembourg’s networks in Q2 2015.
5.4.3 Evaluation results
5.4.3.1 Evaluation process
The basis of the evaluation was made using the datasets that were derived from the field
tests that were carried out between March 2014 and October 2014.
Unfortunately the KPIs regarding the eCall flag could not be tested. Although the eCall Flag
has been updated in POST Luxembourg’s switches the bits setting for the routing was not
D4.3 Final results
17/12/2014 72 v 1.0
defined and this means that 112 eCalls cannot be routed as required. (An update that will
resolve this this will be available to POST in Q2 2015.)
In the following section, special cases in the evaluation are described.
5.4.3.1.1 KPI5
To compute the time until the MSD is presented in the PSAP (KPI_5) two calculation
methods were used:
1. IVS protocol available? The time from when the activation of the eCall was initiated in
the IVS (button pressed) until the ACK-MSD was received by the IVS was used.
2. The typical time for call setup of 3 seconds was added to the time when the call
reached the PSAP and the time when the MSD is presented to the operator. This
typical value was derived from the median of the delay measured by the IVS.
5.4.3.1.2 Manual and automatic tests
Only manual tests have been executed. For the manual tests all planned KPIs are evaluated.
5.4.3.1.3 KPI 13
The evaluation of the heading was made, in the manual tests, by comparing the board
instrument of the car and the heading presented to the operator when the car was stationary.
The aim of the evaluation of this KPI was to determine if it makes sense to transmit the
heading as a separate value besides the last three positions.
5.4.3.1.4 KPI 9
In the definition of KPIs there are two possibilities for specifying the result of this KPI (value
of the accuracy in meters or the success rate of the accuracy; success is defined as the
accuracy is within 100m). Here the success rate of the accuracy was measured by a
comparison of the real location and the presented location. The real position was reported to
the operator by the driver during the voice communication. The operator compared the
shown location with the real position and entered the result into the report.
The Fujitsu IVS showed a systematic error in the location. In the first eCall after powering up
the location was correct. For the second eCall the location of the first eCall was transmitted
and only when requesting a retransmission of the MSD the correct location was transmitted.
The manufacturer is working on this topic. As it is unrealistic that two eCall are issued directly
after each other as it is executed in the tests, we assumed the location as correct, when the
location was correct after the retransmission of the MSD.
D4.3 Final results
17/12/2014 73 v 1.0
5.4.3.1.5 KPI_15 Success rate of VIN decoding with EUCARIS
KPI 15 could not be tested as EUCARIS access was not available in Luxembourg as of
August 2014.
5.4.3.1.6 KPI_16 Time for VIN decoding with EUCARIS
KPI 16 could not be tested as EUCARIS access was not available in Luxembourg as of
August 2014.
5.4.3.1.7 KPI_28a Number of cross border tests
As the 112 eCall flag was not available at the border of Luxembourg to Belgium and
Germany cross border tests could not be executed.
5.4.3.1.8 KPI_28b Number of interoperability tests
Some interoperability tests have been executed. The results are shown in chapter 5.4.4.
5.4.3.1.9 KPI_30 Number of calls flagged as dangerous good
The handling of dangerous goods transports in European eCall has been defined in this
project. It was decided that instead of a flag only additional information should be provided in
the additional information field of the MSD. This standardisation was more complex than
expected and needed more time. The ESA project that should provide the implementation of
the dangerous goods tracking service was delayed until end of 2014Therefore test
implementations of IVS were not available during the project.
5.4.3.1.10 KPI_31 Number of successful access of dangerous goods information
See above
5.4.3.1.11 KPI_32 Number of Dormant IVS tests
No tests with dormant SIM cards have been executed.
D4.3 Final results
17/12/2014 74 v 1.0
5.4.3.2 Evaluation results
FUJITSU man FICOSA 1 man FICOSA 2 man Unit
KPI_001a - - -
KPI_001b 317 263 98 -
KPI_002b 62% 67% 79% %
KPI_003 62% 67% 79% %
KPI_004 100% 100% 100% %
KPI_005 11.6 s 14.4 s
KPI_006 98% % 97% %
KPI_007 8.9 s 12.1 s
KPI_009 97% 94% 100% %
KPI_013 97% 94% 100% %
Table 13: Summary of Results (LU)
5.4.3.2.1 Comments
In July FICOSA provided updated Firmware for the IVS. The new IVS (FICOSA) shows some
improvements in MSD transmission, especially in difficult environments. (Not visible in the
KPIs)
KPI 5 (MSD presentation time) is longer than KPI 7 (voice channel blocking time). This is as
expected. The FICOSA IVS has a significant higher MSD presentation time and Voice
Channel Blocking than Fujitsu. This is caused by the long synchronisation and codecs
selection time before transmitting the MSD
The accuracy of the GPS positioning is as good as expected and is beside of few outliers
between 3 and 5 meters.
The information of the heading shows the actual direction of the stationary vehicle and not
the recent direction of travel, when the vehicle was driving. So the implementation of the
heading information is not in conformance to the requirements of the standard. It does not
provide the expected additional benefit for the rescue team to know on which side of the
highway the incident happened.
D4.3 Final results
17/12/2014 75 v 1.0
5.4.3.3 Time series
Figure 35: KPI5 FUJITSU IVS (LU)
Figure 36: KPI5 FUJITSU IVS (LU)
D4.3 Final results
17/12/2014 76 v 1.0
Figure 37: KPI5 FICOSA 2 (LU)
Due to a problem with the eCall router logging only a few values could be calculated.
In Figure 36 the chronological sequence of KPI 5 during the manual test is drawn.
5.4.3.4 Statistical evaluation
The statistical evaluation was made according to the agreed procedures in Chapter 4.1.
KPI 5 KPI7
FUJITSU FICOSA1 FICOSA2 FUJITSU FICOSA1 FICOSA2
Minimum 5.0 12.0 12.0 3.0 10.0 10.0
Maximum 25.0 20.0 16.0 24.0 19.0 14.0
Mean 11.6 14.4 14.2 8.9 12.2 11.6
Median 9.0 14.0 14.0 6.0 12.0 12.0
Std dev 5.9 1.8 1.1 6.0 2.0 1.1
Skewness 1.4 1.7 -0.4 1.8 2.4 0.5
Table 14: Statistics of manual initiated calls (LU)
D4.3 Final results
17/12/2014 77 v 1.0
Figure 38: MSD presentation time (LU)
Figure 39: MSD presentation time (LU)
D4.3 Final results
17/12/2014 78 v 1.0
Figure 40: Voice channel blocking time (LU)
In Figure 39 the distribution of KPI 7 shows that most data is between 4 and 8 seconds. The
values of the FICOSA IVS are little smaller after the software update but there is a wider
scattering. In principle both IVS show a similar distribution.
5.4.4 Interoperability tests
In this chapter the results of the interoperability tests of the Luxembourg IVS are
documented. The tests done with the Luxembourg PSAP are documented in the chapters of
the home country of the testing IVS.
PSAP IVS 1 IVS2
BG 5 0
DE 10 10
BE 10 7
RO 5 0
Table 15: KPI 1b with own IVS and foreign PSAPs (LU)
PSAP KPI 2b KPI 3 KPI4 KPI6
IVS 1 IVS2 IVS1 IVS2 IVS1 IVS2 IVS1 IVS2
D4.3 Final results
17/12/2014 79 v 1.0
BG 100 0 80 0 80 0 80 0
DE 100 100 80 70 80 70 80 70
BE 100 100 80 100 80 100 80 100
RO 100 0 60 0 60 0 60 0
Table 16: KPI 2b, 3, 4 and 6 with foreign PSAPs (LU)
5.4.5 Recommendations
The following recommendations are given:
Voice channel blocking time and MSD transmission times are too high for both IVS.
The new version of the FICOSA SW reduces the time, but it is still too long. The
problem seems to be in the synchronisation of the IVS and the PSAP modem. We
recommend reviewing the synchronisation process to reduce this time.
The heading information does not provide the direction of travel as intended in all
implementations. The FUJITSU and the FICOSA units provided the heading
information from the current location of the car when the eCall was triggered. If the
car had turned around during the accident this is no longer the direction of travel. The
last three positions in the MSD should be used to determine the direction of travel
only. The PSAP SW needs to show the last three positions on the map in addition to
the last location
When an IVS is triggered to retransmit the MSD, the IVS provides updated content of
the MSD at the moment of request for retransmission. As the MSD containing the
updated position provides valuable information to PSAPs in case of a retransmission
of the MSD (especially for identifying false alarms), the content should always be
updated.
Luxembourg project partner POST Luxembourg particularly recommends that MNOs
planning on testing the eCall switch firstly check that the correct software is installed
that will allow the subsequent routing of the calls through Bits 6 and 7. After
“swapping stories” with several of the other MNO partners in theHeERO2 project this
has been shown to be a relatively common problem and the responses of the MSC
software suppliers can vary greatly as to availability and pricing.
D4.3 Final results
17/12/2014 80 v 1.0
5.4.6 Conclusion
The success rates of completed calls and received MSD could be improved in comparison
with the results of Phase 1. As such the correctness of the modifications in the standard has
been validated.
The IVS are still prototypes and so there are minor problems with stability etc. which should
be solved when the final overall introduction of eCall gets closer. The biggest problem is the
success rate of the MSD transmission and the long voice channel blocking time that needs to
be improved.
The testing proved valuable where it revealed problems that were not necessarily anticipated
to be problems; specifically the Bit routing software’s importance to the success of the call
transmission for many partners.
5.5 Spain
This chapter outlines the Spanish results of the eCalls performed in Phase 1 and Phase 2.
Sub-phase A in Phase 1 was executed in October 2013 and Sub-phase B in Phase 1
between January and March 2014. This phase was used to properly configure all the system
in order to assure the correct data logging at each stage of the e-Call chain. Phase 2 was
undertaken in October 2014 with all the logging issues solved as well as some operational
improvements learned from Phase 1,
5.5.1 General
Unlike other test sites, Spanish test site has an intermediate PSAP belonging to the Spanish
National Traffic Authority (DGT) and located in Madrid capital, in charge of managing all the
eCalls coming from Spanish territory. Once the data from the IVS gets this PSAP at DGT and
the voice channel is established, it decides what regional 112 can support the vehicle
according to its position. At that same time, the PSAP is able to communicate directly with
the IVS and retrieve the significant data of the IVS (the decoded MSD) in its PCs. Tests in
Madrid were done with Madrid and Castilla y León PSAPs and tests in Galicia with Galicia
and Castilla y León PSAPs, which allowed testing cross-regional aspects too.
Figure 41: Call chain from the IVS to final PSAP
D4.3 Final results
17/12/2014 81 v 1.0
Madrid
All KPIs have been calculated according to the information and data registered at the IVS
and intermediate PSAP at DGT. Unfortunately, some KPIs related to communication
between intermediate PSAP at DGT and regional 112 PSAPs have not been calculated due
to the absence of the necessary timestamps at the regional 112 PSAPs.
In addition, in any the tests done, the 112 number has not been used. Always a long number
was configured at the intermediate PSAP at DGT. For this reason, KPI_002a could not be
calculated.
Another KPI that could not be resolved is KPI_011. In this case, due to absence of
information from the device, this KPI is not possible to be determined.
A remark related to KPI_013 follows: No information related with the accuracy of the heading
has been recovered. Heading information provided by the IVS is considered not significant,
as no reference system is used on-board to conclude on a % of success.
In case of Call-back-related KPIs (KPI_021, KPI_022), the call-backs are considered from
intermediate PSAP at DGT to the IVS.
The protocol followed by the intermediate PSAP at DGT and by the regional 112 PSAPs was
agreed between the different participant PSAPs in the Spanish HeERO2 pilot and is
comparable to the protocol presented for the Galician Test Site.
The scenarios where the tests were conducted were similar to the ones used in the Galician
Test Site, including:
Standard Environmental Conditions (STD) with normal cellular network coverage and good
GNSS signal conditions
Weak GSM signal scenario (WEAK) where the environments chosen had low GSM coverage
High traffic load of GSM data scenario (HTRA): this environment was reproduced in areas with
high traffic load of GSM data, in urban areas.
Driving in regional cross border (REGC): it was tested with neighbouring region of Castilla y
León
High and low speed of vehicle (HIGH, LOW): the car was driving at high speed and at low
speed or stopped
Galicia
In order to get comparable results, the intermediate PSAP uses an answering protocol to set
up the basics of the communication among IVS, intermediate PSAP and final PSAP. The
Galician IVS can initiate the eCall in two modes: manual and automatic:
There is a button to push in case of manual eCall and other button to push in automatic mode
that simulates the airbag activation.
Once the eCall is initiated, the intermediate PSAP receives the data, creates the log entry and
answers the incoming call to establish voice communication with the passenger.
D4.3 Final results
17/12/2014 82 v 1.0
At this moment, the intermediate PSAP confirms some data with the IVS, such as VIN
decoding, type of call or position in GIS.
The intermediate PSAP can request a MSD retransmission if there are discrepancies between
IVS and PSAP information.
If there is a break in the eCall the intermediate PSAP can restore the contact with a call-back.
Based on the location of the IVS, the intermediate PSAP decides what PSAP is going to
attend the eCall (Galicia or Castilla y León) and creates the data for transmission
The intermediate PSAP contacts with the final PSAP and forwards the call from IVS
Final PSAP and IVS establish voice transmission
Figure 42: Galician scenarios (ES)
Five different scenarios were employed to conduct the tests, taking into account the climate
conditions of Galicia, which included rainy and cloudy days when the tests were carried out:
- Standard environmental conditions (STD): IVS drove around Porriño town, N550 road and A55 motorway. These are busy areas in south western Galicia with normal network coverage and GPS signal conditions.
- Weak GSM signal scenario (Weak): these tests were performed in environments far for population nucleus in south western Galicia
- High traffic load of GSM data scenario (HTRA): to test this scenario IVS went to Vigo city. It is a great urban area, with more than 300.000 inhabitants, where there is a high traffic load of GSM data.
- Driving in international cross border (INTC): because of the situation of the Galician region, it was possible to test the GSM coverage with the neighbouring country of Portugal.
- Driving in regional cross border (REGC): it was tested with the neighbouring region of Castilla y León.
D4.3 Final results
17/12/2014 83 v 1.0
Figure 43: IVS CTAG in the regional cross border
P2W
Also, Powered 2-wheelers (P2W) have been tested. The P2W test has been completed in 2
phases. First an emulator of PSAP has been used for validation test, and in the second
round of tests the IVS communicated with the real intermediate PSAP in DGT.
Test architecture
The test setup for the IVS of the P2W is described below:
The IVS has 3 main parts, the helmet itself, the accident detection unit and the call
unit.
o The intelligent helmet measures the impacts in the head and provides an
accident severity index, which may be useful for PSAP operators.
o The on-board accident detection unit sends the order for making a call to the
call unit. For the detection it uses several inertial sensors and an algorithm for
processing the information given by them. A manual activation button is
included in this module for a manual emergency call. It receives the severity
index from the helmet by Bluetooth.
o The call unit is the responsible for making the call and sending the MSD to the
PSAP. An audio channel is established with the helmet via Bluetooth.
D4.3 Final results
17/12/2014 84 v 1.0
The PSAP emulator is composed by an electronic board and a PC with a PSAP
software. The Software provides the main functionality for a PSAP operator: MSD
decoding, call tracking, etc.
The call between IVS and the PSAP emulator is a real call under a real Network, which
increases the cost of each test, but results more realistic and closer to the final behaviour.
The schema of the test architecture is shown in the next figure.
Figure 44: P2W Test architecture
Figure 45: P2W with IVS installed
The test results can be shown in the next table, where calls has been divided in successful
calls (OK), Failed Calls and other where the voice connection has been established, but with
some errors in the MSD, usually related to positioning.
Type of call Total Calls OK Failed Calls Connected with MSD error
Accident Detection Unit
call +MSD GPS
PSAP emulator
Accident Severity
Index
Voice
D4.3 Final results
17/12/2014 85 v 1.0
Manual 40 39 (98%) 0 (0%) 1 (3%)
Automatic 40 38 (95%) 0 (0%) 2 (5%)
Total 80 77 (96%) 0 (0%) 3 (4%)
5.5.2 Recommended KPIs
Phase 1
Data obtained may not be representative enough to be taken into account for a complete
evaluation. Table 17 includes the KPIs that could be measured during Phase 1
KPI Description
1a Number of manually initiated eCalls
1b Number of manually initiated eCalls
2b Success rate of completed eCalls using long number
3 Success rate of received MSDs
4 Success rate of correct MSDs
5 Duration until MSD is presented in PSAP
6 Success rate of established voice transmissions
8 Time for call establishment
9 Accuracy of position
10 Number of usable satellites
11 Geometric Dilution of Precision (GDOP)
12 Time between successful positioning fixes
14 Success rate of VIN decoding without EUCARIS
20 Success rate of presented incident data in TMC
21 Number of successful call-backs
22 Success rate of call-backs
28c Number of cross regional tests
D4.3 Final results
17/12/2014 86 v 1.0
Table 17: KPIs measured during Phase 1 (ES)
Phase 2
Galicia and Madrid, as Spanish test sites, planned to calculate the KPIs included in ‘D4.1
Final agreed KPIs, test specification and methodology’. The great number of eCalls
performed allowed obtaining all the committed KPIs and managing them in a proper
statistical way. KPI_19 (Dispatch time of incident data to TMC) and KPI_20 (Success rate of
presented incident data in TMC) were not obtained because of the particularities of Spanish
eCalls where TMC is actually the intermediate PSAP. In this sense, for the Spanish Test
Site, KPI_19 would be the same as KPI_07a, and KPI_20 would be the same as KPI_02a.
In the case of the Spanish pilot, KPI 2a has not been tested due to the fact that the eCall
Flag has not been deployed yet in Spain. Our device is able to do this type of eCall but only
in case of a short number configured as PSAP (for example number 112).
5.5.3 Evaluation results
Phase 1
Table 18 summarizes the KPIs calculated in Spain during Phase 1.
Galicia Madrid Unit
KPI_001a 37 115 -
KPI_001b 137 117 -
KPI_002b 72.41 62.5 %
KPI_003 83.70 64.34 %
KPI_004 83.70 62.5 %
KPI_005 -- 15 s
KPI_006 82.09 62.5 %
KPI_008 -- 17 s
KPI_009 -- 100 %
KPI_010 8.74 9 Satellites
KPI_011 1.44 -- DOP
KPI_012 1 0.1 s
KPI_014 96.77 -- %
D4.3 Final results
17/12/2014 87 v 1.0
KPI_020 83.70 62.5 %
KPI_021 93 32 -
KPI_022 98.94 84 %
KPI_028c 38 45 -
Table 18: Summary of calculated KPIs during Phase 1 (ES)
Phase 2
Table shows the KPIS calculated in Phase 2.
Galicia Madrid P2W Unit
KPI_001a 163 258 81 -
KPI_001b 194 216 79 -
KPI_002b 88 75.3 100 %
KPI_003 89 76.8 96.2 %
KPI_004 90 100 100 %
KPI_005 20.9 18.2 11.7 s
KPI_006 99 75.3 -- %
KPI_007a 10.7 10.6 -- S
KPI_007b 1.9 9.7 -- s
KPI_008 15.9 12.8 3,6 s
KPI_009 11.9 100 -- %
KPI_010 7.8 9 -- Satellites
KPI_011 1.4 -- -- DOP
KPI_012 165.,3 1 -- s
KPI_013 39 -- -- %
KPI_014 88 100 100 %
KPI_020 -- -- -- %
KPI_021 -- 54 0 -
KPI_022 -- 72 0 %
D4.3 Final results
17/12/2014 88 v 1.0
KPI_023 60 -- -- s
KPI_028a 42 -- -- -
KPI_028b 42 69 -- -
KPI_028c 68 45 -- -
KPI_029 44.7 -- -- s
Table 19: Summary of calculated KPIs during Phase 2 (ES)
5.5.3.1 Evaluation process
The evaluation process is based on the data collected during tests performed during the two
phases in which Spanish Pilot Site was divided.
In Galicia and Madrid, due to some problems with PSAP in Phase 1, the first part of the tests
done (Sub-phase A) shows a bad behaviour of the duration to connect with the PSAP or the
TMC. But in the second round of tests (Sub-phase B), the tests were OK as most of
problems were solved. All info collected takes into account both situations and the data must
be understood with this observation.
Galicia
Table 20 summarizes the tests performed in Phase 2 in October 2014. A total of 357 eCalls
and 68 call-backs were carried out from Galician IVS. Separating the tests by scenarios 123
were performed in STD scenario, 62 with weak GSM signal, 86 with a high load of GSM
network, 88 in the regional cross-border and 64 in the international cross-border. 81% of the
MSD sent by IVS were successfully received by the intermediate PSAP and 84% of the data
sent by this PSAP reached the final PSAP. The success in the voice communication is
almost 100%.
Da
te
Sc
en
ari
o
IVS PSAP INT 112
eC
all
sen
t
Ma
n.
Au
to
Ca
ll-b
ack s
en
t
Su
cc
es
s c
all-b
ack
MS
D s
en
t
MS
D s
uc
ce
ss
tra
ns.
112
ca
ll s
en
t fr
om
PS
AP
112
ca
ll s
uc
ce
ss f
rom
PS
AP
MS
D i
nfo
rma
tio
n s
en
t
fro
m P
SA
P t
o 1
12
su
ccess r
eceiv
ed
info
rma
tio
n a
t P
SA
P
07-oct STD 40 22 18 10 1 40 31 31 29 31 29
D4.3 Final results
17/12/2014 89 v 1.0
INTC 24 15 9 5 1 24 22 16 13 16 13
08-oct
STD 27 15 12 13 4 26 23 11 14 12 14
Weak 28 28 0 0 0 28 24 21 19 21 19
INTC 25 14 11 10 4 25 21 14 11 15 11
09-oct Weak 29 14 15 5 6 29 17 16 10 16 10
HTRA 50 25 25 1 0 50 50 40 40 40 40
16-oct STD 29 12 17 4 3 27 23 17 17 17 17
HTRA 35 20 15 0 0 35 35 35 35 35 35
17-oct REGC 68 28 40 20 12 59 43 61 34 60 34
Total 355 193 162 68 31 343 289 262 222 264 222
Table 20: Summary of Galician tests
Table 21 shows the Galician KPIs in the format required by ‘D4.1 Final agreed KPIs, test
specification and methodology’. It gives a general overview of the development of the
Galician tests. These KPIs were calculated for each scenario and also taking into account
the overall eCalls.
KPI_01a and KPI_01b show that was intended to conduct a comparable number of tests in
all the scenarios. KPI_02b, KPI_03, KPI_04, KPI_06, KPI_13 and KPI_14 were calculated in
the piece of chain from IVS to PSAP as it is here where the MSD transmission is produced.
KPI Name of KPI
Galician scenarios
Total Unit
STD Weak HTRA INTC REGC
01a Number of automatically initiated eCalls
46 15 40 22 40 163 -
01b Number of manually initiated eCalls
49 42 45 30 28 194 -
02b Success rate of completed eCalls using long number
94 63 100 87 85 88 %
03 Success rate of received MSDs 94 86 100 88 88 89 %
voice call hands-free) and we have asked about the usefulness and acceptability of this
solution as well as the willingness to pay for a helmet compatible with eCall and an
aftermarket device.
4 - User Expectations: we have asked what data / information is considered useful or
desirable that the system would be able to acquire and send to the PSAP, and the relative
importance of these for the emergency management. The objective is to identify what
characteristics or requirements, in terms of information, might be interesting that an eCall
service for motorcycles would consider, and that would differentiate from the standard set of
data that is sent for cars (MSD).
5 - Other issues: privacy of the data has been assessed, especially if the user would
voluntarily opt to use a helmet with sensors capable of acquiring much more data than the
standardised set of data in the MSD; finally, we have asked about the introduction of the
eCall service for motorcycles, and how you could have access to it through aftermarket
devices for those motorcycles that would not have the built-in system because were
produced before mandatory implantation.
From the analysis and further processing of the received responses, we have derived the
following statistics and conclusions.
A 1 Characterization of motorists
A 1.1 Profile of respondents
636 people have responded to the survey. The majority (83.96%) of respondents is males
and only 16.04% are female.
D4.3 Final results
17/12/2014 121 v 1.0
Figure 74: Gender of the respondents
Most respondents are concentrated in the adult age groups (27-30, 31-34, 35-38 and 39-42)
and there are few young motorcycle drivers or older than 50. However, we emphasize that
the women driving motorcycles seem to tend to be younger than men (we have not picked
any response from a woman motorist in the age range from 55 years old), a fact possibly due
to later incorporation of women to driving motorcycles.
Figure 75: Pyramid: ages and gender
This hypothesis seems to be confirmed if we look at the data referring to the years of
experience of drivers, since women report fewer years of experience than men driving
motorcycles:
83,96%
16,04%
Male
Female
14% 12% 10% 8% 6% 4% 2% 0% 2% 4%
15-18
19-22
23-26
27-30
31-34
35-38
39-42
43-46
47-50
51-54
55-58
59-62
63+
Men % Women %
D4.3 Final results
17/12/2014 122 v 1.0
Figure 76: Years of experience driving a motorcycle, by gender
A.1.1.1 Mobility patterns of the respondents
The type of motorcycle more respondents report they use is the "scooter", followed by the
model "naked". Two thirds of motorists surveyed (nearly 67%) responded that they drive a
motorcycle daily. Among these, the scooter type model is the most used (34.51% of users
driving a motorcycle on a daily basis, take a scooter).
Men
25,71%
23,44%23,25%
15,12%
12,48%
0-5
6-10
11-20
21-30
More than 30
Women
32,00%
28,00%
20,00%
18,00%
2,00%
0-5
6-10
11-20
21-30
More than 30
D4.3 Final results
17/12/2014 123 v 1.0
Figure 77: Frequency of use
Figure 78: Frequency of use by type of motorcycle
The average number of driven kilometres per year is according to the following statistic:
66,98%
33,02%
On a daily basis
Only weekends, bank
holidays or sporadically
3,76%
0,70%
23,71%
34,51%
9,39%
12,21%
2,58%
10,56%
0,23%
0,47%
1,88%
-6,67%
-3,81%
-26,19%
-13,81%
-18,10%
-11,90%
-4,76%
-12,86%
-0,48%
-1,43%
40% 30% 20% 10% 0% 10% 20% 30% 40%
Custom
Enduro
Naked
Scooter
Sport
Sport touring
Touring
Trail
Trial
Electric
Other
Daily Only weekends, bank holidays or sporadically
D4.3 Final results
17/12/2014 124 v 1.0
Figure 79: Average number of Km. driven per year
A 1.2 Accident statistics
Nearly a third of respondents have suffered an accident in which medical assistance was
needed and / or traffic police had to intervene.
“Have you suffered any accident that involved the emergency services and/or the
traffic police?”
Figure 80: Accident statistics of respondents
41,20%
32,39%
17,45%
8,96%
< 5.000 Km
5.001 - 10.000 Km
10.001 - 15.000 Km
> 15.000 Km
29,40%
70,60%
Yes
No
D4.3 Final results
17/12/2014 125 v 1.0
Figure 81: Accident statistics of respondents by age group
We have not found a clear relationship between years of experience driving a motorcycle
and the accidents statistics. Among respondents motorists that have had an accident,
statistics on years of experience is very distributed:
D4.3 Final results
17/12/2014 126 v 1.0
Figure 82: Years of experience of motorcycle drivers that have had at least one accident
More than half (55.32%) of motorists who have ever had an accident did not personally call
the emergency services, and 14.36% did not in most cases. In these cases, an automatic
warning when the accident occurs could be particularly useful.
Figure 83: Did you call the emergency services (yourself)?
Negatively surprising are the statistics on the average time of arrival of the emergency
services to the accident spot, with 16.49% of motorists who answer that it took between 30
and 50 minutes, while 4.79% said it took more 50 minutes, which can clearly be improved
(for example, with the help of the eCall service). Only 17.02% said that emergency services
arrived within 10 minutes or less.
12,97%
24,32%
28,65%
18,92%
15,14%
0-5
6-10
11-20
21-30
More than 30
10,64%
19,68%
14,36%
55,32%
Yes, in most cases
Yes, in all cases
No in most cases
No, never
D4.3 Final results
17/12/2014 127 v 1.0
Figure 84: How much time elapsed until the emergency services arrived?
Among respondents who have ever suffered a motorcycle accident, the distribution of these
between urban / interurban area (road) is the following:
Figure 85: Distribution of accidents between urban and interurban area
The average time of arrival of the emergency services depending on the accident location
(urban, city / intercity area, road) is the following:
Urban (City)
17,02%
48,40%
16,49%
4,79%
13,30%
< 10 minutes
10 - 30 minutes
30 - 50 minutes
> 50 minutes
I don't remember
61,70%
38,30%
Urban (City)
Interurban (Road)
D4.3 Final results
17/12/2014 128 v 1.0
Interurban (Road)
Figure 86: Average time of arrival of the emergency services to the accident spot depending on the type of area (urban / interurban)
Statistics shows that in urban areas the time of arrival of the emergency services to the spot
of a motorcycle accident is lower, on average, than on the road. So while in town in about 2
out of every 3 accidents (59%) the time of arrival of the emergency services is between 10
and 30 minutes, in road environment statistics is almost 1 in 3 (32 %) while statistics is the
opposite as to accidents assisted between 30 and 50 minutes after the accident, with only
20,87%
58,26%
6,96%
3,48%
10,43%
< 10 minutes
10 - 30 minutes
30 - 50 minutes
> 50 minutes
I don't remember
11,11%
31,94%
31,94%
6,94%
18,06%
< 10 minutes
10 - 30 minutes
30 - 50 minutes
> 50 minutes
I don't remember
D4.3 Final results
17/12/2014 129 v 1.0
7% in the city and 32% in road environment. Twice as many accidents are attended within 10
minutes compared with inter urban area (21 % vs. 11%).
Overall, it seems clear that the eCall service for motorcycles will be particularly useful to help
shorten assistance to a motorcycle accident especially interurban area, where time of arrival
of emergency services at an accident spot usually takes longer than in urban areas, in part
because there are greater distances to travel between accident locations and emergency
centres.
A 1.3 Acceptance of the eCall service for motorcycles
In this part of the survey was briefly introduced the characteristics of the eCall service by the
following explanation:
"In the future eCall service for motorcycles, the motorcycle will have a built-in device able to
acquire data from the accident and send it to the 112 emergency services (just like the eCall
for cars). Besides, and optionally, if you have an eCall compatible helmet it will be capable of
acquiring additional data and send it, together with the rest of data, to the 112. Moreover, the
helmet will also allow to automatically establishing a hands-free voice call with the PSAP."
D4.3 Final results
17/12/2014 130 v 1.0
Figure 87: Architecture of the eCall service for motorcycles
The following question was asked about the degree of acceptance that such a system would
have among its potential users, and the results were clearly positive, with the vast majority of
motorists who would strongly agree (60.53%) or agree (29.25%) in that the eCall service for
motorcycles would be very useful to reduce mortality caused by motorcycle accidents, and
therefore would like to have this system available on their motorcycle.
"Do you think the eCall for motorcycles would help reduce fatalities and/or injuries
caused by motorcycle accidents and, consequently, would you like that your
motorcycle or scooter would have this system?"
D4.3 Final results
17/12/2014 131 v 1.0
Figure 88: Level of acceptance of the eCall service for motorcycles
Extra details about the feature, optional, which would provide a helmet compatible with the
eCall service, were provided:
"In the prototype system we are proposing, the helmet would have sensors capable of estimating the injure caused on the driver or passenger (severity of the impact on the head). Besides, the helmet will also help know whether there was an additional passenger in the motorcycle (apart from the driver), determine the final position after the accident, etc.
All this information would also be sent to the emergency services, automatically, in case of
accident. "
We asked motorists about the relevance, in its discretion, of this type of information. Again, a
large majority considers this information as relevant (30, 03%) or very relevant (64, 31%):
60,53%
29,25%
7,08%3,14%
Strongly agree
Agree
I little agree
Disagree
D4.3 Final results
17/12/2014 132 v 1.0
Figure 89: Relevance of the information provided by the helmet in case of accident
Unlike the eCall service for cars (which will come fitted as standard equipment in all new cars
from the mandatory implementation of this system at European level) motorcycle drivers will
have to make an additional investment in order to have all the features a compatible helmet
will be able to provide.
While it is not mandatory to replace the helmet, since the built-in eCall service will be able to
send a minimum set of information about the accident (timestamp, location, direction, etc.),
as previously noted, users believe that the additional information that the helmet can send is
not only complementary, but relevant or very relevant for the emergency management, so
the willingness of respondents to buy a new helmet is very big. This is the question we
formulated:
"It is estimated that a helmet compliant with eCall will have an extra cost of around 150€
compared with a “conventional” helmet…
Would you change your helmet in order to be able to send additional information (about the
impact on the head, etc.) and to speak with the 112 after an accident?”
64,31%
30,03%
4,09%
1,57%
Very relevant
Relevant
A bit relevant
No relevant at all
D4.3 Final results
17/12/2014 133 v 1.0
Figure 90: Willingness to purchase a new helmet compatible with eCall
However, we observe that the proportion of respondents who considered the information that
the helmet can send as relevant or very relevant (94.33%) does not match the proportion of
users who would be willing, therefore, to buy a new compatible helmet (77,83%). It is quite
evident that there is a clear relationship between this result and the observed price sensitivity
(of the helmet) that some of the respondents have shown. This is the question we have
made in this regard:
"You think that the approximate extra cost of 150€ is"
And this is the result we have obtained:
Figure 91: Price sensitivity (for the price increase of an eCall compatible helmet)
77,83%
22,17%
Yes
No
14,62%
51,26%
26,42%
7,70%
Unreasonably expensive
Expensive
A fair price
Cheap, considering the
features of the system
D4.3 Final results
17/12/2014 134 v 1.0
2 out of 3 respondents (65.88 %) consider the price increase of a helmet compatible with
eCall as expensive or unacceptably expensive (however, a significant proportion of these
would change their helmet anyway).
We have not observed any noteworthy tendency in the statistics of the willingness to change
the helmet in relation with the type of motorcycle driven by the respondent, having in all
cases a very similar proportion to that in Figure 90: Willingness to purchase a new helmet
compatible with eCall We have not detected a greater willingness to purchase a new helmet
eCall support among those motorists who have had an accident, or less willingness among
those who have never had an accident. In both cases, the proportion is very similar to that of
Figure 90: Willingness to purchase a new helmet compatible with eCall. All this seems to
indicate a very homogeneous and transverse interest for the eCall service for motorcycles,
as users understand its potential to be useful in case of accident.
It is interesting how using data from figures 1-16, 1-17 and 1-18 we come to the following
conclusions:
- Among those who consider that the information that the helmet can send in case of
accident is very relevant but still would not change their helmet, 3 out of 4
respondents consider this price increase as expensive (51.06%) or unacceptably
expensive (25.53%).
- Among those who consider that the information that the helmet can send in case of
accident is relevant but still would not change their helmet, 41.27% of respondents
consider this price increase as expensive while 44.44% consider it unacceptably
expensive.
A 1.4 User Expectations
The eCall service for cars is fully standardized, that is not the case with the eCall for
motorcycles. The development of this system poses challenges at a technical level and at
user level. It is important to have a vision of what the expectations are, in terms of
requirements, manifested by the future users of the system, because this way we can design
a system that is suited to their needs.
D4.3 Final results
17/12/2014 135 v 1.0
We have tackled this topic in the following way:
"In case of accident, the eCall service will automatically send information about it to the
emergency rescue service (112). Among other data, it will send the timestamp and exact
location where the accident has occurred.
A motorcycle accident is usually different to an accident involving a car, for this reason it
might be interesting to send additional data that are currently not part of the eCall service for
cars."
The following types of information could be generated from data collected by the telematics
system embedded on the motorcycle and / or the helmet. We evaluated the importance
motorists give to these types of information on a scale of 0-5.
These are the results we obtained:
Figure 92: Importance respondents give to different types of information that the eCall service for motorcycles could generate
Overall, respondents expressed a considerable interest in the types of information suggested
by CEIT and NZI. If we arrange and analyse in further detail the assessments of the
respondents, we can draw some interesting conclusions:
D4.3 Final results
17/12/2014 136 v 1.0
Type of information Score Comments
Final location of the driver 4,67 Motorists give more importance to be able to
send the final location of the driver (4.67 rating
points, on average, over 5) or of the other
passenger (4.64 rating points, on average, over
5) versus final location of the motorcycle (3.03
rating points, on average, over 5). In fact, this
information is the most valued by respondents.
Final location of the other
passenger (if there was any) 4,64
Number of impacts (against the
ground, obstacles, etc.) suffered
by the driver
4,53
Motorists are clearly more concerned about
information relating to the severity of the impacts
on the driver (number of impacts, deceleration)
as a result of the accident that the same
information on the motorcycle, giving these types
of information a very high score, close to the
maximum of 5 points.
Deceleration caused due to the
impacts on the driver 4,45
Speed of the motorcycle when the
accident happened 3,69
The speed at which the accident occurred
together with the final location of the motorcycle
and the severity of the impacts are important
types of information, because they provide an
estimation of the severity the accident might
have, but get a lower valuation compared with
the information directly related to the severity of
the accident on the driver.
Distance between the motorcycle
and the driver 3,18
Final location of the motorcycle 3,03
Deceleration caused due to the
impacts on the motorcycle 3
Number of impacts (against the
ground, obstacles, etc.) suffered
by the motorcycle
2,92
The main conclusion we draw is the importance motorists confer to the information that a
helmet compatible with eCall can send about the location of the driver (and passenger if any)
and the severity of the impacts suffered by them in the accident, besides information directly
related to the motorcycle.
Additionally, we asked respondents to indicate, on a free text field, what other types of
information they would consider useful to automatically send in case of accident and the
D4.3 Final results
17/12/2014 137 v 1.0
features they would like the system could offer. We have grouped the responses into
thematic blocks. Following is a summary of these:
Proposals from the
respondents Comments
Personal data of the driver
and passenger (if any)
A large number of respondents have stated that it would be interesting
to send the PSAP the following personal information about the
occupants of the motorcycle:
- Name, Date of birth, Weight, Height, etc.
- Medical data: allergies, blood type, medical history, etc.
- Contact numbers of family or friends to notify in case of accident.
- Other data: if the user is a donor (in case of death in the accident).
The data would be linked to the helmet, so that a helmet would be
uniquely associated with a user. For obvious reasons, this is
information that would be sent optionally, provided the user gives
consent.
For the owner of the helmet to update these data, the helmet could
have an associated unique identifier. The user would sign in the
helmet manufacturer's website, enter this unique identifier and fill in a
form with all the data referenced above. These data would be stored in
a database. The website of the manufacturer of the helmet should fulfil
and require acceptance by the user of a privacy and data protection
policy.
In the event of an accident, the helmet could send its unique identifier
encoded in one of the optional fields of the MSD, and emergency
services could access a pan-European database where, entering this
identifier they could retrieve all relevant data of the user for a better
management of the emergency. The access protocol and design of
this database should be standardized at European level.
The user would be responsible to update the data, for example, in
case of change of ownership of the helmet.
D4.3 Final results
17/12/2014 138 v 1.0
Vital signs of the injured
users
A number of respondents indicated that they would like the helmet was
able to collect data about the user state after the accident:
- Level of consciousness
- Heartbeat
- Temperature
- Etc.
The helmet designed by NZI is unable, for the moment, to acquire this
type of data. Feasibility (at technical level) of taking vital data about
the user should be researched, and data should be standardized in the
optional data fields of the MSD. Again, the challenge of data privacy
(and probably more expensive helmet for additional instrumentation to
acquire such data) arises.
Characterization / accident
reconstruction
Some users found it interesting that the eCall service would be able to
function as a black box capable of collecting data to help latter
reconstruction of the accident. For example:
- The possibility to determine the location of the impact/s on
motorcycle (if the accident was due to a frontal impact, side impact,
rear impact ...).
- If it is an accidental fall or accident is mainly due to the impact
against an obstacle or another vehicle.
- The tilt angle of the motorcycle at the time of the accident.
- Operated controls on the motorcycle before the impact (for example,
if the motorcycle broke, which would indicate a previous reaction to the
accident).
- Video Recording / Photos: ability to store photos / video seconds
before and after an accident.
- Trace prior to the accident: store and send several lat-long
coordinates before the accident.
- Involvement of other vehicles in the accident (through correlation with
eCall sent by other vehicles in the same place and time)
Regardless of the feasibility, at technical level, to get this type of data
(and subsequent algorithms needed to process and draw conclusions
from them), the question arises whether the eCall service should, from
D4.3 Final results
17/12/2014 139 v 1.0
a conceptual point of view, serve for this purpose, as the automatic
emergency call has one goal which is to shorten the time for
assistance in case of accident and provide useful information to the
emergency services (fire brigade, ambulance, ...) for a better
emergency management.
It seems clear that this type of information would be useful not so
much for the emergency management but for subsequent claims (for
example, for the insurance company, or if any of those involved in an
accident were to give evidence in a possible lawsuit).
Again, the acquisition data other than the MSD standard poses a
challenge in terms of privacy (the user should be free to allow - or not -
that these data would be sent in case of accident, and to decide which
to which service provider these would be sent).
Finally, it is clear that the installation of a telematics device on the
motorcycle (and helmet, optionally) can serve as a basis for the
development of many applications / additional services to eCall, and
this poses a new challenge to enable service providers (such as auto
clubs, insurance companies, etc.) to develop and offer value-added
services based on the eCall service.
Environmental conditions
at the accident site
Some users have suggested that it might be interesting to transmit
data on the environmental conditions at the accident site.
- Temperature
- Humidity (if it is raining)
- etc.
The system designed by CEIT is unable, for the moment, to get this
type of data. Feasibility (at technical level) of introducing temperature
sensors, humidity, etc. should be researched, and data should be
standardized in the optional data fields of the MSD. Again, this could
increase the cost for the additional system instrumentation to acquire
such data.
Data after the accident Some users have contributed with ideas on data that could be sent
once the accident has occurred, such as:
D4.3 Final results
17/12/2014 140 v 1.0
- Position the driver / passenger after the event.
- Position of the motorcycle after the accident, if it varies, it
could mean that the driver is not unconscious or there is
someone who can provide help.
- Ability to detect if the helmet is in the head of the occupants
after the accident.
As with the cases before, it is very questionable the usefulness of this
type of information for the emergency management, as this must be as
fast as possible and there (in principle) should not exist any other
information that might distract that is not strictly necessary for prompt
assistance. However, for different applications, such as reconstruction
of an accident, these might be useful, and in this case the technical
and privacy challenges noted above would need to be considered.
Other features
- Option that the driver can manually activate or cancel the
eCall (already implemented in the system designed by CEIT).
- Enable light signal (LED or similar) in the helmet if the
accident occurs at night on unlit roads.
- The device should be easily disabled, if the user wishes so
(for example, in order to run in circuits without the eCall
activated).
A 1.5 Other issues
We have evaluated the aspect of data privacy. More than two thirds of respondents do not
care (46.38%) or care little (25.79%) that data is transmitted from the eCall service. This is
the statistic:
D4.3 Final results
17/12/2014 141 v 1.0
Figure 93: Sensitivity to data privacy of the respondents
Finally, in a scenario like that which will occur in the case of cars, where motorcycles without
the eCall service coexist with new motorcycles with the built-in eCall service, we asked about
the willingness to pay for an aftermarket eCall service, obtaining the following results.
"If your motorcycle does not have the built-in eCall service… Would you buy and install in
your motorcycle an aftermarket device that would be compliant with eCall?"
46,38%
25,79%
7,08%
14,78%
5,97%
I am not concerned
I am a bit concerned
I do not know
I am concerned
I am very concerned
D4.3 Final results
17/12/2014 142 v 1.0
Figure 94 : Price sensitivity for an aftermarket eCall systemeCall service
28,46%
47,64%
16,04%
7,86%
Yes, but only if the total cost of the device (including cost of the installation in my
motorcycle) would not exceed 50€
Yes, but only if the total cost of the device (including cost of the installation in my
motorcycle) would not exceed 100€
Yes, but only if the total cost of the device (including cost of the installation in my
motorcycle) would not exceed 200€
No
D4.3 Final results
17/12/2014 143 v 1.0
A 2 Conclusions
The Spanish pilot in HeERO2 is one of the first experiences at European level to develop,
implement and test an automatic eCall emergency call designed for two-wheeled vehicles.
The user acceptance study conducted by the RACC allows us to draw some interesting
conclusions about this system:
- On the sample of surveyed motorcycle drivers: one in three respondents has suffered an
accident that required the emergency services to participate. We have not identified a clear
link between the experience of motorists and accidents, not by age either. More than half of
respondents who have ever had an accident not personally alerted the emergency services,
while the average time of their arrival at the accident spot is significantly higher in interurban
area compared with urban. However, in both cases the time of arrival at the accident spot
could be clearly improved, for example with the help of an automatic warning to the closest
PSAP.
- On the level of acceptance of the eCall service for motorcycles: a large majority of
respondents motorists would like to have eCall in their motorcycles, and in slightly smaller
proportion they would be willing to change their helmet to have the full functionality of the
system, although they consider that the estimated increase in price compared to the price of
a conventional helmet is rather expensive. On the transition to a future eCall service for
motorcycles, users are receptive to pay for aftermarket devices to be able to have eCall on
their motorcycles manufactured before the system is implemented at European level in all
new motorcycles.
- On the expectations of users: surveyed motorists place a high potential to the eCall service
for motorcycles, and require some features that, conceptually, are outside the scope of the
eCall service. In any case, these features (sending of personal data and medical history of
the driver and passenger, etc.) will most likely be implemented as value added services by
private service providers.
From the results of the first laboratory tests, as well as the expectations generated among
future potential users, we can conclude that, although there are still many unresolved issues
(technical, standardization, conceptual, privacy-related, ergonomics-related, of costs, etc.),
D4.3 Final results
17/12/2014 144 v 1.0
this system could be useful for a faster and more efficient management of motorcycle
accidents, helping to reduce fatalities in these vehicles, especially in interurban area.
D4.3 Final results
17/12/2014 145 v 1.0
A 3 Questionnaire used
Survey eCall for motorcycles
eCall for motorcycles: the user point of view
From October 2015 all passenger cars in the EU will come with an electronic safety
system that, in case of a severe accident, will automatically call the emergency
services (112). This system will inform the 112 about the exact location of the accident
and will send additional data, such as the number of passengers in the car, the vehicle
identification number, etc. shortening the overall rescue time and contributing to save
up to 2500 lives a year in Europe.
However, this system won’t be initially available for motorcycles!
Within the HeERO2 European project a pilot system for eCall for motorcycles is being
developed, and RACC Automobile Club is coordinating a study to investigate the
requirements needed to extend the eCall to this kind of vehicles. This study is made in
cooperation with the Directorate-General of Traffic in Spain together with other
partners of the project.
In order to better understand which specific needs the eCall service for motorcycles
should take into account, we would like to ask you, only if you are a motorcycle driver,
that you answer the following questions. It will not take you more than 5 minutes and
it will be a great source of information for us.
D4.3 Final results
17/12/2014 146 v 1.0
Thank you very much for your cooperation.
[All questions are to be defined as mandatory in the Evalandgo tool]
1. Your gender
a) Male b) Female
2. Year of birth
(1900/1901/…/up to 1999)
3. Years of driving experience (motorcycle or scooter)
D4.3 Final results
17/12/2014 147 v 1.0
a) 0-5 b) 6-10 c) 10-20 d) 20-30 e) More than 30
4. How often do you drive a motorcycle or scooter?
a) Daily b) Only weekends, bank holidays or sporadically
5. What kind of motorcycle do you drive most frequently?
a) Custom
b) Enduro / MotoCross
c) Naked
d) Scooter
D4.3 Final results
17/12/2014 148 v 1.0
e) Sport
f) Sport Touring
g) Touring
h) Trail
D4.3 Final results
17/12/2014 149 v 1.0
i) Trial
j) Electric
k) Other: please specify
6. How many km do you usually drive per year by motorcycle or scooter?
a) Less than 1500 b) Between 1500 and 3000 c) Between 3001 and 5000 d) Between 5001 and 10000 e) Between 10001 and 15000 f) Between 15001 and 20000 g) More than 20000
7. Have you suffered any accident that involved the emergency services and/or the traffic police?
a) Yes b) No
[CONDITIONAL, if answer to question 7. is ‘Yes’ continue to question 8. if answer
in ‘No’ skip to question 11: be careful to configure this properly in the “Evalandgo”
tool]
D4.3 Final results
17/12/2014 150 v 1.0
8. Did you call the emergency services (yourself)?
a) Yes, always b) Yes, in the majority of cases c) In the majority of cases, No d) No, never
9. How much time elapsed until the emergency services arrived?
a) Less than 10 minutes b) Between 10 and 30 minutes c) Between 30 and 50 minutes d) More than 50 minutes e) I don’t remember
10. The accident was in…
a) Urban zone b) Interurban zone
In the future eCall service for motorcycles, the motorcycle will have a built-in device
able to acquire data from the accident and send it to the 112 emergency services (just
like the eCall for cars). Besides, and optionally, if you have an eCall compatible helmet
it will be capable of acquiring additional data and send it, together with the rest of
data, to the 112. Moreover, the helmet will also allow to automatically establishing a
hands-free voice call with the 112.
D4.3 Final results
17/12/2014 151 v 1.0
Now we will ask you some more specific questions about this system…
11. Do you think the eCall for motorcycles would help reduce fatalities and/or injuries caused by motorcycle accidents and, consequently, would you like that your motorcycle or scooter would have this system?
a) I do not agree b) I partially agree c) I agree d) I strongly agree
12. In the prototype system we are proposing, the helmet would have sensors capable of estimating the injure caused on the driver or passenger (severity of the impact on the head). Besides, the helmet will also help know whether there was an additional passenger in the motorcycle (apart from the driver), determine the final position after the accident, etc.
All this information would also be sent to the emergency services,
automatically, in case of accident. For the emergency management, you
consider this information as:
D4.3 Final results
17/12/2014 152 v 1.0
a) Not relevant at all b) A bit relevant c) Relevant d) Very relevant
It is estimated that a helmet compliant with eCall will have an extra cost of around
150€ compared with a “conventional” helmet…
13. Would you change your helmet in order to be able to send additional information (about the impact on the head, etc.) and to speak with the 112 after an accident?
a) Yes b) No
14. You think that the approximate extra cost of 150€ is:
a) Too expensive b) Expensive c) A fair price d) Cheap, considering what the system does
In case of accident, the eCall service will automatically send information about it to the
emergency rescue service (112). Among other data, it will send the timestamp and
exact location where the accident has occurred.
A motorcycle accident is usually different to an accident involving a car, for this
reason it might be interesting to send additional data that are currently not part of the
eCall service for cars.
15. Please, rate in a scale from 0 to 5 how important are, in your opinion, the following types of information, where 0 = ‘not important at all’ and 5 = ‘very important’
a) Speed of the motorcycle when the accident occurred b) Number of impacts (ground, obstacles) suffered by the motorcycle c) Number of impacts (ground, obstacles) suffered by the driver d) Deceleration suffered due to the impacts on the motorcycle e) Deceleration suffered due to the impacts on the driver f) Final location of the motorcycle
D4.3 Final results
17/12/2014 153 v 1.0
g) Final location of the driver h) Final location of the passenger (if there is one, and wears a compliant
helmet) i) Distance between the motorcycle and the driver
Please, tell us what other kind of information you would consider useful to be
sent (automatically): [free text field] [NOT mandatory]
16. Are you concerned that such a system would have access to your personal data (for example, your location, in case of accident), even if it is a public service?
a) I am not concerned b) I am a bit concerned c) I do not know d) I am concerned e) I am very concerned
17. If your motorcycle does not have the built-in eCall service… Would you buy and install in your motorcycle an aftermarket device that would be compliant with eCall?
a) No b) Yes, but only if the total cost of the device (including cost of the
installation in my motorcycle) would not exceed 50€ c) Yes, but only if the total cost of the device (including cost of the
installation in my motorcycle) would not exceed 100€ d) Yes, but only if the total cost of the device (including cost of the
installation in my motorcycle) would not exceed 200€
