48
Approach to Evaluation Technology for Connected Product
1. Trend in Connected Product
Recently, a car has advanced in various ways. A business, which analyzes acquired various data such as vehicle conditions and surrounding road conditions on the network via Internet connection or others using mobile data communication and provides appropriate optimal services, has been growing along with the so-called “connected cars” that are the vehicles equipped with communication devices. For example, emergency call , vehicle management , monitoring/operation management, driver assistance/protection, telematics insurance, congestion elimination, entertainment, etc. start to be popularized commonly. In addition, automotive manufacturers are also expanding their service business (vehicle dispatch system/vehicle management, etc.) in collaboration with car sharing and rideshare operators. Changes in environmental surrounding the connected car society is shown in Fig. 1. We, DENSO TEN Limited, are also promoting
mass production of connected products such as a communication type dashboard camera, an emergency call system, TCU (Telematics Control Unit), and a taxi
Recently, a business which provides various services has been growing along with the popularization of so-called “connected cars” that are the vehicles equipped with communication devices.
Various devices are connected, leading to an increase in the amount of data to be handled and communication frequency in future. On the other hand, service for which reliability is required to function stably even in any environmental change and user usage is being expanded.
In order to meet such market demands, achievement for both short-term development and quality improvement of the connected products is an urgent issue. In addition to improvement of design quality, an approach to improvement of evaluation efficiency and evaluation quality are indispensable.
In this article, cases of evaluation technology development by establishment of a simulation environment and process improvement by front loading of evaluation are introduced.
Abstract
Approach to Evaluation Technology for Connected Product
Tatsuo WAKIMOTO
Shinichi SUGIURA
Takuya TSUJIMOTO
Shogo TANAKA
Eisuke HAYAKAWA
Environment Changes Conditions in 2030
Increased popularity of autonomous
driving vehicles
Emergence of sharing services
Powertrain diversification
Start of services using autonomous driving vehicles(From 2020: Level 3 or higher)
Shift in values from "Owning" to "Using"
International consensusregarding global warming
countermeasures (Paris Agreement)
30% of travelingdistance by
autonomous driving
Max. 50% are sharedvehicles
25% are xEV(Based on amount of
vehicles sold)
Increased popularity of
connected cars
Increased popularity of connected cars
(2020: 1/3 of the new cars sold in North America equipped with a remote diagnostics function,
launched virtual assist in the market, and increased popularity of music streaming)
Majority of passengervehicles (90% of newcars) are connected
carsConnected
Automated
Shared& Services
North America: Deregulation by government + Verification testing, businesstrials and practical implementation led by private industry
Europe: Standardization with the goal of changing from MaaS to autonomousdriving through connected technology led by the government
Global Mega-trends Changes in the Environment Surrounding the
Connected Car Society: "CASE"
Conversion to EV
Fig. 1 Changes in the Environment Surrounding the Connected Car Society 1)
49
DENSO TEN Technical Review Vol.3
dispatch system, and are developing various service businesses such as safe driving management along with hardware. Image of connected car society system is shown in Fig. 2.
2. Increase in Product Reliability Requirement
According to the expansion of services, various devices are connected, leading to an increase in the amount of data to be handled and communication frequency. Also, for data communications such as emergency notification, driving management, and dispatch service, high reliability is required to function stably in any site environment change and user usage. In order to ensure its quality, the expected value of evaluation for systems and products has been increasing.
In site environment, there are various state changes in mobile data communication, GPS reception, or the like depending on location and time. Therefore, implementing a comprehensive evaluation to combine the operating states of each device is necessary. Approaches from both evaluation technology development and evaluation process improvement are important in order to meet the requirements of short-term development and achieve quality improvement with limited resources.
3. Approach to Quality Improvement through Evaluation Technology
Figure 3 shows the V-shape process of product development. With the aim of improving efficiency of
evaluation and quality of evaluation, we have approaches to (1) the efficiency improvement of evaluation by establishing a simulation environment and to (2) front loading of issue extraction by evaluating specifications.
(1) Efficiency improvement of evaluation by establishing a simulation environment
The evaluation approach using two cases is introduced in this chapter.①Automatic evaluation of the emergency notification
systemThe emergency call system is a device which
automatically sends an emergency notification to the center about information such as the location where a collision occurs via TEL communication. Regardless of location and time, European legislation requires normal operation even under radio wave environment which changes with every moment.
Fig. 4 is a functional environment evaluation matrix that covers the functions (logic) to be tested and the radio wave environment (environment parameter). For each step from an alarm activation of emergency call to the call termination, it is necessary to test about 300 patterns and check the operation of about 600 items for confirmation of the function in the case of a change in the communication status such as connected, out of service, and reconnected states. Conventionally, all patterns were tested manually.
This time, an evaluation system which performs automatic execution and automatic judgment for this
AutomobileDomain
HMI
Telematics Unit
Sensor
ECUEV
(( ))
Non-Automobile
Domain
5GWiFi V2X
OEM Services
MAP OEM
National/PublicCenter
(( ))
OperatorIT
Overall System for Connected Cars
Autonomous Driving Support
Shared CarCommercial Vehicles Center
Open Services
Autonomous Driving
Control ECU
Entertainment System ECU
- Agent- Music/Video
Streaming- Mobility Services- Car Sharing
- Dynamic Maps- Traffic information
(Wide and narrow ranges)
- Maintenance- Engineering
Fig. 2 Image of Connected Car Society System 2)
Implementation procedure for system eligibility
confirmation test
System method design specifications
Systemrequirement
specifications
Development Process
System methoddesign
Software requirement
analysis
Software method design
Softwaredetail design
Coding
Systemverification and integration test
Software verification test
Softwareintegration test
Software unit test
Implementation confirmationof system
Implementation confirmation in a coupling state of
hardware/software
Implementation confirmation of software requirement
Implementation confirmation ofsoftware method design
Implementation confirmation of module
specifications
System requirement
analysisSystem
validation test
(1) Efficiency improvement of evaluation by establishing a simulation environment
Implementation procedurefor system coupling test
Software method design specifications
Software detail design specifications
Softwarerequirement
specifications
(2) Front loading of issue extraction by evaluating specifications
Fig. 3 V-shaped Process of Product Development
50
Approach to Evaluation Technology for Connected Product
test was established, utilizing a simulator (CRAMAS) developed by our company. Full automation enabled the system to operate 24 hours a day, which achieved the efficiency improvement of evaluation and the period shortening.
Fig. 5 shows the evaluation system structure. It consists mainly of a simulator (CRAMAS), which performs test pattern execution and automatic judgment of results, in addition to a base station simulator which simulates a radio wave environment and a call center, a microphone/speaker set for emergency call, and others. CAN information (vehicle speed and crew information), which is a vehicle signal, and an airbag signal at the time of vehicle collision secured the simplification of system and the synchronism of the test timing by generating a pseudo signal with the simulator (CRAMAS).
Furthermore, the confirmation of LED lighting state of the microphone/speaker kit serving as a human interface was performed by voltage monitoring, and the transmission/reception confirmation of voice was performed by judgment of transmission and reception of a specific frequency. Automatic judgement about whether voice transmission to the center and the center recording were success or failure was realized by this method, which led to the achievement of the fully
automated test. By establishing this evaluation system, the functional test period was shortened from 30 days to 10 days, and the evaluation man-hours were greatly reduced from 480H to 40H.
②Automatic Evaluation of Communication Type Dashboard Camera
The communication type dashboard camera has functions of recording data such as CAN communication, position information, and image in a built-in memory or SD card, and sending it to a server. Depending on the user usage, when the data processing timing overlaps with the time of startup or shutdown of the device, it may result in a problem that data is damaged or wrong data is sent. Since this kind of problem may occur only within a few milliseconds during specific data processing, it is extremely difficult to detect the defect by a normal power supply fluctuation test or a manual operation repetition test. Therefore, we worked on strengthening the evaluation of such power supply systems for establishing the automatic evaluation environment.
Fig. 6 shows the evaluation environment and Fig. 7 shows the evaluation time chart (case of evaluation patterns) of power supply ON/OFF.
電波圏外(無線基地局GSM)(-140dBm)
ネットワーク圏外(無線基地局GSM)(-140dBm)
キャリア圏線基地局(-140dBm
【9】自動緊急通報作動 No.1 No.40 No.91 No.1【10】手動緊急通報作動 No.2 No.41 No.92 No.1【11】圏外且つ自動緊急通報作動 - No.42 No.93 No.1【12】圏外且つ手動緊急通報作動 - No.43 No.94 No.1【13】圏外かつ2分経過 - No.44 No.95 No.1【14】圏内 No.3 No.45 No.96 No.1【15】PSAP側で受話 No.4 No.46 No.97 No.1【16】受話せず No.5 No.47 No.98 No.1【17】MSD送信完了(初回送信) No.6 No.48 No.99 No.1【18】T5タイムアウト(初回送信) No.7 No.49 No.100 No.1【19】T6タイムアウト(初回送信) No.8 No.50 No.101 No.1【20】T7タイムアウト(初回送信) No.9 No.51 No.102 No.1【21】MSD送信中にPSAPで終話(初回送信)
No.10 No.52 No.103 No.1
【22】MSD送信中に電波圏外(初回送信)
- No.53 No.104 No.1
【23】リダイヤル時間終了 - - - -【24】圏内 No.11 No.54 No.105 No.1【25】終話による切断 - - - -【26】CCFT(60分)による切断 - - - -【27】MSD要求 No.12 No.55 No.106 No.1【28】MSD送信完了 No 13 No 56 No 107 No 1
微弱電波GSM(-90dBm)
Func
tion(
Logi
c)
Radio Wave Environment (environment parameter)
Fig. 4 Emergency Call System Functional/ Environmental Evaluation Matrix
eCALL(TCU)
TEL
eCALL OUTPUT
Microphone
Speaker
DC Power Supplyfor TCU_BUB
LAN
LAN
Diagnostic Tool (CAN)
CAN
CAN
RELAY_BOX
Terminal Series 12V TCU
LAN
GPS Antenna
Base-station Simulator
KVASER
PowerSupply12V
【Simulator】
Simulator(CRAMAS)
CAN
Power/Vehicle Signal
Control PC
FailBOX
Fig. 5 Evaluation Environment of Emergency Notification System
51
DENSO TEN Technical Review Vol.3
Setting the time T1 to T4 shown in Fig. 7 in detail with a simulator program and executing a large number of patterns enable a defect with low occurrence frequency to be detected. In addition, the reproducible repeated patterns make it easy to analyze the defect and confirm improvement effects, and also enable the frequency of occurrence in the market to be calculated.
Data such as driving history is required for defect detection. Vehicle data such as vehicle speed and engine speed can be generated by the simulator program. Furthermore, pseudo rapid acceleration/deceleration and collision conditions were entered into the product by using a tilting device capable of program control. The operation check of a device was realized by automatically checking the LED lighting state with a sensor and leaving the timestamp when an image data was obtained as data. The storage of these driving
histories and the verification of data which is uploaded to the server made the quality confirmation of products more efficient.
(2) Front loading of issue extraction by evaluating specifications
Normally, all functions are evaluated after designing the application/viewer software. However, more than 80% of defects and improvements are operation screen transitions, wrong indications related to the display, and difficulties to see. These contents are incorporated at the time when the design specifications are created and can be confirmed before completing the software design. Therefore, front loading was performed to extract defects and improvements by “usability evaluation for specifications” in the design phase of application/viewer specifications. (Fig. 8)
Two new methods of “heuristic evaluation method” and “cognitive walkthrough” were adopted for “usability evaluation for specifications.”
Each function of the Web application was checked for 47 items from 17 viewpoints with “heuristic evaluation method,” and the operation/screen transition, etc. of all functions were confirmed with the “cognitive walkthrough” method.
As a result, improvement can be made in the design phase. Although the modified rate of the items which were pointed out at the time of release was 8% with the conventional method, 90% of the items which were pointed out in the overall evaluation could be improved by introducing “Usability Evaluation for Specifications,”
SD rewriting devicePowersupply12V
Control PC
Connected ProductPower Supply,Vehicle Signal
Simulator(CRAMAS)
G tilting device
LED monitoring(camera image)
G tilting deviceGPS Antenna
【Simulator】
Fig. 6 Evaluation Environment of Communication type Dashboard Camera
Power-supply voltage
13.5V
Vehiclespeed
ACCvoltage
8.0V
T4T1 0.5s
0km/h
60km/h10s 10s 20s 10s
T2
12.0V
3s T3
DrivingHistory
G Value0.6G
0.1s
Fig. 7 Evaluation Time Chart of Power Supply On/OFF
BeforeImprovement
Usability Evaluation
Confirmation of Basic Function
Release☆
EvaluationPlanning
Modified rate of items which were pointed out: 8%
AfterImprovement
Evaluation is conducted at the specification development stage before incorporating all functions
Release☆
Specifications Usability Evaluation
Development Evaluation
Development Evaluation
EvaluationPlanning
Confirmation of Basic Function
ProductsUsability Evaluation
Development of Specifications
Development of Specifications
Modified rate of items which were pointed out: 90%
Usability evaluation is conducted after incorporating all functions⇒Prioritizing the handling of a functional defect. Improvement is likely to be implemented in the next model or later.
Modified rate of items which were pointed out was greatly improved since it could be modified at thespecification stage and also the man-hours for modification were reduced.(Modified 132 items out of 147 items which were pointed out.)
Fig. 8 Front Loading of Usability Evaluation
52
Approach to Evaluation Technology for Connected Product
resulting in the improvement of the completion rate at the time of release.
Figure 9 shows the improvement case in the “Usability Evaluation for Specifications.”
4. Conclusion
In order to meet the high-quality requirements of connected products , technology development such as automatic evaluation which simulates actual environment of automobile and network is indispensable.
I n addit ion to the eva luat ion technology development described above, we would like to contribute to the development of connected products capable of being used safely and comfortably through an approach of developing evaluation technologies related to human interfaces such as gesture operation and voice recognition.
Lastly, we would like to express our sincere appreciation to the relevant people inside and outside the company for their cooperation and guidance in developing this evaluation technology.
・CRAMAS is a registered trademark of DENSO TEN Limited.
Reference1) Hiroyuki Watabe and the other, “Efforts toward
Realization of Connected Car Society” and DENSO TEN Technical Review Vol.1
2) How will the Automobile Industry be Different in 2030 (held February 8, 2017), Nikkei Automotive seminar
WithNotification
When detectingstrong G
Without Notification
Excessive Speed
Sudden Steering RapidDeceleration
Excessive Engine Speed Idling
Lane Departure
Rapid Acceleration
Dangerous Distance between Vehicles
Wandering
Expiry Notification
Violation items subject to notification
【Before Improvement】 Mail Notification Setting Screen
WithNotification
Without Notification
Driver’s license items
hr. min.
OFFON
☑ Excessive Speed
☑ Sudden Steering ☑ RapidDeceleration
☑ Excessive Engine Speed ☑ Idling
□ Lane Departure
☑ RapidAcceleration
□ Dangerous Distancebetween vehicles
□ Wandering
OFFON
Notification time
Changed to clearly legible characters
Changed to check box
When detectingstrong G
Violation items subject to a notification
Notification related to driver’s license Expiry notification
【After Improvement】 Mail Notification Setting Screen
hr. min.Changed to pulldown
Fig. 9 Improvement cases in Usability Evaluation for Specifications
TatsuoWAKIMOTO
ShinichiSUGIURA
TakuyaTSUJIMOTO
ShogoTANAKA
EisukeHAYAKAWA
Connected Business GroupQuality Assurance DivSystem Evaluation Dept
Connected Business GroupQuality Assurance Div
Connected Business GroupQuality Assurance DivSystem Evaluation Dept
Connected Business GroupQuality Assurance DivSystem Evaluation Dept
seconded toDENSO Corporation
Profiles of Writers