11

FIT for Embedded System Test Development

PRESENTER : Jingwen Chen

22

Agile methodology from the desktop world XP Inspired embedded system life-cycle Adapting FIT into embedded system Adapting FIT into embedded system

developmentdevelopment– What is FIT/FitNesseWhat is FIT/FitNesse– EmbeddedFitNesseEmbeddedFitNesse testing frameworktesting framework– MatLab/C++ FitNesse MatLab/C++ FitNesse testing testing

frameworkframework Issues with FIT in embedded design Issues with FIT in embedded design Conclusion Conclusion

Main topicsMain topics

33

What we want to avoid ?What we want to avoid ?

System defectsSystem defects– Software errorsSoftware errors cost $59 billion / year cost $59 billion / year– Baxter Healthcare has to recall all its Baxter Healthcare has to recall all its

256,000 Colleague Volumetric Infusion 256,000 Colleague Volumetric Infusion Pumps because the Pumps because the hardware / software hardware / software problemsproblems that already cause 3 deaths and that already cause 3 deaths and 6 serious injuries.6 serious injuries.

We want to find an appropriate and We want to find an appropriate and efficient defect free development efficient defect free development process for embedded devicesprocess for embedded devices

44

Agile methodology from the Agile methodology from the desktop business worlddesktop business world

Minimize risk by developing software Minimize risk by developing software in short iterations, ech iteration is like in short iterations, ech iteration is like a miniature software project of its owna miniature software project of its own

Extreme Programming (XP) Extreme Programming (XP) http://www.extremeprogramming.org/http://www.extremeprogramming.org/map/project.htmlmap/project.html

55

66

Agile methodology from the Agile methodology from the desktop business worlddesktop business world

Explore design space Explore design space – Test Driven Development (TDD) Test Driven Development (TDD)

Improved defect reductionImproved defect reduction Enhanced communication between the Enhanced communication between the

customers and developers customers and developers – Extremes of knowledge within a multi-Extremes of knowledge within a multi-

disciplinary team disciplinary team – Customer: product end-users and Customer: product end-users and

researchersresearchers– Developer: EngineersDeveloper: Engineers

77

XP inspired (XPI) product XP inspired (XPI) product lifecycle for biomedical lifecycle for biomedical embedded systemembedded systemStage 1 : XPI Product envisagement Stage 1 : XPI Product envisagement

– Customers together with developers Customers together with developers produce “produce “executable system specificationexecutable system specification””

– System acceptance testsSystem acceptance tests

Stage 2 : XPI Proof of conceptStage 2 : XPI Proof of concept– Algorithms developed in MATLAB and Algorithms developed in MATLAB and

validated by the acceptance tests written in validated by the acceptance tests written in first stagefirst stage

– Additional unit test in MATLAB using MUnitAdditional unit test in MATLAB using MUnit

88

XP inspired (XPI) product XP inspired (XPI) product lifecycle for biomedical lifecycle for biomedical embedded systemembedded systemStage 3 : XPI Prototyping of initial Stage 3 : XPI Prototyping of initial

productproduct– Forming the initial product using host Forming the initial product using host

machine simulation / embedded systemmachine simulation / embedded system– System moved into the C/C++ production System moved into the C/C++ production

language and validated via tests written in language and validated via tests written in stage 1 and stage 2stage 1 and stage 2

– Additional embedded unit tests added for Additional embedded unit tests added for new environment.new environment.

Stage 4 : XPI Full production ofStage 4 : XPI Full production of systemsystem– The production of the full system on real The production of the full system on real

embedded system target boardembedded system target board – Satisfy the acceptance tests and unit tests Satisfy the acceptance tests and unit tests

developeddeveloped

99

Build an Build an appropriate Agile Agile development environment for embedded system A mechanism to allow A mechanism to allow customers customers

without embedded system without embedded system knowledge to form a part of the knowledge to form a part of the development teamdevelopment team

FitNesse/FITFitNesse/FIT is one possible is one possible solutionsolution

1010

What is FIT/FitNesse ?

A software testing tool designed for customers with limited IT knowledge

A wiki-based system for creating, A wiki-based system for creating, organizing, and running FIT test tablesorganizing, and running FIT test tables

Fit test tables are easy to be read and written by customer

1111

An example of FIT test table

TestFixtures. CheckLifeSign()

items inputValue testResults?

bloodPressure 95 normal

heartRate 80 normal

temperature 38 high expected

normal actual

1212

Starting from a simple temperature measurement embedded system Test software function: CalculateTemperature ( ) running on the

target board convert pulse width to actual temperature

Target Platform

Hardware LayerAnalog Device

ADSP-BF533 processor

System Software Layer

Application Layer CalculateTemperature( )

VDK real-time operating systemInput device:

TMP03 thermal sensor

Voltage pulse:

Time Low

Time High

1313

Stage 1. Stage 1. XPI Product envisagementXPI Product envisagement

Customers together with the developers Customers together with the developers write the Fit test tablewrite the Fit test table

DSPFixture.VDSPColumnFixtureTest

VoltHigh VoltLow outputTemperature(F)

10 100 195

20 100 155

30 100 115

1414

Stage 2. Stage 2. XP Inspired Proof of XP Inspired Proof of conceptsconcepts MatLabFitNesseMatLabFitNesse testing framework testing framework Developing the function in MatLab Developing the function in MatLab

validated by the fit test in stage 1validated by the fit test in stage 1

Fixture forTest table

FitNesseWeb serverWiki editor and storage

DSPFixture.VDSPColumnFixtureTest

VoltHigh

VoltLowoutputTemperature(F)

10 100 195

20 100 155

30 100 115 expected

135 actual

MATLAB ENGINE

MATLAB API

RunnerFit

Server

1515

Migrate the MatLab code into embedded Migrate the MatLab code into embedded system language validated via tests for stage system language validated via tests for stage 1 and stage 2 using host machine simulation 1 and stage 2 using host machine simulation

HOST MACHINE

EmbeddedFitNesse EmbeddedFitNesse

Fixture forTest table

RunnerFit

Server

C++ Simulation

Environment

Stage 3. Stage 3. XPI initial production XPI initial production systemsystem

FitNesseWeb serverWiki editor and storage

DSPFixture.VDSPColumnFixtureTest

VoltHigh

VoltLowoutputTemperature(F)

10 100 195

20 100 155

30 100 115 expected

135 actual

MATLAB ENGINE

MATLAB API

1616

Running the function on real embedded Running the function on real embedded target platform and satisfying the original target platform and satisfying the original test in stage 1 test in stage 1

HOST MACHINE

EmbeddedFitNesse EmbeddedFitNesse

Fixture forTest table

DSP Interface class

S.M.I.L.E TARGET PLATFORM

Stage 4. Stage 4. XPI full production systemXPI full production system

MATLAB ENGINE

MATLAB API

FitNesseWeb serverWiki editor and storage

DSPFixture.VDSPColumnFixtureTest

VoltHigh

VoltLowoutputTemperature(F)

10 100 195

20 100 155

30 100 115 expected

135 actual

VDSP APIVDSP Environment

RunnerFit

Server

C++ Simulation

Environment

1717

Some tests we have done Some tests we have done for embedded hardware:for embedded hardware:

Lab2TestFixtures.SetCoreTimerAssembly

setCOUNT

setPERIOD

setSCALE

readTCOUNT() readTPERIOD()

readTSCALE()

2000 2000 1 2000 2000 1

2000 3000 3 2000

expected3000 3

3000 actual1000 1000 2 1000 1000 2

An error on the ADSP-BF533 processor was found by undergraduate students taking our embedded systems course.

They were using TDD to develop the register setting function SetCoreTimer ( ) running on the embedded target board.

1818

HOST MACHINE

EmbeddedFitNesse EmbeddedFitNesse

Fixture forTest table

Issues with the full EmbeddedFitNesse testing framework

FitNesseWeb serverWiki editor and storage

Long communication path !

S.M.I.L.E TARGET PLATFORM

DSPFixture.VDSPColumnFixtureTest

VoltHigh

VoltLowoutputTemperature(F)

10 100 195

20 100 155

30 100 115 expected

135 actualDSP Interface class

VDSP APIVDSP Environment

RunnerFit

Server

C++ Simulation

EnvironmentMATLAB ENGINE

MATLAB API

1919

FitNesse in embedded design

How to use FIT tables to specify an embedded system properly ? – Embedded system acceptance tests includes:

system operations external and internal stimulus responses interrupt latency system reliability

– Extend the fit test table to handle diverse test cases.

timing diagrams signal comparison figures

Efficiency and reliability concerns– Make the EmbeddedFitNesse testing frame

work more efficiently

Is FitNesse a good solution for embedded system Agile development ?

2020

XPI Lifecycle embedded system development

EmbeddedFitNesseEmbeddedFitNesse testing testing frameworkframework

MatLab/C++ FitNesse MatLab/C++ FitNesse testing testing frameworkframework

Issues with FIT in embedded design Issues with FIT in embedded design

ConclusionConclusion

2121

Thank you !Thank you !