Workshop TDD4ES

18/11/2010

EP 1

iwt-090191 TDD4ES

Workshop

Building blocks for

Test-Driven Development of Embedded Software

18/11/2010

dr. ing. Jeroen Boydens

ing. Piet Cordemans

ing. Sille Van Landschoot

ing. Wim Catteeuw

@ep_research http://ep.khbo.be

18/11/2010 TDD4ES

Content

Session 1 (Tuesday 16/11/2010)

1. Introduction

2. TDD in practice

3. TDD for embedded software

Session 2 (Thursday 18/11/2010)

1. Test on target (continued)

2. Test on host

3. Remote testing 2

Workshop TDD4ES

18/11/2010

EP 2

18/11/2010 TDD4ES

Session 1 recap

3

Tests

Test on target

Test on host

Remote testing

18/11/2010 TDD4ES

Content: session 2

1. Test on target 1. 3-tier TDD

2. Hardware-aware TDD

3. Hardware-specific TDD

4. Pros-cons

5. Do it yourself

6. Exercise

7. Mocking

4

Workshop TDD4ES

18/11/2010

EP 3

18/11/2010 TDD4ES

Content: session 2 cont’d

2. Test on host 1. Test on host in practice

2. 3-tier TDD

3. Mock verification

4. Embedded TDD cycle

5. Pros-cons

6. Do it yourself

7. Exercise

5

18/11/2010 TDD4ES

Content: session 2 cont’d

3. Remote testing 1. Remoting

2. Remote testing in practice

3. Pros-cons

4. Do it yourself

5. Exercise

6

Workshop TDD4ES

18/11/2010

EP 4

18/11/2010 TDD4ES

1 Test on target

7

Tests

1. 3-Tier TDD

2. Hardware-aware TDD

3. Hardware-specific TDD

4. Pros-cons

5. Do it yourself

6. Exercise

7. Mocking

18/11/2010 TDD4ES

1.1 3-Tier TDD

8

Hardware aware

Hardware specific

• Relative simple unit & integration tests

• Unit test isolation (with stubs)

• Automation of hardware events

Workshop TDD4ES

18/11/2010

EP 5

18/11/2010 TDD4ES

1.2 Hardware-aware TDD

9

1. Identify hardware-aware functionality

2. Write test first

1. Select test values

2. Add stub replacing driver input

3. Write assertions

3. Add minimal implementation

4. Pass test

5. Refactor

18/11/2010 TDD4ES

1.3 Hardware-specific TDD

10

• Follow hardware-aware process

– Writing test-first can be time consuming

– Leads to brittle tests

OR

• Write driver first

Then encapsulate driver behavior with tests

– Treats drivers as third party code

– Changes are quickly notified

Workshop TDD4ES

18/11/2010

EP 6

18/11/2010 TDD4ES

1.4 Pros-cons

11

Advantages

• No cross-compiling

Disadvantages

• Isolating hardware behavior typically enlarges test cases (stubs)

- Processing power

- Memory footprint

• Slow development cycle

- Frequent flash programming

- …

18/11/2010 TDD4ES

1.5 Do it yourself

12

DS1820 - sign - magnitude

+ read() + getTemperature(): int

temperature sensor driver

hardware dependency

DS1820 # sign # magnitude

+ read() + getTemperature(): int

minimal changes quick & dirty

DS1820Mock + read() + setSign(int) + setMagnitude(int)

stub implementation

Workshop TDD4ES

18/11/2010

EP 7

18/11/2010 TDD4ES

1.5 Do it yourself

13

DS1820 + getState() + getTemperature(): int

state pattern: a bit more work & memory

hardware dependency is replaced by a state object

DS1820State + read()

DS1820StateMock + read() + setSign(int) + setMagnitude(int)

DS1820StateReal + read()

18/11/2010 TDD4ES

1.5 Do it yourself

1. Create a new mock class

2. Adapt the interface

3. Implement the mock

4. Write test for class with mock

14

Workshop TDD4ES

18/11/2010

EP 8

18/11/2010 TDD4ES

1.6 Exercise

15

04_TDDonTarget_TempMockStart

Requirements • Button class (in the drivers folder)

• Test the getState method

• Mock the hardware dependent method

Goals • Isolating hardware dependent functionality

• Test automation

18/11/2010 TDD4ES

1.6 Exercise

16

Button # getCurrentState

+ getState : ButtonState

Button driver hardware

dependency

Button # getCurrentState

minimal changes quick & dirty

ButtonMock + getCurrentState + setCurrentState

stub implementation

ButtonStates: RELEASED, PRESSED, REPEAT

Workshop TDD4ES

18/11/2010

EP 9

18/11/2010 TDD4ES

1.7 Mocking

• Software stub replacing the hardware

• Isolates the unit under test

• Mock object -> also validates interaction with the stub

1. Polymorphism

2. Preprocessor directives

3. Function pointers

4. Link-time polymorphism

17

18/11/2010 TDD4ES

1.7 Mocking in C++

1. Polymorphism

Advantages

• Minimal changes to source code

• No code duplication

Disadvantages

• Introduces overhead in production code

18

DS1820 # sign # magnitude

+ read() + getTemperature(): int

DS1820Mock + read() + setSign(int) + setMagnitude(int)

Workshop TDD4ES

18/11/2010

EP 10

18/11/2010 TDD4ES

2. Preprocessor directives

19

1.7 Mocking in C

Conditional compiling per file Conditional compiling per project

Disadvantage • Code duplication ->

unscalable

#ifdef TESTONHOST

#include “mockdriver.c”

#endif

#ifndef TESTONHOST

#include “realdriver.c”

#endif

#ifdef TESTONHOST

#define functionUT(arg1, arg2)

#endif

/* code containing function to be

mocked */

#ifndef TESTONHOST

#include “unittestframework.h”

int main() {

/* run unit tests & report*/

}

#endif

18/11/2010 TDD4ES

1.7 Mocking in C

3. Function pointers

Disadvantages

• Introduces overhead in production code

• Less scalable than polymorphism

20

/* list of functionsToMock with real or

mock address */

struct functionsToMock

{

void (*function1)(arg1);

}

extern struct functionsToMock FTM;

(*FTM.function1)(arg1);

Workshop TDD4ES

18/11/2010

EP 11

18/11/2010 TDD4ES

4. Link-time polymorphism

• Own responsibility to present only one implementation to the linker

21

1.7 Mocking in C

/* Function.h*/

void FunctionName(void);

/* Function.c*/

void FunctionName(void)

{

/*real implementation*/

}

/* FunctionMock.c*/

void FunctionName(void)

{

/*fake implementation*/

}

18/11/2010 TDD4ES

1.7 Mocking in C

4. Link-time polymorphism

Advantages

• No changes to source code

• No overhead in production code

Disadvantages

• Separate build configurations

• Manual selection of build

Demonstration with Eclipse

22

Scripting or IDE

Workshop TDD4ES

18/11/2010

EP 12

18/11/2010 TDD4ES

2 Test on host

23

1. Test on host in practice

2. 3-tier TDD

3. Mock verification

4. Embedded TDD cycle

5. Pros-cons

6. Do it yourself

7. Exercise

18/11/2010 TDD4ES

2.1 Test on host in practice

24

Tests

Program

Test Framework

Interfaces

Mock driverReal driver

Workshop TDD4ES

18/11/2010

EP 13

18/11/2010 TDD4ES

2.2 3-Tier TDD

25

Hardware aware

Hardware specific

• Relative simple unit & integration tests

• Mock hardware drivers

• Mock hardware interaction (per case ROI analysis)

OR

• Encapsulate layer with tests

18/11/2010 TDD4ES

2.3 Mock verification

26

1. Develop test on host

– Mock hardware

– Correspondence with driver on target

2. Migrate test to target

– Mock is replaced with real hardware driver

Test passes: mock & implementation = OK

OR

Test fails: mock & implementation != OK

Workshop TDD4ES

18/11/2010

EP 14

18/11/2010 TDD4ES

2.4 Embedded TDD cycle

27

1. Red bar 2. Green bar 3. Refactor

1. Compile for target

2. Fix

1. Unit tests in target

2. Fix

1. Manual tests

2. Fix

+/- 10 minutes

Host Target

Few hours Few days Daily

© J. Grenning, 2004, Progress before hardware

18/11/2010 TDD4ES

2.5 Pros-cons

28

Advantages

• Fast programming cycle

• Hardware does not need to be available

• Migration supported by test suite

Disadvantages

• Verification on target needed

– Cross-compilation issues

– Mock driver verification

Workshop TDD4ES

18/11/2010

EP 15

18/11/2010 TDD4ES

2.6 Do it yourself

29

1. Develop on host

1. Write a test for the driver

2. Define an interface

3. Implement a mockdriver

4. Use the mockdriver in the program

5. Write a test for program functionality

6. Implement program functionality

7. Run tests on host

2. Verify on target

1. Run tests on target

18/11/2010 TDD4ES

2.7 Exercise

30

05_TDDonHost

Requirements • Test the Button class on host

• Mock the Button class

• Verify on target

• Test the Button in the Program on host

• Verify on target

Goals • TDD on host

• Driver verification

Workshop TDD4ES

18/11/2010

EP 16

18/11/2010 TDD4ES

3 Remote testing

31

1. Remoting

2. Remote testing in practice

3. Pros-cons

4. Do it yourself

5. Exercise

18/11/2010 TDD4ES

3.1 Remoting

32

Background

• Inter Process Communication (IPC)

• Remote procedure call (RPC)

• Common Object Request Broker Architecture (CORBA)

Operation

• Tests / framework on host

• Program / drivers on target

• Management mechanism on both

Workshop TDD4ES

18/11/2010

EP 17

18/11/2010 TDD4ES

SKELETONSSKELETONS

STUBSSTUBS

3.2 Remote testing in practice

33

target host

Program Drivers Tests

Test Framework

TDD

BROKER COMMUNICATION CHANNEL BROKER

STUBSSKELETONS

Architecture

18/11/2010 TDD4ES

3.2 Remote testing in practice

34

Operation

target host

C L A S S

T E S T S

B R O K E R

init broker, init runner

init broker init

setup

run

tear-down

BROKER

LIST

ID Class

Stub

ID

Skeleton

OBJ

Workshop TDD4ES

18/11/2010

EP 18

18/11/2010 TDD4ES

3.3 Pros-cons

35

Advantages

• Scalable alternative for test on target (ROI)

• Fast prototyping

– Stable driver base on target

– On host development of higher level tiers

Disadvantages

• Remote testing system

– Development

– Overhead: memory / processing power

18/11/2010 TDD4ES

3.4 Do it yourself

36

• Hello test framework: (MSUnit)

Visual Studio Unit Testing Framework

using namespace Microsoft::VisualStudio::TestTools::UnitTesting;

[TestClass]

public ref class UnitTest1

{

[TestMethod]

public void MyTest()

{

Assert::IsTrue(true);

}

}

Workshop TDD4ES

18/11/2010

EP 19

18/11/2010 TDD4ES

Assert::IsTrue(true); //succeeds

Assert::AreEqual(1, 2); //fails

Assert::AreEqual(1.25, 2.35, 0.1); //fails

Assert::IsNotNull(object); //succeeds

37

• Grouping

• Setup & Teardown

• Assertions

3.4 MSUnit

[TestClass]

public ref class UnitTest1{}

[ClassInitialize()]

static void MyClassInitialize

(TestContext^ testContext) {/*setup*/};

[TestInitialize()]

void MyTestInitialize() {/*teardown*/};

18/11/2010 TDD4ES 38

3.4 Managed runtime

O/S

.Net Runtime GC

C++/CLI • ref class • ^ heap pointer • gcnew • IDisposable • %

C++ • class • * stack pointer • new • delete • &

Managed Unmanaged

ref class Person{}

Person ^p = gcnew Person();

class Person{}

Person *p = new Person();

Workshop TDD4ES

18/11/2010

EP 20

18/11/2010 TDD4ES

3.4 Do it yourself

39

06_RemoteTesting_DiY • Write new test on host • Run test on target • Review results on host

18/11/2010 TDD4ES

3.5 Exercise

40

06_RemoteTesting

Requirements • Testing target implementation on host

• Led test

• DS1820 test (environment temperature)

Goals • TDD with remote testing

• Testing hardware aware functionality

• Testing hardware independent functionality

Workshop TDD4ES

18/11/2010

EP 21

18/11/2010 TDD4ES

3.5 Exercise

41

06_RemoteTesting

LED + on + off + get : int + set(int)

DS1820 + getTemperature : int + double getTemperature : double

• Led test • DS1820 test (environment temperature)

18/11/2010 TDD4ES

TDD4ES overview

42

Test on target

Test on host

Remote testing

1. Limited number of tests

2. Minimalistic framework

3. Program code

1. Unlimited number of tests

2. Full option framework

3. Program code with hardware virtualization

1. Unlimited number of tests

2. Full option framework with remote testing functionality

3. Program code with remote testing functionality

Workshop TDD4ES

18/11/2010

EP 22

18/11/2010 TDD4ES

TDD4ES overview

43

Memory

Dependency

Slow

Test on host Test on target Remote testing

+++ Test & Program

on host

+++ Real drivers

+++ Real drivers

+++ Development limited to host

+/- Program &

Broker on target

--- Test & Program

on target

--- Only virtual

drivers

--- Frequent flash programming

- Frequent flash programming*

18/11/2010 TDD4ES

Literature

• K. Beck. Test-Driven Development: By Example. Addison-Wesley, 2003.

• K. Beck and C. Andres. Extreme Programming Explained: Embrace Change (2nd ed.). Addison-Wesley, 2004.

• B.W. Boehm. Software Engineering Economics (Prentice-Hall Advances in Computing Science & Technology Series). Prentice Hall PTR, October 1981.

• J. Boydens, P. Cordemans, and E. Steegmans. Test-Driven Development of Embedded Software. In European Conference on the Use of Modern Information and Communication Technologies, 2010.

• E. Dijkstra. The humble programmer. Commun. ACM, 15(10):859866, 1972.

• M. Feathers. Working Effectively with Legacy Code. Prentice Hall PTR, 2005.

• M. Fowler. UML Distilled (3rd ed.). Addison-Wesley, 2004.

• M. Fowler. Refactoring: Improving the design of existing code. Addison-Wesley, 1999.

• E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.

• J. Grenning. Progress before hardware. Agile Alliance Newsletter, 4:74_79, 2004.

44

Workshop TDD4ES

18/11/2010

EP 23

18/11/2010 TDD4ES

Literature continued

• M. Karlesky, W. Bereza, and C. Erickson. Effective test driven development for embedded software. In IEEE 2006 Electro/Information Technology Conference, 2006.

• R. Koss, and J. Langr. Test-Driven Development in C. In C/C++ Users Journal, 2002.

• M. Müller and F. Padberg. About the return on investment of test-driven development. In International Workshop on Economics-Driven Software Engineering Research EDSER-4, 2003.

• R. Osherove. The Art of Unit Testing. Manning, 2009.

45

18/11/2010 TDD4ES

Thank you

Follow-up e-mail

– Publish handouts & code

– List of participants

– Evaluation form

Follow project on http://ep.khbo.be/TDD4ES

46