Date post: | 18-Dec-2015 |
Category: |
Documents |
View: | 213 times |
Download: | 0 times |
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
1
Automated Testing Environment
Concepts required for testing embedded systems adopted in this course
(quizzes, assignments and laboratories)
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
2 / 28
To be tackled today
Why test, and what kinds of tests are there? Difference between
TLD – Test Last Development TDD – Test Driven Development
(Test First Development) How do you add tests? More details on the testing process
Other kinds of available tests – time and access
Design of custom TESTs and ASSERTS
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
3 / 28
Why test?
Unit test I AM CURRENTLY BUILDING A NEW FUNCTION –
does it work the way I expect? Testing my and my partner’s understanding of the
Design WE ARE DESIGNING A NEW PRODUCT
-- If we can’t explain (to ourselves via a test) the product ideas – how do we know when product is working?
Regression testing MY PARTNER ADDED A NEW FEATURE to a product –
how do I know it did not “break something” I wrote? System testing
PROOF TO THE CUSTOMER that the product (Lab. or assignment) works the way that was requested.
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
4 / 28
What are the problems with this sort of test?
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
5 / 28
VisualDSP development environment(IDDE)
DebuggingInterface
Processor
Peripherals
BF533 Evaluation Board
VisualDSP++
program
Debugging Interface
HPPCIce
Lab. station (ICT318 / 320)
Each time we want to send a message (via printf( ) or cout) we must “stop” the processor from doing what we want it to do, and then send messages over the interface – Slow, plus the testing can impact on peripheral performance.
E.g. no sound while printing in Assignment 2 and Lab. 1
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
6 / 28
More issues with this sort of test!
Tests are mixed up “with production code” Must remove tests before release to customer
Unreliable – may result in test messages at wrong time Difficult to “add” and “remove tests” Can’t be automated – therefore unlikely to have tests
run often during development – leads to unreliable product
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
7 / 28
E-TDD Tool
E-TDD – embedded test driven development Build-the-tests-first ideas for product development has
been made popular by the “Agile” approach Requires an automated testing approach
In this class, we are going to use the E-TDD tool for its automated behaviour Will use in both “test last” development (what most
people currently do) and in “test first” development I (customer) will provide tests so that you can check
that you have developed the “right” stuff at the “high” level
You will build additional tests to check your own code at the “low level”
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
8 / 28
Test Environment
Can download “LabTestEnvironment.zip” file from the web
LabX directory iswhere you keepyour code
LabXTests is whereyou keep the testsfor LabX
E-TDDIncludes directory contains “everything” needed to make test environment work
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
9 / 28
Build a new VDSP project “Assignment1Test” and place in “AssignmentTests” directory
Now add certain standard files to the project – same set of files for all assignments and laboratory
All found in E-TDDIncludes directory
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
10 / 28
Now add your Assignment1 subroutines files to the Assignment1Test project
Don’t move or copy the files into the test directory – just “add” them to the project
Note the use of “Assignment1.h” which has the prototypes for all “C++” and “assembly code” functions used (from Assignment dir.)
This files are “added” NOT copiedfrom the Assignment1 directory
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
11 / 28
Now we need to develop and add “Testmain.cpp”
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
12 / 28
Now we need to develop and add Assignment1Tests.cpp”
1
2
3
4
5
6
7
8
9
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
13 / 28
Tests have a standard format
1
3
4
8
9
TESTGROUP NAME
TESTGROUP NAME, TEST TYPE
TEST TYPETEST NAME
Add further tests at this location
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
14 / 28
Tests then need to be customized
1
2
3
4
5
6
7
8
9
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
15 / 28
Test Components
The ACTUAL Test (This is a TEST MACRO)CHECK(result1 == result2) Would provided better test messages if we
“self-documented the code”CHECK(result_CPP == result_ASM);
Test CONTROL statements (ALL MACROS) TEST_CONNECT( test_name) LINK_TEST( test_name, test_type) TEST_LEVEL( which_test_level)
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
16 / 28
Stages to “automating” the tests
1) Activate the E-TDD Gui (E-TDDIncludes directory)
This causes the GUI to appear, and adds new “menus” to VisualDSP
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
17 / 28
Stages to “automating” the tests
2) Select TDD Connection | Refresh Connection List
This automatically builds the Force Connect Test file
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
18 / 28
Now “build and run” the testsQuick test failure
visualization
Detailed test failure information
Clicking on the “detailed test info”automatically takes you to the “test” that failed
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
19 / 28
Test information modes arecontrolled via “ActivateTests.cpp” Show Failures only
Show Failures and Successes
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
20 / 28
Add a separate test to show that result2(from the ASM code) is equal to 7
Add further tests at this location
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
21 / 28
Write the tests for “self-documenting” messages(What does “result1 == result2” mean if 200 tests present?)
HAVE
WANT
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
22 / 28
Write the “CPP” code to get to workbut “Write the ASM for speed”
FAIL
SUCCESS
SUCCESS
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
23 / 28
Did the ASM code “destroy” or even “use” the CPParray[ ] defined in CPPassignment1.cpp?
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
24 / 28
Available ASSERTS
LONG INT tests – 32 bit – typical for “C++” CHECK(condition) – Success if passes CHECK_EQUAL(expected, actual) EXPECTED_FALSE(condition) – Success if fails ARRAYS_EQUAL(expected, actual, length) CHECK_VALUES(expectedTemp, condition, actualTemp)
DOUBLES_EQUAL(expected, actual, threshold)
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
25 / 28
ASSERTS for embedded systems
Short int tests – 16 bit – typical for embedded systems SHORTS_EQUAL(expected, actual) SHORTS_CHECK(expected, condition, actual) USHORTS_EQUAL(expected, actual) USHORTS_CHECK(expected, condition, actual)
Timing of routine – Blackfin Specific MAXTIME_ASSERT(max_time, function) Useful utility
MEASURE_EXECUTION_TIME(timetaken, function)
Examples available in the tests associated with Lab. 1
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
26 / 28
You can add your own ASSERTSASSERT FORMAT#define CHECK_VALUES(expectedTemp, condition, actualTemp)\if (!((expectedTemp) condition (actualTemp))) \{\
FAILURE1(#expectedTemp##" !"#condition##" "#actualTemp);\}\else {\ SUCCESS1(#expectedTemp##”""#condition##" "#actualTemp);\}
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
27 / 28
Practice – Assert Design
Design an assert that will determine whether the lower 8 bits of a 32-bit variable is identical to the lower 8 bits of a second 32-bitvariable.
The two variables should remain unchanged by the assert (since they may be reused later)
The upper 24 bits of each variable MUST be ignored in the assert
Does it matter if the variables were defined in the .data section of an assembly code routine or in an C++ code routine?
04/18/23 Concept of Test Driven Development applied to Embedded Systems M. Smith University of Calgary, Canada
28 / 28
Handled today – some easy quiz and midterm questions Why test, and what kinds of tests are there? Difference between
TLD – Test Last Development TDD – Test Driven Development
(Test First Development) How do you add tests for Assignment 1? More details on the testing process
Other kinds of available tests – time and access
Design of custom TESTs and ASSERTS