Date post: | 17-Jan-2018 |
Category: |
Documents |
Upload: | doreen-owens |
View: | 213 times |
Download: | 0 times |
PROGRAMMING
TESTING201300071-1B MODULE 2: SOFTWARE SYSTEMS
22 NOVEMBER 2013
Week 1• Values, conditions• Classes, objects
Week 2• Specifications• Testing
Week 3• Abstraction,
inheritance
Week 4• Interfaces,
abstract classes• Arrays
Week 5• Collections• Generic structures
Week 6• Exceptions• Stream I/O• GUIs
Week 7• Concurrency• Networking
Week 8• Security
Week 9/10Project
Software Systems - Programming 2
OVERVIEW PROGRAMMING LINE
System testing & unit testing Test plan Test framework Test coverage
Nino & Hosch: Chapter 6Tool support: Emma (EclEmma for Eclipse)Self study: Nino & Hosch: Chapters 7 & 8
Software Systems - Programming 3
CONTENTS
Software Systems - Programming
System testing (functional testing) Complete system, final configuration Test system behavior, e.g., use cases Typically performed by testing team
Unit testing Class, method, etc. in isolation Test unit’s behavior Typically performed by developer of unit
Further tests: Performance , Stress , Platform , Usability∼ ∼ ∼ ∼ Out of scope for this course
4
OVERVIEW: TESTING TERMINOLOGY
Test whether the system or unit Behaves as intended When used in the specified way
Plan: List of execution scenarios Describe how to perform scenario Describe expected outcome
Software Systems - Programming 5
TEST PLAN
Use cases as execution scenarios, consider Alternative flows in use cases Different values used in use cases Additional requirements
Treat system as “black box” Internal structure is unknown Validation by observing behavior
Software Systems - Programming 6
TEST PLAN: SYSTEM TESTING
One test plan per class or method, can also be Per group of logically related methods Per group of classes in the same hierarchy
Consider method contracts, test that The postcondition of methods holds When the precondition is satisfied
Treat method as “white box” Internal structure is known E.g., consider conditionals Validation: by inspecting method result and/or instance/class variables
Software Systems - Programming 7
TEST PLAN: UNIT TESTING
Lecture will further focus on unit testing.
For the project, you need to perform both, system
and unit tests!
Identify relevant cases
Example: Lock Four cases
Attempt to open in open state Attempt to open in closed state Attempt to close in open state Attempt to close in closed state
Actually many more situations: Do we have to test for all possible values? 1000 possibilities for lockCode 1000 possibilities for code passed to enterDigits
→ isOpen()→ isOpen() <==> code correct→ not isOpen()→ not isOpen()
Software Systems - Programming 8
TEST CASES
Lock-int lockCode+boolean isOpen()+void close()+void enterDigits(int code)
> 2,000,000combinations
Do we have to test for all possible values? NO! Pick representative values
What is representative? Group values for which the unit under test behaves the same Pick samples from this group
Random value Boundaries (e.g,. maximum and minimum)
E.g,. for the Lock: pick 0, 123, 999
Software Systems - Programming 9
TEST CASES
Developers are like to make mistakesat boundaries
So called off-by-one errors
Example: ModuloCounter Specification
Count from 0 up to 9 When increasing with a value of 9,
reset to 0 instead Test
Invoke increase: zero times, five times, nine times, ten times
Software Systems - Programming 10
BOUNDARY CASES
ModuloCounter-int value+int getValue()+void increase()
this.value = this.value + 1;if (value >= 9) { this.value = 0:}
off-by-one error
Primarily test for correct behaviour when input is within specified bounds
Sometimes we want to test that nothing bad happens when input is wrong E.g., data entered through the user interface Test robustness of software
Can only be tested if the system behavior for wrong input is specified E.g., usage of default values instead of wrong input
Beware: documenting behaviour outside specified precondition encourages to ignore preconditions! Specification of exceptional behaviour: see week 6
Software Systems - Programming 11
TESTING ILLEGAL DATA
One class per test plan Implement each test case as method Test method uses unit under test and asserts correct behaviour
Compare actual outcome with expected outcome Often test cases share preparatory tasks, e.g., creating instance of class
under test Extract shared code into setUp() method setUp() method executed before each test method Establishes defined initial situation
Software Systems - Programming 12
COMMON STRUCTURE OF UNIT TEST
JUnit (www.junit.org) Provides framework for writing and running unit tests Uses Java features not introduced in this course
Provided: simplified version of Junit Main class (Test) Test class per class-under-test
Main class is executable (has main method) Instantiates and test classes and invokes their method runTest()
Test classes implement test plan Method runTest() must: invoke each test method invoke setUp() before each test method
Software Systems - Programming 13
TEST FRAMEWORK
Software Systems - Programming 14
If my code passes all tests, I’m done!
Is this correct?
How do I assure a good quality of my tests? Systematically develop tests based on contracts Review tests Measure coverage Other techniques, not covered in this course
QUALITY OF TESTS
Statement Each statement executed at least once
Branch Each branch traversed (and every entry point taken) at least
once
Path All program paths traversed at least once
Software Systems - Programming 15
TYPES OF COVERAGE
100% coverage difficult to achieve Especially for branch or path coverage, too many test cases would be
needed Synthetic code generated by compiler Dead code (should be eliminated when discovered) Code handling exceptional situations
High statement coverage generally possible (e.g., > 80%)
Software Systems - Programming 16
GOALS FOR CODE COVERAGE
Open-source tool Supports class, method, and line coverage.
“Fractional” line coverage supported, but not branch coverage. Eclipse plugin EclEmma also available
http://www.eclemma.org
Software Systems - Programming 17
EMMA
Software Systems - Programming 18
EMMA COVERAGE REPORT
Software Systems - Programming 19
EMMA SOURCE CODE ANNOTATIONS
Software Systems - Programming 20
FRACTIONAL LINE COVERAGE
Only partof conditionalexecuted
Loop incrementnot executed
Java compiler may emit multiple copies of code, e.g., for instance field initializer Emma expects
coverage of all copies
Software Systems - Programming 21
ANOMALIES IN REPORTED COVERAGE
Java compiler may generate default code, e.g., for missing constructors Emma also reports
coverage of such code
Iterative process Perform unit tests always after functionality is complete Test from the beginning of the implementation Extending implementation requires extending test plan Test results lead to further coding Test coverage results lead to extending test plan
Philosophies “Code a little, test a little” “Test a little, code a little” (test-driven development, test first)
Software Systems - Programming 22
TESTING / DEVELOPMENT PROCESS
Test plan System tests based on use cases Unit tests based on method contracts
Test representatives and boundary cases Confirm quality of your tests by measuring their code coverage
Software Systems - Programming 23
MAIN POINTS