Date post: | 27-Apr-2018 |
Category: |
Documents |
Upload: | truongtuong |
View: | 216 times |
Download: | 1 times |
Testing in the 21st Century Software defines behavior
– network routers, finance, switching networks, other infrastructure
Today’s software market : – is much bigger
– is more competitive
– has more users
Embedded Control Applications – airplanes, air traffic control
– spaceships
– watches
– ovens
– remote controllers
Agile processes put increased pressure on testers – Programmers must unit test – with no training or education!
– Tests are key to functional requirements – but who builds those tests ? 2
– PDAs
– memory seats
– DVD players
– garage door openers
– cell phones
Industry is going through a revolution in what testing means to
the success of software products
4
Software Fault : A static defect in the software
Software Failure : External, incorrect behavior with respect to the requirements or other description of the expected behavior
Software Error : An incorrect internal state that is the manifestation of some fault
Faults in software are equivalent to design mistakes in hardware.
Software does not degrade.
Software Faults, Errors & Failures
A Concrete Example
5
public static int numZero (int [ ] arr)
{ // Effects: If arr is null throw NullPointerException
// else return the number of occurrences of 0 in arr
int count = 0;
for (int i = 1; i < arr.length; i++)
{
if (arr [ i ] == 0)
{
count++;
}
}
return count;
}
Fault: Should start
searching at 0, not 1
Test 1
[ 2, 7, 0 ]
Expected: 1
Actual: 1
Test 2
[ 0, 2, 7 ]
Expected: 1
Actual: 0
Error: i is 1, not 0, on
the first iteration
Failure: none
Error: i is 1, not 0
Error propagates to the variable count
Failure: count is 0 at the return statement
Spectacular Software Failures
6
Intel’s Pentium FDIV fault : Public relations nightmare
THERAC-25 radiation machine : Poor testing of safety-critical software can cost lives : 3 patients were killed
Mars Polar
Lander crash
site?
THERAC-25 design
Ariane 5:
exception-handling
bug : forced self
destruct on maiden
flight (64-bit to 16-bit
conversion: about
370 million $ lost)
We need our software to be dependable Testing is one way to assess dependability
NASA’s Mars lander: September 1999, crashed due to a units integration fault
Ariane 5 explosion : Very expensive
Northeast Blackout of 2003
7
Affected 10 million
people in Ontario,
Canada
Affected 40 million
people in 8 US
states
Financial losses of
$6 Billion USD
508 generating
units and 256
power plants shut
down
The alarm system in the energy management system failed
due to a software error and operators were not informed of
the power overload in the system
Testing in the 21st Century More safety critical, real-time software
Embedded software is ubiquitous … check your pockets
Enterprise applications means bigger programs, more users
Paradoxically, free software increases our expectations !
Security is now all about software faults
– Secure software is reliable software
The web offers a new deployment platform
– Very competitive and very available to more users
– Web apps are distributed
– Web apps must be highly reliable
8
Industry desperately needs our inventions !
What Does This Mean?
9
Software testing is getting more
important
What are we trying to do when we test ?
What are our goals ?
Validation & Verification (IEEE)
Validation : The process of evaluating software at the end of software development to ensure compliance with intended usage
Verification : The process of determining whether the products of a given phase of the software development process fulfill the requirements established during the previous phase
IV&V stands for “independent verification and validation”
10
11
Testing Goals Based on Test Process Maturity
Level 0 : There’s no difference between testing and debugging
Level 1 : The purpose of testing is to show correctness
Level 2 : The purpose of testing is to show that the software doesn’t work
Level 3 : The purpose of testing is not to prove anything specific, but to reduce the risk of using the software
Level 4 : Testing is a mental discipline that helps all IT professionals develop higher quality software
Where Are You?
12
Are you at level 0, 1, or 2 ?
Is your organization at work at level
0, 1, or 2 ?
Or 3?
We hope to teach you to become
“change agents” in your workplace …
Advocates for level 4 thinking
Tactical Goals : Why Each Test ?
13
Written test objectives and requirements must be documented
What are your planned coverage levels?
How much testing is enough?
Common objective – spend the budget … test until the ship-date …
– Sometimes called the “date criterion”
If you don’t know why you’re conducting each test, it won’t be very helpful
14
Cost of Not Testing
Testing is the most time consuming and expensive part of software development
Not testing is even more expensive
If we have too little testing effort early, the cost of testing increases
Planning for testing after development is prohibitively expensive
Poor Program Managers might say: “Testing is too expensive.”
Cost of Late Testing
15
60
50
40
30
20
10
0
Fault origin (%) Fault detection (%) Unit cost (X)
Software Engineering Institute; Carnegie Mellon University; Handbook CMU/SEI-96-HB-002
Assume $1000 unit cost, per fault, 100 faults
Summary: Why Do We Test Software ?
16
A tester’s goal is to eliminate faults
as early as possible
• Improve quality
• Reduce cost
• Preserve customer satisfaction
Complexity of Testing Software No other engineering field builds products as complicated
as software
The term correctness has no meaning
– Is a building correct?
– Is a car correct?
– Is a subway system correct?
Like other engineers, we must use abstraction to manage complexity
– This is the purpose of the model-driven test design process
– The “model” is an abstract structure
18
20
Testing & Debugging
Testing : Evaluating software by observing its execution
Test Failure : Execution of a test that results in a software failure
Debugging : The process of finding a fault given a failure
Not all inputs will “trigger” a fault into causing a
failure
21
Fault & Failure Model Three conditions necessary for a failure to be observed
1. Reachability : The location or locations in the program that contain the fault must be reached
2. Infection : The state of the program must be incorrect
3. Propagation : The infected state must cause some output or final state of the program to be incorrect
Software Testing Activities Test Engineer : An IT professional who is in charge of one
or more technical test activities – Designing test inputs
– Producing test values
– Running test scripts
– Analyzing results
– Reporting results to developers and managers
Test Manager : In charge of one or more test engineers – Sets test policies and processes
– Interacts with other managers on the project
– Otherwise supports the engineers
22
23
Traditional Testing Levels
Class A
method mA1()
method mA2()
Class B
method mB1()
method mB2()
main Class P Acceptance testing :
Is the software acceptable to the user?
Integration testing : Test how modules interact with each other
System testing : Test the overall functionality of the system
Module testing (developer testing) : Test each class, file, module, component
Unit testing (developer testing) : Test each unit (method) individually
This view obscures underlying
similarities
24
Object-Oriented Testing Levels
Class A
method mA1()
method mA2()
Class B
method mB1()
method mB2()
Intra-class testing : Test an entire class as sequences of calls
Inter-class testing : Test multiple classes together
Inter-method testing : Test pairs of methods in the same class
Intra-method testing : Test each method individually
Coverage Criteria Even small programs have too many inputs to fully test
them all
– private static double computeAverage (int A, int B, int C)
– On a 32-bit machine, each variable has over 4 billion possible values
– Over 80 octillion possible tests!!
– Input space might as well be infinite
Testers search a huge input space
– Trying to find the fewest inputs that will find the most problems
Coverage criteria give structured, practical ways to search the input space
– Search the input space thoroughly
– Not much overlap in the tests
25
Advantages of Coverage Criteria Maximize the “bang for the buck”
Provide traceability from software artifacts to tests
– Source, requirements, design models, …
Make regression testing easier
Gives testers a “stopping rule” … when testing is finished
Can be well supported with powerful tools
26
Test Requirements and Criteria Test Criterion : A collection of rules and a process that
define test requirements
Cover every statement
Cover every functional requirement
Test Requirements : Specific things that must be satisfied or covered during testing
– Each statement is a test requirement
– Each functional requirement is a test requirement
27
Testing researchers have defined dozens of criteria, but
they are all really just a few criteria on four types of
structures …
1. Graphs
2. Logic expressions
3. Input domains
4. Syntax descriptions
Types of Test Activities Testing can be broken up into four general types of
activities
1. Test Design
2. Test Automation
3. Test Execution
4. Test Evaluation
Each type of activity requires different skills, background knowledge, education and training
No reasonable software development organization uses the same people for requirements, design, implementation, integration and configuration control
28
Why do test organizations still use the same people for all
four test activities??
This clearly wastes resources
1.a) Criteria-based
1.b) Human-based
Other Activities Test management : Sets policy, organizes team, interfaces
with development, chooses criteria, decides how much automation is needed, …
Test maintenance : Save tests for reuse as software evolves
– Requires cooperation of test designers and automators
– Deciding when to trim the test suite is partly policy and partly technical – and in general, very hard !
– Tests should be put in configuration control
Test documentation : All parties participate
– Each test must document “why” – criterion and test requirement satisfied or a rationale for human-designed tests
– Ensure traceability throughout the process
– Keep documentation in the automated tests
29
Using MDTD in Practice
This approach lets one test designer do the math
Then traditional testers and programmers can do their parts
– Find values
– Automate the tests
– Run the tests
– Evaluate the tests
Just like in traditional engineering … an engineer constructs models with calculus, then gives direction to carpenters, electricians, technicians, …
30
Test designers become the technical
experts
Model-Driven Test Design
31
software artifact
model / structure
test requirements
refined requirements /
test specs
input values
test cases
test scripts
test results
pass / fail
IMPLEMENTATION ABSTRACTION
LEVEL
DESIGN ABSTRACTION
LEVEL
test requirements
Model-Driven Test Design – Steps
32
software artifact
model / structure
test requirements
refined requirements /
test specs
input values
test cases
test scripts
test results
pass / fail
IMPLEMENTATION ABSTRACTION
LEVEL
DESIGN ABSTRACTION
LEVEL
analysis
criterion refine
generate
prefix
postfix
expected
automate execute evaluate
test requirements
domain
analysis
Model-Driven Test Design–Activities
33
software artifact
model / structure
test requirements
refined requirements /
test specs
input values
test cases
test scripts
test results
pass / fail
IMPLEMENTATION ABSTRACTION
LEVEL
DESIGN ABSTRACTION
LEVEL
Test Design
Test
Execution Test
Evaluation
Raising our abstraction level makes test design MUCH easier
Small Illustrative Example
34
Software Artifact : Java Method /**
* Return index of node n at the
* first position it appears,
* -1 if it is not present
*/
public int indexOf (Node n)
{
for (int i=0; i < path.size(); i++)
if (path.get(i).equals(n))
return i;
return -1; }
4 5
3
2
1 i = 0
i < path.size()
if
return i return -1
Control Flow Graph
Example (2)
35
4 5
3
2
1
Graph Abstract version
Edges
1 2
2 3
3 2
3 4
2 5
Initial Node: 1
Final Nodes: 4, 5
6 requirements for
Edge-Pair Coverage
1. [1, 2, 3]
2. [1, 2, 5]
3. [2, 3, 4]
4. [2, 3, 2]
5. [3, 2, 3]
6. [3, 2, 5]
Test Paths
[1, 2, 5]
[1, 2, 3, 2, 5]
[1, 2, 3, 2, 3, 4]
Find values …
What is Test Automation?
Reduces cost
Reduces human error
Reduces variance in test quality from different individuals
Significantly reduces the cost of regression testing
37
The use of software to control the execution of tests, the
comparison of actual outcomes to predicted outcomes, the
setting up of test preconditions, and other test control and
test reporting functions
Components of a Test Case A test case is a multipart artifact with a definite structure
Test case values
Expected results
38
The result that will be produced when executing the test if
the program satisfies it intended behavior
The values that directly satisfy one test requirement
39
What is JUnit?
Open source Java testing framework used to write and run repeatable automated tests
JUnit is open source (junit.org)
A structure for writing test drivers
JUnit features include:
– Assertions for testing expected results
– Test features for sharing common test data
– Test suites for easily organizing and running tests
– Graphical and textual test runners
JUnit is widely used in industry
JUnit can be used as stand alone Java programs (from the command line) or within an IDE such as Eclipse
40
Writing Tests for JUnit
Need to use the methods of the junit.framework.assert class
– javadoc gives a complete description of its capabilities
Each test method checks a condition (assertion) and reports to the test runner whether the test failed or succeeded
The test runner uses the result to report to the user (in command line mode) or update the display (in an IDE)
All of the methods return void
A few representative methods of junit.framework.assert
– assertTrue (boolean)
– assertTrue (String, boolean)
– fail (String)
41
JUnit Test Fixtures
A test fixture is the state of the test
– Objects and variables that are used by more than one test
– Initializations (prefix values)
– Reset values (postfix values)
Different tests can use the objects without sharing the state
Objects used in test fixtures should be declared as instance variables
They should be initialized in a @Before method
Can be deallocated or reset in an @After method
42
Simple JUnit Example public class Calc
{
static public int add (int a, int b)
{
return a + b;
}
}
import org.junit.Test;
import static org.junit.Assert.*;
public class calcTest
{
@Test public void testAdd()
{
assertTrue (“Calc sum incorrect”,
5 == Calc.add (2, 3));
}
}
43
Testing the Min Class import java.util.*;
public class Min
{
/**
* Returns the mininum element in a list
* @param list Comparable list of elements to search
* @return the minimum element in the list
* @throws NullPointerException if list is null or
* if any list elements are null
* @throws ClassCastException if list elements are not mutually comparable
* @throws IllegalArgumentException if list is empty
*/
…
}
public static <T extends Comparable<? super T>> T min (List<? extends T> list)
{
if (list.size() == 0)
{
throw new IllegalArgumentException ("Min.min");
}
Iterator<? extends T> itr = list.iterator();
T result = itr.next();
if (result == null) throw new NullPointerException ("Min.min");
while (itr.hasNext())
{ // throws NPE, CCE as needed
T comp = itr.next();
if (comp.compareTo (result) < 0)
{
result = comp;
} }
return result;
} }
MinTest Class Standard imports
for all JUnit classes :
44
import static org.junit.Assert.*;
import org.junit.*;
import java.util.*;
Test fixture and pre-test setup method (prefix) :
Post test teardown method (postfix) :
private List<String> list; // Test fixture
// Set up - Called before every test method.
@Before
public void setUp()
{
list = new ArrayList<String>();
}
// Tear down - Called after every test method.
@After
public void tearDown()
{
list = null; // redundant in this example!
}
45
Min Test Cases: NullPointerException
@Test public void testForNullList()
{
list = null;
try {
Min.min (list);
} catch (NullPointerException e) {
return;
}
fail (“NullPointerException expected”);
}
@Test(expected = NullPointerException.class)
public void testForNullElement()
{
list.add (null);
list.add ("cat");
Min.min (list);
} This NullPointerException
test uses the fail assertion
This NullPointerException test
decorates the @Test
annotation with the class of the
exception
This NullPointerException
test catches an easily
overlooked special case
@Test(expected = NullPointerException.class)
public void testForSoloNullElement()
{
list.add (null);
Min.min (list);
}
46
Remaining Test Cases for Min @Test(expected = ClassCastException.class)
@SuppressWarnings("unchecked")
public void testMutuallyIncomparable()
{
List list = new ArrayList();
list.add ("cat");
list.add ("dog");
list.add (1);
Min.min (list);
}
@Test(expected = IllegalArgumentException.class)
public void testEmptyList()
{
Min.min(list);
}
Note that Java
generics don’t
prevent clients from
using raw types!
Special case: Testing for the
empty list
Finally! A couple of
“Happy Path” tests
@Test
public void testSingleElement()
{
list.add ("cat");
Object obj = Min.min (list);
assertTrue("Single Element List", obj.equals("cat"));
}
@Test
public void testDoubleElement()
{
list.add ("dog");
list.add ("cat");
Object obj = Min.min (list);
assertTrue ("Double Element List", obj.equals ("cat"));
}
47
Question: Where Does The Data Come From? Answer:
– All combinations of values from @DataPoint annotations where assume clause is true
– Four (of nine) combinations in this particular case
– Note: @DataPoint format is an array
@DataPoints
public static String[] string = {"ant", "bat", "cat"};
@DataPoints
public static Set[] sets = {
new HashSet (Arrays.asList ("ant", "bat")),
new HashSet (Arrays.asList (“bat", “cat", “dog“, “elk”)),
new HashSet (Arrays.asList (“Snap”, “Crackle”, “Pop"))
};
48
JUnit Theories Need BoilerPlate import org.junit.*;
import org.junit.runner.RunWith;
import static org.junit.Assert.*;
import static org.junit.Assume.*;
import org.junit.experimental.theories.DataPoint;
import org.junit.experimental.theories.DataPoints;
import org.junit.experimental.theories.Theories;
import org.junit.experimental.theories.Theory;
import java.util.*;
@RunWith (Theories.class)
public class SetTheoryTest
{
… // See Earlier Slides
}
49
Running from a Command Line
This is all we need to run JUnit in an IDE (like Eclipse)
We need a main() for command line execution …
50
AllTests import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import junit.framework.JUnit4TestAdapter;
// This section declares all of the test classes in the program.
@RunWith (Suite.class)
@Suite.SuiteClasses ({ StackTest.class }) // Add test classes here.
public class AllTests
{
// Execution begins in main(). This test class executes a
// test runner that tells the tester if any fail.
public static void main (String[] args)
{
junit.textui.TestRunner.run (suite());
}
// The suite() method helpfs when using JUnit 3 Test Runners or Ant.
public static junit.framework.Test suite()
{
return new JUnit4TestAdapter (AllTests.class);
}
}
51
How to Run Tests
JUnit provides test drivers
– Character-based test driver runs from the command line
– GUI-based test driver-junit.swingui.TestRunner
• Allows programmer to specify the test class to run
• Creates a “Run” button
If a test fails, JUnit gives the location of the failure and any exceptions that were thrown
52
Summary
The only way to make testing efficient as well as effective is to automate as much as possible
JUnit provides a very simple way to automate our unit tests
It is no “silver bullet” however … it does not solve the hard problem of testing :
What test values to use ? • This is test design … the purpose of test criteria
54
Changing Notions of Testing
Old view focused on testing at each software development phase as being very different from other phases
– Unit, module, integration, system …
New view is in terms of structures and criteria
– Graphs, logical expressions, syntax, input space
Test design is largely the same at each phase
– Creating the model is different
– Choosing values and automating the tests is different
55
New : Test Coverage Criteria
Test Requirements : Specific things that must be satisfied or covered during testing
Test Criterion : A collection of rules and a process that define test requirements
A tester’s job is simple : Define a model of the
software, then find ways
to cover it
Testing researchers have defined dozens of
criteria, but they are all really just a few criteria
on four types of structures …
Criteria Based on Structures
56
Structures : Four ways to model software
1. Graphs
2. Logical Expressions
3. Input Domain Characterization
4. Syntactic Structures
(not X or not Y) and A and B
if (x > y)
z = x - y;
else
z = 2 * x;
A: {0, 1, >1}
B: {600, 700, 800}
C: {swe, cs, isa, infs}
57
Old View : Black and White Boxes
Black-box testing : Derive tests from external descriptions of the software, including specifications, requirements, and design
White-box testing : Derive tests from the source code internals of the software, specifically including branches, individual conditions, and statements
Model-based testing : Derive tests from a model of the software (such as a UML diagram)
MDTD makes these distinctions less important.
The more general question is:
from what abstraction level do we derive tests?
Source of Structures These structures can be extracted from lots of software
artifacts
– Graphs can be extracted from UML use cases, finite state machines, source code, …
– Logical expressions can be extracted from decisions in program source, guards on transitions, conditionals in use cases, …
This is not the same as “model-based testing,” which derives tests from a model that describes some aspects of the system under test
– The model usually describes part of the behavior
– The source is usually not considered a model
58
59
1. Graph Coverage – Structural
6
5
3
2
1 7
4
Node (Statement)
Cover every node
• 12567
• 1343567
This graph may represent
• statements & branches
• methods & calls
• components & signals
• states and transitions
Edge (Branch)
Cover every edge
• 12567
• 1343567
• 1357
Path
Cover every path
• 12567
• 1257
• 13567
• 1357
• 1343567
• 134357 …
60
Defs & Uses Pairs
• (x, 1, (1,2)), (x, 1, (1,3))
• (y, 1, 4), (y, 1, 6)
• (a, 2, (5,6)), (a, 2, (5,7)),
(a, 3, (5,6)), (a, 3, (5,7)),
• (m, 2, 7), (m, 4, 7), (m, 6,
7)
1. Graph Coverage – Data Flow
6
5
3
2
1 7
4 This graph contains:
• defs: nodes & edges
where variables get values
• uses: nodes & edges
where values are accessed
def = {x, y}
def = {a , m}
def = {a}
def = {m}
def = {m}
use = {x}
use = {x}
use = {a}
use = {a}
use = {y}
use = {m}
use = {y}
All Defs
Every def used once
• 1, 2, 5, 6, 7
• 1, 2, 5, 7
• 1, 3, 4, 3, 5, 7
All Uses
Every def “reaches”
every use
• 1, 2, 5, 6, 7
• 1, 2, 5, 7
• 1, 3, 5, 6, 7
• 1, 3, 5, 7
• 1, 3, 4, 3, 5,7
61
1. Graph - FSM Example Memory Seats in a Lexus ES 300
Driver 1
Configuration
Driver 2
Configuration
[Ignition = off] | Button2
[Ignition = off] | Button1
Modified
Configuration
sideMirrors () [Ignition = on] |
lumbar () [Ignition = on] |
seatBottom () [Ignition = on] |
seatBack () [Ignition = on] |
New
Configuration
Driver 1
New
Configuration
Driver 2
[Ignition = on] | Reset AND Button1
[Ignition = on] | Reset AND Button2
Ignition = off
Ignition = off
(to Modified)
Guard (safety constraint) Trigger (input)
62
2. Logical Expressions
( (a > b) or G ) and (x < y)
Transitions
Software Specifications
Program Decision Statements Logical
Expressions
63
2. Logical Expressions
Predicate Coverage : Each predicate must be true and false
– ( (a>b) or G ) and (x < y) = True, False
Clause Coverage : Each clause must be true and false
– (a > b) = True, False
– G = True, False
– (x < y) = True, False
Combinatorial Coverage : Various combinations of clauses
– Active Clause Coverage: Each clause must determine the predicate’s result
( (a > b) or G ) and (x < y)
64
2. Logic – Active Clause Coverage
( (a > b) or G ) and (x < y)
1 T F T
2 F F T
duplicate 3 F T T
4 F F T
5 T T T
6 T T F
With these
values for G and
(x<y), (a>b)
determines the
value of the
predicate
65
3. Input Domain Characterization
Describe the input domain of the software
– Identify inputs, parameters, or other categorization
– Partition each input into finite sets of representative values
– Choose combinations of values
System level
– Number of students { 0, 1, >1 }
– Level of course { 600, 700, 800 }
– Major { swe, cs, isa, infs }
Unit level
– Parameters F (int X, int Y)
– Possible values X: { <0, 0, 1, 2, >2 }, Y : { 10, 20, 30 }
– Tests F (-5, 10), F (0, 20), F (1, 30), F (2, 10), F (5, 20)
66
4. Syntactic Structures Based on a grammar, or other syntactic definition
Primary example is mutation testing
1. Induce small changes to the program: mutants
2. Find tests that cause mutant programs to fail: killing mutants
3. Failure is defined as different output from the original program
4. Check the output of useful tests on the original program
Example program and mutants
if (x > y)
z = x - y;
else
z = 2 * x;
if (x > y)
if (x >= y)
z = x - y;
z = x + y;
z = x – m;
else
z = 2 * x;
67
Coverage Overview
Four Structures for Modeling Software
Graphs Logic Input Space Syntax
Use cases
Specs
Design
Source
Applied to
DNF Specs
FSMs Source
Applied to
Input
Models
Integ
Source
Applied to
68
Coverage
Infeasible test requirements : test requirements that cannot be satisfied
– No test case values exist that meet the test requirements
– Dead code
– Detection of infeasible test requirements is formally undecidable for most test criteria
Thus, 100% coverage is impossible in practice
Given a set of test requirements TR for coverage
criterion C, a test set T satisfies C coverage if and only
if for every test requirement tr in TR, there is at least
one test t in T such that t satisfies tr
69
Two Ways to Use Test Criteria
1. Directly generate test values to satisfy the criterion
– often assumed by the research community
– most obvious way to use criteria
– very hard without automated tools
2. Generate test values externally and measure against the criterion usually favored by industry
– sometimes misleading
– if tests do not reach 100% coverage, what does that mean?
Test criteria are sometimes called
metrics
70
Generators and Recognizers Generator : A procedure that automatically generates
values to satisfy a criterion
Recognizer : A procedure that decides whether a given set of test values satisfies a criterion
Both problems are provably undecidable for most criteria
It is possible to recognize whether test cases satisfy a criterion far more often than it is possible to generate tests that satisfy the criterion
Coverage analysis tools are quite plentiful
Criteria Summary
71
• Many companies still use “monkey testing” • A human sits at the keyboard, wiggles the mouse and
bangs the keyboard
• No automation
• Minimal training required
• Some companies automate human-designed tests
• But companies that use both automation and criteria-based testing
Save money
Find more faults
Build better software
Summary of Part 1’s New Ideas Why do we test – to reduce the risk of using the software
– faults, failures, the RIP model
– Test process maturity levels – level 4 is a mental discipline that improves the quality of the software
Model-Driven Test Design
– Four types of test activities – test design, automation, execution and evaluation
Test Automation
– Testability, observability and controllability, test automation frameworks
Criteria-based test design
– Four structures – test requirements and criteria
72
Earlier and better testing can empower the
test manager