Date post: | 03-Jan-2016 |
Category: |
Documents |
Upload: | lydia-evans |
View: | 219 times |
Download: | 2 times |
DR. HARIS MOURATIDIS02/11/08
Software Testing and Extreme Programming
1
© H. Mouratidis
© H. Mouratidis
2
What we do today?
Learning Objectives Provide an understanding of
Development and implementation of a testing strategy eXtreme Programming
Lecture Layout Testing
Concepts Testing strategies
Agile Development Extreme Programming
Testing
“Testing is the process of finding differences between the expected behaviour specified by system models and the observed behaviour of the implemented system” [Bruegge, 2004]
Another definition “Testing is a systematic attempt to find faults in a planned
way” Contrary to all other activities: find defects and reveal
problems It is impossible to completely test any nontrivial module or
any system Prohibitive in time and cost
“Testing can only show the presence of bugs, not their absence” (Dijkstra)
© H. Mouratidis
3
Some testing truths
Testing often viewed as dirty/easy work.To develop an effective test, one must have:
Detailed understanding of the system Knowledge of the testing techniques Skill to apply these techniques in an effective and efficient manner
Testing is done best by independent testers We often develop a certain mental attitude that the
program should work in a certain way when in fact it does not.
Programmers often stick to the data set that makes the program work "Don’t mess up my code!"
A program often does not work when tried by somebody else. Don't let this be the end-user.© H. Mouratidis
4
Testing concepts
Component Part (object, subsystem) of the system that can be
isolated for testing Fault (bug, defect)
Design or coding mistake that can cause abnormal component behaviour
Erroneous state A fault during the execution of a system
Failure Deviation between the specification and the
actual behaviour© H. Mouratidis
5
Testing concepts (II)
Test case Set of inputs and expected results that exercises
a component with the purpose of causing failures and detecting faults
Test driver Partial implementation of components that
depends on the tested component (calls the tested component)
Test stub Partial implementation of components on which
the tested component depends (are called by the tested component)
Correction A change to a component whose purpose is to
repair a fault© H. Mouratidis
6
Graphical Representation
© H. Mouratidis
7
Component
Test Driver
Test Stub
Depends on
Depends on
Test Case
Five attributes Name: name of the test case Location: full path name of executable Input: input data or commands Oracle: expected test results Log (output): output produced by the test
You have to specify also the associations!
© H. Mouratidis
8
Test Case Associations
© H. Mouratidis
9
Test Case A
Test Case E
Test Case W
Test Case D
Precede
As less associations as possible
Aggregation
Precede
Classification of test cases
© H. Mouratidis
10
Which aspect of system model is tested?
Blackbox test Focuses on the input/output of the component
Whitebox test Internal structure of the component
InputOutput
Testing technical activities
Component inspection Finds faults in individual components through
manual inspections of codeUsability testing
Differences between the system and the user’s expectations
Unit testing Finds faults by isolating individual components
using a test caseIntegration testing
Finds faults by integrating several components together
System testing Focuses on the complete system© H. Mouratidis
11
Component inspection
Find faults by reviewing source code in a formal meeting
Fagan’s inspection method Overview (Purpose and scope of component) Preparation
Reviewers become familiar with component code Inspection meeting
Identify the presence of faults Rework
Revise the component Follow-up
Check the revisionGood but Time consuming
Parna’s method Find faults during preparation© H. Mouratidis
12
Usability testing
Tests the user’s understanding of the system
Does not compare the system against specifications
Three types Scenario test
Users presented with visionary scenario of the system
Prototype test Users presented with a prototype version of the
system Product test
Users presented with a functional version of the system
Test objectives: identify useful features, compare user interaction styles, type of training information
© H. Mouratidis
13
Unit testing
Motivations Reduces complexity of overall test activities Makes it easier to identify/ correct faults Saves time, since each component can be tested
independently of the others
In principle all components should be tested!Not feasible!
© H. Mouratidis
14
Unit testing techniques
Equivalence testing (blackbox) Systems usually behave in similar ways for all
members of a class Test one member of the class If an object is supposed to accept a negative
number, testing one negative number is enough+ Minimises number of test cases- Do not explore combinations of test input data
Boundary testing Special case of equivalence testing Testing elements from the “edges”
© H. Mouratidis
15
Unit testing techniques (II)
Path testing (whitebox) By exercising all possible paths through the code
at least once, most faults will trigger failures Flow graph Test cases to check transitions
State based testing Recently developed for O-O systems Compares the resulting state of the system with
the expected state Not part of the state of practice
© H. Mouratidis
16
Integration testing
Suitable for finding faults on the interfaces between the components
Focuses on groups of components…whole system
Detects faults that have not been detected during unit testing by focusing on small group of components
Couple of components are integrated and tested…more added to the group
© H. Mouratidis
17
Integration testing strategies
Big bang testing All components tested individually, then all
together as a single system+ Simple strategy- Expensive!
Bottom-up testing Split system into layers Test each component of the bottom layer…then
integrates them with the components of the next layer up
+ Interface faults can be more easily found- Interface components are tested last
© H. Mouratidis
18
Integration testing techniques (II)
Top-down testing Similar to bottom-up…tests the components of
the top layer first…then integrates the components of the next layer down
+ Starts with user interface components…same set of tests as requirements
- Test stubs take time and are prone to errorsSandwich testing
Combination of top-down and bottom-up Systems is decomposed into three layers
Target layer (the meat!) Layer above the target layer (the top slice of bread) Layer below the target layer (the bottom slice)
Top-down and bottom-up can be done in parallel© H. Mouratidis
19
System Testing
Ensures the system complies with its functional and non functional requirements
Different activities Functional testing
Functional requirements Performance testing
Non-functional requirements Pilot testing
Selected set of users test the common functionality Acceptance testing
Tests against acceptance criteria Installation testing
Tests performed in the target environment
© H. Mouratidis
20
Functional testing
Differences between functional requirements and the system
Test cases are derived from the use case model
Quality of use cases determines the ease of functional testing
Different than usability testing Differences between use case model and system
behaviour not user expectations of the system
© H. Mouratidis
21
Performance testing
Finds differences between design goals and the system
Different kind of tests Stress testing
Simultaneous requests Volume testing
Large amount of data Security testing
Security faults Timing testing
Identify time constraints Recovery tests
Ability to recover© H. Mouratidis
22
Pilot/acceptance/installation tests
Pilot System is installed and used by selected users Alpha test: development environment Beta test: target environment
Acceptance Client tests the system
Installation Most times repeats previous tests in the target
environment
© H. Mouratidis
23
Plan testing
Testing should be left for the end of the project…Wrong!!!
Start testing as soon as possible! But remember to update!
Conduct testing activities in parallel But keep them consistent
In summaryStart early, update, keep testing consistent!
© H. Mouratidis
24
Testing life cycle
© H. Mouratidis
25
Establish the test objectives
Design the test cases
Write the test cases
Test the test cases
Execute the tests
Evaluate the test results
Change the system
Documenting Testing
Test Plan Documents the scope, approach, resources and
schedule of testing activities Requirements and components to be tested
Test Case Specification Inputs, drivers, stubs, expected outputs
Test incident report Actual results of tests and differences
Test report summary Lists all failures
© H. Mouratidis
26
Automated testing
Manual testing can be: Costly Error prone Time consuming
Automated testing is focused on executionMany infrastructures to support automated
testing JUnit: writes and executes unit tests for Java classes
© H. Mouratidis
27
A historical view
1963-1968 The emerging software crisis
1968-1970 Birth of software engineering
The most important goals were to get control of the software development process and to improve software performance and software reliability
28
© H. Mouratidis
What changed? Did it help?
The introduction of software engineering led to New approaches to the management of software
engineering New techniques of software development
The introduction of software engineering helped to Better understand the issues involved in the
development of systems Successfully build large, reliable systems
Software Engineering has worked! (??)
29
© H. Mouratidis
Software Development challenges
Shorter development lifecycles
More ambitious requirements
Less certain requirementsMore challenging
environments Dynamic systems Open systems Security issues
30
© H. Mouratidis
Dealing with the challenges
Better processes Faster development Light methods
Better models Abstractions Architectures Components
Agile Development
Agent Oriented Software Engineering
31
© H. Mouratidis
Agile development approaches
AD – Agile Database Techniques The goal of the Agile Data (AD) methodology is to define strategies
that IT professionals can apply in a wide variety of situations to work together effectively on the data aspects of software systems.
AM – Agile Modelling Agile Modeling (AM) is a practice-based methodology for effective
modeling and documentation of software-based systems. Simply put, Agile Modeling (AM) is a collection of values, principles, and practices for modeling software that can be applied on a software development project in an effective and light-weight manner.
XP – Extreme Programming Extreme Programming is a discipline of software development based
on values of simplicity, communication, feedback, and courage. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation
32
© H. Mouratidis
Extreme Programming - XP
“XP is a light weight methodology for small-to-medium-sized teams developing software in the face of vague or rapidly changing requirements” (Beck, 2000)
XP is based on four values Communication Simplicity Feedback Courage
33
© H. Mouratidis
Value 1: Communication
Good communication is very important for the successful completion of projects
XP aims to keep the right communication flowing by employing practices that cannot be done without
communicating employing a coach whose job is to notice when people
aren’t communicating and reintroduce them
34
© H. Mouratidis
Value 2: Simplicity
XP argues for a simplistic point of view“It is betting that it is better to do a simple thing
today and pay a little more tomorrow to change it if it needs it, than to do a more complicated thing today that may never be used anyway” (Beck, 2000)
Simplicity supports communication and vice versa The more you communicate the more you
understand about the system and what it doesn’t need to be done
The simpler your system is the less you have to communicate
35
© H. Mouratidis
Value 3: Feedback
According to XP feedback works at different time scales Scale of minutes/days
Feedback provided to customers, developers, progression
Scale of weeks/months Check the overall velocity of the project
Feedback works together with communication and simplicity The more feedback you have the easier is to
communicate
36
© H. Mouratidis
Value 4: Courage
Within the context of the first three values, courage is also needed
Throw code away when necessary Working on code for whole day, it doesn’t work, five
minutes the next morningCommunication supports courage because it opens
the possibility for more high-riskSimplicity supports courage because you can afford
to be much more courageous with a simple systemCourage supports simplicity because as soon as
you see the possibility of simplifying the system, you try it
37
© H. Mouratidis
The XP principles
The four values are too vague to give us much help in deciding which practices to use
We need principlesXP has a set of fundamental principles and a set of
“less central” principlesFundamental principles
Rapid Feedback Assume Simplicity Incremental change Embracing change Quality Work
38
© H. Mouratidis
Principle 1: Rapid Feedback
Research indicates that the time between an action and feedback is critical to learning
Important to get feedback, interpret it, put what is learned back to the system as soon as possible
39
© H. Mouratidis
Principle 2: Assume Simplicity
“Treat every problem as if it can be solved with simplicity”
“Do a good job of solving today’s job today and trust your ability to add complexity in the future where you need it.”
40
© H. Mouratidis
Principle 3: Incremental change
Making big changes at one, most likely it will not work
Everything should be changed incrementally through a series of smallest changes The plan changes a little at a time The design changes a little at a time The code changes a little at a time
41
© H. Mouratidis
XP practices
1. The planning game2. Small releases3. Metaphor4. Simple design5. Testing6. Refactoring7. Pair Programming8. Collective ownership9. Continuous Integration10.40-hour week11.On-site customer12.Coding standards
42
© H. Mouratidis
Practice 1: The planning game
The goal of the planning game is to maximise the value of software produced by the team
How XP does this? The customer provides business stories Developers estimate the time to build software to
implement the stories The customer decides which stories provide the most
business value The developers implement these stories
43
© H. Mouratidis
Practice 2: Small releases
Put a simple system into production quickly Release early and release often
Releases are not prototypes A release implements a couple of stories A release is thoroughly tested
44
© H. Mouratidis
Practice 3: Metaphor
Figure of speech used to refer to something that it does not literally denote to suggest a similarity
Guide all the development with a simple shared story of how the whole system works
To do this you need To have concrete feedback quickly about whether the
metaphor works Both the customer and the developers are comfortable
talking about the system in terms of the metaphor
45
© H. Mouratidis
Practice 4: Simple Design
The system should be designed as simply as possible at any given moment
Right (simple) design at any given time is one that Runs all the tests Has no duplicated logic States every intention important to programmers Has the fewest possible classes and methods
46
© H. Mouratidis
Practice 5: Testing
Programmers continually write unit tests Unit tests always run 100% If a test is broken, fixing it is the most important job
Customers write tests demonstrating that features are finished. What would have to be checked before I would be
confident that this story was done? Each scenario turns into a test
47
© H. Mouratidis
Practice 6: Refactoring
Programmers restructure the system without changing its behaviour
Used mainly to simplify code to make it easier to understand and maintain
After refactoring all the tests should still run 100%
E.g. If there is an one-minute way to get a test working, and a ten-minute way with a simpler design -> go for the ten-minute!
48
© H. Mouratidis
Practice 7: Pair Programming
All production code is written with two programmers at one machine
What pair programming is not? It is not one person programming and the other
watching
Dialog between two people that analyse, design, program, test and improve the program together!
49
© H. Mouratidis
Practice 8: Collective Ownership
Anyone can change any code anywhere in the system at any time
Without any tests, it would be a dead ideaImportant to run the tests after any changeIt tends to
Prevent complex code writing Spread knowledge of the system
50
© H. Mouratidis
Practice 9: Continuous Integration
Integrate and build the system many times a day, every time a task is completed
Is this possible?It is if:
You run the tests quickly so you know you haven’t broken anything
You program in pairs, so there are half as many streams of changes to integrate
You refactor, so there are more smaller pieces, reducing the chance of conflicts
51
© H. Mouratidis
Practice 10: 40-hour week
Work no more than 40 hours per week as a rule Never work overtime a second week in a row Can this work?
Valuable work is identified by the Planning Game
The combination of the Planning game and the testing reducing the frequency of surprises
The practices help you program at top speed
52
© H. Mouratidis
Practice 11: On site customer
Include a real, live user on a team, available to answer questions
Very difficult to have a full time on-site customer!
But if you can…They can produce value for the project
by writing functional tests By making priorities and scope decisions for
programmers.
53
© H. Mouratidis
Practice 12: Coding standards
Programmers write all code in accordance with rules emphasizing communication through the code
Difficult because Individual programmers Different styles
Be part of a winning team
54
© H. Mouratidis
Roles on an XP project
7 different roles Programmer Customer Tester Tracker Coach Consultant Big Boss
55
© H. Mouratidis
Programmer
The programmer is at the heart of XPLooks like being a programmer within other
software development disciplinesExtra skills required
Pair programming Simplicity Refactor Unit test Set aside the feeling of individual ownership
56
© H. Mouratidis
Customer
Being an XP customer is not easyThere are skills to be learned
Writing good stories The team can help with continuous feedback
Make decisions You have to be confident enough to decide fast for
something Write tests
You have to work close with the team to learn the kind of tests which are helpful
Spend time You need to be available …always
57
© H. Mouratidis
Tester
Mainly help customer choose and write testsRun all the tests regularlyBroadcast the test resultsMake sure the test tools run well
58
© H. Mouratidis
Tracker
Tracks and provides feedback regarding estimates Lots of practice and feedback Usually at the beginning you do a lot of estimations
and see how reality conformed to your guessesKeep an eye on the bigger picture, i.e. the
project as a wholeThe most important skill is to get the
necessary information without disturbing the rest of the team too much!!!
59
© H. Mouratidis
Coach
Responsible for the process as a wholeYou notice when people are deviating from
the team’s processes and bring this to the team’s attention
Working on a thin line Every time you guide the team, you make them that
much less self-reliant Too much steering and they lose the ability to work
with you On the other side, you must be direct sometimes
60
© H. Mouratidis
Consultant
Consultants are not always needed in XP projects
When are needed they provide deep technical knowledge
Not solve the problem alone, but help the team to solve it
61
© H. Mouratidis
Big Boss
Provide courageProvide confidencePersistence to check that the team does what
they say they doCorrect the team, if what they do does not
make sense
62
© H. Mouratidis
References about XP
Extreme Programming explained, Kent Beck, (005.1 BEC – library)
http://www.extremeprogramming.org/index.html
Extreme programming for Web projects, Wallace et al., (005.72 – Library)
Various web pages
63
© H. Mouratidis
Conclusions
Testing Concepts Testing strategies
Agile Development Extreme Programming
© H. Mouratidis
64