Testlife Cycle

8/13/2019 Testlife Cycle

http://slidepdf.com/reader/full/testlife-cycle 1/94

n Introduction to Testing

Life Cycle Process



Contents

Testing Life Cycle Overview

Test Strategy

Test Planning Test Case Design

Test Execution

Testing Types

Testing Tools Overview



Testing Life Cycle Overview Test Strategy

Test Planning

Test Case Design

Test Execution Testing Types






•Normally, testing of any Large Systems will be in TWOparts. The functional verification and validation against

the Requirement Spec and Performance evaluationagainst the indicated requirements

Testing activity is involved right from the beginning of theproject

V&V Process model to successfully deliver a project ontime

This model is followed for a complete project life cyclemodel

TestingLife CycleOverview

TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes

TestingToolsOverview



Testing Life Cycle - V&V

Process Model

HLD/LLD for each

application

involved

Interface design

Application

Integration

Integrated

Solution

Release

Project Planning

Master Test Strategy

Test Planning

Strategy for

individual

applications, Test

case design

Performance

Testing,

Business Cycletesting

Acceptance Tests

&

Certification

Incremental

Integration Testing

System Testing

(Application level

Testing)

Solution

Mapping/

Development/

Customization

Business

requirements,

Solution

Architecture


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Process Concept

The "V &V " concept relates the build components ofdevelopment to the test components that occur during

that build phase.

Testing related activities during Requirement phase

Creation and finalization of testing templates

Creation of over-all Test Plan and Test Strategy

Capturing Acceptance criteria and preparation of Acceptance

Test Plan

Capturing Performance criteria of the software requirements


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Process Concept …

• Testing activities in System Testing phase – System test is done for validating the product with respect to client

requirements

– Testing can be in multiple rounds

– Defect found during system test should be logged into DefectTracking System for the purpose of tracking.

– Test logs and defects are captured and maintained.

– Review of all the test documents TestingLife CycleOverview

TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Process Concept…

Testing activities in Design phase Develop Test cases to ensure that product is on par with

Requirement Specification document.

Verify Test Cases & test scripts by peer reviews.

Preparation of traceability matrix from system requirements


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Process Concept …

• Testing activities in Integration Testing Phase – This testing is conducted in parallel with integration of various

applications (or components)

– Testing the product with its external and internal interfaceswithout using drivers and stubs.

– Incremental approach while integrating the interfaces.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Process Concept …

• Performance Testing – This is done to validate theperformance criteria of the product / application.This is non-functional testing

• Business Cycle testing – this refers to end to end

testing of real life-like business scenarios.• Testing activities during Release phase

– Acceptance testing is conducted at the customer location.

– Resolves all defects reported by the customer duringAcceptance Testing

– Conduct Root Cause Analysis (RCA) for those defectsreported by customer during acceptance testing


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Testing Life Cycle - Team

Structure

• An effective testing team includes a

mixture of members who has

– Testing expertise

– Tools expertise – Database expertise

– Domain/Technology expertise TestingLife CycleOverview

TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Testing Life Cycle - Team

Structure (Contd…)

• The testing team must be properly structured,with defined roles and responsibilities that allowthe testers to perform their functions withminimal overlap.

• There should not be any uncertainty regardingwhich team member should perform whichduties.

• The test manager will be facilitating any

resources required for the testing team.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Testing Life Cycle - Roles and

Responsibilities

• Clear Communication protocol should bedefined with in the testing team to ensureproper understanding of roles and

responsibilities.

• The roles chart should contain both on-siteand off-shore team members.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Responsibilities• Test Manager

– Single point contact between onsite andoffshore team

– Prepare the project plan

– Test Management

– Test Planning

– Interact with onsite lead, Client QA manager

– Team management

– Work allocation to the team

– Test coverage analysis


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Responsibilities

• Test Manager cont..

– Co-ordination with onsite for issue resolution.

– Monitoring the deliverables

– Verify readiness of the product for release through release

review

– Obtain customer acceptance on the deliverables

– Performing risk analysis when required

– Reviews and status reporting

– Authorize intermediate deliverables and patch releases tocustomer.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Responsibilities

• Test Lead

– Resolves technical issues for the product group

– Provides direction to the team members

– Performs activities for the respective product group

– Review and Approve of Test Plan / Test cases – Review Test Script / Code

– Approve completion of Integration testing

– Conduct System / Regression tests

– Ensure tests are conducted as per plan

– Reports status to the Offshore Test Manager


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Responsibilities

• Test Engineer

– Development of Test cases and Scripts

– Test Execution

– Result capturing and analysing – Defect Reporting and Status reporting


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Testing Life Cycle - Term

Definitions• Testing: It is a process of executing software in a

controlled manner, in order to answer the question“Does the software behave as specified?”

• Verification: Says “Are we building the productright?” All QC activities throughout the life cycle that

ensure that interim deliverables meet their inputspecification.

• Validation: Says “Are we building the right product?” The test phase of the life cycle which assures that theend product meets the user‟s needs.

• Test Case: “An input operation and the correspondingexpected output” in order to test a small unit of work.

• Test Script: “A logical group of test cases which, whentaken together, test a particular function or unit of asystem”


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Definitions (Contd.)• Test Bed: “A group of test scripts which, when takentogether, test all functions of an entire system”

• Audit: An inspection/assessment activity that verifiescompliance with plans, policies and procedures

• Baseline: A quantitative measure of the current level of

performance• Bug: A catch all term for all software defects or errors

• Certification: Acceptance of a software by anauthorized agent or its validity been demonstrated to anauthorized agent

• Cost of Quality (COQ): Money spent above and beyondexpected production costs, this involves prevention,appraisal, internal & external failure costs

• Cyclomatic Complexity: The number of decisionstatements plus one.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Definitions (Contd.)• Defect: It is “A deviation from specification orstandard” or “Any thing that causes customer

dissatisfaction”.

• Desk Check : Verification technique conducted by theauthor of the artifact to verify the completeness

• Dynamic Assertion: A dynamic analysis technique thatinserts into the program code assertions about therelationship between program variables

• Force field Analysis: A group technique used toidentify both driving and restraining forces thatinfluence a current situation

• Inspection: A formal assessment of a work productconducted by one or more qualified independentreviewers to detect defects, violation of developmentstandards, etc. Inspection identifies defects but doesnot attempt to correct them.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Definitions (Contd.)• Instrumentation: The insertion of the additional code

into a program to collect information about aprogram behavior during program execution.

• Life Cycle Testing: The process of verifying theconsistency, completeness and correctness of software

at each phases of the development life cycle.• Phase Containment: Method of control put in place

within each stage of the development process topromote error identification and resolution so thatdefects are not propagated downstream to subsequentstages of the development process

• Productivity: The ratio of the output of a process tothe input


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes





Definitions (Contd.)• Quality Assurance: The set of activities (includingfacilitation, training, measurement, and analysis)

needed to provide adequate confidence that process areestablished continuously improved to produceproducts that meet specifications and are fit for use.

• Quality Control: The process by which product qualityis compared and detected w.r.t requirements and otherrelevant specifications, focus is in detection andremoval

• Test Driver: A program that directs the execution ofanother program against a collection of test data sets

• Test item: A software item that is an object of testing

• User: The customer that actually uses the productreceived


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy


Test Planning

Test Case Designing





Test Strategy

• Test strategy is statement of overall approach oftesting to meet the business and test objectives.

• It is a plan level document and has to be preparedin the requirement stage of the project.

• It identifies the methods, techniques and tools to be

used for testing .• It can be a project or an organization specific.

• Developing a test strategy which effectively meetsthe needs of the organization/project is critical to

the success of the software development• An effective strategy has to meet the project and

business objectives

• Defining the strategy upfront before the actualtesting helps in planning the test activities


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Elements of Test Strategy

• A test strategy will typically cover the followingaspects – Definition of test objective

– Strategy to meet the specified objective

– Overall testing approach

– Test Environment – Test Automation requirements

– Metric Plan

– Risk Identification, Mitigation and Contingency plan

– Details of Tools usage

– Specific Document templates used in testing


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Project/Business

Objectives

• All the business and test objectives areidentified and test strategy shall be defined

in order to meet those objectives.

• As far as possible all the test objectives to

be met are defined in quantitative terms inthe test strategy.

• Examples of test objectives can be like 90 %

of code coverage, finding at least a

minimum of 100 defects etc.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Test Approach

• Test approach will be based on the objectivesset for testing

• Test approach will detail the way the testing to

be carried out

• Types of testing to be done viz Unit,Integration and system testing

• The method of testing viz Black – box, White-

box etc.,

• Details of any automated testing to be done


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Test Environment

• All the Hardware and Software requirementsfor carrying out testing shall be identified in

detail.

• Any specific tools required for testing will also

be identified• If the testing is going to be done remotely, then it

has to be considered during estimation TestingLife CycleOverview

TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Metric Plan

• Metric plan will contain the measures that willindicate the effectiveness of testing .

• All the metrics to be tracked for testing like

defect density ,residual defect density ,code

coverage etc are identified in test strategy .• The objectives set for testing are also measured

in the metric plan.

Note: More details are covered in Test Execution section


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Risks

• Risk analysis should carried out for testing phase• The risk identification will be accomplished byidentifying causes-and-effects or effects-and-causes

• The identified Risks are classified into to Internal andExternal Risks. The internal risks are things that the

test team can control or influence. The external risksare things beyond the control or influence of the testteam

• Once Risks are identified and classified, the followingactivities will be carried out – Identify the probability of occurrence

– Identify the impact areas – if the risk were to occur

– Risk mitigation plan – how avoid this risk?

– Risk contingency plan – if the risk were to occur what do wedo?


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Tools Usage

• Tools required in the project and rationale to usethat particular tool(s) should be documented.

• Automation requirements should be identified

and analyzed

– Which phases need to be automated – How much of testing needs to be automated

• Based on the automation requirements, tools

strategy should be prepared. TestingLife CycleOverview

TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Templates

• Any specific templates to be used for testing apartfrom the normal templates based on client or project

requirements are identified in the test strategy.

• In some case client provides the templates and we‟ll

use that.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Acceptance criteria

• Acceptance test plan can be prepared either byus or by the customer, as agreed.

• Customer will be involved in preparation, review

and approval of acceptance test cases

• This plan specifies the criteria for clientacceptance of the final tested product including

Features/functionalities to be tested and

traceability information. TestingLife CycleOverview

TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Strategy - Summary

• Test strategy is a high level document whichstates in detail way testing is to be carried out.

• It is done in line to meet the test and business

objectives

• Tools ,Methods ,and techniques to be used arestated in the test strategy.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Planning


Test Planning

Test Case Designing





Test Planning - Documenting

assumptions

• If any assumptions are there, it needs to bedocumented to avoid any confusions

– Testers has understanding of V&V-process model or any

SDLC model

– Testers are aware of test phases of the project

– A good understanding of the Domain

– Standard naming conventions are followed in

documentation TestingLife CycleOverview

TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Planning -Traceability

• Requirements tracing is the process ofdocumenting the links between the userrequirements for the system you are building andthe work products developed to implement andverify those requirements.

• Work products include software requirements,design specifications, software code, test plans andother artifacts of the systems development process


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Planning -Traceability

• Traceability is documenting of – the dependencies and logical links between individual

requirements and other system elements

– traceability to greatly facilitate change impact

analysis.

– traceability is that it can help identify the propagationof change that can result when a specific requirement

is deleted or modified.

– specific requirements mapped into tasks in a work-

breakdown structure, you know those tasks that will

be affected if and when a requirement is changed ordeleted.


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Planning - Schedule/

Milestones

• Test plan captures the details about differentmilestones of the project and the schedule

details.

• Define the schedule for each event for offshore

and onsite. Events may include familiarization,training, test case preparation, test scripting,

test execution etc.,

• Completion criteria for each milestone should

be identified and documented


TestStrategy

TestPlanning

Test CaseDesigning

TestExecution

TestingTypes




Test Planning - Resources

• Staffing: – Check what types of personnel should be

considered for inclusion on the test team

– Check for the skills of the tester

– Ensure a good ratio of testers to developers.

• Steps to form the best team:

– Identify potential test team members

– Recruit test team members and developtentative test assignments

– Define individual work assignments


TestStrategy

Test

Planning

Test CaseDesigning

TestExecution

TestingTypes




Test Planning - Test run plan

• The execution order of testcases depends on manyfactors and is shown asbelow:

TC priorityExecution

Order

ComplexityExecution

rate

Impacted

Functionality

Unstable

functionality

Customer

Priority

Test

ExecutionRun Plan

Release Plan 4

0

200

400

600

800

1000

1200

1400

7/4 7/5 7/6 7/7 7/8 7/9 7/10 7/11 7/12 7/13 7/14 7/15 7/16 7/17

# of Days

C u m u l a t i v e T C s

E x e c u t e d

Plan 106 257 575 669 767 877 983 1065 1119 1119

Actual 297 483 632 737 855 939 1015 1079 1140 1147

4-Ju l 5-Ju l 6-Ju l 9-Ju l 10-Ju l 11-Ju l 12-Ju l 13-Ju l 16-Ju l 17-Ju l

Functionality

Dependency

Execution

Order

TC Inter-

dependency


TestStrategy

Test

Planning

Test CaseDesigning

TestExecution

TestingTypes


T Pl i Ch



Test Planning - Change

management

• Version control - Versioning test cases beforeand after release to customer

• Change control - Authority to approve andprioritize changes

• Configuration auditing - Ensure changes made

properly

• Reporting

• Addition/Updating of test cases/plans because ofbugs which are identified by the customer and

that are not available in test plans


TestStrategy

Test

Planning

Test CaseDesigning

TestExecution

TestingTypes




Test Planning – Suspension and

Resumption criteria• Deadlines (release/testing deadlines etc.) are

met and quality of the product is as per the

agreed norms.

• Test cases completed with certain percentage

passed• Test budget depleted

• Coverage of code/functionality/requirements

reaches specified point

• Beta or Alpha testing period ends


TestStrategy

Test

Planning

Test CaseDesigning

TestExecution

TestingTypes




Determination of Test Completion criteria • Most popular method of establishing a test

phase completion criteria is by detecting a pre-

defined number of bugs or when a pre-defined

time elapses• E.g.: Unit testing of a unit is not deemed

complete unless at least 5 errors are detected

• Testing completion criteria could be detection

and fixing of 30 bugs, or elapsed time of 30days, whichever is later.


TestStrategy

Test

Planning Test CaseDesigning

TestExecution

TestingTypes



Resumption criteria



Estimation of number of errors• How to determine the number of errors to be

detected?

– An estimate of total number of errors in theprogram.

– An estimate of what percentage of these errors canbe feasibly found through testing.

– Estimates of what fraction of errors originated in aparticular phase, during what testing phases these

are likely to be detected.


TestStrategy

Test


TestExecution

TestingTypes



Resumption criteria



• Methods of estimation – Rough estimate of total number of errors can be obtained

by previous experience on similar programs.

– Error-seeding techniques – involves intention introduction(seeding) of error in the code. After some period of testingtime, the product of the number on non-seeded errors

found during testing and the ratio of total seeded errors tothe number of seeded errors detected during testing, givesan estimate of the number of non-seeded errors remainingin the program.


TestStrategy

Test


TestExecution

TestingTypes



Resumption criteria



• Estimation of errors in a particular phase - Example – Existing data reveals that in large programs, 40% of errors

are coding and logic-design mistakes and the rest aregenerated in earlier phases.

– Example to illustrate estimation of total number of errors –

– We have a 10,000 statement program, and the number oferrors remaining in the program after code review is 5 per100 statements. Our objective is to find 98% of coding andlogic design errors and 95% design errors. Therefore-

• Total number of estimated errors = 500

• Coding and Logic design errors = 200 (40% of 500)

• Design errors = 300.• We need to find at least:

• 196 Coding and Logic Design errors

• 285 Design errors.


TestStrategy

Test


TestExecution

TestingTypes



Resumption criteria

i S i



Coding & LogicDesign errors

Design Errors

Unit Test 65% 0%

ModuleIntegration Test

30% 60%

System Test 3% 35%

Total 98% 95%

Estimation of error distribution in different phases


TestStrategy

Test


TestExecution

TestingTypes



Resumption criteria

T Pl i S i d



Establishment of completion criteria• Exit Criteria for Unit testing would be detection and

correction of 130 errors (65% of estimated 130coding and design logic errors)

• Exit Criteria for Module Integration testing would

be detection and correction of 240 errors (30% of200 plus 60% of 300) or elapse of one month; whichever is later.

• Exit Criteria for System Testing is when 111 errors

are detected or when 2 months elapses; which ever islater.

Assumption: We have established one month for Integration

Testing and 2 months for System Testing.


TestStrategy

Test


TestExecution

TestingTypes



Resumption criteria

T Pl i S i d



• Resolution in case of over estimation of defects• In the previous example, let us say that 240

errors could not be detected even after onemonth of integration testing is completed. Thenthe project manager should take the following

action – With the help of the appropriate tool, he/she should

analyze the test cases and decide whether • There are inadequate Test cases, or

• Excellent Test cases, but lack of errors to detect


TestStrategy

Test


TestExecution

TestingTypes



Resumption criteria

T Pl i S i d



• Determination of completion criteria through a graphical representation• A graph is plotted using number of errors found per unit time during a

testing phase. By examining the shape of the curve, one can determine

whether to continue the test phase or end it.

• Assume a program is being Integration tested, and the number of errors per

day is plotted. Lets say on the 7h day the graph looks as follows:

No. of days 7

N u m b e r o f

e r r o r s AT the end of 7th day, even if

the number of errors detectedhas crossed the criteria mark, itwould be wise to continueIntegration Testing. This isbecause, the error rate is stillvery high.


TestStrategy

Test


TestExecution

TestingTypes



Resumption criteria

T t Pl i S i d



• Determination of completion criteria through a graphicalrepresentation

• On the hand if the graph looks as shown, then this shows that

error rate has decreased, and that testing can be stopped.

No. of days 7

N

u m b e r o f e r r o r s


TestStrategy

Test


TestExecution

TestingTypes



Resumption criteria

Test Planning Document



Test Planning – Document

templates • Typical Test Plan

– Test Plan identifier

– Introduction

– Test items

– Features to be tested

– Features not to be tested

– Approach

– Testing strategy

– Item pass/fail criteria

– Suspension criteria and resumption criteria

– Test deliverable

– Automation scope – Testing tasks

– Effort estimation

– Environmental needs

– Roles and Responsibilities


TestStrategy

Test


TestExecution

TestingTypes




Test Case Design


Test Planning

Test Case Design





Test Case Design (Contd…)

Development Of Test Objectives:• At very First stage of testing , you need to set the

goals and objectives of what you need to achieve.

• This Simplifies the limits or borders of testing

process.• Test Objectives will be contain the list of tasks you

are going to do and the achievements.

Test Case Writing:

• A specific set of steps and data along with expectedresults for a particular test objective. A test case

should only test one limited subset of a feature or

functionality.


TestStrategy

Test

Planning Test CaseDesign

TestExecution

TestingTypes





A typical test case sheet contains these fields… • Test Case ID: It is unique number given to test case in

order to be identified.

• Test description: The description if test case you are

going to test.• Revision history: Each test case has to have its revision

history in order to know when and by whom it is

created or modified.

• Function to be tested: The name of function to betested.

• Priority/Build/time: The priority of the Test Case,

Build version of the test case and the Minimum time-

to-execute the test case is also mentioned.


TestStrategy

Test


TestExecution

TestingTypes





• Environment: It tells in which environment you aretesting.

• Test Setup: Anything you need to set up outside of your

application for example printers, network etc.,

• Test Execution: It is detailed description of every stepof execution.

• Expected Results: The description of what you expect

the function to do.

• Actual Results: Pass /Failed.If pass – What actually happens when you run the test

If failed - Description of what you've observed.


TestStrategy

Test


TestExecution

TestingTypes





Characteristics of a Good Test: • Tests are likely to catch bugs

• No redundancy

• Not too simple or too complex.

TIP: Test case is going to be complex if you have

more than one expected results. TestingLife CycleOverview

TestStrategy

Test


TestExecution

TestingTypes





Test Data Inputs:• Before testing, the tester should plan what

kind of data he is giving for test.

• Give data inputs as functional, boundary,

stress, performance, usability values etc.


TestStrategy

Test


TestExecution

TestingTypes





Test Scripts:• A test script is the executable form of a test.

• It defines the set of actions to carry out in order

to conduct a test and it defines the expected

outcomes and results that are used to identify anydeviance in the actual behavior of the program

from the logical behavior in the script (errors

during the course of that test).

• In essence it is a program written for a human

computer (tester) to execute.


TestStrategy

Test


TestExecution

TestingTypes




Test Execution


Test Planning

Test Case Design





Test Execution

• Test execution is culmination of testingactivities which involves executing the

planned test cases and conducting of the

tests.

• Test execution phase broadly involves

execution and reporting.


TestStrategy

Test


TestExecution

TestingTypes




Test Execution - Activities

• Test execution consists of following activitiesto be performed

– Creation of test setup or Test bed

– Execution of test cases on the setup

– Test Methodology used

– Collection of Metrics

– Defect Tracking and Reporting

– Regression Testing TestingLife CycleOverview

TestStrategy

Test


TestExecution

TestingTypes




Test Execution Cycle


TestStrategy

Test


TestExecution

TestingTypes




Test Execution - Testing

• Only exhaustive testing can show a program that it isdefect free ,but exhaustive testing is impossible

• Tests should exercise the systems capabilities fully

• Testing typical situations is more important thanboundary value conditions


TestStrategy

Test


TestExecution

TestingTypes


Test Execution -Test Data and



Test Execution Test Data and

Test Cases• Test Cases: Inputs to test the system and predictedoutput from the system are detailed, further will

contain the expected result and test topology

• Test Data: Inputs which are to exercise the system

under test.


TestStrategy

Test


TestExecution

TestingTypes


Test Execution -Types of Testing



Test Execution Types of Testing

and Methodologies• There are three major types of testing andmethods used are based on the testing type

– Unit testing

– Integration testing

– System testing


TestStrategy

Test


TestExecution

TestingTypes




Test Execution - Unit Testing

• Unit testing is typically done as white box testing.• Unit testing is done at module level.

• Derivation of Test cases are based on the programstructure .Knowledge of the program is used toidentify the test cases .

• Objective is to exercise all parts of the program .

• Test coverage measurements are done to verify thatall program parts are exercised

• Drivers are used to feed the data to the program to

test it, wherever required.• Test stubs are used to stub for the functions which

are outside the module


TestStrategy

Test


TestExecution

TestingTypes


Test Execution - Integration



Test Execution Integration

Testing• Tests partial systems composed of integratedcomponents

• Integration testing is black box testing

• Main difficulty in integration testing is

localizing errors• There two approaches to integration testing

– Top-down approach

– Bottom - up approach


TestStrategy

Test


TestExecution

TestingTypes


Integration testing- Top Down



Integration testing Top Down

approach• Start from the top level module and integrate

individual components and stub the rest of the

system


TestStrategy

Test


TestExecution

TestingTypes


Integration testing - Bottom –



Integration testing Bottom

Up approach

• Integrate individual components from lowerlevel module until whole system is integrated .


TestStrategy

Test


TestExecution

TestingTypes


Test Execution - System



Test Execution System

Testing

• System testing is typically “Black Box “ testingwhere the system is taken as black – box and

tested based only on its functionality

• Test cases are based on the system specification

• It is critical part of the testing process as thesystem is verified as a whole


TestStrategy

Test


TestExecution

TestingTypes




Test Execution - Defect



Test Execution - Defect

Tracking and Reporting• All the defects/bugs found during testing is

logged and they are classified based on theseverity and priority.

• They are assigned to developers for fixing.

• Each defect found is logged and tracked to

closure.• Test summary report shall be prepared

containing the details like numbers of test casespassed/failed ,number of cycles of testing.

• Generally test defect logging and tracking toolsare used for this purpose.


TestStrategy

Test


TestExecution

TestingTypes


Test Execution - Start / Stop



Test Execution Start / Stop

Criteria• Testing is done in rounds. Software passes

though several rounds before it gets released.• Testing is stopped when there are many major

bugs and critical system errors, when furthertesting is not feasible

• Testing is stopped when the system becomesun stable during test.

• When those critical errors are fixed andsystem is stable, the next round of testing isstarted .

• The test cases are executed all over again ineach round.


TestStrategy

Test


TestExecution

TestingTypes


T E i M i



Test Execution - Metrics

• Metrics are important to collect for any project as ithelps in monitoring the health of project /product/application under test.

• Metrics are collected during testing depending on theobjective of testing.

• There are different kinds of metrics which can beused in testing. – For example LOC (Lines of Code) is a metric for Unit

testing. i.e., Code related metrics comes in the category ofUnit testing.

• Test case related metrics can be used for all kinds of

testing types, since for every type of test, we willwrite test cases.

• Normally source of errors/defects is requirements,design or Implementation. Testing can not be thesource of errors/defects.


TestStrategy

Test


TestExecution

TestingTypes


T E i M i




• Typical testing metrics – No of LOC tested per day

– No of test cases executed per day

– Number of test cases failed

– Number of bugs reported

– Number of rejects of bugs reported

– Number of persons working on the test plan duringthe month

– Total Effort spent on test execution in person hours

– Number of simulations made during the month

– Number of out of norms per month

– Average time taken to find a problem categorized bypriority

– Defect density

– Review efficiency


TestStrategy

Test


TestExecution

TestingTypes


T t E ti M t i




• Having a complete Metrics data helps inbuilding the “Trustworthy, Capable, Reliableand Predictable” Product or Application.

– Productivity Increase

– Field error rate Decrease

– Customer Overhead reduces – Improvement release on release

– On time delivery

– Risks in Program identified and proactively intimated

– Decrease in Cost of Quality

– Decrease in Re-work – Helps in identifying the area which needs focus in the

project/organization for improvement

– Helps in arriving at better estimates for the project


TestStrategy

Test


TestExecution

TestingTypes


Test Execution – Good test



engineer?• What makes a good test engineer?

– A good test engineer has a 'test to break' attitude.

– An ability to take the point of view of the customer

– A strong desire for quality, and an attention to detail.

– Tact and diplomacy for maintaining a cooperative relationship withdevelopers.

– An ability to communicate with both technical (developers) and non-

technical (customers, management) people. – Previous software development experience can be helpful, gives the

tester an appreciation from the developers' point of view, and reducethe learning curve in automated test tool programming.

– Judgment skills are needed to assess high-risk areas of an applicationon which to focus testing efforts when time is limited.

– Notice little things that others miss/ignore (See symptom not bug) – Good testers use files, Databases and all the other accoutrements ofan organized mind

– Testers are fundamentally honest and incorruptible

– Last but not the least is “Patience”


TestStrategy

Test


TestExecution

TestingTypes


T t E ti T ti R l ?



Test Execution – Testing Rules?

• Software testing 10 rules? – Test early and test often.

– Integrate the application development and testing lifecycles.

– Formalize a testing methodology; you'll test everything thesame way and you'll get uniform results.

– Develop a comprehensive test plan; it forms the basis forthe testing methodology.

– Use both static and dynamic testing.

– Define your expected results.

– Understand the business reason behind the application.You'll write a better application and better testing scripts.

– Use multiple levels and types of testing (regression,systems, integration, stress and load).

– Review and inspect the work, it will lower costs.

– Don't let your programmers check their own work; they'llmiss their own errors.

Source: SQAtester.com


TestStrategy

Test


TestExecution

TestingTypes


T t E ti S



Test Execution - Summary

• Test execution consists of preparing of testbed/setup ,execution of planned test cases

,verification of results ,Test tracking and

reporting and metrics collection.

• Apart from that ,it involves applying different

test methods like black box to system testing

,white box at unit testing level and top – down and

bottom – up in case of integration testing .


TestStrategy

Test


TestExecution

TestingTypes


Test Execution – Summary



y

Contd..

• Testing will involve several rounds based on thestability and bugs found on the software and

start /stop criteria identified .

• Regression testing will be carried out when ever

the new bugs are found and fixed in the tested

software to verify the impacted items.

• Test execution will involve preparation of test

reports ,collection of different metrics and

meeting the test objectives .


TestStrategy

Test


TestExecution

TestingTypes


T ti T l O i





Test Strategy

Test Planning

Test Case Design

Test Execution

Testing Types





g

(contd…)

• Testing tools are used to automate the testingprocess.

• These tools will have features like

– Record and play back

– Test Script generation

– Virtual users

And so on… TestingLife CycleOverview

TestStrategy

Test


TestExecution

TestingTypes





g

(contd…)

• Different vendors released different testingtools in the market. Some of them are

Functional / GUI Testing Tools:

• Win Runner• Rational Robot

• Visual Test TestingLife CycleOverview

TestStrategy

Test


TestExecution

TestingTypes





g

(contd…)

Performance Testing: • Rational Quantify

• True time

Web Testing:

• Silk Test

• Astra Quick Test

• eTest Suite


TestStrategy

Test


TestExecution

TestingTypes





g

(contd…)

Java Testing Tools: • Jtest

• JProbe

• QStudio

• Optimizeit

• JStyle TestingLife CycleOverview

TestStrategy

Test


TestExecution

TestingTypes




Appendix 1: Example for test life



pp p

cycle cont..• Test strategy to test HRWEB->Resume page includes overview of

– Document the test requirements

– User Interface testing

– Test for associated links

– Error Handling and Exceptions

– Functionality testing

– Boundary conditions

– Test for Load, Performance, Security issues

– Deciding of tools and test set-up to test

• Test Planning to test HRWEB->Resume includes

– Resume page and all associated links will be tested – Specify how it will be tested - Test Cases

– Identify resources for testing and finalize timeline

– Specify Hardware, Software configuration requirement




pp p

cycle cont..

• Test Case Development – Several Test Cases will be generated as per the test strategy and plan

for User Interface, Functionality, Error Handling of Resume Section in

HRWEB

– Integration and System Test cases will be generation to test for the

whole application including Resume Section.

• Test report:

– Test Cases will be executed and Test Results will be recorded and

tracked.

– After bugs are fixed, the system is tested again to verify the same

– The whole system is also tested to verify that new bugs have not been

introduced.




pp p

cycle cont..Sample User Interface Test Case for Resume Section

• Perform the necessary steps to view the Resume Section onTEDWEB

• Verify that the menu TEDWEB Menu displays at the top andthe Resume Menu displays at the left of the screen

• Verify the presence of the following buttons - Add Resume,Modify Resume, Add project, Modify Project etc..

Sample Functionality Test Case for Resume Section

• Click on „Add Resume‟ link

• Verify a new screen appears to the right that allows the user

to add his resume details• Put the cursor in Educational Qualifications field and verify it

is enabled for user input

• Put the cursor in Summary field and verify it is enabled foruser input.




cycle cont..

Sample Error Handling Test Case for Resume Section• Click on „Add Resume‟ when the resume already exists.

Verify the message displayed is „Profile Already Exists‟

• Click on „Modify Resume‟ when no resume has been added.Verify that a proper error message is displayed that no

resume exists.

Appendix 2: Testing Standards



Appendix 2: Testing Standards

• I EEE Standard for Software Test Documentation

(ANSI/IEEE Standard 829-1983). This is a summary of theANSI/IEEE Standard 829-1983.

• It describes a test plan as: “A document describing thescope, approach, resources, and schedule of intended testingactivities. It identifies test items, the features to be tested,

the testing tasks, who will do each task, and any risksrequiring contingency planning.”

Appendix 3: Additional readings



Appendix 3: Additional readings

• Please go through the following documents, which areavailable in Interops KM Page.

– Inter101_002-Intro Software Testing

– Inter101_003-Test Plan & Test Case

– Inter101_004-SDLC&TLC

– Inter101_005-Unit & Integration Testing

– Inter101_006-Functional & Regression Testing

– Inter101_007-General Testing Types

– Inter101_008-Test Report Automation Tools

– Inter101_009-Review & Walk thru – Inter101_010-Reliability terminology

Date post:	04-Jun-2018
Category:	Documents
Upload:	rohit-mehta
View:	215 times
Download:	0 times

Testlife Cycle

Documents