SOFTWARE Testing Introduction - bvicam.inbvicam.in/sites/default/files/subject/LearningMannuals/ST...

MCA-303, Software Testing

© Bharati Vidyapeeth’s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1.1

SOFTWARE TestingIntroduction

© Bharati Vidyapeeth’s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jian U1. 1

UNIT I

Software Testingprocess of executing a program with the intent

© Bharati Vidyapeeth’s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1. ‹#›

p g p gof finding errors

Software TestingWhy it is hard?


Why it is hard?



An Example

• The program is designed to add two numbers,

which you enter. Each number should be one or

two digits. The program will echo your entries, then

print the sum. Press <Enter> after each number. To


print the sum. Press Enter after each number. To

start the program type Adder

Guidelines

1. Start with an obvious and Simple test

2. Make Some notes about what else needs testing

3. Check the valid cases and see what happens


4. Do some testing on fly

5. Summarize what you know about the program and

its problems

Question of Interest

• Can you test this program exhaustively? If so, how many test cases are required?

• Assuming that each test case can be executed and analyzed in one second, how long would it take for


you to run all the tests?



Limitation of Testing• Can we test the program completely? Some text Many managers Test coverage analyzer Salesperson Some tester


• No matter how hard we try, how cleverly we plan, howmuch time we spend, and how many staff and computerwe use, we still cannot do enough testing. We still missbugs

Reasons

• Theoretical Foundation

No general purpose testing or analysis procedure can be

used to prove program correctness

• Impracticality of Testing All Data


Impracticality of Testing All Data

The domain of possible inputs is too large to test

• Impracticality of Testing All Paths

There are too many possible paths through the program

to test

• No absolute proof of correctness Howden suggests that there are proofs of equivalency

unless a formal specification can be shown to be correct and,indeed, reflects exactly the user's expectations, no claims ofproduct correctness can be made.

• The user interface issues are too complex to completely

Reasons..


• The user interface issues are too complex to completelytest

• The challenge is to achieve maximum coverage throughminimum test cases.

• It is rightly said that testing process is as efficient as thetest data.

•



• Apart from well designed test cases it is important to:

Analyze the results properly

Establish relation between faults and incidents

Report the detected fault effectively

Are Well Designed Test Cases Sufficient…


Report the detected fault effectively

Association with a module

Steps for regenerating the problem encountered

Optionally a probable impact

Software Testing

• Consumes at least half of the labor

• Process of testing software product

• Contribute to


Contribute to Delivery of higher quality product

More satisfied users

Lower maintenance cost

More accurate and reliable results

TERMINOLOGY

• Error

• Fault/Defect

• Failure

• Incident

• Test


Test

• Test Case



Error

• People make errors.

• Typographical error

• Misreading of a specification

Mi d t di f f ti lit f d l


• Misunderstanding of functionality of a module

• A good Synonym is Mistake.

• When people make mistakes while coding we

call these mistakes “bugs”

Fault/Defect

Representation of an error DFD Hierarchy chart Source Code An error may lead to one or more faults


Fault of Omissions If certain specifications have not been programmed

Fault of Commission If certain program behavior have not been specified

• Defects generally fall into one of the following three categories Wrong

Missing

Extra

Fault/Defect..




Failure A particular fault may cause different failures, depending on how it

has been exercised

Incident When a failure occurs, it may or may not readily apparent to the user

Failure, Incident


When a failure occurs, it may or may not readily apparent to the user

Incident is the symptom associated with a failure that alerts the userto the occurrence of a failure

Test, Test CaseTest:

A test is the act of exercising S/W with test cases

Test Case: A test case has an identity and is associated with a program

behaviour

It h t f i t d li t f t d t t


It has a set of inputs and a list of expected outputs

Test Case Template

Test Case ID:

Section –I

(Before Execution)

Section –II

(After Execution)

Purpose: Execution History:

Pre Condition: (If Any) Result:


Inputs: If Fails, any possible reason(Optional):

Expected Outputs: Any Other observation:

Post Conditions: Any Suggestions:

Written by: Run By:

Date: Date:



S/W Testing• Testing is the process of demonstrating that errors are not

present

• The purpose of testing is to show that a program performsits intended functions correctly


• Testing is the process of establishing confidence that aprogram does what it is supposed to do

• Testing is the process of executing a program with theintent of finding errors

Testing Life Cycle


• Three phases Putting Bugs IN

• Testing Phase Finding Bugs

• Last Three phases are Getting Bugs OUT

Testing Life Cycle..


• The fault resolution step is another opportunity for

errors. When a fix causes formerly correct software

to misbehave, the fix is deficient.



Testing Principles

1. Testing shows presence of defects

2. Exhaustive testing is impossible

3. Early testing

© Bharati Vidyapeeth’s Institute of Computer Applications and Management, New Delhi-63, by Anukiran Jain U1. ‹#›h

4. Defect clustering

5. Pesticide paradox

6. Testing is context depending

7. Absence – of – errors fallacy

Relative Costs to Fix Errors

Cost to fix an error increases as it is found later and later

40

50

60

70

80

What can you infer from this graph?


later and later in the software

lifecycle

0

10

20

30

Requ

irem

ents

Des

ign

Implem

entation

Testing

Mainten

ance

Testing Cost Curve




Testing ProcessInput domain

Construct InputOperational Profile

Test Plan

Execute Program

SpecificationsI b h i

Behaviour

Use

Use

Input data

Test Case

Use


p

File Test Report

Terminate testing?

Is behavior as expected

Cause of Error?

Debug program

Error to beFixed now?File pending error

ReportFix error

yes No

yes

NoNo

yes

Assignment 1Q1. The following were some of the statements made by people in a

product development organization. Identify the fallacies if any inthe statements and relate it to the principles discussed

1. “The code for this product is generated automatically by a CASEtool- it is therefore defect –free.”

2 We are certified according to the latest process models we do


2. We are certified according to the latest process models-we donot need testing.

3. I have run all the tests that I have been running for the last tworeleases and I don’t need to run any more tests.

4. This automation tool is being used by our competitors- hence weshould also use the same tool

Assignment 1 Cont..Q2. Assume that each defect in gathering requirements

allowed to go to customers cost $10,000, and that thecorresponding costs for design defects and coding defectsare $1000 and $100, respectively. Also assume that currentstatistics indicate that on average ten new defects comefrom each of the phases. In addition, each phase also letsthe defects from the previous phase seep through What is


the defects from the previous phase seep through. What isthe total cost of the defects under the current scenario? Ifyou put a quality assurance process to catch 50% of thedefects from each phase not to go to the net phase, whatare the expected cost saving? Assume the cost of fixing bugin the same phase is $10 only.



Venn Diagram

• to show relationship between the set T of test cases and the sets S and P of specified and implemented behaviors.

S P


S

T

The goal of tester is to make the region where these sets all intersect as large as possible

• specified behaviors exist for which no test case areavailable, the testing is necessarily incomplete.

• If certain test cases correspond to unspecifiedbehaviors, two possibility arise

Venn Diagram..


test case is unwarranted or

the specification is deficient.

Role of Testing in Software Evolution

• whole life-cycle process- must be applied at each

stage in the software process.

• The development approach determines the specific

intermediate products to be created


intermediate products to be created

• objectives must be identified for each of the product

created



Verification & Validation• Traditionally testing used to be an End-of-the-development-cycle

activity

• With Life-Cycle testing model there is a requirement to testdeliverables of each phase.

The associated activities can be classified as Verification &Validation


Validation.

• Thus, software testing is essentially the verification and validationprocesses put together.

• Both the processes are complementary in nature and providetheir own set of error filters

Verification

• Verification: Are we building the product right?

• Static and proactive in nature (QA).

• Checks if product (result of a particular phase of SDLC) conforms withits specifications.


its specifications.

• Is static in nature. Involves manual checking, doesn’t include executionof code.

• The target documents are Req. Specs., HLD, DB Design

• The methods used are Inspection, Walkthrough etc.

Validation• Validation: Are we building the right product?

• Dynamic and reactive in nature (QC).

• Checks the software to insure it meets customer’s requirements.


• Is dynamic in nature. Involves execution of code.

• The target could be a component, module, a set of integratedmodules or system as a whole.

• The methods used are Black box, white box testing etc.



Verification & Validation

Req Specs

HLD

Detailed Design

Verify

Verify

Verify


Code

Integration

Verify & Validate

System Testing

UAT

Verify & Validate

V Model

• The V-model is a software development process whichcan be presumed to be the extension of the waterfallmodel.

• Instead of moving down in a linear way, the process


steps are bent upwards after the coding phase, to formthe typical V shape.

• The V-Model demonstrates the relationships betweeneach phase of the development life cycle and itsassociated phase of testing.

V Model

• The V-model deploys a well-structured method where each phase can be implemented by the detailed

documentation of the previous phase.

• Testing activities like test designing start at theb i i f th j t ll b f di d


beginning of the project well before coding andtherefore Saves a huge amount of the project time and thus helps

in cost reduction

Assures better validation

Assures involvement of people with appropriate skills.



V Model


V Model

• The V-Model further promotes the notion of earlytest preparation. finds faults in baselines and is an effective way of

detecting faults early

Th V d l t th id th t


• The V-model promotes the idea that the dynamic test stages (on the right hand side of the

model) use the documentation identified on the left handside as baselines for testing.

V Model: Problems• Derivative of Waterfall model => no provision for increments

• V Model focuses on specific dynamic test stages

And has no mention of the effectiveness of static tests such as reviews,inspections, static code analysis and so on.

This is a major omission and the V-Model does not support the broaderview of testing as a constantly prominent activity throughout the


g y p y gdevelopment lifecycle.

• No mention of important activities like:

Planning

Post implementation maintenance

Testing Quality Control



W Model


Objectives

• In general, the objective is to insure that the product

satisfies the user needs.

The discovery of defects in a system

The assessment of whether or not the system is usable in


The assessment of whether or not the system is usable in

operational situations.

Verification Techniques




Recall

• There are two basic types of testing

Execution-based testing

Non-execution-based testing

• “V & V”

Verification


Determine if the workflow was completed correctly

Validation

Determine if the product as a whole satisfies itsrequirements

• Remember

The terms “verify” / “review” are also used for all non-execution-based testing

Case Study ‘log in’ Project

• Consider the requirement for ‘log in’ project as

follows

Ability to login into the system using the correct user id

and password


p

Facility to change the password

Ability to reset the password in case the user forgets the

password

Case Study ‘log in’ Project cont..

• Refined Requirement The purpose of the application is to allow the users to login into

the system.

The user will be required to enter a user is and password to login

into the system


y

Changing the password will be allowed any time

In case the user forgets his/her password, there will be a provision

of recovering the passwords. The recovery of the password will be

dependent on the user specifying their email ids correctly. The link

to reset the password will be sent to their email ids.




• System design

Designing the screen elements

Designing the look & feel of the screen

Design database


Design database


Login ID Login Id

Password Password

Login Screen


Log In Reset

Forgot Password? Change Password?


New Password

Change Password Screen


New Password

Change Reset



• Login Table

Row no

User ID

Name



Name

Password

emailid

• Coding

Login Screen 8h

Change Password screen 6h



Reset Password Screen 4h

• Testing 60 min

Defect in change password screen





• Defect found at testing phase Update requirement document 10 min

Update Design document 60 min

Update code 90 min

Retest 30 min

T t l ti t fi d f t 190 i

Scenario 1


Total time to fix defect 190 min

Total time to complete change password screen

Actual Coding Time 6h

First time testing 60min

Fix bug 190

Total time 610min

Scenario 2

• Using Review Technique

• Defect not found at the time of requirement review

• Defect found at design review


• Total time to implement Change password Screen?

Reviews

• A review is any activity in which a work product is distributed /presented to reviewers who examine it and give feedback.

• In execution-based testing, one defect may mask another so severalexecutions might be required.

On the other hand, many different defects may be discovered in asingle review.


• Because of the reuse of domain and programming knowledge,reviewers are likely to have seen the types of error that commonlyarise

• In a peer review, co-workers of a person who created a software workproduct examine that product to identify defects and correctshortcomings



What Can be Reviewed

• All interim and final development work products,including: Requirements specifications

User interface specifications and designs

HLD, LLD


HLD, LLD

Source code

Test plans, designs, cases, and procedures

Software development plans, includingproject management plan,

configuration management plan, and

quality assurance plan

Objective of Review

• Verify whether the work product

Correctly satisfies the specifications found in any predecessorwork product.

Adheres to standards.

• Suggest improvement opportunities to the author


Suggest improvement opportunities to the author.

• Gain consensus amongst the project team members.

• Secure approval from stakeholders.

• Promote the exchange of techniques and education of theparticipants.

Review: Categories

• Technical Ensure conformance to specifications. Check if development is in accordance to plans, standards

and guidelines. Ensure changes to software elements are implemented

properly.


…

• Managerial Check if project making progress as per plan. Identify and evaluate alternatives. Maintain global control through adequate resource allocation. …



Techniques

• Pass-around/ Informal review

• Walkthrough

• Inspection


• Technical Review

Verification Techniques…


…Passaround/ Informal Review

Participants & Entry-Criteria

• Participants The author selects the participants in a passaround. No specific roles are assigned. However, it can be beneficial to

ask individual reviewers to examine the work product fromspecific perspectives like: checking internal cross-references, confirming interface definitions, checking conformance to standards


checking conformance to standards, or identifying grammatical and typographical errors.

• Entry Criteria The author selected a passaround review approach. The author has stated his objectives for the review. The document has been spell-checked.



Tasks & Responsibilities

Task Responsible

Either distribute a physical or electronic copy of the work productto each reviewer, or create a file to which all reviewers haveread/write access.

Author

Notify reviewers that work product is available and indicate the Author


date by which review comments are to be provided.

Either hand-write comments directly on the work product, orenter comments into the work product file, using a revision marksconvention or an annotation feature of the tool being used. Ifworking with an individual copy of the work product, deliver thework product with comments to the author after completing thereview.

Reviewers


Task Responsible

After the final date of the review period, remove work productfrom the shared file location and examine added comments.

Author

Based on reviewer comments, perform any necessary rework ofthe work product.

Author


If a shared file was used to accumulate comments, delete eachcomment from the work product as it is resolved. Leave anycomments that require discussion in the work product untilagreement is reached on actions to be taken.

Author

Deliverables, Verification, Exit-Criteria

• Deliverables

Modified work product

• Verification

No verification of rework is required.

The author is responsible for making appropriate decisions on


The author is responsible for making appropriate decisions onissues and for correctly performing any rework.

• Exit-Criteria

The author has addressed all review issues raised..



Review Techniques…


…Walkthrough

Walkthroughs

• A walkthrough is an informal way of presenting a technicaldocument in a meeting.

• The author runs the walkthrough: calling the meeting,inviting the reviewers, soliciting comments and ensuring thateveryone present understands the work product.

• After the meeting the author should follow up with


• After the meeting, the author should follow up withindividual attendees who may have had additionalinformation or insights.

• The walkthrough is preceded by preparation bydesigners/developers Lists of itemsItems not understoodItems that appear to be incorrect

Managing walkthroughs

• Document driven Person responsible for document walks the reviewers

through the document

Reviewers interrupt with comments triggered by thepresentation


• Interactive process

• Not to be used for the evaluation of participants



Participants & Entry Criteria

• Participants The author selects the participants in a walkthrough. No

specific roles are assigned.

• Entry Criteria The author selects a walkthrough review approach for


The author selects a walkthrough review approach forthe product being reviewed.

The author has stated his objectives for the review.


Task Responsible

Select review participants, obtain their agreement to participate, and schedule a walkthrough meeting.

Author

Distribute work product to reviewers prior to the meeting. Author

Describe the work product to the reviewers during the meeting in Author


Describe the work product to the reviewers during the meeting in any appropriate way. Lead discussion on the topics of interest or concerns about the work product.

Author

Present comments, possible defects, and improvement suggestions to the author.

Reviewers

Based on reviewer comments, perform any necessary rework of the work product.

Author

Deliverables, Verification, Exit-Criteria

• Deliverables Modified work product

• VerificationN ifi ti f k i i d Ch


No verification of rework is required. Changes aremade at the author’s discretion.

• Exit-Criteria The author has made any appropriate changes in the

work product.





Inspections

Participants

• In general, a work product should be reviewed by: The author of any predecessor document or

specification

Someone who must base their subsequent work on thework product


p

Peers of the author

Anyone responsible for a component to which the workproduct interfaces

• Attendance by anyone with supervisory authorityover the author is by invitation of the author only.

Roles - Responsibilities

• Author: Creator or maintainer of the work product to be

inspected.

Initiates the inspection process by asking the peerreview coordinator to assign a moderator.


g

States his objectives for the inspection.

Delivers work product and its specification orpredecessor document to moderator.

Works with moderator to select inspectors and assignroles.

Addresses items on the Issue Log.

Reports rework time and defect counts to moderator.




• Moderator Plans, schedules, and leads the inspection events.

Works with author to select inspectors and assign roles.


Assembles inspection package and delivers it toinspectors at least 3 days prior to the inspectionmeeting.

Determines whether preparation is sufficient to hold themeeting. If not, reschedules the meeting


• Moderator (Contd.) Facilitates inspection meeting. Corrects any

inappropriate behavior. Solicits input from inspectors as reader presents each

section of the work product. Records any action items or side issues that arise during


Records any action items or side issues that arise duringthe inspection.

Leads inspection team in determining the work productappraisal.

Serves as verifier or delegates this responsibility tosomeone else.

Delivers completed Inspection Summary Report to theorganization’s peer review coordinator.

Roles - Responsibilities• Reader

Presents portions of the work product to the inspection teamto elicit comments, issues, or questions from inspectors.

• Recorder

Records and classifies issues raised during inspectionmeeting.


meeting.

• Inspector

Examines work product prior to the inspection meeting to finddefects and prepare for contributing to the meeting.

Records preparation time.

Participates during the meeting to identify defects, raiseissues, and suggest improvements.




• Verifier Performs follow-up to determine whether rework has

been performed appropriately and correctly.

• Peer Review CoordinatorC t di f th i ti ’ i ti t i


Custodian of the organization’s inspection metricsdatabase.

Maintains records of inspections conducted and datafrom the Inspection Summary Report for eachinspection.

Generates reports on inspection data for management,process improvement team, and peer review processowner.

Entry Criteria• The author selected inspection as the approach for the product

being reviewed.

• All necessary supporting documentation is available

• The author has stated his objectives for this inspection.


• Reviewers are trained in the peer review process.

• Documents to be inspected are identified with a versionnumber. All pages are numbered and line numbers aredisplayed. The documents have been spell-checked.

Entry Criteria• Source code to be inspected is identified with a version

number. Listings have line numbers and page numbers.

• Code compiles with no errors or warning messages usingthe project’s standard compiler switches. Errors foundusing code analyzer tools have been corrected.


• For a re-inspection, all issues from the previous inspectionwere resolved.

• Any additional entry criteria defined for the specific type ofwork product are also satisfied.



Planning: Tasks & Responsibilities

• Give moderator the work product to be inspected andsupporting documents, such as specifications,predecessor documents, or pertinent test documentation:

Author

• Determine whether work product satisfies inspection entry


criteria:

Moderator

• Based on the size and complexity of the work product,determine how many inspection meetings will be required.

Moderator and Author

Planning: Tasks & Responsibilities

• Select inspectors and assign roles to individuals. Gain agreement fromthe other inspectors to participate:

Moderator and Author

• Determine whether an overview meeting is required:

Author


• Schedule the inspection, and possibly overview, meetings anddistribute a meeting notice:

Moderator

• Distribute the inspection package to the participants at least 3 workingdays prior to the inspection meeting:

Moderator or Author

Overview: Tasks & Responsibilities

• Describe the important features of the workproduct to the rest of the inspection team. Stateinspection objectives: Author


• Evaluate the assumptions, history, and context ofthe work product: Inspectors



Preparation: Quiz• Ask individual inspectors to prepare with specific objectives in mind,

such as: checking for interface errors; checking traceability to, andconsistency with, predecessor specifications; or checkingconformance to standards. Moderator and Author

• Examine the work product, to understand it, find defects, and raisequestions about it. Use the appropriate defect checklist to focus


attention on defects commonly found in the type of product beinginspected. Use other analysis methods to look for defects asappropriate. Inspectors

• Log minor defects found, such as typographical errors or styleinconsistencies, on the Typo List. Deliver this to the author at or priorto the inspection meeting. Inspectors

Inspection Meeting: Process - Quiz

• Open the Meeting: Introduce the participants (if necessary) and statetheir roles, state the purpose of the inspection, and direct inspectors tofocus their efforts toward finding defects, not solutions. Remindparticipants to address their comments to the work product underreview, not to the author.

Moderator

• Establish Preparedness: Ask each inspector for his preparation time


• Establish Preparedness: Ask each inspector for his preparation timeand record the times on the Inspection Summary Report. Ifpreparation is insufficient, reschedule the meeting.

Moderator

• Present Work Product: Describe portions of the work product to theinspection team.

Reader


• Raise Defects and Issues: Point out concerns, potential defects,questions, or improvement opportunities after the reader presentseach section.

Inspectors

• Record Issues: Capture the information on the Issue Log for eachissue raised. State aloud what was recorded to make sure it was


recorded accurately.

Recorder

• Answer Questions: Respond briefly to any specific questions raised,and contribute to defect detection based on special understanding ofthe work product.

Author




• Make Product Appraisal: After all meetings scheduled for a giveninspection are complete, decide on the work product appraisal. If theinspectors disagree, assign the most conservative appraisal offeredby any of the inspectors. Inspectors

• Sign Inspection Summary Report: All participants sign theI ti S R t t i di t th i t ith th


Inspection Summary Report to indicate their agreement with theinspection outcome. Inspectors

• Collect Inspection Feedback. Ask the inspectors to evaluate theinspection and suggest improvements, using the Inspection LessonsLearned Questionnaire. Moderator

Information to Record for Each Defect Found

Origin Development phase in which the defect was introduced

Type Missing (something needs to be there but is not) Wrong (something is erroneous or conflicts with something else) Extra (something unnecessary is present) Usability Performance Non-defect issue (question, point of style, suggestion,

clarification needed)


clarification needed)

Severity Major (could cause product failure or cost significantly more to correct in the future)

Minor (non-fatal error, cosmetic problem, annoyance, or a workaround is available)

Location Page and line or section number where the defect is located

Description Concise description of the issue or possible defect

Inspected Work Products: Possible Appraisals

Appraisal Meaning

Accepted As Is Modifications may be required in the work product, but verification of the modification is not necessary.

AcceptConditionally

Defects must be corrected, and the changes must be verified by the individual named on the Inspection Summary Report.


Re-inspectFollowingRework

A substantial portion of the product must be modified, or there are many changes to make. A second inspection is required after the author has completed rework.

Inspection NotCompleted

A significant fraction of the planned material was not inspected, or the inspection was terminated for some reason.



ReworkTask Responsible

Correct defects and typos found, resolve issues raised, and modify work product accordingly. Mark issues list to indicate action taken.

Author

Correct any other project documents based on defects identified in the inspected work product.

Author


Record any uncorrected defects in the project’s defect tracking system.

Author

If rework verification is not needed, report the number of major and minor defects found and corrected and the actual rework effort to the moderator.

Author

Record the actual rework effort on the Inspection Summary Report.

Moderator

Follow-UpTask Responsible

• Confirm that the author has addressed every item on the Issue Log. Determine whether the author made appropriate decisions as to which defects not to correct and which improvement suggestions not to implement.

Verifier

• Examine the modified work product to judge whether the Verifier


rework has been performed correctly. Report any findings to the author, so rework can be declared complete, incorrect rework can be redone, or items that were not originally pursued can be addressed.

• Report the number of major and minor defects found and corrected and the actual rework effort to the moderator.

Author

Follow-Up

Task Responsible

Check whether the exit criteria for the inspection and for the peer review process have been satisfied. If so, the inspection is complete.

Moderator

Check the baselined work product into the project’s configuration Author


Check the baselined work product into the project s configuration management system.

Author



Measurements• The moderator shall collect the data items from each

inspection. These data items are used to calculate theprocess metrics and to monitor and improve the inspectionprocess.

• The moderator shall record the data items in the appropriatespaces on the Inspection Summary Report and Issue Log and


spaces on the Inspection Summary Report and Issue Log andreport them to the organization’s peer review coordinator.

• The peer review coordinator shall maintain these metrics in arepository and produce periodic reports of summary data forpractitioners and managers

Data to be Collected

Data Item Definition

Effort: Planning total labor hours spent by the moderator and author inplanning, scheduling meetings, assembling, duplicating,and distributing materials, and any other related tasks

Effort: Overview total labor hours spent by the participants in an overviewmeeting, if one was held


g,

Effort: Preparation total labor hours spent by the inspectors and authorpreparing for the inspection

Effort: Rework total labor hours the author spent correcting defects in theinitial deliverable and making other improvements;include verification time from the follow-up stage

Time: Meeting duration of the inspection meeting in hours

Data to be Collected

Data Item Definition

Defects Found:Major, Minor

total number of major and minor defects found by theinspection team; do not include non-defect issues raised,such as questions, requests for clarification, points ofstyle, or items from the Typo Lists

Defects Corrected: total number of major and minor defects corrected during


Defects Corrected:Major, Minor

total number of major and minor defects corrected duringrework

Size:Planned, Actual

total physical lines of code (not including comments andblank lines) or number of document pages that wereplanned for inspection and that were actually inspected

Number of Inspectors number of active participants in the inspection meeting

Inspection Appraisal inspection team’s decision about disposition of theinspected work product (accepted as is, acceptedconditionally, re-inspect following rework)



Rules to be followed

• Inspection should be carried out by 3-6 participants

• All participants can act as inspector

• The author shall not act as inspection leader

• The author should not act as reader or recorder


The author should not act as reader or recorder

• Roles may be shared among the team members

• Individual participant may act in more than one role

Deliverables

1. Baselined work product

2. Completed Inspection Summary Report

3. Completed Issue Log

4 Completed Typo Lists


4. Completed Typo Lists

5. Counts of defects found and defects corrected

Exit-Criteria1. All of the author’s inspection objectives are satisfied.2. Issues raised during the inspection are tracked to closure.3. All major defects are corrected.4. Uncorrected defects are logged in the project’s defect tracking

system.5. The modified work product is checked into the project’s

configuration management system.


6. If changes were required in earlier project deliverables, thosedeliverables have been correctly modified, checked into theproject’s configuration management system, and anynecessary regression tests were passed.

7. Moderator has collected and recorded the inspection data.8. Moderator has delivered the completed Inspection Summary

Report and defect counts to the peer review coordinator



Inspection Vs. Walkthroughs

Walkthrough Inspection

Adv. Circulation of Documents

Not Required Yes

Adv. Preparation by No Yes


Adv. Preparation by reviewers

No Yes

Presentation in the meeting by

Authors Readers

Inspection Vs. Walkthroughs

Walkthrough Inspection

Rigor Informal to formal Formal

Purpose Judge quality, find defects,training

Measure/improve quality ofproduct and process


g p p

Effectiveness Low to medium Low to very high, dependingon training and commitment

Which Review Method to Use

• Work products that fit in one or more of categories discussed,are considered high risk.

• A product is considered low risk if an undetected error will notsignificant affect the project’s ability to meet its schedule, quality, cost, and feature objectives.


• Use inspections for high-risk work products, or the high-risk portions of large

products, and for major work products that are about to be baselined.

• Less formal reviews are acceptable for other work products.



Identifying ‘High Risk’ Components

• Components that use new technology, techniques, or tools

• Key architectural components

• Complex logic or algorithms that are difficult to understand butmust be accurate and optimized


• Mission-critical / security-critical components with dangerousfailure modes.

• Components having many exception conditions or failuremodes.

• Components that are intended to be reused

Identifying ‘High Risk’ Components

• Components that will serve as models or templates for othercomponents

• Components that affect multiple portions of the product

• Complex user interfaces


• Complex user interfaces

• Components created by less experienced developers

• Code modules having high complexity

• Modules having a history of many defects or changes



Code Review



Code Review

• A code review is a special kind of inspection inwhich the team examines a sample of code andfixes any defects in it.

• In a code review, a defect is a block of code whichdoes not properly implement its requirements


does not properly implement its requirements,which does not function as the programmerintended, or which is not incorrect but could beimproved

• For example, it could be made more readable or itsperformance could be improved

Good candidates for code review

• A portion of the software that only one person has theexpertise to maintain

• Code that implements a highly abstract or tricky algorithm

• An object, library or API that is particularly difficult to work with


• Code written by someone who is inexperienced or has notwritten that kind of code before, or written in an unfamiliarlanguage

• Code which employs a new programming technique

• An area of the code that will be especially catastrophic if thereare defects

Checklist for Code Review: Example

Data declarationAll variables declared?

Default values understood?

Arrays and strings initialized?

V i bl ith i il ?

Control flowEach loop terminates?

DO/END statements match?


Variables with similar names?

Correct initialization? Input/output

OPEN statements correct?

Format specification correct?

End-of-file case handled?





Deskchecks

Deskchecks

• A deskcheck is a simple review in which the author

of a work product distributes it to one or more

reviewers.

The author sends a copy of the work product to selected


py p

project team members. The team members read it, and

then write up defects and comments to send back to the

author.

Deskchecks

• Unlike an inspection, a deskcheck does notproduce written logs which can be archived with thedocument for later reference.

• Deskchecks can be used as predecessors to


inspections. In many cases, having an author of a work product pass

his work to a peer for an informal review will significantlyreduce the amount of effort involved in the inspection.





Pair Programming

Types of Review: Pair Programming

• Pair programming is a technique in which two programmers worksimultaneously at a single computer and continuously review eachothers’ work.

• Although many programmers were introduced to pair programming as apart of Extreme Programming, it is a practice that can be valuable in anydevelopment environment.


• Pair programming improves the organization by ensuring that at leasttwo programmers are able to maintain any piece of the software.

Types of Review: Pair Programming

• In pair programming, two programmers sit at one computer towrite code. Generally, one programmer will take control andwrite code, while the other watches and advises. Some teams have found that pair programming works best

for them if the pairs are constantly rotated; this helpsdiffuse the shared knowledge throughout the organization.

Others prefer to pair a more junior person with a moresenior for knowledge sharing.


• The project manager should not try to force pair programmingon the team; it helps to introduce the change slowly, and whereit will meet the least resistance. It is difficult to implement pair programming in an

organization where the programmers do not share thesame nine-to-five (or ten-to-six) work schedule.

Some people do not work well in pairs, and some pairs donot work well together.



Testing Software Requirements

• User being the dominant party, should takeresponsibility for requirement phase testing

• In case of multiple users, responsibility may beassigned to a committee, which may be the same


committee that develops the requirements

Objective• Most cost effective phase to detect a system flaw

• Primary objectives of testing Determine that the requirements fairly represent what the

user needs

Determine that the needs have been defined anddocumented


Verify that a cost/benefit study has been performed andthat it is reasonable

Determine the business problem has been solved

Verify that the control requirements have been specified

Verify that a reasonable process was followed indeveloping the business solution

Input

• The most common deliverables from the requirement phase

• Proposal to management describing the problem, the

alternatives, and proposing a solution

• Cost/benefit study describing the economics of the proposed


solution

• Detailed description of the recommended solution, highlight

the recommended method for satisfying those needs

• List of system assumptions, such as the life of the project, the

value of the money, the average skill of the user and so on



Do procedure• Prepare a risk matrix

• Perform a Test Factor Analysis for the Requirement Phase

• Conduct Requirements Walkthrough


Do procedure cont..

• For example need to verify

Requirements Comply with Methodology (Methodology Test

Factor)

Functional Specification Defined (correctness test factor)

Usability Specifications determined (Ease-of-use Test factor)


Maintenance Specifications Determined (Maintainable Test

factor)

Portability needs Determined (portable Test factor)

System Interface Defined (Coupling Test Factor)

Performance Criteria Established(Performance Test factor)

Check Procedures and output

• Only output is a report indicating requirementsdeficiencies




Design Phase Testing


Provides the opportunity to test the structure

Overview• Involves the close working relationship between the user

and the system designer

• System design is an IT responsibility


Objective

• Design phase concerns are described and a testingprocess is proposed to assess whether thoseconcerns have been adequately addressed by theproject team


• Scoring the design success factor, conducting adesign review and inspecting design deliverables



Inputs• The common design phase deliverables for testing

• Input specifications

• Processing specifications

• File specifications

• Output specifications


• Output specifications

• Control specifications

• System flowcharts

• Hardware and software requirements

• Manual operating procedure specifications

• Data retention policies

Concerns to be analyzed

• Data integrity controls designed

• Authorization rules designed

• File integrity controls designed

• Contingency plan designed


Contingency plan designed

• Method to achieve service level designed

• Access procedure defined

• Design complies with methodology

Concerns to be analyzed cont..

• Design conforms to requirements

• Design facilitates use

• Design is maintainable

• Design is portable


Design is portable

• Interface design is complete

• Design achieve criteria

• Needs communicated to operations



Output

• The moderator’s certification of the product,releasing the product to the next phase of theprocess


Test Software Design

• Some important concerns to be analyzed during this phase are:• Conformance to requirements

• Compliance with methodology

• Design of

Data Integrity Controls


Data Integrity Controls,

Authorization Rules,

Contingency Plans

Audit Trails,

• Maintainability

• Portability

User ManualTesting


Provides the opportunity to test the documentation



Effective Documentation

• Improves Usability

• Lowers customer support

• Improves reliability

• Increase Maintainability


Increase Maintainability

• Improve installability

• Enhance salability

• Reduces liability

Tester’s Objective

• Checking the technical accuracy of every word

• Verify that writer describe the product as it is

• Look out for missing features

• No authority to demand stylistic changes


No authority to demand stylistic changes

Benefits

• You will find more bugs than you expect

• An important source of real world combination test cases


• Bug reports arising out of documentation testing are particularly credible



How to do

• Use the program exactly as the manual says

• Try every suggestion

• Check every statement of fact and every obvious


Check every statement of fact and every obvious inference from the stated facts, instructions and suggestions

Project Audits(审计)

Project Audits


Project Audits

• In a process oriented approach for software development ,

two key task are:

Process definition

Process implementation.

• The process definition activities deal with identifying &


The process definition activities deal with identifying &

specifying processes, which when followed will give Good

Quality & Productivity.

• The process implementation activities ensure that the

defined process are followed in the project.



Cont…

• Unless the projects adhere to the standard process

• one cannot find the “Capability” of the process.

• Therefore without knowing the capability of a process one

cannot Improve the process.


• Since the Process are executed by people, there is a

possibility that the process may be skipped.

Cont…

• The reason for not following the process are:

People tend to take shortcuts

It maybe do to pressure of deadline

Or maybe some process are not clear

It may also be due to overconfidence


And also people resist changes, because it restrict their freedom

• Taking shortcut or skipping a process may not always lead

to project failure.

• A project could potentially fail if the processes are not

followed.

Cont…

• An analogy for this could be Traffic Rules.

• Just because a rash driver failed to observe

traffic rule & reaches home safely does not mean

traffic rules is unnecessary


traffic rules is unnecessary.

• Therefore a active effort is needed to ensure

compliance to the defined process.

• The basic purpose of AUDIT is to ensure

compliance to the defined process.



Cont…

• Audits are essential part of Quality Assurance KPA

of Level 2

• It is also a requirement of ISO 9000 Quality System

• There are many ways of Organizing AUDIT in an


There are many ways of Organizing AUDIT in an

Organization.

• We shall study the Audit Mechanism employed at

Infosys.

Project Audit (The Challenge)

• As the projects become larger and more complex,

understanding and providing effective validation of the

project management processes is a significant

challenge for today’s information systems


organizations.

• In addition to the awareness and implementation of the

project management procedures, recognition of the

quality of the standards and practices is critical for

continued performance improvement.

The Objective of Audit:

• Insight into the project team’s use of the project

management standards

• Identification of the project’s “Project Management” related

risks


• Detail corrective action plan for addressing the risks,

incomplete procedures and standards training

• Awareness of the areas of opportunity for improvement of

the “Project Management” methods and behaviors



Audit Process

• Auditing is a systematic & independent examination of

various activities of project execution.

• It is intended to determine compliance with the quality

system of a organization


• Its main focus is on implementing the process of an

Organization.

• It is also used to determine the effectiveness of a process &

to identify area of Improvement.

Cont…

• Audit can be Internal or External

• External Audit is generally employed for some type of

certification

• Internal audit is conducted by the Organization using people


of the Organization.

• The primary goal is to ensure compliance with

Organization’s process & help in process improvement.

Cont…

• To ensure reasonable degree of

compliance with the defined process,

audit must be done regularly.

• The people performing audit could be

b f SEPG h h


member of SEPG or person who has

maturity & stature to assess the

implementation on a project

objectively.

• It could also be performed by other

project members.



Cont…

• When others perform Audit, they can learn & give

advice.

• Since they become “Law Keepers” they also

respect & appreciate “Law”


respect & appreciate Law .

• The Audit activity has three components:

Planning

Auditing

Follow-up

Planning

• Before audits are conducted, they must be carefully

planned to achieve optimal result.

• Planning of Audit operates at three level:

Strategy


gy

High-level plan

Detailed schedule.

Audit strategy

• The Audit strategy defines how Audit will be scheduled &

planned, so as to monitor compliance & the effectiveness of

processes.

• The strategy at Infosys are:

A dit d t d thl


Audit are conducted monthly

During a audit, a sample of the project is selected for audits, but the

projects selected are not known before the month of Audit

Each month has some focus area for audit, which will be examined in

detail during audit.



Audit PLAN• The Audit PLAN is an implementation of the audit strategy

for a specific period say 6 to 12 months.

• The PLAN specifies how strategy will be implemented,

such as

Selection of focus area


Selection of focus area

Selection of projects

Selection of auditors

So…on..

• The plan covers some of the key areas for project that are

closely related with CMM.

Auditing

• A team of two people normally conduct the internal

Audit who are selected from auditor’s pool.

• Auditor’s pool is a set of people who are trained to

conduct audit.


• One standby Auditor is also selected

• A remainder is sent to Auditors & project Leaders,

whose project will be Audited, one day before the audit

begins.

Cont…

• On the day of audit, auditors meet the Quality advisor

associated with the project to get views about the process

used in the project

• The team plans out its audit strategy

Wh t ti t k


What question to ask

Who will they interview

What artifacts are needed

• In actual audit, the auditors focus more attention on whether

the defined process is followed in the project.

• A check list can also be used



Sample checklist

• Project Planning Checklist

• Is the project plan documented in the standard project plan

template?

• Has the project plan been group reviewed?


• Has the project plan been approved & baselined?

• Is it under configuration management?

• Is there a signed Contract?

• Have the commitments to the customers or other group

been reviewed?

Cont…

• Is there an estimated effort for the project that is

based on historical data?

• Have the effort estimates & the schedule been

reviewed?


reviewed?

• Has the quality plan been reviewed

• Is the life cycle used in the project identified &

documented?

Cont…

• Are personnel identified & responsibility for each work

element defined & tracked?

• Are reestimation triggers such as scope changes and

required corrective actions defined?


• Are deliverables to the customer, including user

documentation, clearly identified?

• Are risk & risk mitigation plans identified & properly

documented?

• Are reviews, progress reporting, tracking, & approval

mechanism identified?



Cont..

Requirement Management Checklist

• Is there a requirements document that includes

technical & non techinical requirements?

• Have the requirements been reviewed & are the


Have the requirements been reviewed & are the

review records available?

• Has the requirements document been signed off by

the customer & other affected groups?

• Are changes to requirement logged?

Cont…

• Has traceability to changed requirement been established in

other work products?

• Has requirement change threshold been negotiated with

customer?


• Is status of changed requirement available & maintained

properly?

• Are acceptance criteria defined & signed off by customer?

• Is there record of the re-estimation of size, effort, & other

critical resources?

Cont…

• The audit process is said to be completed when the audit team has

asked all questions at whatever artifacts they require.

• An noncompliance report (NCR) is issued if the evidence suggest that

the organization-wide process or authorized process for the project is

not being followed.


• The questions & checklist aid in unearthing noncompliance.

• An important point that is stressed during training of auditors is to focus

on process & process improvement and problems found should be

attributed to process factors & not people.

• A report is then sent to coordinator of audit (SEPG member) within three

days of conducting an audit.



Follow-up

• The audit report & NCR’s are sent to the coordinator of

audits, who is a member of SEPG, at the end of AUDIT.

• For each NCR, corrective action are taken.

• The audit coordinator get approved from the auditors for


each corrective action taken.

• An audit may also reveal weakness in process

• In such cases, the auditors may recommend modification to

SEPG team.

Audit Analysis• The data from audit of different project together offer valuable data

about the state of the implementation of the process across the

organization.

• These data can be very useful in analyzing the effectiveness of the

processes and offer scope for improvement.

• Summaries are produced for:


Number of Audits scheduled vs number conducted.

Total no of NCR given

Closer date of NCR

Distribution of NCR by severity

• These summaries yield information about the health of the audit

system and seriousness with which it is conducted.

• Its also offers visibility into the implementation of audit process.




Objective Questions

1. Define the following

i. Software Testing

ii. Mistake

iii. Fault

i F il


iv. Failure

v. Test case

vi. Test suite

2.Define Incident

3. Define Pair Programming

4. Differentiate Fault of Omission vs. Fault of Commission.

Objective Questions5. Differentiate Verification vs. Validation

6. Provide a sample Test Case template

7. Differentiate Error, Fault, Incident and Failure with example?

8. Make a Venn Diagram that reflects a part of the following

statement:”.. we have left undone that which we ought to have

done and we have done that which we ought not to have


done, and we have done that which we ought not to have

done..”.

9. What argument would you give to a person who says that the

testing job is not challenging.

10.Differentiate between “Testing as a Service” and “Product

Testing” Organization

Short Questions

1. Provide do Procedure for testing software requirements

2. What are the important concern(s), need to be analyze

during Design phase Testing?

3. How Walkthrough is different from Inspection technique,

4 Provide a sample check list for code review


4. Provide a sample check list for code review.

5. What is the purpose of Requirement Traceability matrix?

6. How defects from early phases to later phases of SDLC

add to the cost?



Long Questions1. Discuss the limitation of software testing with reasons.

Support your answer with example.

2. Write short note on V model for Software Development

process

3. Discuss Software Testing Life Cycle

4 Write short note on Project Audits


4. Write short note on Project Audits.

5. Discuss the different roles & responsibilities in Inspection

process

6. Discuss the importance of any two graphs in testing with

example

i) Program Graph ii) Finite State Machine

iii) State Chart

Research Problems

Use venn Diagram to show relationship between the

set T of test cases and the sets S and P of

specified and implemented behaviors. Represent

the following


•Test cases which are not for specified behavior

•Test cases which are not for implemented behavior

•Test cases which are not for specified as well as

implemented behavior

1. Paul C. Jorgenson, Software Testing A Craftsman’s approach,CRC Press, 1997.

2. Roger S. Pressman, “Software Engineering – A Practitioner’sApproach”, , McGraw-Hill International Edition, New Delhi, 5th

Ed., 2001.3. Boris Beizer, “Black-Box Testing – Techniques for Functional

References


3. Boris Beizer, Black Box Testing Techniques for FunctionalTesting of Software and Systems”, John Wiley & Sons Inc.,New York, 1995.

4. K. K. Aggarwal & Yogesh Singh, “Software Engineering”, NewAge International Publishers, New Delhi, 3rd Ed., 2003.

5. Marc Roper, “Software Testing”, McGraw-Hill Book Co.,London, 1994.



6. Gordon Schulmeyer, “Zero Defect Software”, McGraw-Hill,New York, 1990.

7. Watts Humphrey, “Managing the Software Process”, AddisonWesley Pub. Co. Inc., Massachusetts, 1989.

8. Boris Beizer, “Software System Testing and QualityAssurance”, Van Nostrand Reinhold, New York, 1984.

References


Assurance , Van Nostrand Reinhold, New York, 1984.9. Glenford Myers, “The Art of Software Testing”, John Wiley &

Sons Inc., New York, 1979.10.Elfriede Dustin, “Effective Software Testing: 50 Specific ways

to improve your Testing”, Pearson, 2003.11.Dorothy Graham, Erik Van Veenendaal, Isabel Evans and Rex

Black, “Foundation of Software Testing, ISTQB Certification”,PHI, 8th Ed., 2009.

Date post:	03-May-2018
Category:	Documents
Upload:	hoangngoc
View:	220 times
Download:	2 times