Software Testing Slides

E&R Infosys 1ER/CORP/CRS/SE04/003

Software Testing

2E&R InfosysER/CORP/CRS/SE04/003

Objectives

Quality – a cultureSoftware Testing Life CycleSoftware Testing Techniques Types of testingIntroduction to Test automation and Test toolsTest Planning and Strategizing


Agenda

What is quality?What is testing?Testing life cycleTesting TechniquesTypes of TestingTest AutomationTest Tools


What is Quality?

“The degree to which a system, component or process meets requirements”

or“The degree to which a system, component or process meets customer or user needs or expectations”

-I.E.E.E.


Quality is…

What the customer says it is… Features Cost Time

Not a goal, but a means for continuous improvement


Dimensions of Software Quality

Functionality Completeness Correctness Compatibility

Performance Time Resources

Maintainability Correctability Expandability Testability


Dimensions of Software Quality (contd.)

Portability Hardware independence Interoperability

Reliability Error tolerance Availability

Usability Understandability Learnability Operability Communicativeness


Software Quality Issues

Scope Creep / Churning requirementsFuzzy RequirementsLack of Skilled ResourcesLack of a most suitable processFlaws in DesignIncomplete Test PlanningLate TestingUnreliable Data for TestingImproper Defect Tracking

User or Business needs not MetPoor End-user ExperienceDifficult-to-Maintain SystemsBuild and Release IssuesModules Don’t IntegratePoor Performance under LoadFlaws Discovered Very LateDefect Leakage

Challenges Problem Symptoms


Testing – What Is it?

A process of demonstrating that errors are not present?

ORA way of establishing confidence that a program does what it is supposed to do?

ORA means of achieving an error-free program by finding all errors?


Testing is…

A process of executing a program with the intent of finding errors

A “DESTRUCTIVE”, yet creative process


Why Testing?

Verifies that all requirements are implemented correctly (both for positive and negative conditions)Identifies defects before software deploymentHelps improve quality and reliabilityMakes software predictable in behaviorReduces incompatibility and interoperability issuesHelps marketability and retention of customers


Cost of Ineffective Testing

Time Late Releases Projects need to be reworked or abandoned

Money Budget over-runs Defects are 100 to 1000 times more costly to

find and repair after deploymentQuality

Developers unsure of product quality Products released with undiscovered or

unresolved defects


Testing LifecycleIteration

nDevelopment LifecycleProject Planning

Requirements Capture

Implementation

Build 0 Build 1 Build 2

Test Lifecycle

Plan Test

Analysis & Design

Design Test

Setup Test Bed

Execute TestEvaluate Test

Defect Tracking/Enhancement Request


Testing LifecycleRequirements

Capture

Analysis

Test planning & Scenario Design

Test Case Development

Test Execution

Test Result Analysis

Test Cycle Closure

Defect Fixing Cycle

Defects

New Version

The Scenarios Design and the Test Case Development could normally start in parallel with the Development Cycle.Test Execution Synchs up with the Development Cycle during the functional Testing Phases


Testing Lifecycle

Test requirements identification Validate for testability

Test Planning and Scenario design Develop Test Objectives Identify Test Items Resources and Schedules

Test Case Development Test Case Specification Pre-requisites, post-requisites and Acceptance

criteria


Testing Lifecycle (contd.)

Prepare Test Bed Test Data Preparation Test Environment Setup

Test Execution Run Tests and Validate Results Bug Reporting Bug fixes and retesting

Test Result Analysis Defect Analysis Determining Test Set Coverage and

Effectiveness


Start Testing Early

There are two basic forms of testing Verification Testing Validation Testing

Verification: “Are we building it right?”

Validation: “Are we building the right thing?”


Testing Stages – the V-ModelRequirement Analysis

Functional Specifications

High-Level Design

Detailed Design

Code

User Acceptance Testing

System Testing

Integration Testing

Unit Testing

Verification ValidationReviews


Testing Techniques

Static TestingDynamic TestingWhite Box TestingBlack Box Testing


Testing Techniques (contd.)

Static Testing/Reviews Scope

Requirements FeasibilityTechnical ArchitectureDesignProgram specificationsTest PlansTest Cases and DataUser Documents



Static Testing Advantages

Capture defects early, so saves rework costChecklist-based approachFocuses on coverageGroup perspectiveHighest probability of finding defects

DisadvantagesTime-consumingCannot test data dependenciesHigh skill levels required



Dynamic Testing White-box/Glass-box Testing

Unit TestingIntegration/Interface Testing

Black-box TestingSystem TestingAcceptance testing


White-Box/Glass-box Testing

Structural Testing processProgram Logic-driven TestingDesign-based TestingExamines the internal structure of program


White-Box Testing – Advantages

High Code coverage – Exhaustive (thorough) path testingProgram logic is testedInternal boundaries are testedTesting is algorithm specific


Black-box Testing

Specifications-based TestingBusiness Transaction-driven TestingNo concern to internal behavior and program structure


Black-box Testing – Advantages

User’s perspectiveFocus on features, not implementationBig-picture approach


Levels of Testing

Unit TestingIntegration TestingFunction Testing Performance Testing SystemAcceptance Testing TestingInstallation Testing}

E&R Infosys 28

Unit Testing


Testing Stages – the V-modelRequirement Analysis


High-Level Design

Detailed Design

Code


System Testing

Integration Testing

Unit Testing



What is Unit Testing?

Testing of individual units or groups of related code is known as Unit Testing Unit tested with a Unit Test Plan


Unit Test Plan should test for:

Standards specified (Ex. GUI related)Field validations, boundary valuesBasic functionality of the component (add/delete/modify/query activities)Code coverage and exception handlingNegative or destructive testingMemory leaksData computation/integrity


Unit Test Plan should test for:

Standards specified (Ex. GUI related)Field validations, boundary valuesBasic functionality of the component (add/delete/modify/query activities)Code coverage and exception handlingNegative or destructive testingMemory leaksData computation/integrity

Usability

Functionality

Performance


Unit Testing – White Box Technique

Activities Identification of Testable units/modules Focus on code and logic Design logic-based Test cases and prepare Test

data Run tests and verify output through path

traversing


Unit Testing – White Box Technique

Logic Coverage method includes: Statement coverage Decision (branch) coverage Condition Coverage Path Coverage


Unit Testing - Black Box Technique

Commonly used: Equivalence partitioning Boundary value analysis Error Guessing

Less commonly used: Cause effect graphing State-transition testing


Equivalence Partitioning

Systematic process of identifying a set of input conditions to be tested

The two distinct steps involved: Identify equivalence classes Identify test cases



Equivalence Classes – Valid Data User supplied command Responses to system prompts File names Computational data Physical parameters, bounding values,

initialization values Output data formatting commands Responses to error messages Graphical data (e.g. mouse picks)



Equivalence Classes – invalid Data Data outside bounds of the program Physically impossible data Proper value supplied in the wrong place


Boundary Value Analysis

Check edge values of all Output domain User queries Numerical data Mouse picks on menu Output format requests Responses to prompts Command key input


Error Guessing

Ad Hoc method to identify tests likely to expose errors based on experience and intuitionSome areas to guess are:

Empty or Null lists / strings Zero instances / Occurrences Blanks or null characters in strings Negative numbers


Advantage/Disadvantage

Equivalence Partitioning Significantly reduces number of input

conditions to be tested Does not test combinations of input conditions

Boundary Value Analysis Boundary conditions sometimes subtle and

difficult to identify Does not test combinations of input conditions


Exercise

Writing a unit test plan…


Recap

What is qualityDimensions of QualityTesting TechniquesUnit Test

E&R Infosys 44

Integration Testing




High-Level Design

Detailed Design

Code


System Testing

Unit Testing

Integration Testing



Integration Testing

Combining and testing multiple components togetherIntegration of modules, programs and functionsTests Internal Program interfacesTests External interfaces for modules


Integration TestingBanking System

CRM HR Financial

Java Based Based on C, C++ VB based

Online banking Loans Account Opening

External interface

Inter module interface

Intra module interface

Unit Module

External System


Approach for testing interfaces Intra-Module Inter-Module External

Test Techniques

White Box Black Box Black Box

Test Environment Setup

Development environment

Isolated development environment

Simulate live environment

Test Approach Check for parameter / procedure called.Database updation

Identify pre-process and approach to Interface.Identify the post-process to validate the interface

Identify external / internal interface and validate against external system


Techniques based on order of Integration

Big-bang Integration (non-incremental)Random Incremental IntegrationTop-Down IntegrationBottom-up Integration


Top-down vs. Bottom-up

Top-down Integration Advantage

Key Interface defects trapped earlier Disadvantage

Core functionality tested late in the cycleBottom-up Integration

AdvantageCore functionality tested early

DisadvantageKey Interface defects trapped late in the cycle

E&R Infosys 51

System Testing




High-Level Design

Detailed Design

Code


Integration Testing

Unit Testing

System Testing



System Testing – What is it?

A Black-box Testing techniqueSpecifications-based TestingTypically independent team testingSimulated environment testingLive/Simulated user dataTests the whole systemFunctional and non-functional requirements tested Business transaction-driven testingCompatibility errors uncoveredPerformance limitations uncovered


System Testing – Different Types

Functional testing (Sanity / Regression )Performance and Scalability testingLoad/Stress testingVolume TestingUsability testingConfiguration testingInstallability testingDisaster and Recovery testingReliability / Availability testingStatistical Testing based on Usage Models


Functional Testing – Sanity Test

Very basic minimal number of tests to verify the product for the feature / protocol complianceCould be a subset of regression test used to validate only high level functionalityTypically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effortSanity testing will be performed in scenarios when cursory testing is sufficient to prove that the application is functioning properly


Why Sanity Testing?

Regression testing is more time taking and more effort is required whereas Sanity testing is quicker focusing on important functionality of softwareSaves time as only selected high level functionality areas are only tested. Bugs found this way would take almost triple the time when found through regression testingEfficient in scenarios wherein releases comes in pretty often with very few changes


Functional Testing – Regression

Re-execution of one or more tests in subsequent build of the application/product to ensure

Revisiting and testing all prior bug-fixes in response to a new fix/enhancement

Re-testing all programs that might be affected by the fix/enhancement

Hidden Bugs are uncoveredThe baseline for Regression Testing grows with every build


Regression Testing

Should Cover Business Process depending on criticality User friendliness Cross functional dependencies

Activities System appreciation Preparation of Test case repository Automate test cases Execute Regression test suite


Without Regression Testing – Hidden Bugs

Fix

PASS PASS? PASS?

FAIL FAIL PASS

PASS BUG? BUG?

Test Requirement 1

Unknown

Test Requirement 2

Test Requirement 3

New Problem

Hidden Bug

Unknown

Fix

CBA

Software Builds


With Regression Testing – No Hidden Bugs

Fix

PASS PASS PASS

FAIL FAIL PASS

PASS BUG PASS

Test Requirement 1

Test Requirement 2

Test Requirement 3

New Problem

Fix

CBA

Software Builds

Fix


Acceptance Testing

Demonstrates that the system meets mutually agreed Acceptance criteria

Critical Requirements Minimum Performance level Maximum Defect Detection Rate

Typically, a sub-set of System testing


Alpha / Beta Testing

Forms of Acceptance testingTesting in the production environmentAlpha testing is performed by end users within a company but outside development groupBeta testing is performed by a sub-set of actual customers outside the company


Performance Testing

Number of concurrent users accessing at any point in given time System’s performance under high volume of data Stress testing for systems, which are being scaled up to larger environments or implemented for the first time Operational intensive transactions. (Most frequently used transactions) Volume intensive transactions (for both volume and stress testing)


Installation Testing

Basic installationInstallation of various configurationsInstallation on various platformsRegression testing of basic functionality


System TestingA Banking System

CRM HR Financial

Java Based Based on C, C++ VB based

Online banking Loans Account Opening


When do you stop testing?

When time runs out Specific test case design methodsTarget test coverage attainedCertain number of errors found*Error detection rate drops below a threshold*Predicted density of errors left drops below a threshold

Sequence test cases in order of priority

* Infosys Process Capability Baseline


Test Artifacts

Test PlanTest Environment/BedTest CaseTest DataTest ToolsTest ScriptsTest LogBug Reports


Traits of a good Tester

Destructive creativity Detective skills Appreciating the users’ perspective Adapt to and Understand Requirements change A skeptical, but not hostile attitude An eagerness to embrace new technologies


Test Engineers

Role - Execute tests, record test scripts, maintain statistics and metrics, check test data setup, test environment setup, execute re-testsKnowledge - Understanding of the importance of testing, awareness of tools, how to progress against a planSkills - Observation, accuracy, methodical, co-ordination, problem solverExperience - Following instructions, problem reporting and solving, and relevant testing tools


Test Automation

Good candidates: Complex and time-consuming tests Tests requiring a great deal of precision Tests involving many simple, repetitive tests Tests involving many data combinations

Lower priority candidates: One-time only tests Testing peripheral devices Subjective assessment tests (look and feel

based)


Test Automation

Evolvable

Effective

Exemplary

Economic

Manual Test

First Run of Automated Test

Automated Test (After many runs)


Simple Vs Sophisticated Automation

COST

Simple Automation

Sophisticated Automation

Maintenance CostImplementation Cost

E&R Infosys 73

Testing Tools


Types of Testing Tools

The software testing tools broadly of the following type:

Functional / regression Load /Stress: Performance testing under high

load / stress Web Testing: testing the Web Sites Data Generation: Generation of test data


Benefits

Reduces manpower and timeMore coverage within the same testing time.Testing products which are very difficult to test manually. Example, Load TestsCan look inside the software for memory leaks, redundant code etc.


Test Tools (Client Server)Category Tool Vendor

Source Code Testing BoundsChecker NumegaPure Coverage RationalPurify RationalJprobe Sitraka SoftwareATTOLCoverage ATTOL Software

Functional Testing QA Run CompuwareWin Runner MercuryTeam Test RationalQA Center Compuware

Performance Testing OptimizeIT VM Gear

Load Runner Mercury

QA Load Compuware


Test Tools (Web Applications)

Category Tool VendorFunctional Testing Rational Robot Rational

Silk Test Segue SoftwarePerformance Testing Silk Performer Segue Software

Web Load RadviewLink & HTML Testing NetMechanic Monte Sano

Linkbot TetranetSite InspectorDoctor HTML Imageware


Test Tools (Network & Security)

Category Tool Vendor

Security & Communication

SNMP Test Suite Interworking Labs Inc.

Sniffer Pro Network Associates Inc.

Lophtcrack

Saint, Satan, Sara SwTech

TCP Wrappers

TCP Dump

E&R Infosys 79

Going Forward…


Testing LifecycleIteration

nDevelopment LifecycleProject Planning

Requirements Capture

Implementation

Build 0 Build 1 Build 2

Test Lifecycle

Plan Test

Analysis & Design

Design Test

Setup Test Bed

Execute TestEvaluate Test

Defect Tracking/Enhancement Request


Test Planning/Strategizing

Includes – Testing Objectives and Goals Test Strategy/Approach based on customer priorities Test Environment (Hardware, Software, Network,

Communication etc.) Features to test with priority/criticality Test Deliverables Test Procedure – Activities and tools Test Entry and Exit criteria Test Organization and Scheduling Testing Resources and Infrastructure Test Measurements/Metrics


Test Planning / Strategizing

Benefits Sets clear and common objectives Helps prioritize tests Facilitates Technical tasks Helps improve coverage Provides structure to activities Improves communication Streamlines tasks, roles and responsibilities Improves test efficiency Improves test measurability


Optimizing TestingCategorize entire gamut of testing into Sanity / Regression /Performance and StressEfficient Review Process Use Tools to do the testing (both Automation and Simulators )Look out for Automation if it helps ( leverage 24 hours a day ) Use Traceability MatrixTracking and Reporting of bugs and defects – Defining a process Revisit the Test coverage and categorization on an ongoing basisRisk strategy Retest strategy


Recap

Integration TestingSystem TestingTest AutomationTesting Tools

E&R Infosys 85

Date post:	22-Nov-2014
Category:	Documents
Upload:	sheetal-giriya
View:	139 times
Download:	1 times