Computer Science and Software Engineering

University of Wisconsin - Platteville

Note 4. Organization

Structure and Test

DocumentsYan Shi

Lecture Notes for SE 3730 / CS 5730

Outline

QA organizational structure

Test Documents

— Test plan

— Test specifications

— Context driven test planning

Good requirements to test

http://bobcesca.thedailybanter.com/files/2013/08/fox_henhouse_12.jpg

“The guardians of the hen house can’t be in the employment of the fox.”

Independent QA Organization

http://ibagroupit.com/about/structure/

Independent QA Organization

QA organizations are generally separate from organizations such as Development, Marketing, and Sales and report to different VPs.

This maintains independence and allows specialization of skills.

This independence supports:

— Decision making that primarily reflect a company’s software quality goals rather than development, financial, marketing, … goals.

QA Roles in the Lifecycle (1)

Process cops: — Is the process followed? PDCA?

Requirements and/or use cases:— current, validated and formally documented?

Test Cases: — generated and reviewed after requirements are

confirmed!

Configuration Management:— QA owns the code once it’s baselined!

Build Control:— clean and controlled process and environment.— documented and repeatable?

QA Roles in the Lifecycle (2)

During the testing process:— Unit testing (white-box): done by developers,

audited by QA

— Integration testing

— System testing

— Acceptance testing: written by the customer

— TDD: assist customer to produce automated acceptance tests.

— Simulators/Test Harness: build and maintain.

— STE(special test equipment).

Test Document Types

Test plan— Test goals, overall plans, schedules, special equipment and resources— But not the test cases! — Completed during INCEPTION or early ELABORATION phase

Test specification— Detailed test cases— Created during ELABORATION phase after requirement is detailed.— Completed during early CONSTRUCTION phase.— Evolve during CONSTRUCTION and early TRANSITION phases.

Test plan and specification are often merged into one test document.

IEEE Std 829-2008: Standard for Software Test Documentation

Test Document Styles

Inclusive test documents: — chapters for everything from unit tests to acceptance

tests.

Individual test documents:— Each software phase has its own test plan and

specification— Perhaps an overview document to link all phases

together

Unit test documents:— Sometimes it is a part of a Detailed Design

document.

Test Document Template

Introduction:— Purpose— Test approach— Definition of test environment— Definition of the build environment and version control

Test cases— Test setup and preconditions— Test inputs and steps— Expected result— Pass/Fail— Estimated and actual test time— Teardown procedure: cleanup after testing so that the running of

this test does not contaminate subsequent tests

Traceability to requirements Traceability to software

Test Document for Phone Calling

Context Driven Test Planning

• Monitor major test planning challenges

• Clarify your mission

• Analyze the product

• Analyze product risk

• Design the test strategy

• Plan logistics

• Share the plan

Seven task

themes to assist with test planning:

Kaner, Bach and Pettichord, “Lessons Learned in Software Testing”

How to Evolve a Context Driven Development Plan

Good Requirements to Test

Good requirements are essential for producing good test cases!

Forms of Requirement Spec.

Raw individual statements

Requirement Specifications:— Hierarchical structure starting from the top:

system subsystems major components

Use Cases:— Specify a “sequence of transitions performed by a

system, which yields an observable result or value for a particular actor”

— More natural; easier to understand by all parties

— Easier to validate, to write test cases for

Examples of Requirements

A1: The system shall have a wonderful users’ interface. A2: The user interface shall require four or fewer mouse clicks to initiate any transaction and follow Company XYZ’s user interface standards.A3: It would be nice to have white lettering on a blue background.

B1: The system shall deliver rapid responses to user queries.B2: The system shall deliver 98% of all users queries within 3 seconds while sustaining transaction rates of 10,000 transactions per second on the target hardware. B3: The system shall maintain a maximum transaction rate of 9,000 transactions per second.

C1: The control mechanism shall use E.W.M.A. (with a lambda of 0.3), and L.C.L and U.C.L. of 99.98 and 101.23, respectively.

SMART Requirement

SMART Requirements— Specific

Valid – meets user expectations and needs

Complete and non-ambiguous – where interpretation is necessary, there will be different interpretations.

• “I saw the man in the park with a telescope.”

Technical level –User technical terms, algorithms, business logic need to be clearly defined (generally in a glossary). C1

— Measurable Testable – the system can test and have very little uncertainty whether the

requirement was satisfied or not satisfied. (A1 or A2) and (B1 or B2)

— Attainable (Achievable, Actionable, Appropriate) Non-conflicting – one requirement should not conflict with another requirement.

(B2 or B3) Actually needed – not just wishes

— Realistic— Time-bound (Timely, Traceable)

Well organized – cross reference related requirement.

Summary

Independent QA organization QA roles in the software life cycle Test Documents:

— Test plan— Test specification— Other styles— IEEE 829-1998 standard— Context driven test planning

Good requirements to test— Valid, testable, non-conflicting, non-ambiguous, low

technical level, necessary, cross-reference