SPI Software Process & Infrastructure

Quality Assurance (QA)http://spi.cern.ch/qa

QA reports generation - 22 June 2005

Johanne BENARDjohanne.benard@cern.ch

J.Benard SPI - Software Process & Infrastructure 2

QA reports generation

Test coverage reports• Basics• Some examples• Other utilities

Savannah Quality Assurance (QA) reports• The web interface• Capabilities• Some examples

J.Benard SPI - Software Process & Infrastructure 3

Test coverage reports - Basics

• Aim at:• Showing what percentage of an application has been

executed by the test process,• Measuring effectiveness of a system test.

• Two different principles in GCOV:

Statement coverage analysis:• Code break down into

basic blocs, • Block solicitations counting

Branch coverage analysis:• Conditional possible

paths• Path usage frequency

J.Benard SPI - Software Process & Infrastructure 4

Test coverage reports - Basics

Runtime

Compilation time

File.cxx, .c , .h, etc.

File.bb

File.bbg

File.da

Coverage analysis

File.info

Code instrumentalisation:

-fprofile-arcs

-ftest-coverage

LTP GCOV

Suite (*):

lcov, genhtml, etc.

(*) LTP: Linux Test ProjectLCOV is an extension of GCOV, a GNU tool which provides information about what

parts of a program are actually executed while running a particular test case.

J.Benard SPI - Software Process & Infrastructure 5

Some examples - POOL

J.Benard SPI - Software Process & Infrastructure 6

Some examples - POOL

J.Benard SPI - Software Process & Infrastructure 7

Some examples - CLHEP

J.Benard SPI - Software Process & Infrastructure 8

Some examples - PI

J.Benard SPI - Software Process & Infrastructure 9

Some examples - SEAL

J.Benard SPI - Software Process & Infrastructure 10

Some examples - ROOT

J.Benard SPI - Software Process & Infrastructure 11

Test Coverage - Other utilities Tools

• Genhtml:• Creates an HTML view from coverage data found in INFOFILE

• Geninfo:• Converts .da files into .info files

• Genpgn:• Creates an overview image for a given source code file of either

plain text or .gcov file format

• Gendesc:• Converts a test case description file into a format as understood

by genhtml INPUTFILE

J.Benard SPI - Software Process & Infrastructure 12

QA Savannah Reports

J.Benard SPI - Software Process & Infrastructure 13

QA Savannah report – Web Form

Bugs, Tasks, Support and

Patches

Default: Project beginning

Today

.ini file: header, Style sheet and options

Category,Assigned To

Severity, etc.

J.Benard SPI - Software Process & Infrastructure 14

QA Savannah report – Web Form

• Potential users: projects or experiments that actively use Savannah trackers (bugs, tasks, support, patch)

• Current users: JRA1 coordination, JRA1 Middleware, LCG 2 Sites

• Ready for LCG projects:- POOL, - PI, - SEAL

• Tried on several other projects: - Athena, - Gaudi, - Cobra, - Iguana

J.Benard SPI - Software Process & Infrastructure 15

QA Savannah report - Capabilities

Options used

Globalprocessing

BreakdownInto

Savannahfields

J.Benard SPI - Software Process & Infrastructure 16

QA Savannah report - Capabilities

Statistics on a given period

Translated intograph

J.Benard SPI - Software Process & Infrastructure 17

Some examples – LCG 2 Sites

J.Benard SPI - Software Process & Infrastructure 18

Some examples – JRA1 Middleware

J.Benard SPI - Software Process & Infrastructure 19

Some examples - POOL

J.Benard SPI - Software Process & Infrastructure 20

Some examples - SEAL

J.Benard SPI - Software Process & Infrastructure 21

Some example - PI

J.Benard SPI - Software Process & Infrastructure 22

SPI

Quality Assurance

J.Benard SPI - Software Process & Infrastructure 23

Web site: http://spi.cern.ch/qa

J.Benard SPI - Software Process & Infrastructure 24

Conclusion

Thank you for your attention!

Question?

• Proposed services:• To generate customized test coverage reports• To set parameters for Savannah QA reports• To repeat this talk for projects / experiments