Efficient Coverage Metrics That Won’t Leave You Exposed

Who are we?

2

Nikhil Kaul Rick Almeida

On our plate today

3

• Why use coverage metrics?

• Different types of coverage metrics

• Where these coverage metrics may go wrong

• Coverage goals and where to start?

Coverage metrics can come in handy in a variety of ways

4

1 Areas not covered by tests

Indirect measure of quality 2

3

Identifying redundant test cases

Variety of coverage metrics are available to achieve these results

5

Coverage Metrics

Statement Branch Path

Let’s start with Statement Coverage

6

No of statements run Statement Coverage = -------------------------------- * 100

Total number of statements

Statement Coverage

Branch Coverage

Path

Coverage

Source Code

100% statement coverage doesn’t mean effective testing

7

Statement Coverage

Branch Coverage

Path

Coverage

TestCase

Statement Coverage

100%

Statement coverage comes with other challenges

8

Terminal condition Do-While Switch Statements

Statement Coverage

Branch Coverage

Path

Coverage

Branch Coverage

9

Statement Coverage

Branch Coverage

Path

Coverage

• Each branch (if statements, loops) has been executed at least once

• Every “if statement” has two distict branches

• No branches lead to abnormal behavior

Source Code

To cover 100% of branches, we need one more test case

10

Statement Coverage

Branch Coverage

Path

Coverage

TestCase

Statement Coverage

100%

Branch Coverage

50%

However, branch coverage can ignore short circuit operators

11

Statement Coverage

Branch Coverage

Path

Coverage

No call to function1

Test Case 1: Condition 1 = true Condition 2 = true

Test Case 2: Condition 1 = false Condition 2 = false

Next comes path coverage

12

Statement Coverage

Branch Coverage

Path

Coverage

T F

T F T F

1 2 3 4

Source Code

50% path coverage with TT and FF

13

Statement Coverage

Branch Coverage

Path

Coverage

TestCases

Branch Coverage

100%

Path Coverage

50%

You are missing 2 paths

14

Statement Coverage

Branch Coverage

Path

Coverage

T F

T F T F

1 2 3 4

Path coverage is not always optimal

15

Statement Coverage

Branch Coverage

Path

Coverage

Exponential increase in branches with added conditionals

End up creating extra test cases

Where to start with coverage

16

Find evident issues early and often

Don’t focus on hitting high coverage metrics for individual parts of you app

Test some of each part of you application

What to do once you get started?

17

Call at least one method in every file or class

Attain 80% to 90% branch coverage in each method

Invoke 8 out of 10 methods, ignoring branch coverage

TestComplete for automated functional testing

18

AQTime for code coverage

Optimizes code for developers & testers

Supports multiple languages & frameworks Integrates with IDEs and TestComplete

DEMO

20

QUESTIONS?

21