+ All Categories
Home > Software > From jUnit to Mutationtesting

From jUnit to Mutationtesting

Date post: 13-Apr-2017
Category:
Upload: sven-ruppert
View: 208 times
Download: 0 times
Share this document with a friend
185
prepare for… Mutation Testing ..from jUnit to Mutation-Testing
Transcript
Page 1: From jUnit to Mutationtesting

prepare for…

Mutation Testing..from jUnit to Mutation-Testing

Page 2: From jUnit to Mutationtesting

@SvenRuppert has been coding java since 1996

Fellow / Head of R&D

reply Group

Germany - Munich

2

Page 3: From jUnit to Mutationtesting

@SvenRuppert has been coding java since 1996

3

Page 4: From jUnit to Mutationtesting

@SvenRuppert has been coding java since 1996

Projects in the field of:•Automobile-industry•Energy•Finance / Leasing•Space- Satellit-•Government / UN / World-bank

Where?

•Europe•Asia - from India up to Malaysia

3

Page 5: From jUnit to Mutationtesting

4

Save harbor statement

Page 6: From jUnit to Mutationtesting

4

Save harbor statement

The following is intended for information purposes only. Ican not be held responsible for the overuse of effects andanimations in this presentation. If any person in this roomhas a medical condition that is triggered by fast movingobjects on the screen and/or explosions, he/she shouldprobably better leave now…

(I got carried away by the topic.)

Page 7: From jUnit to Mutationtesting

5

The Environment @SvenRuppert

Codebase is > 13 years old

Page 8: From jUnit to Mutationtesting

5

The Environment @SvenRuppert

Codebase is > 13 years old no test coverage

Page 9: From jUnit to Mutationtesting

5

The Environment @SvenRuppert

Codebase is > 13 years old no test coverage

no dedicated refactoring budget

Page 10: From jUnit to Mutationtesting

5

The Environment @SvenRuppert

Codebase is > 13 years old no test coverage

no dedicated refactoring budgetdecrease complexity

Page 11: From jUnit to Mutationtesting

5

The Environment @SvenRuppert

Codebase is > 13 years old no test coverage

no dedicated refactoring budgetdecrease complexity

but… lets start with the basics

Page 12: From jUnit to Mutationtesting

6

TDD with jUnit @SvenRuppert

Page 13: From jUnit to Mutationtesting

6

TDD with jUnit @SvenRuppert

are you using jUnit?

Page 14: From jUnit to Mutationtesting

6

TDD with jUnit @SvenRuppert

are you using jUnit?

assume that the following would make sense.. ;-)

Page 15: From jUnit to Mutationtesting

6

TDD with jUnit @SvenRuppert

are you using jUnit?

assume that the following would make sense.. ;-)

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } } }

Page 16: From jUnit to Mutationtesting

6

TDD with jUnit @SvenRuppert

are you using jUnit?

assume that the following would make sense.. ;-)

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } } }

How many tests

you will need ?

Page 17: From jUnit to Mutationtesting

6

TDD with jUnit @SvenRuppert

are you using jUnit?

assume that the following would make sense.. ;-)

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } } }

How many tests

you will need ?

it depends ;-)

Page 18: From jUnit to Mutationtesting

7

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

it depends ;-)

Page 19: From jUnit to Mutationtesting

7

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

it depends ;-)

for line 100% coverage

Page 20: From jUnit to Mutationtesting

7

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

it depends ;-)

for line 100% coverage 2

Page 21: From jUnit to Mutationtesting

7

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

it depends ;-)

for line 100% coverage 2

but will this be enough?

Page 22: From jUnit to Mutationtesting

7

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

it depends ;-)

for line 100% coverage 2

but will this be enough? No

Page 23: From jUnit to Mutationtesting

8

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

for line 100% coverage 2

but will this be enough? No

it depends ;-)

Page 24: From jUnit to Mutationtesting

8

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

for line 100% coverage 2

but will this be enough? No

how to find out, what will be enough?

it depends ;-)

Page 25: From jUnit to Mutationtesting

8

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

for line 100% coverage 2

but will this be enough? No

how to find out, what will be enough?how to find the right tests?

it depends ;-)

Page 26: From jUnit to Mutationtesting

9

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

Page 27: From jUnit to Mutationtesting

9

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

@Test public void testAdd001() throws Exception { final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0); }

Page 28: From jUnit to Mutationtesting

9

TDD with jUnit @SvenRuppert

public class Service { public int add(int a, int b){ if(a<2){ return (a+b) * -1; } else { return a+b; } }}

How many tests

you will need ?

@Test public void testAdd001() throws Exception { final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0); } @Test public void testAdd002() throws Exception { final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3); }

Page 29: From jUnit to Mutationtesting

10

Mutation Testing @SvenRuppert

Page 30: From jUnit to Mutationtesting

10

Mutation Testing @SvenRuppert

Mutation Testing is a structural testing method

Page 31: From jUnit to Mutationtesting

10

Mutation Testing @SvenRuppert

Mutation Testing is a structural testing method

we want to find a way to write "good" tests

Page 32: From jUnit to Mutationtesting

10

Mutation Testing @SvenRuppert

Mutation Testing is a structural testing method

we want to find a way to write "good" testshow to find "good" tests?

Page 33: From jUnit to Mutationtesting

10

Mutation Testing @SvenRuppert

Mutation Testing is a structural testing method

we want to find a way to write "good" testshow to find "good" tests?

let the machine find the targets

Page 34: From jUnit to Mutationtesting

10

Mutation Testing @SvenRuppert

Mutation Testing is a structural testing method

we want to find a way to write "good" testshow to find "good" tests?

let the machine find the targets

let´s mutate it... but how?

Page 35: From jUnit to Mutationtesting

11

Mutation Testing - the Idea @SvenRuppert

Page 36: From jUnit to Mutationtesting

11

Mutation Testing - the Idea @SvenRuppert

a mutation is a small change in the code

Page 37: From jUnit to Mutationtesting

11

Mutation Testing - the Idea @SvenRuppert

a mutation is a small change in the code.. small enough to be a small defect

Page 38: From jUnit to Mutationtesting

11

Mutation Testing - the Idea @SvenRuppert

a mutation is a small change in the code.. small enough to be a small defect

P will be the program

Page 39: From jUnit to Mutationtesting

11

Mutation Testing - the Idea @SvenRuppert

a mutation is a small change in the code.. small enough to be a small defect

P will be the programT will be the collection of all tests / Test Suite

Page 40: From jUnit to Mutationtesting

12

Mutation Testing - the Idea @SvenRuppert

P will be the programT will be the collection of all tests / Test Suite

Page 41: From jUnit to Mutationtesting

12

Mutation Testing - the Idea @SvenRuppert

P will be the programT will be the collection of all tests / Test Suite

we will create a sequence of mutations / P1,P2,P3...

Page 42: From jUnit to Mutationtesting

12

Mutation Testing - the Idea @SvenRuppert

P will be the programT will be the collection of all tests / Test Suite

we will create a sequence of mutations / P1,P2,P3...

.. Px will have only one mutation compared to P

Page 43: From jUnit to Mutationtesting

12

Mutation Testing - the Idea @SvenRuppert

P will be the programT will be the collection of all tests / Test Suite

we will create a sequence of mutations / P1,P2,P3...

.. Px will have only one mutation compared to P

running all tests from T against Px

Page 44: From jUnit to Mutationtesting

12

Mutation Testing - the Idea @SvenRuppert

P will be the programT will be the collection of all tests / Test Suite

we will create a sequence of mutations / P1,P2,P3...

.. Px will have only one mutation compared to P

running all tests from T against Px

green: T will kill the mutation

Page 45: From jUnit to Mutationtesting

12

Mutation Testing - the Idea @SvenRuppert

P will be the programT will be the collection of all tests / Test Suite

we will create a sequence of mutations / P1,P2,P3...

.. Px will have only one mutation compared to P

running all tests from T against Px

green: T will kill the mutation.. at leased one test from T will fail

Page 46: From jUnit to Mutationtesting

12

Mutation Testing - the Idea @SvenRuppert

P will be the programT will be the collection of all tests / Test Suite

we will create a sequence of mutations / P1,P2,P3...

.. Px will have only one mutation compared to P

running all tests from T against Px

green: T will kill the mutation.. at leased one test from T will fail

red: if all tests are green

Page 47: From jUnit to Mutationtesting

13

Mutation Testing - the Idea @SvenRuppert

Page 48: From jUnit to Mutationtesting

13

Mutation Testing - the Idea @SvenRuppert

if we kill k out of n mutants

Page 49: From jUnit to Mutationtesting

13

Mutation Testing - the Idea @SvenRuppert

if we kill k out of n mutants -> we are not good enough ;-)

Page 50: From jUnit to Mutationtesting

13

Mutation Testing - the Idea @SvenRuppert

if we kill k out of n mutants -> we are not good enough ;-)

we are perfect enough if we are reaching : k == n

Page 51: From jUnit to Mutationtesting

13

Mutation Testing - the Idea @SvenRuppert

if we kill k out of n mutants -> we are not good enough ;-)

we are perfect enough if we are reaching : k == n

how to create all versions of Px ?

Page 52: From jUnit to Mutationtesting

13

Mutation Testing - the Idea @SvenRuppert

if we kill k out of n mutants -> we are not good enough ;-)

we are perfect enough if we are reaching : k == n

how to create all versions of Px ?

.. the good thing..

Page 53: From jUnit to Mutationtesting

13

Mutation Testing - the Idea @SvenRuppert

if we kill k out of n mutants -> we are not good enough ;-)

we are perfect enough if we are reaching : k == n

how to create all versions of Px ?

.. the good thing..

we could almost generate/

automate everything

Page 54: From jUnit to Mutationtesting

14

Mutation Testing @SvenRuppert

practical TDD with Mutation Testing

Page 55: From jUnit to Mutationtesting

14

Mutation Testing @SvenRuppert

generating the mutants and

practical TDD with Mutation Testing

Page 56: From jUnit to Mutationtesting

14

Mutation Testing @SvenRuppert

generating the mutants and

practical TDD with Mutation Testing

running all junit tests

Page 57: From jUnit to Mutationtesting

14

Mutation Testing @SvenRuppert

generating the mutants and

practical TDD with Mutation Testing

running all junit testscheck the reports

Page 58: From jUnit to Mutationtesting

14

Mutation Testing @SvenRuppert

generating the mutants and

practical TDD with Mutation Testing

running all junit testscheck the reports

write more / better tests

Page 59: From jUnit to Mutationtesting

14

Mutation Testing @SvenRuppert

generating the mutants and

practical TDD with Mutation Testing

running all junit testscheck the reports

write more / better tests

loop until quality target reached

Page 60: From jUnit to Mutationtesting

15

Mutation Testing @SvenRuppert

mutants are a good approach / model to estimate the default rate of defects per 1k lines of the P

Page 61: From jUnit to Mutationtesting

15

Mutation Testing @SvenRuppert

mutants are a good approach / model to estimate the default rate of defects per 1k lines of the P

estimate that:

Page 62: From jUnit to Mutationtesting

15

Mutation Testing @SvenRuppert

mutants are a good approach / model to estimate the default rate of defects per 1k lines of the P

estimate that: the defects are independent

Page 63: From jUnit to Mutationtesting

15

Mutation Testing @SvenRuppert

mutants are a good approach / model to estimate the default rate of defects per 1k lines of the P

estimate that: the defects are independent normaly ;-)

Page 64: From jUnit to Mutationtesting

16

Mutation Testing @SvenRuppert

Page 65: From jUnit to Mutationtesting

16

Mutation Testing @SvenRuppert

no need to know that Mutation Testing will be done, independend creation of T

Page 66: From jUnit to Mutationtesting

16

Mutation Testing @SvenRuppert

no need to know that Mutation Testing will be done, independend creation of T

for K==n we need a high number of mutants ( P1, P2, .., Px)

Page 67: From jUnit to Mutationtesting

16

Mutation Testing @SvenRuppert

no need to know that Mutation Testing will be done, independend creation of T

for K==n we need a high number of mutants ( P1, P2, .., Px)

.. mostly it will lead into exponential numbers of Px

Page 68: From jUnit to Mutationtesting

16

Mutation Testing @SvenRuppert

no need to know that Mutation Testing will be done, independend creation of T

for K==n we need a high number of mutants ( P1, P2, .., Px)

.. mostly it will lead into exponential numbers of Px

.. how to find YOUR barrier you

have to reach?

Page 69: From jUnit to Mutationtesting

16

Mutation Testing @SvenRuppert

no need to know that Mutation Testing will be done, independend creation of T

for K==n we need a high number of mutants ( P1, P2, .., Px)

.. mostly it will lead into exponential numbers of Px

.. how to find YOUR barrier you

have to reach?

but.. what is a mutation?

Page 70: From jUnit to Mutationtesting

17

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutationchanging constants,

loop bounds (adding/subtracting values)

Page 71: From jUnit to Mutationtesting

18

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation

Decision Mutation for example < will be changed to <=

Page 72: From jUnit to Mutationtesting

19

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation Decision Mutation

for example swapping/deleting/duplicating lines of code

Statement Mutation

Page 73: From jUnit to Mutationtesting

20

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation Decision MutationStatement Mutation

Page 74: From jUnit to Mutationtesting

20

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation Decision MutationStatement Mutation

for Java you could think about more language spec. mutations

Page 75: From jUnit to Mutationtesting

20

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation Decision MutationStatement Mutation

for Java you could think about more language spec. mutations

.. changing modifiers

Page 76: From jUnit to Mutationtesting

20

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation Decision MutationStatement Mutation

for Java you could think about more language spec. mutations

.. changing modifiers.. changing between static / non static

Page 77: From jUnit to Mutationtesting

20

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation Decision MutationStatement Mutation

for Java you could think about more language spec. mutations

.. changing modifiers.. changing between static / non static

.. delete member initialization

Page 78: From jUnit to Mutationtesting

20

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation Decision MutationStatement Mutation

for Java you could think about more language spec. mutations

.. changing modifiers.. changing between static / non static

.. delete member initialization .. delete this.

Page 79: From jUnit to Mutationtesting

20

Mutation Testing - Kinds of Mutation @SvenRuppert

but.. what is a mutation?

Value Mutation Decision MutationStatement Mutation

for Java you could think about more language spec. mutations

.. changing modifiers.. changing between static / non static

.. delete member initialization .. delete this... argument order change

Page 80: From jUnit to Mutationtesting

21

Mutation Testing - in short words @SvenRuppert

Page 81: From jUnit to Mutationtesting

21

Mutation Testing - in short words @SvenRuppert

mutation testing is an add on to normal jUnit TDD

Page 82: From jUnit to Mutationtesting

21

Mutation Testing - in short words @SvenRuppert

mutation testing is an add on to normal jUnit TDD

tools are supporting it well

Page 83: From jUnit to Mutationtesting

21

Mutation Testing - in short words @SvenRuppert

mutation testing is an add on to normal jUnit TDD

tools are supporting it well

generating and running all tests are time consuming

Page 84: From jUnit to Mutationtesting

21

Mutation Testing - in short words @SvenRuppert

mutation testing is an add on to normal jUnit TDD

tools are supporting it well

generating and running all tests are time consuming

but most important

Page 85: From jUnit to Mutationtesting

21

Mutation Testing - in short words @SvenRuppert

mutation testing is an add on to normal jUnit TDD

tools are supporting it well

generating and running all tests are time consuming

but most important

will effect your project structure

Page 86: From jUnit to Mutationtesting

22

Mutation Testing - Frameworks @SvenRuppert

Page 87: From jUnit to Mutationtesting

22

Mutation Testing - Frameworks @SvenRuppert

muJava

Page 88: From jUnit to Mutationtesting

22

Mutation Testing - Frameworks @SvenRuppert

muJava2003. First released as JMutation (Java Mutation System).2004. The name was changed to MuJava (Mutation System for Java).2005. Software Copyright Registration, ALL RIGHTS RESERVED.2005. Version 2 released with several fault fixes and modified mutation operators.2008. Version 3 released with minimal support for Java 1.5 and 1.6.2013. Version 4 released to support JUnit tests and Java 1.6 language features, including generics, annotations, enumerations, varargs, enhanced for-each loops, and static imports.2015. Additional and improved error messages. Bug fixes for OpenJava. Licensing changed to the Apache license.

Page 89: From jUnit to Mutationtesting

22

Mutation Testing - Frameworks @SvenRuppert

muJava2003. First released as JMutation (Java Mutation System).2004. The name was changed to MuJava (Mutation System for Java).2005. Software Copyright Registration, ALL RIGHTS RESERVED.2005. Version 2 released with several fault fixes and modified mutation operators.2008. Version 3 released with minimal support for Java 1.5 and 1.6.2013. Version 4 released to support JUnit tests and Java 1.6 language features, including generics, annotations, enumerations, varargs, enhanced for-each loops, and static imports.2015. Additional and improved error messages. Bug fixes for OpenJava. Licensing changed to the Apache license.

https://cs.gmu.edu/~offutt/mujava/

https://github.com/jeffoffutt/muJava/graphs/contributors

Page 90: From jUnit to Mutationtesting

22

Mutation Testing - Frameworks @SvenRuppert

muJava2003. First released as JMutation (Java Mutation System).2004. The name was changed to MuJava (Mutation System for Java).2005. Software Copyright Registration, ALL RIGHTS RESERVED.2005. Version 2 released with several fault fixes and modified mutation operators.2008. Version 3 released with minimal support for Java 1.5 and 1.6.2013. Version 4 released to support JUnit tests and Java 1.6 language features, including generics, annotations, enumerations, varargs, enhanced for-each loops, and static imports.2015. Additional and improved error messages. Bug fixes for OpenJava. Licensing changed to the Apache license.

https://cs.gmu.edu/~offutt/mujava/

https://github.com/jeffoffutt/muJava/graphs/contributors

inactive

Page 91: From jUnit to Mutationtesting

23

Mutation Testing - Frameworks @SvenRuppert

Page 92: From jUnit to Mutationtesting

23

Mutation Testing - Frameworks @SvenRuppert

2012. started around 2012 with a small codebase.2014. very active since 2014

Page 93: From jUnit to Mutationtesting

23

Mutation Testing - Frameworks @SvenRuppert

2012. started around 2012 with a small codebase.2014. very active since 2014

http://pitest.org/

Page 94: From jUnit to Mutationtesting

23

Mutation Testing - Frameworks @SvenRuppert

2012. started around 2012 with a small codebase.2014. very active since 2014

http://pitest.org/

active ;-)

Page 95: From jUnit to Mutationtesting

24

Mutation Testing - Hello World @SvenRuppert

http://pitest.org/

Page 96: From jUnit to Mutationtesting

24

Mutation Testing - Hello World @SvenRuppert

http://pitest.org/assume the following would make sense ;-)

Page 97: From jUnit to Mutationtesting

24

Mutation Testing - Hello World @SvenRuppert

http://pitest.org/assume the following would make sense ;-)

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }}

Page 98: From jUnit to Mutationtesting

25

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }}

Page 99: From jUnit to Mutationtesting

25

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }} how many test you will need for..

Page 100: From jUnit to Mutationtesting

25

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }} how many test you will need for..

100% Line Coverage… and…

Page 101: From jUnit to Mutationtesting

25

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }} how many test you will need for..

100% Line Coverage… and… to be save ?

Page 102: From jUnit to Mutationtesting

25

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }} how many test you will need for..

100% Line Coverage… and… to be save ?

2 for Line Coverage

Page 103: From jUnit to Mutationtesting

25

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }} how many test you will need for..

100% Line Coverage… and… to be save ?

2 for Line Coverage we will see ;-)

Page 104: From jUnit to Mutationtesting

26

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }}

100% Line Coverage… and…

Page 105: From jUnit to Mutationtesting

26

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }}

100% Line Coverage… and…

we have one if statement

Page 106: From jUnit to Mutationtesting

26

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }}

100% Line Coverage… and…

we have one if statement with an else branch

Page 107: From jUnit to Mutationtesting

26

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }}

100% Line Coverage… and…

we have one if statement with an else branch

this will lead to 2 jUnit Tests to get 100 %

Page 108: From jUnit to Mutationtesting

27

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }} 100% Line Coverage… and…

Page 109: From jUnit to Mutationtesting

27

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }} 100% Line Coverage… and…@Testpublic void testAdd001() throws Exception { final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0);}

Page 110: From jUnit to Mutationtesting

27

Mutation Testing - Hello World @SvenRuppert

public class Service { public int add(int a, int b){ if (a<2) { return (a+b) * -1; } else { return a+b; } }} 100% Line Coverage… and…@Testpublic void testAdd001() throws Exception { final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0);} @Test

public void testAdd002() throws Exception { final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);}

Page 111: From jUnit to Mutationtesting

28

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0);

final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);

Page 112: From jUnit to Mutationtesting

28

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0);

final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);

Page 113: From jUnit to Mutationtesting

28

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0);

final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);

Page 114: From jUnit to Mutationtesting

29

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0); final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);

Page 115: From jUnit to Mutationtesting

29

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0); final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);

we got 100% Line Coverage

Page 116: From jUnit to Mutationtesting

29

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0); final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);

we got 100% Line CoverageHow good these tests are?

Page 117: From jUnit to Mutationtesting

29

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0); final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);

we got 100% Line CoverageHow good these tests are?

How to measure if these test are good?

Page 118: From jUnit to Mutationtesting

29

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0); Assertions.assertThat(add).isEqualTo(0); final int add = new Service().add(3, 0); Assertions.assertThat(add).isEqualTo(3);

we got 100% Line CoverageHow good these tests are?

How to measure if these test are good?

How to find the good tests?

Page 119: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?

Page 120: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation report

Page 121: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation reportwith maven : pitest: mutationCoverage

Page 122: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation reportwith maven : pitest: mutationCoverage

>> Generated 54 mutations

Page 123: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation reportwith maven : pitest: mutationCoverage

>> Generated 54 mutations Killed 3 (6%)

Page 124: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation reportwith maven : pitest: mutationCoverage

>> Generated 54 mutations

org.pitest……mutators.ConditionalsBoundaryMutator

Killed 3 (6%)

Page 125: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation reportwith maven : pitest: mutationCoverage

>> Generated 54 mutations

org.pitest……mutators.ConditionalsBoundaryMutatororg.pitest……mutators.IncrementsMutator

Killed 3 (6%)

Page 126: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation reportwith maven : pitest: mutationCoverage

>> Generated 54 mutations

org.pitest……mutators.ConditionalsBoundaryMutatororg.pitest……mutators.IncrementsMutatororg.pitest……mutators.ReturnValsMutator

Killed 3 (6%)

Page 127: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation reportwith maven : pitest: mutationCoverage

>> Generated 54 mutations

org.pitest……mutators.ConditionalsBoundaryMutatororg.pitest……mutators.IncrementsMutatororg.pitest……mutators.ReturnValsMutatororg.pitest……mutators.MathMutator

Killed 3 (6%)

Page 128: From jUnit to Mutationtesting

30

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

final int add = new Service().add(3, 0);

How to find the good tests?let´s generate a the mutation reportwith maven : pitest: mutationCoverage

>> Generated 54 mutations

org.pitest……mutators.ConditionalsBoundaryMutatororg.pitest……mutators.IncrementsMutatororg.pitest……mutators.ReturnValsMutatororg.pitest……mutators.MathMutatororg.pitest……mutators.NegateConditionalsMutator

Killed 3 (6%)

Page 129: From jUnit to Mutationtesting

31

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutations Killed 3 (6%)

Page 130: From jUnit to Mutationtesting

31

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutations Killed 3 (6%)

Page 131: From jUnit to Mutationtesting

31

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutations Killed 3 (6%)

Page 132: From jUnit to Mutationtesting

32

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutations Killed 3

Page 133: From jUnit to Mutationtesting

32

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutations Killed 3

Page 134: From jUnit to Mutationtesting

32

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutations Killed 3

Page 135: From jUnit to Mutationtesting

Killed 3

33

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutations

Page 136: From jUnit to Mutationtesting

Killed 3

33

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutationsfinal int add = new Service().add(2, 0);

Page 137: From jUnit to Mutationtesting

33

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutationsfinal int add = new Service().add(2, 0);

Page 138: From jUnit to Mutationtesting

Killed 4

33

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutationsfinal int add = new Service().add(2, 0);

Page 139: From jUnit to Mutationtesting

Killed 4

33

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutationsfinal int add = new Service().add(2, 0);

Page 140: From jUnit to Mutationtesting

Killed 4

33

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutationsfinal int add = new Service().add(2, 0);

Page 141: From jUnit to Mutationtesting

Killed 4

33

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);final int add = new Service().add(3, 0);

>> Generated 54 mutationsfinal int add = new Service().add(2, 0);

Page 142: From jUnit to Mutationtesting

Killed 4

33

Mutation Testing - Hello World @SvenRuppert

final int add = new Service().add(0, 0);

>> Generated 54 mutationsfinal int add = new Service().add(2, 0);

Page 143: From jUnit to Mutationtesting

Killed 4

33

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutationsfinal int add = new Service().add(2, 0);

Page 144: From jUnit to Mutationtesting

Killed 4

34

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);

Page 145: From jUnit to Mutationtesting

Killed 4

34

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

Page 146: From jUnit to Mutationtesting

34

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

Page 147: From jUnit to Mutationtesting

Killed 5

34

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

Page 148: From jUnit to Mutationtesting

Killed 5

34

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

killed 9:1

Page 149: From jUnit to Mutationtesting

Killed 5

35

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

killed 9:1

Page 150: From jUnit to Mutationtesting

Killed 5

35

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

killed 9:1

final int add = new Service().add(2, 2);

Page 151: From jUnit to Mutationtesting

35

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

killed 9:1

final int add = new Service().add(2, 2);

Page 152: From jUnit to Mutationtesting

Killed 6

35

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

killed 9:1

final int add = new Service().add(2, 2);

Page 153: From jUnit to Mutationtesting

Killed 6

35

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

killed 9:1

final int add = new Service().add(2, 2);

Page 154: From jUnit to Mutationtesting

Killed 6

35

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);

killed 9:1

final int add = new Service().add(2, 2);

Page 155: From jUnit to Mutationtesting

Killed 6

35

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(2, 0);final int add = new Service().add(1, 1);final int add = new Service().add(2, 2);

killed 11:1

Page 156: From jUnit to Mutationtesting

Killed 6

35

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(1, 1);final int add = new Service().add(2, 2);

killed 11:1

Page 157: From jUnit to Mutationtesting

Killed 6

36

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(1, 1);final int add = new Service().add(2, 2);

Page 158: From jUnit to Mutationtesting

Killed 6

36

Mutation Testing - Hello World @SvenRuppert

>> Generated 54 mutations

final int add = new Service().add(1, 1);final int add = new Service().add(2, 2);

Page 159: From jUnit to Mutationtesting

37

Mutation Testing - Lesson Learned @SvenRuppert

Page 160: From jUnit to Mutationtesting

37

Mutation Testing - Lesson Learned @SvenRuppert

mutation tests are often leading to

Page 161: From jUnit to Mutationtesting

37

Mutation Testing - Lesson Learned @SvenRuppert

mutation tests are often leading to

…cleaner code compared to jUnit only

Page 162: From jUnit to Mutationtesting

37

Mutation Testing - Lesson Learned @SvenRuppert

mutation tests are often leading to

…cleaner code compared to jUnit only… smaller modules (shorter mutation runtime)

Page 163: From jUnit to Mutationtesting

37

Mutation Testing - Lesson Learned @SvenRuppert

mutation tests are often leading to

…cleaner code compared to jUnit only… smaller modules (shorter mutation runtime)

and something nice… helps to find useless code

Page 164: From jUnit to Mutationtesting

38

Example of useless Code @SvenRuppert

Page 165: From jUnit to Mutationtesting

38

Example of useless Code @SvenRuppert

Page 166: From jUnit to Mutationtesting

38

Example of useless Code @SvenRuppert

Page 167: From jUnit to Mutationtesting

39

Mutation Testing - How to start @SvenRuppert

Page 168: From jUnit to Mutationtesting

39

Mutation Testing - How to start @SvenRuppert

you need jUnit - to generate the reference

Page 169: From jUnit to Mutationtesting

39

Mutation Testing - How to start @SvenRuppert

you need jUnit - to generate the reference add the pitest-plugin to the build section

Page 170: From jUnit to Mutationtesting

39

Mutation Testing - How to start @SvenRuppert

you need jUnit - to generate the reference add the pitest-plugin to the build section

configure the plugin

Page 171: From jUnit to Mutationtesting

39

Mutation Testing - How to start @SvenRuppert

you need jUnit - to generate the reference add the pitest-plugin to the build section

configure the plugingenerate the reference -> clean , install

Page 172: From jUnit to Mutationtesting

39

Mutation Testing - How to start @SvenRuppert

you need jUnit - to generate the reference add the pitest-plugin to the build section

configure the plugingenerate the reference -> clean , install

run pitest: mutationCoverage

Page 173: From jUnit to Mutationtesting

39

Mutation Testing - How to start @SvenRuppert

you need jUnit - to generate the reference add the pitest-plugin to the build section

configure the plugingenerate the reference -> clean , install

run pitest: mutationCoveragereport will be under target/pit-reports

Page 174: From jUnit to Mutationtesting

40

Mutation Testing - How to start @SvenRuppert

pom.xml - example - build<plugin> <groupId>org.pitest</groupId> <artifactId>pitest-maven</artifactId> <configuration> <outputFormats> <outputFormat>XML</outputFormat> <outputFormat>HTML</outputFormat> </outputFormats> <targetClasses> <param>org.rapidpm.*</param> </targetClasses> <targetTests> <param>org.rapidpm.*</param> <param>junit.org.rapidpm.*</param> </targetTests> </configuration> </plugin>

Page 175: From jUnit to Mutationtesting

41

Mutation Testing - How to start @SvenRuppert

pom.xml - example - reporting<reporting> <plugins> <plugin> <groupId>org.pitest</groupId> <artifactId>pitest-maven</artifactId> <reportSets> <reportSet> <reports> <report>report</report> </reports> </reportSet> </reportSets> </plugin> </plugins> </reporting>

Page 176: From jUnit to Mutationtesting

42

Mutation Testing - practical usage @SvenRuppert

Page 177: From jUnit to Mutationtesting

42

Mutation Testing - practical usage @SvenRuppert

Start with some tests

Page 178: From jUnit to Mutationtesting

42

Mutation Testing - practical usage @SvenRuppert

Start with some testsgenerate the pitest report

Page 179: From jUnit to Mutationtesting

42

Mutation Testing - practical usage @SvenRuppert

Start with some testsgenerate the pitest report

write more tests to kill mutations

Page 180: From jUnit to Mutationtesting

42

Mutation Testing - practical usage @SvenRuppert

Start with some testsgenerate the pitest report

write more tests to kill mutationsif you have time, eliminate useless tests

Page 181: From jUnit to Mutationtesting

42

Mutation Testing - practical usage @SvenRuppert

Start with some testsgenerate the pitest report

write more tests to kill mutationsif you have time, eliminate useless tests

do it one by one

Page 182: From jUnit to Mutationtesting

42

Mutation Testing - practical usage @SvenRuppert

Mutation 001Mutation 002

Mutation 003Mutation 004 Survived

SurvivedSurvived

Survived

Start with some testsgenerate the pitest report

write more tests to kill mutationsif you have time, eliminate useless tests

do it one by one

Page 183: From jUnit to Mutationtesting

42

Mutation Testing - practical usage @SvenRuppert

Mutation 001Mutation 002

Mutation 003Mutation 004 Survived

Killed

SurvivedSurvived

SurvivedKilled

KilledKilled

Start with some testsgenerate the pitest report

write more tests to kill mutationsif you have time, eliminate useless tests

do it one by one

Page 184: From jUnit to Mutationtesting

Summary

43

@SvenRuppert

If you are interested…

have a look at GITHUB

ProxyBuilder

Dynamic-Dependency-Injection

Java-Microservice

or contact me ;-) @SvenRuppert

Page 185: From jUnit to Mutationtesting

Summary

43

@SvenRuppert

If you are interested…

have a look at GITHUB

ProxyBuilder

Dynamic-Dependency-Injection

Java-Microservice

or contact me ;-) @SvenRuppert

Thank You !!!


Recommended