CONSOLIDATED TEST CASESEXECUTION

Dmitry Hes

SEPTEMBER, 2016

2CONFIDENTIAL

CONSOLIDATED TEST CASE EXECUTION

INTRODUCTION

3CONFIDENTIAL

INTRODUTION

4CONFIDENTIAL

INTRODUTION

Takes much time

• long-running process (> 5 min)

• modify database significantly

Integration processes

5CONFIDENTIAL

INTRODUTION

tim ecount

… integration processes are invoked many

times

6CONFIDENTIAL

INTRODUTION

How to speed up?

7CONFIDENTIAL

INTRODUTION

8CONFIDENTIAL

CONSOLIDATEDAPPROACH

9CONFIDENTIAL

TestcaseArrangement

test data(create e-

mail)

Sync processes

(Set data in DB)

Build-mode

lVerify data

REAL EXAMPLETestcase

Arrangement test data(create e-

mail)

Sync processes

(Set data in DB)

Build-mode

lVerify data

…Testcase

Arrangement test data(create e-

mail)

Sync processes

(Set data in DB)

Build-mode

lVerify data

many times with unique test data

10CONFIDENTIAL

REAL EXAMPLE

Could we use parallelization? (setup scope of environments and run TCs on each)

No.• We are unable to run build-model against one server• Only one process must run at the same time• Therefore we need setup whole environment on each VM• It takes about 20 minutes (some installation scripts

execution require• So no sense • Moreover we don’t have so much free VMs

11CONFIDENTIAL

REAL EXAMPLE

Execution time of 1 test case: 10–15 minutes

Number of test cases: 70

Total running time: 875 minutes (14.5 hours)

12CONFIDENTIAL

SOLUTION

Sync processes

(Set data in DB)

0Build-mode

l

Test

case Arrangement

test data(create e-

mail)

Test

case Arrangement

test data(create e-

mail)

…

Test

case Arrangement

test data(create e-

mail)

Test

case Verify

data

Test

case Verify

data

…Te

stca

se Verify data

13CONFIDENTIAL

CONSOLIDATEDFRAMEWORK

14CONFIDENTIAL

CONSOLIDATED FRAMEWORK

Test data csv-file should be created/updated before

running consolidated Test caseby executing

ParameterExportByTestCaseID test method.

Test class implementationStructure

[TestMethod]public void MasterTestSuite()

- testCaseData - <variables to store test data> ...

<Test class>

CNTestCaseBase

+ setup()reads test data from the appropriate csv file into and stores it to local dictionary and variables of method instance testCaseData =

CNTestSuiteExtensions .LoadTestCaseData();+ exec()

create and setup test data needed to TC like create mails, contacts, meeting, etc.

+ validate()method that perform validation steps (after processes execution)

+ cleanup()clean up environment after TC execution

Test caseCSV file:

../../../CNAutomationProject /TFS.Parameters/<test case id>.csv

[TestMethod]<Test class>()

Start

Create TC…

classes instances

For each instanceexecute setup();execute exec()

Execute sync- processes

Execute build-model process,iisreset

For each instanceexecute

validate()

End

For each instanceexecute

cleanup()

[TestClass]TFSParameterExportByTestCaseID[TestMethod]+ ParameterExportByTestCaseID() Gets test parameters data from TFS for specified test cases and stores them in appropriate csv-file

15CONFIDENTIAL

CONSOLIDATED FRAMEWORK

0Build-model0

CRM sync

process

Sync to CRM

example 2

16CONFIDENTIAL

CONSOLIDATED FRAMEWORK

Test class implementation

Structure[TestMethod]public void BatchExecution_TestMethod ()

+ ctor <Test class>

Invokes setup()

<Test class>

TestCaseBase

+ setup()reads test data from the appropriate csv file into and stores it to local dictionary and variables of method instance

[ + Validation1() ]

if required

[ + Validation2() ]

if required

[TestClass]<Test class>()

Start

Create TC…

classes instances

ExecuteProcess 1

End

Implementation 2

For each instanceExec Validation

1

ExecuteProcess 2

For each instanceExec Validation

2…ExecuteProcess N

For each instanceExec Validation

N

…[ + ValidationN() ]

if required

17CONFIDENTIAL

PROS AND CONS

Advantages Disadvantagessignificantly reduced TC run time complex test case structure

able to manage execution order test data should be unique

test data is read from local CSV huge test log

difficult to add new methods

MS test methods like Class/Test initialize or cleanup are not executedhard to adopt existing TCs to consolidated approach

18CONFIDENTIAL

NEXT STEP

19CONFIDENTIAL

1. Make test case more flexible

• ability to change testing workflow process by managing pool of executable methods

2. Add execution of ClassInitialize, TestInitialize, ClassCleanup, TestCleanup accordingly

NEXT STEP