Cerberus

What is it ?Web Application .

Developed by La Redoute since 2011, then published in Open source.

Source code and documentation available on github and sourceforge :https://github.com/vertigo17/CerberusAlmost 2600 commits by 10 contributeurs

Centralized test repository.Automate multi-technologies. (Web, Application Mobile, Heavy Client, SOAP, Rest, SQL…)

Multi-environment (DEV, QA, UAT, PROD….)Multi-langages (FR, UK, RU, IT, PL, …)

Business

Development TeamTest Team

Where does it come from ?• From the observation of the limits of a traditional organization.

• Silo Organisation.• Test definition during or after development phase.

The possible interpretation of a functional specification can lead to a number of significant go back .

The test case, correctly described in earlier in the development phase , is the most effective way to prevent the poor quality .

Need for a common test repository to the various players , fed earlier in the development phase.

Where does it come from ?• From the experimentation of new organizations : Agile / Continuous Integration /

Devops

time

risk

timeShort iteration

Long iteration

Need for test automation

No automated Tests = No quick feedback = No short iteration = Increasing risk to production deploy.Regression risk evolution vs delay between each deploy

DevOps is an effort to create business value by involving all stakeholders in the software development ( from design to production , including the developers ) through short iterations and recognize the value closely , consists of experimentation and rewind.

Where does it come from ?• From the experience in test automation.

To guarantee test maintenability.

a.Avoid code duplication Use step library.

b.Guaranty functional test coverage. Functional test coverage should be defined by business/MOA team.

c. Deploy one unique repository. Link functional information with technical implementation to allow to easily maintain one unique

repository.

Need for a test framework, including functional and technical information.

Login

Product

Basket

Test Automation

Web (Selenium)IOS and Android Apps (Appium)

Heavy Client / GUI (sikuli)Web-services (xml unit)

Databases (sql connectors - jdbc)

Why do it ?• A common test repository to the various players , fed earlier in the development

phase.• Test automation whatever the technology.• Test framework, including functional and technical information.

Test Repository

Functional description ACTION / CONTROLES / DATATest centralisationStep libraryTest description standardisation.

Testing Management

Testcase execution Dashboard.Performance metrics.

Log (screenshot).Standardize test reporting.

Link to ticketing system.

Cerberus Usage: In the context of the project.

Business

Dev TeamQA Team

Business

Dev TeamQA Team

Cerberus usage : in the context of the project

Team exchanges around a centralized tool.

Become

Cerberus usage : in the context of the project1 – Test specification allowing to guaranty the quality of the project

• Who ? Business, AMOA.

• Where ?In the description fields.

• How ?Following the frame (Action/Control).Using step library and data library.

• When?As soon as possible in the project.Ideally during functional specification.

Cerberus usage : in the context of the project2 – Test Automation

• Who ? Developers, QA Team.

• Where ?In the script fields.

• How ?Following the frame.

• When?During development phase.When user interface are designed.

Cerberus usage : in the context of the project3 – Test Execution

• Who ? All project actors.

• Where ?Manual & Automated. In the test execution page.

• How ?Manual: Allow to define status for each action.Automated : Launched manually or for continuous integration chain.

• When ?During the test phase.

Manual execution

Cerberus usage : in the context of the project4 – Test project management

• Who? Every project actors.

• Where ?Reporting page

• How ?Follow execution status.Interface with several ticketing tools.

• When?During the test phase.

Cerberus usage : in the context of the project

Synthesis:

1 Test specification.• Suivant le cadre défini (Data/Actions/Contrôles)• En utilisant les données et étapes de libraires

Test automation (20/80)• Pour le périmètre qui n’est pas en librairie / données

manquantes• Permettant de valider les cas passant et quelques cas

bloquant

2

Project test execution (manual)3

Business

Dev TeamQA Team

3

GO/NO GO on common reporting

Project testing become regression testing for next projects.

3 Project test execution (automated)

Regression test execution (automated)

444

5

Regression Testing : key figures

La Redoute :For the perimeter www.laredoute.xx (10 countries) and m.laredoute.xx (10 countries) : 3500 tests launched twice a day (UAT and PreProd). 4 production deploy for each application. Speed of test creation increasing, due to library usage.

Evolution du périmètre des TNRs

Aout 2014180 Tests

Aout 20152000 Tests

Janvier 20162900 Tests

Avril 2016 3500 Tests

Another usage: Functional Monitoring

Functional monitoring Run functional to guaranty .

1

23

1

Functional test execution(Example : 10 scenarii every 5 minutes)

Technical data records• Response time per action• Network Trafic (HAR file)

2

Data exploitation• Services allowing to raise alert on

non OK status.• Publish data into Elastic Search

/Kibana• Business Activity Monitoring

3

Functional monitoring : key figures

Follow the availability of the key scenarios:Homepage / Authenticate / Account creation / Navigation / Product List / Product Page / Comparison / Basket / Delivery / Payment

At a pertinent frequency:Every 5 minutes, 21h a day

>> 24.000.000 step every year for1 application

Decorelate the cost of the monitoring from the execution frequancy allow to avoid to opposite quality with cost of the necessary solution.

Integration into IT service

Cerberus in an Information Technology service

Cerberus can be interfaced with a lot of popular tools already in place in most of the IT service

Oracle SQLMySQLPostGreSQLDB2Microsoft SQLServerSSAS

Can be launched by any task scheduler.• Via 1 api REST• $U, cron, Jenkins….• Execute 1 test or even a campaign

Push execution status.• OK/KO for a go/no go status

Push into ES/Kibana

Automatic ticket opening.• In Mantis / Redmine / ….• Forward testcase information

Can be intefaced with Centreon

Allow to Read/Write any Database.

Architecture

Application serverActif / Actif(Glassfish / Tomcat / Jboss)

DatabaseActif/Passif(MySQL / MariaDB)

Robots Selenium / Sikuli(GUI Testing)

Front

Back

Robot

Core

Architecture : For a POC

Technical information:1 VM (Linux)4 vCPU8 GB RAM50 Go Disk Space

Application :MySQL (v5.6.xx)Glassfish 4.1.1

Technical information:3 VM (Linux)2 vCPU4 GB RAM10 Go Disk Space

Application :Selenium serverFirefox

Front

Back

Robot

Core

RoadMap 2016

RoadMap 2016

https://github.com/vertigo17/Cerberus/issues

• GUI Refactoring (70% Done)• Bootstrap• Interface Internationalisation

• Interfacing with Jmeter• Load test scenario centralisation• JMX generation (in study)• Trigger Execution

• TestCase Versionning

• Plugin management• Add action and control as plugin