ADMIRE 2018 WELCOME! -...

ADMIRE 2018WELCOME! Tuesday June 12, 2018Holiday Inn Diegem

The 'Shift Left' and how to get there with ALM Octane and LeanFTPartner Sponsored Session by Bjorn Boisschot – Solution Architect CTG

SPONSORED BY

© 2018 CTG, Inc. 3

Presenter

Presentation Notes

In the past, a majority of software teams worked with a waterfall model, the workflow looked something like this example from Winston Royce's "Managing The Development of Large Software Systems", first published in 1976. This model resulted that Testing was done at the end of the development lifecycle (which could take months even years) and thus resulting in defects found at the end of the lifecycle. In the early 90’ the V-model was introduced, this model focussed on applying testing/verification on the different stages of the Development Life Cycle. The problem was that most companies interpreted this model kind of the same as the waterfall approach and only applying testing at the end of the development cycle. Unit tests were not written, no system integration testing and a lot of focuss on system and acceptance testing. In general the iterations in the V-model took somewhere around 4 months or more, resulting in quarterly releases to production. This generally results in many defects found late in the cycle, worst of all found in production and a slow release cadence to production.

© 2018 CTG, Inc. 4

Presenter

Presentation Notes

As we all know, finding bugs late, costs us more money. Barry Boehm delivered us this marvelous graph in his study: Software Engineering Economics This graphs shows the cost of finding and correcting a defects, related to its development phases. You’ll notice that the earlier you find a defect, the cheaper it is to solve. You have less items to correct AND you avoid encountering false positives.

© 2018 CTG, Inc. 5

Presenter

Presentation Notes

Frameworks such as Agile and SAFe try to improve on this by using regular intervals on which working software is demonstrated to end customers. This limits the time between the ideation of a new feature and the first time the business actually sees the implementation of this feature. These frameworks also have quality built in each iteration, quality is part of the defenition of done and is everyone’s responsability. What we do see in real life projects is that this solves part of the puzzle being the early discovery of defects and that there is a mismatch between the idea and what is actually delivered.

© 2018 CTG, Inc. 6

Presenter

Presentation Notes

Next to the issue of finding bugs late in the cycle, i’m certain that most of you know this slide. There is still a huge gap in software development projects between what is demanded by a cutomer (or what he believes he needs) and what is finally delivered to the customer. Resulting in a lot of rework and projects running out of time & budget.

© 2018 CTG, Inc. 7

• 21% of projects fail due to requirementsStandish Group CHAOS Report

• 57% of defects are made before coding startsLauesen & Vinter – Preventing Requirements Defects

• Working Software over Comprehensive DocumentationAgile Manifesto

Presenter

Presentation Notes

Research has demonstrated that one of the main reasons for software development projects to fail is bad or unclear requirements. This is where approaches such ATDD, BDD, specification by example and others try to solve

© 2018 CTG, Inc. 8

Developer Testing User Testing

TDD ATDDBDD

DEV METHODOLOGY

USER STORIES & SCENARIOS

DOMAIN SPECIFIC LANGUAGE

CUSTOMER CENTRIC APPROACH

LIVING DOCUMENTATION, EXAMPLE BASED

AUTOMATED ACCEPTANCE CRITERIA

Presenter

Presentation Notes

BDD is a methodology for developing software through continuous example-based communication between developers, testers and business analysts. More than anything else, the primary purpose of BDD is to encourage communication amongst stakeholders of the project so that the context of each feature is correctly understood by all members of the team before development work starts.

© 2018 CTG, Inc. 12

Overview Process

Build Server

Feature Files

Compile

BuildTest

Test AutomationDiscover & Define

User Stories

Add Feature Files to versioning

GUI Tests

Jenkins

Unit Tests

NUnit, MS Unit Test, …

Formalize

Three Amigo’sGherkin Feature files & Scenario’s

Add Features & User Stories

Planning & Tracking

Add Automated Tests to versioning

Run

Push Test Results back to Test Manangement System

LeanFT

LeanFT

Add Feature File

© 2018 CTG, Inc. 13

Overview ProcessDiscover & Define

Discover & Define

Three Amigo’s


Planning & Tracking

© 2018 CTG, Inc. 14

Overview Feature

© 2018 CTG, Inc. 15

Add User Story to Feature

Presenter

Presentation Notes

Name: Color Filtering As an end user I would like to be able to filter on color So that i can only see products in that color

© 2018 CTG, Inc. 16

Overview User Story

© 2018 CTG, Inc. 17

Overview ProcessFormalize

Discover & Define

User Stories

Formalize



Planning & Tracking

Add Feature File

© 2018 CTG, Inc. 18

Dissecting the GherkinOverview Structure

Feature

Scenario

Given – When – Then

Background

Scenario outline

© 2018 CTG, Inc. 19

Dissecting the GherkinScenario Keywords

GIVEN

WHENAn ActionThe test action that will be executedTest action = the user input action

THENThe expected consequenceThe expected consequence of the action performed (WHEN) in the state (GIVEN) mentioned above

The ContextCreating a state of an application in which we want to execute the scenario

© 2018 CTG, Inc. 20

Dissecting the GherkinScenario Example

Feature: Functionality of the Login PageIn order to shop on the siteAs an end-userI need to log-in

Scenario: Login with correct credentialsGiven the user navigated to the home pageWhen the user fills in a usernameAnd fills in a correct passwordAnd clicks on log-inThen the user should be logged-in

© 2018 CTG, Inc. 21

Dissecting the GherkinBackground Example

Scenario: Buy MiceGiven the user navigated to the home pageWhen the user fills in a usernameAnd fills in a correct passwordAnd clicks on log-inAnd he adds a mice to his shopping cardThen the mice should be added to his shopping card

Scenario: Buy KeyboardGiven the user navigated to the home pageWhen the user fills in a usernameAnd fills in a correct passwordAnd clicks on log-inAnd he adds a keyboard to his shopping cardThen the keyboard should be added to his shopping card

© 2018 CTG, Inc. 22


Background: Given the user navigated to the home pageAnd the user fills in a usernameAnd fills in a correct passwordAnd clicks on log-in

Scenario: Buy MiceGiven the user navigates to the products pageWhen he adds a mice to his shopping cardThen the mice should be added to his shopping card

Scenario: Buy KeyboardGiven the user navigates to the products pageWhen he adds a keyboard to his shopping cardThen the keyboard should be added to his shopping card

© 2018 CTG, Inc. 23


Background: Given the logged-in

Scenario: Buy MiceGiven the user navigates to the products pageWhen he adds a mice to his shopping cardThen the mice should be added to his shopping card

Scenario: Buy KeyboardGiven the user navigates to the products pageWhen he adds a keyboard to his shopping cardThen the keyboard should be added to his shopping card

© 2018 CTG, Inc. 24

Dissecting the GherkinOutline Example

Background: Given the logged-in

Scenario Outline: Buy HardwareGiven the user navigates to the products pageWhen he adds a <product> to his shopping cardThen the <product> should be added to his shopping card

Examples:|product ||mice ||keyboard |

© 2018 CTG, Inc. 25

Add Gherkin Test to User Story

© 2018 CTG, Inc. 26

Create Gherkin Test

Presenter

Presentation Notes

Name: Color Filter Validation Validate the different acceptance criteria for the filter functionality: Only items with the correct collor should appear The price of the items should be the same for all colors The Order must be based on Price Descending If no items are found, display all the items with a clear message that the filter didn’t return any results

© 2018 CTG, Inc. 27

Overview Gherkin Tests covering User Story

© 2018 CTG, Inc. 28

Gherkin Test Details

© 2018 CTG, Inc. 29

Gherkin Test Script: Feature File

© 2018 CTG, Inc. 30

Gherkin Test Automation Status

© 2018 CTG, Inc. 31

Gherkin Test Start Manual Run

© 2018 CTG, Inc. 32

Gherkin Test Manual Execution

© 2018 CTG, Inc. 33

Gherkin Test Manual Run Overview

© 2018 CTG, Inc. 34

Overview Process: Test Automation

Feature Files


User Stories


Formalize



Planning & Tracking


LeanFT

Add Feature File

© 2018 CTG, Inc. 35

Test Automation Tools Used

Technology UseGherkin/Cucumber BDD Scenarios – Test casesLeanFT (Java) Automation framework – implement test casesTestNG Test Framework – test set and test suite runnerMaven Project build – manage libraries, build configurationsGIT Source Code Versioning

LeanFT

© 2018 CTG, Inc. 36

Gherkin Stack Flow

© 2018 CTG, Inc. 37

Steps in the automation process

1. The Test Automater (TA) will create a new Maven project in Intellij/Eclipse IDE2. Download the feature file from Octane and add it to the Maven project3. Create the LeanFT Test4. Create the Step Definitions 5. Create the ObjectModel 6. Create the necessary Helpers & Framework7. Create the necessary test data8. Execute the test cases and refactor untill they pass (Red-Green-Refactor)9. Once the test cases are automated, the TA will store them into the versioning system

© 2018 CTG, Inc. 38

Download Feature file from Octane to IntelliJ

© 2018 CTG, Inc. 39

Create LeanFT Test

© 2018 CTG, Inc. 40

Create Step Definitions

© 2018 CTG, Inc. 41

Create Object Model

© 2018 CTG, Inc. 42

Create Object Model

© 2018 CTG, Inc. 43

Use Object Model in Step Definitions

© 2018 CTG, Inc. 44

Add Changed Code to GIT

© 2018 CTG, Inc. 45

Overview Process

Build Server

Feature Files

Compile

BuildTest


User Stories


GUI Tests

Jenkins

Unit Tests

NUnit, MS Unit Test, …

Formalize



Planning & Tracking


Run

Push Test Results back to Octane

LeanFT

LeanFT

Add Feature File

© 2018 CTG, Inc. 46

Steps in the CI & automation process

1. The tests will be executed through a Jenkins job1. Execution can be triggered by any change in source control (continuous build)2. Execution can be triggered by a timed build (nighly/weekly/... build)3. Execution can be triggered manually (start build manually from Octane or Jenkins)

2. Once the job has started, Jenkins will start the execution of the automated LeanFT tests1. Tests will be launched using the Cucumber Runner & LeanFT2. Can be executed locally, in a docker container, in StormRunner Functional, etc.

3. After the test execution:1. Report of test results will be saved locally on the Jenkins master2. The status of the test case execution will be pushed back to Octane3. Octane will provide an overview of the jobs ran in the Jenkins pipeline

© 2018 CTG, Inc. 47

Run the Pipeline Manually from Octane

© 2018 CTG, Inc. 48

Run the Pipeline Manually from Jenkins

© 2018 CTG, Inc. 49

Overview Pipeline in Octane

© 2018 CTG, Inc. 50

Failure Analysis Pipeline Build

© 2018 CTG, Inc. 51

Gherkin Run Results in Octane

© 2018 CTG, Inc. 52

LeanFT Report in Jenkins

© 2018 CTG, Inc. 53

Details Piepline Build

© 2018 CTG, Inc. 54

Commits Pipeline Build

© 2018 CTG, Inc. 55

Backlog Items Pipeline Build

© 2018 CTG, Inc. 56

Failure Analysis Pipeline Build

© 2018 CTG, Inc. 57

Dashboard Pipelines

© 2018 CTG, Inc. 58

© 2018 CTG, Inc. 59

Q&A

Bjorn BoisschotSolution Architect Testing

+32 (0)485 50 77 [email protected]

Date post:	26-Jun-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

ADMIRE 2018 WELCOME! -...

Documents