+ All Categories
Home > Technology > Testing automaton

Testing automaton

Date post: 12-Jan-2017
Category:
Upload: coldfusionconference
View: 225 times
Download: 0 times
Share this document with a friend
of 33 /33
Testing Automaton Luis Majano @lmajano
Transcript
Page 1: Testing automaton

Testing AutomatonLuis Majano @lmajano

Page 2: Testing automaton

Who am I

• Luis Majano

• Computer Engineer

• Imported from El Salvador

• CEO of Ortus Solutions

• Creator of all things Box

Page 3: Testing automaton

Why people don’t testWhy we need to test

How to testHow to automate

Page 4: Testing automaton

Where to start?

Write Tests

Page 5: Testing automaton

WHY PEOPLE DON’T TEST

COMFORT

Page 6: Testing automaton

WHY PEOPLE DON’T TEST

New MethodologyNew Learned Behavior

It is a leap….

Page 7: Testing automaton

BIGGEST LIE IN SOFTWARE DEV

Don’t worry, we will create the

tests and refactor it later!

Page 8: Testing automaton
Page 9: Testing automaton

Bugs cost money

Cost To Fix Time detected Requirements Design Building Testing Post-Release

Time Introduced

Requirements 1x 3x 5-10x 10x 10-100x

Design --- 1x 10x 15x 25-100x

Building -- -- 1x 10x 10-25x

^ Kaner, Cem; James Bach, Bret Pettichord (2001). Lessons Learned in Software Testing: A Context-Driven Approach. Wiley. p. 4. ISBN 0-471-08112-4.

Page 10: Testing automaton

• Just do it!

• You will get dirty

• It can hurt (a little)

• Learned behavior

NO MORE EXCUSES

IT WILL ACCELERATE YOUR DEVELOPMENT

Page 11: Testing automaton

TDD/BDD=ProcessContinuousIntegrationContinuousDelivery

ContinuousImprovement

AUTOMATION

Page 12: Testing automaton
Page 13: Testing automaton

TestBox : What you get

API Docs

Core

Test Browser

Test Harness

Test Runner

Samples

install testbox —saveDev

install testbox@be —saveDev

Page 14: Testing automaton

Test Harness

Automated test results!

xUnit/BDD Test Bundles

Harness bootstrap

HTML Runner

ANT Runner

Page 15: Testing automaton

Test Bundle CFCs• Inherits from testbox.system.BaseSpec

•TesBox will then execute all tests/specs/suites within 1 or more bundles

Page 16: Testing automaton

RUN TESTS

• Execute bundles via the URL• http://localhost/tests/bundle.cfc?method=runRemote

• SOAP Runner• HTTP/REST Runner• ANT Runner• NodeJS Runner• CommandBox Runner• Custom Runner

Page 17: Testing automaton

TESTBOX REPORTERS

•ANTJunit : Variant of JUnit for Jenkins•Codexwiki : Mediawiki syntax•Console : Server console•Doc : Semantic HTML•Dot : Awesome Dots•JSON•JUnit•Raw : CFML Struct•Simple : A basic HTML reporter•Text : Back to the 80’s, Text•MinText : Great for CommandBox Runners•XML •Tap : A test anything protocol reporter•Min : Minimalistic Report

Page 18: Testing automaton

MANUAL TESTING

BundleCFC Runner

Page 19: Testing automaton

CommandBox Runnerpackage set testbox.runner = "http://127.0.0.1:57538/tests/runner.cfm"

testbox run

1

2

Page 20: Testing automaton

{ "runner": "http://127.0.0.1:57538/tests/runner.cfm", "directory": "/tests/specs", "recurse": true}

NodeJS Test Runnernpm install -g testbox-runner

create .testbox-runnerrc

1

2

Run it : testbox-runner2

Page 21: Testing automaton

ANT TESTING

Page 22: Testing automaton

DEVELOPER AUTOMATION

1.Grunt Watchers + LiveReload2.Grunt Watchers + testbox-runner3.Grunt Watchers + commanbox4.ColdBox Elixir

Page 23: Testing automaton

CONTINOUS INTEGRATION

Page 24: Testing automaton

CI SERVERS

Page 25: Testing automaton

Jenkins + CommandBoxwget http://mirrors.jenkins-ci.org/war-stable/latest/jenkins.war

box server start warpath=jenkins.war

1

2

Page 26: Testing automaton

Travis CI - Sign Uphttps://travis-ci.org/

Sign Up

1

2

Page 27: Testing automaton

Accounts

Activate For Repositories

1

2

Create .travis.yml3

Commit to repository4

Travis CI - Getting Started

Page 28: Testing automaton

TRAVIS + ANTlanguage: javabranches: only: - developmentsudo: requireddist: trustybefore_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- box install- box server startbefore_script:# Startup the app- curl http://localhost:49616/script: - ant -f workbench/build.xml

Page 29: Testing automaton

TRAVIS + COMMANDBOXlanguage: javasudo: requireddist: trustybefore_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- cd tests- box install- box server startbefore_script:# Startup the app- curl http://localhost:49615/script: > testResults="echo $(box testbox run)"; echo "$testResults"; if grep -i "\[Failures: [1-9][0-9]\?[0-9]\?\]\|\[Errors: [1-9][0-9]\?[0-9]\?\]\|<t[^>]*>\|<b[^>]*>" <<< $testResults; then exit 1; fi

Page 30: Testing automaton

MULTIPLE VERSIONS

MULTIPLE ENGINES

Page 31: Testing automaton

TRAVIS + MULTIPLE ENGINESlanguage: javasudo: requiredjdk:- oraclejdk8env: matrix: - [email protected] - ENGINE=lucee@5 - ENGINE=adobe@2016 - ENGINE=adobe@11 - ENGINE=adobe@10before_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- box installbefore_script:- box server start cfengine=$ENGINE port=8500script: > testResults="$(box testbox run runner='http://127.0.0.1:8500/tests/runner.cfm' )"; echo "$testResults"; if grep -i "\[(Failures|Errors): [1-9][0-9]\?[0-9]\?\]\|\[Errors: [1-9][0-9]\?[0-9]\?\]\|<t[^>]*>\|<b[^>]*>" <<< $testResults; then exit 1; finotifications: email: false

Page 32: Testing automaton

Resources

• TestBox : ortussolutions.com/products/testbox

• CommandBox: ortussolutions.com/products/commandbox

• Slack: boxteam.herokuapp.com

• CFML Slack: #box-products

• travis-ci.org

• jenkins.io

Page 33: Testing automaton

Thank you!


Recommended