Who am I
Gavin Pickin – developing Web Apps since late 90s
● New Addition to Ortus Solutions● ContentBox Evangelist
What else do you need to know?● Blog - http://www.gpickin.com● Twitter – http://twitter.com/gpickin● Github - https://github.com/gpickin
Let’s get on with the show.
Trick Question - You are all testing your apps.
● Most people look like this guyClicking around in the browser yourself
● Setup Selenium / Web Driver to click around for you
● Structured Programmatic Tests
Bugs Hurt
•Bugs hurt – the later in the process, the harder to
fix.
•Test Early and Often
–Find them before they rot your foundation
•Testable Code is Maintainable Code
Types of Testing
● Black/White Box● Unit Testing● Integration Testing● Functional Tests● System Tests● End to End Tests● Sanity Testing
● Regression Test● Acceptance Tests● Load Testing● Stress Test● Performance Tests● Usability Tests● + More
Important Testing Types
•Unit Testing
–Test behavior of individual objects
•Integration Testing
–Test Entire Application from Top Down
•UI verification testing
Verification via HTML/Visual elements
Important Testing Tools
•TestBox (Run BDD and MXUnit style)
•IDE - CF Builder / Eclipse
•Mocking Framework
•ANT
•Jenkins, Bamboo, Teamcity, other Cis
•Selenium
•Jmeter or Webstress Tool, Apache AB
What is TestBox?
TestBox is a next generation testing framework for ColdFusion (CFML) that is based on BDD (Behavior Driven Development) for providing a clean obvious syntax for writing tests. It contains not only a testing framework, runner, assertions and expectations library but also ships with MockBox, A Mocking & Stubbing Framework. It also supports xUnit style of testing and MXUnit compatibilities.
More info: https://www.ortussolutions.com/products/testbox
Integration Tests
● Integration Tests several of the pieces together
● Most of the types of tests are variations of an Integration Test
● Can include mocks but can full end to end tests including DB / APIs
What is Unit Testing
“unit testing is a software verification and validation method
in which a programmer tests if individual units of source code
are fit for use. A unit is the smallest testable part of an
application”
- wikipedia
Unit Testing
•Can improve code quality -> quick error discovery
•Code confidence via immediate verification
•Can expose high coupling
•Will encourage refactoring to produce > testable code
•Remember: Testing is all about behavior and expectations
Style of Testing - TDD vs BDD
● TDD = Test Driven Development● Write Tests● Run them and they Fail● Write Functions to Fulfill the Tests● Tests should pass● Refactor in confidence● Test focus on Functionality
Style of Testing - TDD vs BDD
● BDD = Behavior Driven Development● Actually similar to TDD except:● Focuses on Behavior and Specifications● Specs (tests) are fluent and readable● Readability makes them great for all levels of testing in the
organization
● Hard to find TDD examples in JS that are not using BDD describe and it blocks
BDD Example
Describe( ‘Email Address’, function(){
It(‘should not be blank’, function(){
expect(email).not.toBe(“”);
});
});
BDD Matchers
expect(true).toBe(true);
expect(true).toBe(true);
expect(true).toBe(true);
expect(true).toBe(true);
BDD Matchers
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
Matcher Examples
expect(true).toBe(true);
expect(a).not.toBe(null);
expect(a).toEqual(12);
expect(message).toMatch(/bar/);
expect(message).toMatch("bar");
expect(message).not.toMatch(/quux/);
expect(a.foo).toBeDefined();
expect(a.bar).not.toBeDefined();
BDD Example
describe("Hello world function", function() {
it(”contains the word world", function() {
expect(helloWorld()).toContain("world");
});});
New BDD Examplefeature( "Box Size", function(){
describe( "In order to know what size box I need
As a distribution manager
I want to know the volume of the box", function(){
scenario( "Get box volume", function(){
given( "I have entered a width of 20
And a height of 30
And a depth of 40", function(){
when( "I run the calculation", function(){
then( "the result should be 24000", function(){
// call the method with the arguments and test the outcome
expect( myObject.myFunction(20,30,40) ).toBe( 24000 );
});
});
});
});
});
});
Installing Testbox
Install Testbox – Thanks to Commandbox - this is easy.
# box install testbox
Next, we need to decide how you want to run Testbox
Create a Runner.cfm File to Run your tests<cfsetting showDebugOutput="false">
<!--- Executes all tests in the 'specs' folder with simple reporter by default --->
<cfparam name="url.reporter" default="simple">
<cfparam name="url.directory" default="tests.specs">
<cfparam name="url.recurse" default="true" type="boolean">
<cfparam name="url.bundles" default="">
<cfparam name="url.labels" default="">
<!--- Include the TestBox HTML Runner --->
<cfinclude template="/testbox/system/runners/HTMLRunner.cfm" >
Create a test suite// tests/specs/CFCTest.cfc
component extends="testbox.system.BaseSpec" {
function run() {
it( "will error with incorrect login", function(){
var oTest = new cfcs.userServiceRemote();
expect( oTest.login( '[email protected]', 'topsecret').result ).toBe('400');
});
}
}
Create 2nd Test Suite// tests/specs/APITest.cfccomponent extends="testbox.system.BaseSpec" { function run() { describe("userService API Login", function(){ it( "will error with incorrect login", function(){ var email = "[email protected]"; var password = "topsecret”; var result = ""; http url="http://www.testableapi.local.com:8504/cfcs/userServiceRemote.cfc?method=login&email=#email#&password=#password#" result="result”;
expect( DeserializeJSON(result.filecontent).result ).toBe('400'); }); });
}}
Run tests with Grunt
● Install Testbox Runner – Thanks Sean Coyne
# npm install testbox-runner
● Install Grunt Shell
# npm install grunt-shell
Then, we need to add Grunt Configuration
Add Grunt Config - 1
module.exports = function (grunt) {
grunt.loadNpmTasks('grunt-shell');
grunt.initConfig({ … })
}
Add Grunt Config - 3
shell: {
testbox: {
command: "./node_modules/testbox-runner/index.js --colors --runner http://www.testableapi.local.com:8504/tests/runner.cfm --directory /tests/specs --recurse true”
}
}
Add Grunt Config - 4
grunt.registerTask("testbox", [ "shell:testbox" ]);
grunt.loadNpmTasks('grunt-contrib-jasmine');
grunt.loadNpmTasks('grunt-contrib-watch');
Grunt Config Gists
Jasmine + Testbox
https://gist.github.com/gpickin/9fc82df3667eeb63c7e7
TestBox JSON output
Testbox has several runners, you have seen the HTML one, this Runner uses the JSON runner and then formats it.
http://www.testableapi.local.com:8504/tests/runner.cfm?reporter=JSON&directory=%2Ftests%2Fspecs&recurse=true
Running TestBox in Sublime 2
Install PackageControl into Sublime Text
Install Grunt from PackageControl
https://packagecontrol.io/packages/Grunt
Update Grunt Sublime Settings for paths{
"exec_args": { "path": "/bin:/usr/bin:/usr/local/bin” }
}
Then Command Shift P – grunt
Run TestBox with CommandBox
Run the tests in the box.json file.
# testbox run
Watch the folders and run tests on change
# testbox watch
More info on TestBox usage from CommandBox: https://www.ortussolutions.com/blog/using-testbox-watch-to-automate-your-testing-suite
More Testing - Mocking with MockBox
"A mock object is an object that takes the place of
a ‘real’ object in such a way that makes testing
easier and more meaningful, or in some cases,
possible at all"
by Scott Bain - Emergent Design
More Testing - IntegratedStart using an easy, fluent API for your integration tests!
More Info: https://github.com/elpete/integrated
Api - https://elpete.github.io/integrated/
Some real life exampleshttps://github.com/framework-one/fw1/tree/develop/tests maybe?
https://github.com/Ortus-Solutions/TestBox/tree/development/tests
https://github.com/foundeo/cfdocs/tree/master/tests
https://github.com/foundeo/cfmlparser/tree/master/tests/tests
https://github.com/foundeo/toscript/tree/master/tests
https://github.com/foundeo/bolthttp/tree/master/test
https://github.com/foundeo/cfml-security/tree/master/tests/tests/secureupload
Some real life examples cont.https://github.com/ryanguill/emit/tree/master/tests
https://github.com/ryanguill/cfmlBase62/tree/master/tests
https://gitlab.com/ryanguill/FPcfc/tree/master/tests
https://gitlab.com/ryanguill/template-cfc/tree/master/tests
https://github.com/MotorsportReg/sidecar/tree/master/tests/basic
https://github.com/MotorsportReg/sidecar/tree/master/tests/basic