20-21 August 2008, Queenstown, New Zealand
Kia ora Koutou
Greetings everyone!
About me
• CTO, Straker Software, New Zealand
• Involved in web development since 1996– development, content, project management
• Using ColdFusion for over a decade; Flex, 3 years
• Currently in love– Groovy (http://groovy.codehaus.org/)
– Testing processes– Getting Things Done (GTD) methodology
About Straker Software
• Adobe technologies - ColdFusion and Flex
• ShadoCMS (http://shadocms.com)– a large content management software written in ColdFusion
• ZoomFlex (http://zoomflex.com)– A rapid, Flex application generation software
• Been working at Straker for about five years
makeShadoCMS()makeZoomFlex()
ColdFusionFlex
Straker
Agenda
• Touch upon high-level issues in using testing
• Share some experiences of implementing a testing process
• Look at some tools that can help
• Basic idea: When you leave here, you have a fairly decent idea of what does it take to set up testing processes that work
Recognize the need for testing
• THE first step
• Ask questions like... what happens to the existing software if– New functionality has to be added– Fixing an issue in commonly used component– Departure of a developer
• How do you handle these?
• Ideally, how would you like to handle these?
We test anyway...
• All of us test our code
• In ColdFusion– Create a component (CFC), with methods– Create a script (CFM)– Instantiate the CFC in CFM– Fire methods of the instantiated CFC– Eyeball results
• Commonly used approach
But what is wrong with this?
• Unsystematic
• Unorganized
• Dependent on an individual
• Not repeatable in different situations
• Does not lend itself to automation
Why do we continue like this?
Organizational attitude
• Perhaps the most important one to change
• If an organization does not value testing, developers won’t do it
• Build a case for return on investment– Reduce time to market– Reduce ongoing support– Build quality software
• Identify in-house lobbyist
?
Consider this...
• 80% of the avoidable rework comes from 20% of the defects
• 90% of the downtime comes from 10% of the defects
• Disciplined personal practices can reduce defect introduction rates by up to 75%
• Peer reviews catch 60% of the defects
• http://www.cebase.org/www/frames.html?/www/researchActivities/defectReduction/top10/index.html
Individual factors
• All of the significant battles are waged within the selfSheldon Kopp (If You Meet the Buddha on the Road, Kill Him, 1976)
• Recognizing that the only Constant is Change– From being a developer to a technologist– From being “lazy” to proactive
• Quick exercise: Think of what the internet was when you first experienced it. Compare it to how you use it now.
Technological change
• Perhaps the easiest aspect
• Formal testing is becoming more mainstream in scripting languages
• Multiple frameworks: testing, mocking
• Integration with IDEs
• Integration with automation tools
Testing Frameworks #1
• ColdFusion Unit testing– MXUnit (http://mxunit.org)
– CFUnit (http://cfunit.sourceforge.net/)
– RocketUnit (http://rocketunit.riaforge.org/)
– CFCUnit (http://www.cfcunit.org/)
• HTML User Interface (UI) Testing– Selenium (http://selenium.openqa.org/)
– Canoo Webtest (http://webtest.canoo.com/)
Testing Frameworks #2
• Flex Unit Testing– FlexUnit (http://code.google.com/p/as3flexunitlib/)
– DPUnit (http://code.google.com/p/dpuint/)
– ASUnit (http://www.asunit.org/)
• Database Unit Testing– DBUnit (http://dbunit.sourceforge.net/)
Choosing a Framework
• Different needs for– Different organizations– Different technologies
• Defining criteria for choosing a framework is important
Choosing a ColdFusion Framework
• Integration with IDE (Eclipse)• Support for automation (Ant, Continuous Integration tools)• Reporting (Standard formats like JUnit)• Extendability (Creating own assertions)• Easy generation of dummy tests• Community support
• MXUnit
HTML UI Testing Framework
• Easy recording tests• Conversion of test scripts into Java/Groovy• Reporting
– Standard formats like JUnit• Support for automation• Community support
• Selenium
Writing tests
• Tests for new functionality– Test generators play a big role– Create tests, watch them fail, add test logic, watch them pass– Remember, the earlier you catch issues, the lesser the cost later
on
• Tests for existing codebase– Time consuming– Code does not lend itself easily to be tested– Setting aside regular time for writing tests - difficult to justify/
implement
Leverage IDE support
• Eclipse is a great IDE– CFEclipse for ColdFusion development– Plugins for other languages: Groovy
• MXUnit integrates beautifully– CFUnit comes pretty close
• Provide views for loading and running tests
• Setup for loading application and session scopes before running tests
Automation
• Manual running of tests does not scale
• Use Apache Ant for automating running of tests
• Use continuous integration server like Hudson to– Create build > Run tests > View reports
• Ensures that new changes do not break existing code
• Host on fast/multiple servers
Source Control Management
• How many use a source control management system (SCM)?
• Subversion is very good
• Works well with ColdFusion development teams– Haven’t made up my mind about next generation SCM tools like
Mercurial, Git, Baazar
• Great client tools -- TortoiseSVN, Subclipse, SmartSVN
• Host on fast servers -- either in-house or externally
Code Review Systems• Remember peer-reviews catch up to 60% of defects
• Establishing a Code Review System - Review Board
• Works in conjunction with an SCM system
• Allows peer review before code is checked into a source control system
• A little involved to set them up, but well worth the effort
SPEED
LIMIT
55
Groovy -- the language to learn
• Simple scripting language that compiles into Java
• Easy to learn; rapid application development
• Java libraries written in Groovy can be used directly in CF
• In-built support for Ant– Using Ant with scripting rather than XML
• Integrates very well with CI servers like Hudson
Testing Champion
• Someone in the organization needs to advocates testing
• Someone with perceived competence and experience
• Sets up testing processes and guidelines
• Reviews processes and refines
• Occasionally brings out the whip
Summing up
• “If you are not testing, you are crazy.” - Thomas Burleson
• Keep in mind the organizational, individual and technological
factors that impact testing processes
• Get organizational buy-in
• Choose frameworks carefully
• Testing and Continuous Integration go hand in hand
• Be the Testing Champion in your organization
• Embrace change