Date post: | 05-Dec-2014 |
Category: |
Documents |
Upload: | david-odowd |
View: | 1,004 times |
Download: | 0 times |
1
“As a part of Liberty Mutual Information Systems enable Liberty Mutual Group to exceed their business objectives by delivering high value, market responsive IT solutions”
Testing During The Development Phase Using Open Source Tools
March 2008
Richard Thompson (Verification Technologist)
2
Agenda
IntroductionWhy Open SourceExamples Through the LifecycleLessons in Implementing Open Source SuccessfullyQuestions
3
Overview of Liberty-IT
Belfast based software development & testing house with ~230 staff Our parent company is Liberty Mutual Group
a leading global insurerLiberty Mutual ranks 95th on the Fortune 500 list of largest U.S. corporations (based on 2006 revenue). $85.5 billion in consolidated assets
Technologies includeJava, .Net, Struts, IBM PortalHibernate, Spring, Web Services -> SOAMultiple database platformsDedicated Testing practice focusing on technical testing solutions
4
Disclaimer
The views expressed within this presentation belong solely to Richard Thompson and do not necessarily represent those of Liberty Mutual or Liberty-IT.
5
Why Open Source?
6
Business Demands
FasterStrategic advantage to be first to marketQuarterly -> Monthly -> Fortnightly Releases
BetterLower and lower tolerance for poor qualityExpect higher productivityDemand more visibility into project status
CheaperContinually looking to reduce costsEverything must have a ROI (Return on Investment)
7
Perception of high vendors pricesGrowing number of high quality, mature open source products
Which are highly reliableHave large world wide support
More and more large companies are using Open Source tools/ components
AmazonGoogleIBMConducted by Boston-based Forrester Research Inc. in February 2004, a study revealed that 65 of the 140 companies questioned -- almost 50 percent -- are using open source applications.
Growing Role of Open Source
8
Examples Through the Lifecycle
9
Open Source Solutions
Unit Testing JUnit, HttpUnit, HTMLUnit, MaxQ
Automated Functional/ Regression Testing
Watir, Samie, Pamie, Jiffie
Performance Testing
OpenSTA, JMeter, WebInject, Grinder
Web Service Testing SoapUI, HttpUnit, XMLUnit
Code CoverageEmma, Cobertura
Test Management TestLink,Test Case Web
Static Analysis CheckStyle, PMD, FindMoreBugs
Continuous Integration
Cruise Control
Configuration Management
CVS, Subversion
Build Management ANT
Project Management XPlanner, dotProject
Application Monitoring
Big Brother, Nagios
Defect Tracking Mantis, Bugzilla
10
RequirementsRequirements DevelopmentDevelopment
Unit/ComponentTesting
RequirementsReview & Testability
IS TestingIS Testing ProductionProduction
Performance Testing
Smoke Testing
Application Monitoring
Defect Tracking
Reports
SystemTesting
Code Review & Static Analysis
PostImplementation
Validation
Code Coverage
Business TestingBusiness Testing
Acceptance Testing
RegressionTesting
Testing Through the Lifecycle
11
Automated Unit Test Reporting
JUnit• Simple framework for writing automated tests• Written in Java to test Java, so Developers only need to know 1 language• Most widely used automated unit testing framework for Java• Provides fast/ repeated feedback on quality of code
12
PMD – Static Analysis
PMD scans Java source code and looks for potential problems like:•Possible bugs - empty try/catch/finally/switch statements •Dead code - unused local variables, parameters and private methods •Suboptimal code - wasteful String/StringBuffer usage •Overcomplicated expressions - unnecessary if statements, for loops that could be while loops •Duplicate code - copied/pasted code means copied/pasted bugs
13
Continuous Integration Process
Code Repository
Change Request
Cruise Control Server
(Continuous Build & Test)
Build(Automated Build
Script)
Automated Unit Test
Automated Code Analysis
1. Code Coverage, 2. Static Analysis, 3.Coding Standards
Reporting DB
Developer
14
Code Coverage
15
Cruise Control Dashboard
16
Continuous Integration – Daily builds
17
Pre-commit validationEnsures that comment containsa) BUGID:b) BUGNOTE:c) TEST PLAN/DESCRIPTION
Post-commit validationWrites out the followinginformation to a DB tablea) BUGID:b) BUGNOTE:c) TEST PLAN/DESCRIPTIONd) CVS Committere) Files updated (with full path)
CVS
Comment OK
Comment InvalidDeveloper
Checks In code
MantisBugnote
CVS Check-In Process
18
Watir – Browser AutomationWatir•Web Application Testing in Ruby •Watir is a toolkit used to automate browser-based tests in Internet Explorer using IE’s COM interface
19
OpenSTA•Open Source alternative to LoadRunner/ Rational Performance Manager•Large community, excellent online documentation•Option to purchase guaranteed support (for a price ☺)
Performance Testing
20
Lessons In Implementing Open Source Successfully
21
Factors to consider when selectingMain Places to look
Sourceforge (http://sourceforge.net/ )Freshmeat (http://freshmeat.net/ )
What to look forActive communityRegular releases> 90% ActivityDocumentationDo the technologies used match those used within the company?
Perform a PilotTry on a single project, review lessons learned, move forward
Determining ROIMeasure cost to implement on project verses benefits gainedAlso factor in ongoing maintenance
ImplementationTypically requires a higher level of technical ability than Vendor solutions
22
• Perform a pilot implementation and measure the setup time, learning curve & value added
• For example: Piloting Watir on a particular project resulted in:• 38% less defects identified during Business Testing
• Testing Coverage significantly increased
• Developers free to perform other activities
• Automated tests reused within Regression & Smoke Testing
• Reduction in Defects found in Production
Pilot & Measure The Value
0102030405060708090
Requirements Development Testing Production
%
MayAug
23
Questions?