A SUPPLEMENT TO

OCTOBER 1 2006

Leading ToolsAnd SolutionsFor SoftwareQualityAssurance

Testing is essential in everysoftware development

project Yet while computerscience programs teach devel-opers about software architec-ture object-oriented designalgorithms and programmingthey offer little formal empha-sis on testing and qualityassurance

Every day managers im-plore their programmers tocode faster But do they pro-vide them with the tools andknowledge they need to writebetter code Do they equiptheir test teams with the bestresources to stamp out defectsand vulnerabilities All toooften testing is neglected atevery stage

Testing is an art A develop-er or tester needs to understandwhere quality assurance fitsinto the application develop-ment life cycle and why therersquosmore to quality than passing atest suite or meeting some arbi-trary metric

Testing is a science Overthe past 30 years a significant

body of knowledge has evolvedbest practices for functionaltesting unit testing and per-formance testing but nowthose classic techniques arebeing augmented by recentinnovations in the field ofdefect management testautomation software config-uration management metricstest design and securityvul-nerability testing

Great software requiresgreat tools and great serviceproviders ldquoThe Art amp Scienceof Software Testingrdquo profilesleading testQA solutions tohelp you choose the right part-ners for your projects Thesecompanies can help yourdevelopment and test teamsmake better software

We hope you enjoy thisspecial testQA supplement toSD Times ampamp

1 october 2006 the artampamp science of software testing u 3

The ArtampampScienceOf Software Testing

Editorial DirectorAlan Zeichickalanbzmediacom

Managing EditorPatricia Saricapsaricabzmediacom

Art DirectorLuAnn T Palazzo

Copy EditorLaurie OrsquoConnell

Lead WriterGeorge Walshgwalshbzmediacom

Customer ServiceSD Times Subscriptions+1-847-763-9692sdtimeshalldatacom

Article ReprintsLisa Abelson +1-516-379-7097labelsonbzmediacom

BZ Media LLC7 High Street Suite 407Huntington NY 11743+1-631-421-4158 bull fax +1-631-421-4130wwwbzmediacom bull infobzmediacom

PresidentTed Bahr

Executive Vice PresidentAlan Zeichick

Cover Photograph by Elena Korenbaum

A SUPPLEMENT TO SD TIMESOCTOBER 1 2006

Copyright copy 2006 BZ Media LLCAll Rights Reserved

SUBSCRIBE TODAYwwwsdtimescom

ArtScienceBoth

Platinum

Thanks to Our Sponsors

Diamond

Gold

Alan Zeichick

Editorial Director SD Times

4 CEO PERSPECTIVERethinking Our Approach to Software QualityADAM KOLAWA CO-FOUNDER AND CEO PARASOFT

6 DIAMOND SPONSORParasoft Catches Bugs Before They HatchmdashWith AutomationPARASOFT

11 TEST SOLUTIONS LEADERIBM

15 APPLICATION LIFE CYCLE MANAGEMENTTechExcel Takes the Guesswork Out of Quality AssuranceTECHEXCEL

17 DEFECT TRACKINGAxosoft Keeps Development Projects on the Fast TrackAXOSOFT

19 SOFTWARE CONFIGURATION MANAGEMENTPerforce Makes SCM Fast Easy and First-ClassPERFORCE

21 WEB SECURITYKeeping the Bad Guys at Bay With Cenzic SolutionsCENZIC

23 TESTING SERVICESStelligent Brings Objectivity to Quality MeasurementSTELLIGENT

introduction

t a b l e o f c o n t e n t s

The primary mission of information technology is toincrease profits through improved business processesCompanies are constantly rethinking and struggling

with how to use IT to a competitive advantage reduce IT oper-ating and maintenance costs and reduce the total cost of own-ershiphellipall while attempting to deliver increased value

Most of these challenges are directly linked to makingsoftware workmdashwithout incurring unreasonable costs Manypeople in the industry would agree that low IT productivi-ty is the reason software development is so costly But whyare IT teams with all their expertise and hard work suf-fering from low productivity The root cause of low produc-tivity is errors made throughout the software developmentlife cycle

These errors include everything from performance errorsto security errors to misimplemented functionality to errorsthat crash an entire system They essentially stifle IT teamsrsquoability to produce working software in a reasonable time andat reasonable costs In fact if you look at virtually any ITteam you will see that its members spend about 80 percentof their time chasing and fixing bugs and only about 20percent of their time on tasks that deliver value and improvethe business This practice is far from productive

Adding to this inefficiency is the traditional softwaredevelopment approach of leaving testing till late in the devel-opment life cycle It is only then that QA does the testingnecessary to ensure that bugs are found requirements metand reliability performance and security goals achievedbefore an application is deployed into production Findingand fixing errors late in the development cycle is exponen-tially more costly time-consuming and inefficient thanaddressing them early and throughout the cycle Maintainingthis approach is a primary reason that we continue to strug-gle with quality and low productivity in the software world

Many other industries have struggled with low qualityhigh costs and low productivity as a result of human errorThe automotive industry for example recognized that althoughmistakes cannot be entirely eliminated they can be controlledThose automotive manufacturers who by taking a holisticand preventative approach to the problem making quality anintegrated focus throughout the production process and evenmodifying their production lines to prevent as many errorsas possible from ever entering the products addressed theirmost critical problems and have remained viable

The software industry still has not learned this impor-

tant lesson Many people think that error prevention is notpossible in the software industry they believe that becauseeach piece of software is different the lessons learned fromworking on one piece of software cannot be applied to oth-er pieces Instead of trying to prevent errors from enteringsoftware the industry tries to test errors out at the end ofthe development life cycle

First we build a product then we test at the end of theproduction cycle to determine whether the product worksand finally we remove any errors that testing exposesThroughout this process we cross our fingers and hope thatthe most insidious and embarrassing problems will be iden-tified before the release However a consideration of thestaggering number and impact of software errors reportedannually and their cost to the US economy suggests thatthis quality-through-end-of-cycle-testing approach is notyielding the desired results

The belief that our traditional software testing approachcan create quality software is a fundamental problem Wedonrsquot think of the whole process of building and deployingsoftware in a way that would prevent errors because we donrsquotbelieve that it can actually be done Yet this error preven-tion approach is not only possible it is necessary The increas-

ing complexity of software systems the push for faster nearcontinuous release cycles and the expanding dependencyon software for nearly every phase of business executionrequire that error prevention be addressed

If the software industry is serious about reducing theerror rate and resolving the issues that stem from errors wecanrsquot afford to continue hoping that our current approachto testing will miraculously start yielding quality softwareInstead we need to follow in the footsteps of other indus-

4 t the artampamp science of software testing 1 october 2006

Rethinking Our ApproachTo Software Quality

Testing in general must

become the responsibility

of every team member

c e o p e r s p e c t i v e

1 october 2006 the artampamp science of software testing u 5

tries and start preventing errors throughout the softwaredevelopment life cycle

Achieving Software QualityAchieving a consistently high level of quality starts with com-mitting to a practice of an end-to-end quality process Whilethere is no single silver bullet for producing reliable high-quality software there are proven steps that software devel-opment organizations can and should employ to help preventsoftware errors and improve development productivity Themost significant of these are addressed below

Establish a Quality Initiative And Group Culture Organizations need to establish a group culture that places ashared focus and importance on quality Many companies con-tinue to treat development and testing as independent disci-plines This separation of development and QA leads to manysoftware problems and inefficienciesmdashdevelopers might writecode assuming that someone in the QA department will injectquality into the software When problems occur the code justbounces back and forth between departments without anyonetaking responsibility

The manufacturing world learned long ago that it couldnot separate the responsibility of production from theresponsibility of verification and expect to achieve qualityThese responsibilities go hand in hand Organizations shouldplace development and QA under the same managementor merge the two groups completely to facilitate owner-

ship and responsibility for code quality In an establishedgroup culture developers will show that they care about

the code because caring about the code is synonymouswith caring about the group

Adopt Quality PracticesOrganizations must adopt software error preven-

tion practices from the earliest stages of devel-opment Well-known software error prevention

practices such as coding standards unit test-ing and regression testing are not regularly

followed despite their recognized value in catch-ing errors at the code level early in the cycle when it

is easiest and least costly to find and fix them These prac-tices should be an integrated part of an organizationrsquos devel-opment process Testing in general must become the respon-sibility of every team member

The trend toward service enablement of applications(eg Web services SOA) and faster shorter software releasecycles mandates that quality assurance and testing can nolonger be treated as a set event handled as an independ-ent discipline and relegated to a single proscribed phaseof the development life cycle It must become a continu-ous integrated part of the development process enabledthrough the application of known software development

quality practices

AutomateEffective adoption of software quality practicesrequires automation of as many testing tasks as possibleDevelopment organizations are increasingly burdened toproduce more code faster and with the same or fewerresources If they are to adopt software quality practicesthey will need to use software testing tools and developmentmethodologies that allow them to automate many of thesepractices and integrate them easily into their developmentprocesses Fortunately there are a growing number of prod-uct offerings that deliver this automation and integrationand that allow developers to take a ldquotest-as-you-coderdquoapproach to developmentmdashto readily create reusable testobjects and operate within a framework that facilitates cre-ating high-quality software

If their companies are to remain competitive softwareorganizations must improve their productivity by control-ling their costs while ensuring the high quality of their deliv-erables To do so requires rethinking and restructuring theway we have traditionally approached software developmentand delivery and necessitates a heightened commitment toquality throughout the software development life cycle and

Adam KolawaCo-Founder and CEO Parasoft

Everyone knows that bugs exist But at what stage of thedevelopment cycle should they be isolated and killed

Parasoft says ldquoTest early and often when bugs are easiestand least costly to find and fixrdquo Parasoftrsquos Jtest software forJava developers uses automation to make testing fast easy

and practical for developersto perform during the de-velopment life cyclersquos cod-ing stages when testing is usually the furthestthing from a program-merrsquos mind

ldquoIf you look at adeveloperrsquos resume thelast thing yoursquoll see list-ed is testing becausedevelopers just donrsquotlike to testrdquo says BrianHunt Parasoftrsquos VP of sales and actingCOO (wwwparasoftcom) ldquoHowever atsome point youhave to prove that

the software worksWe help developersvalidate what theyrsquove

built from the point of cre-ation to the completion of development That

validation starts at the desktop in the same way that spellcheckers are run against text documentsrdquo

Jtest provides Java development teams an automated unittesting and code analysis tool suite that performs compre-hensive test and analysis of Java source code exposing bugsand errors in code structure execution and design at thesource or unit level Used as a plug-in to the developerrsquos IDE(such as Eclipse WSAD Rational RAD) or integrated witha central build process Parasoft Jtest is designed to be usedby development teams in a ldquotest-as-you-coderdquo strategy tofind and eliminate errors early in the development processbefore they can infect the main application codebaseldquoUncovering errors early and at their source or root causemakes them quicker and less costly to fix and helps revealand resolve design errors that could have extended negativeimpact on an application if gone undetectedrdquo states Hunt

The newly released Jtest 80 adds new testing innovationsto help teams automatically verify the functionality of com-

plex constantly changing enterprise systems like Java EESOA and Web services reducing the risks of system down-time and security vulnerabilities At the same time teamscan find more defects with their existing resources increas-ing productivity while adhering to budget parameters

One of the most exciting new features of Jtest is itsBugDetective By automatically tracing and simulating exe-cution paths BugDetective exposes runtime defects thatwould be difficult or even impossible to find through man-ual testing or inspections With BugDetective you can nowfind diagnose and fix classes of software errors that routine-ly evade standard analysis and unit testing techniques

Jtest also lets development teams automatically generateand run tests using the popular Apache Cactus test frame-work This gives organizations early development-level defectexposure that might go unnoticed until QA deployment orproduction time when itrsquos a lot more expensive and prohib-itive to find and fix problems Another new technology isJtest Tracer which creates realistic functional JUnit test cas-es that reflect an applicationrsquos correct functional behaviorWith Jtest Tracer organizations can quickly create librariesof regression test cases that can be run to ensure that newcode changes donrsquot inadvertently break existing applicationfunctionality

ldquoThe key to reducing testing time is automationrdquo Huntdeclares ldquoJtest can even perform testing overnight to scanthe code find errors and report those errors to the devel-opers when they start working in the morning It lets themdrill straight through the results to the lines of code thatneed to be fixed It can also perform automatic functionaltests that run the code to make sure that it does what itrsquossupposed to do Because we write these tests in an openformat you can modify and extend them to meet your spe-

cific needsrdquoJtest integrates with com-

plementary Parasoft prod-ucts to provide automatedsystemwide testing solutions

for Web applications Web services and other n-tier systemsMoreover Jtest works as part of a comprehensive teamwideAutomated Error Prevention solution that provides central-ized administration and application of test practices man-agement dashboards and metrics for real-time analysis thathelp managers evaluate code compliance code readiness andteam productivity

With Jtest you catch bugs before they hatchmdashearly andoften ampamp

Parasoft Catches Bugs BeforeThey HatchmdashWith Automation

6 t the artampamp science of software testing 1 october 2006

d i a m o n d s p o n s o r

Brian Hunt

VP of Sales and Acting COO

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

1 october 2006 the artampamp science of software testing u 5

tries and start preventing errors throughout the softwaredevelopment life cycle

Achieving Software QualityAchieving a consistently high level of quality starts with com-mitting to a practice of an end-to-end quality process Whilethere is no single silver bullet for producing reliable high-quality software there are proven steps that software devel-opment organizations can and should employ to help preventsoftware errors and improve development productivity Themost significant of these are addressed below

Establish a Quality Initiative And Group Culture Organizations need to establish a group culture that places ashared focus and importance on quality Many companies con-tinue to treat development and testing as independent disci-plines This separation of development and QA leads to manysoftware problems and inefficienciesmdashdevelopers might writecode assuming that someone in the QA department will injectquality into the software When problems occur the code justbounces back and forth between departments without anyonetaking responsibility

The manufacturing world learned long ago that it couldnot separate the responsibility of production from theresponsibility of verification and expect to achieve qualityThese responsibilities go hand in hand Organizations shouldplace development and QA under the same managementor merge the two groups completely to facilitate owner-

ship and responsibility for code quality In an establishedgroup culture developers will show that they care about

the code because caring about the code is synonymouswith caring about the group

Adopt Quality PracticesOrganizations must adopt software error preven-

tion practices from the earliest stages of devel-opment Well-known software error prevention

practices such as coding standards unit test-ing and regression testing are not regularly

followed despite their recognized value in catch-ing errors at the code level early in the cycle when it

is easiest and least costly to find and fix them These prac-tices should be an integrated part of an organizationrsquos devel-opment process Testing in general must become the respon-sibility of every team member

The trend toward service enablement of applications(eg Web services SOA) and faster shorter software releasecycles mandates that quality assurance and testing can nolonger be treated as a set event handled as an independ-ent discipline and relegated to a single proscribed phaseof the development life cycle It must become a continu-ous integrated part of the development process enabledthrough the application of known software development

quality practices

AutomateEffective adoption of software quality practicesrequires automation of as many testing tasks as possibleDevelopment organizations are increasingly burdened toproduce more code faster and with the same or fewerresources If they are to adopt software quality practicesthey will need to use software testing tools and developmentmethodologies that allow them to automate many of thesepractices and integrate them easily into their developmentprocesses Fortunately there are a growing number of prod-uct offerings that deliver this automation and integrationand that allow developers to take a ldquotest-as-you-coderdquoapproach to developmentmdashto readily create reusable testobjects and operate within a framework that facilitates cre-ating high-quality software

If their companies are to remain competitive softwareorganizations must improve their productivity by control-ling their costs while ensuring the high quality of their deliv-erables To do so requires rethinking and restructuring theway we have traditionally approached software developmentand delivery and necessitates a heightened commitment toquality throughout the software development life cycle and

Adam KolawaCo-Founder and CEO Parasoft

Everyone knows that bugs exist But at what stage of thedevelopment cycle should they be isolated and killed

Parasoft says ldquoTest early and often when bugs are easiestand least costly to find and fixrdquo Parasoftrsquos Jtest software forJava developers uses automation to make testing fast easy

and practical for developersto perform during the de-velopment life cyclersquos cod-ing stages when testing is usually the furthestthing from a program-merrsquos mind

ldquoIf you look at adeveloperrsquos resume thelast thing yoursquoll see list-ed is testing becausedevelopers just donrsquotlike to testrdquo says BrianHunt Parasoftrsquos VP of sales and actingCOO (wwwparasoftcom) ldquoHowever atsome point youhave to prove that

the software worksWe help developersvalidate what theyrsquove

built from the point of cre-ation to the completion of development That

validation starts at the desktop in the same way that spellcheckers are run against text documentsrdquo

Jtest provides Java development teams an automated unittesting and code analysis tool suite that performs compre-hensive test and analysis of Java source code exposing bugsand errors in code structure execution and design at thesource or unit level Used as a plug-in to the developerrsquos IDE(such as Eclipse WSAD Rational RAD) or integrated witha central build process Parasoft Jtest is designed to be usedby development teams in a ldquotest-as-you-coderdquo strategy tofind and eliminate errors early in the development processbefore they can infect the main application codebaseldquoUncovering errors early and at their source or root causemakes them quicker and less costly to fix and helps revealand resolve design errors that could have extended negativeimpact on an application if gone undetectedrdquo states Hunt

The newly released Jtest 80 adds new testing innovationsto help teams automatically verify the functionality of com-

plex constantly changing enterprise systems like Java EESOA and Web services reducing the risks of system down-time and security vulnerabilities At the same time teamscan find more defects with their existing resources increas-ing productivity while adhering to budget parameters

One of the most exciting new features of Jtest is itsBugDetective By automatically tracing and simulating exe-cution paths BugDetective exposes runtime defects thatwould be difficult or even impossible to find through man-ual testing or inspections With BugDetective you can nowfind diagnose and fix classes of software errors that routine-ly evade standard analysis and unit testing techniques

Jtest also lets development teams automatically generateand run tests using the popular Apache Cactus test frame-work This gives organizations early development-level defectexposure that might go unnoticed until QA deployment orproduction time when itrsquos a lot more expensive and prohib-itive to find and fix problems Another new technology isJtest Tracer which creates realistic functional JUnit test cas-es that reflect an applicationrsquos correct functional behaviorWith Jtest Tracer organizations can quickly create librariesof regression test cases that can be run to ensure that newcode changes donrsquot inadvertently break existing applicationfunctionality

ldquoThe key to reducing testing time is automationrdquo Huntdeclares ldquoJtest can even perform testing overnight to scanthe code find errors and report those errors to the devel-opers when they start working in the morning It lets themdrill straight through the results to the lines of code thatneed to be fixed It can also perform automatic functionaltests that run the code to make sure that it does what itrsquossupposed to do Because we write these tests in an openformat you can modify and extend them to meet your spe-

cific needsrdquoJtest integrates with com-

plementary Parasoft prod-ucts to provide automatedsystemwide testing solutions

for Web applications Web services and other n-tier systemsMoreover Jtest works as part of a comprehensive teamwideAutomated Error Prevention solution that provides central-ized administration and application of test practices man-agement dashboards and metrics for real-time analysis thathelp managers evaluate code compliance code readiness andteam productivity

With Jtest you catch bugs before they hatchmdashearly andoften ampamp

Parasoft Catches Bugs BeforeThey HatchmdashWith Automation

6 t the artampamp science of software testing 1 october 2006

d i a m o n d s p o n s o r

Brian Hunt

VP of Sales and Acting COO

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

Everyone knows that bugs exist But at what stage of thedevelopment cycle should they be isolated and killed

Parasoft says ldquoTest early and often when bugs are easiestand least costly to find and fixrdquo Parasoftrsquos Jtest software forJava developers uses automation to make testing fast easy

and practical for developersto perform during the de-velopment life cyclersquos cod-ing stages when testing is usually the furthestthing from a program-merrsquos mind

ldquoIf you look at adeveloperrsquos resume thelast thing yoursquoll see list-ed is testing becausedevelopers just donrsquotlike to testrdquo says BrianHunt Parasoftrsquos VP of sales and actingCOO (wwwparasoftcom) ldquoHowever atsome point youhave to prove that

the software worksWe help developersvalidate what theyrsquove

built from the point of cre-ation to the completion of development That

validation starts at the desktop in the same way that spellcheckers are run against text documentsrdquo

Jtest provides Java development teams an automated unittesting and code analysis tool suite that performs compre-hensive test and analysis of Java source code exposing bugsand errors in code structure execution and design at thesource or unit level Used as a plug-in to the developerrsquos IDE(such as Eclipse WSAD Rational RAD) or integrated witha central build process Parasoft Jtest is designed to be usedby development teams in a ldquotest-as-you-coderdquo strategy tofind and eliminate errors early in the development processbefore they can infect the main application codebaseldquoUncovering errors early and at their source or root causemakes them quicker and less costly to fix and helps revealand resolve design errors that could have extended negativeimpact on an application if gone undetectedrdquo states Hunt

The newly released Jtest 80 adds new testing innovationsto help teams automatically verify the functionality of com-

plex constantly changing enterprise systems like Java EESOA and Web services reducing the risks of system down-time and security vulnerabilities At the same time teamscan find more defects with their existing resources increas-ing productivity while adhering to budget parameters

One of the most exciting new features of Jtest is itsBugDetective By automatically tracing and simulating exe-cution paths BugDetective exposes runtime defects thatwould be difficult or even impossible to find through man-ual testing or inspections With BugDetective you can nowfind diagnose and fix classes of software errors that routine-ly evade standard analysis and unit testing techniques

Jtest also lets development teams automatically generateand run tests using the popular Apache Cactus test frame-work This gives organizations early development-level defectexposure that might go unnoticed until QA deployment orproduction time when itrsquos a lot more expensive and prohib-itive to find and fix problems Another new technology isJtest Tracer which creates realistic functional JUnit test cas-es that reflect an applicationrsquos correct functional behaviorWith Jtest Tracer organizations can quickly create librariesof regression test cases that can be run to ensure that newcode changes donrsquot inadvertently break existing applicationfunctionality

ldquoThe key to reducing testing time is automationrdquo Huntdeclares ldquoJtest can even perform testing overnight to scanthe code find errors and report those errors to the devel-opers when they start working in the morning It lets themdrill straight through the results to the lines of code thatneed to be fixed It can also perform automatic functionaltests that run the code to make sure that it does what itrsquossupposed to do Because we write these tests in an openformat you can modify and extend them to meet your spe-

cific needsrdquoJtest integrates with com-

plementary Parasoft prod-ucts to provide automatedsystemwide testing solutions

for Web applications Web services and other n-tier systemsMoreover Jtest works as part of a comprehensive teamwideAutomated Error Prevention solution that provides central-ized administration and application of test practices man-agement dashboards and metrics for real-time analysis thathelp managers evaluate code compliance code readiness andteam productivity

With Jtest you catch bugs before they hatchmdashearly andoften ampamp

Parasoft Catches Bugs BeforeThey HatchmdashWith Automation

6 t the artampamp science of software testing 1 october 2006

d i a m o n d s p o n s o r

Brian Hunt

VP of Sales and Acting COO

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

SPONSORED WHITE PAPER

When it comes to software testing two key words are speedand accuracymdashnot only of the application yoursquore test-

ing but also of the QA process itself Thatrsquos what teams needmdashand what TechExcel delivers in DevTest one of the three toolsin its DevSuite DevTest offers your developers test standardi-zation test reuse and powerful defect analysis capabilities whilegiving managers a birdrsquos-eye view of the entire process

ldquoDevTest is an integrated solution that allows QA teams toimprove standardization and leverage existing testing knowl-edge to carefully monitor test executionrdquo says Tieren Zhou TechExcelrsquos CEO and chief software architect ldquoDevTest accomplish-es that goal by focusing knowledge management test librarycreation planning and scheduling and test execution and analy-sis I think our approach is truly unique to the marketrdquo

DevTest is a one-stop information resource for your teamstoring all related test documentation including requirementsdocuments specifications automation scripts screen shotsand other essential components in a central repository This

ldquoknowledge viewrdquo can beused in test creation andexecution so that the testteam is always equipped to plan and execute itsassignments

A test library is built onthis knowledge founda-tion letting your teamreuse tests for new testassignments new soft-ware versions or evenentirely different soft-ware products Thesestandard test pro-cedures called test templates can belinked to supporting

documentation in theknowledge view Testtemplates can be or-

ganized and classifiedbased on products applicable

environments functional areas or any otherstructure on which a team needs to focus

Once a test library has been created QA managers and teamleads use a wizard-driven interface to assign tasks to test teamshelping them leverage everything in the test library includingprevious test assignments and defect history to aid in the plan-

ning process The test team receives its test assignment in theDevTest interface executes the items assigned to it and sub-mits defects directly from the interface into an integrated defect-tracking tool Meanwhile DevTest tracks the test results in areal-time dashboard and in presentation-quality custom reports

In addition to DevTest TechExcel (wwwtechexcelcom) offerstwo other products in the DevSuite DevTrack and DevPlan

DevTrack tracks andmanages product defectschange requests and oth-er issues facilitating team-work among users teams

and customers DevTrack also provides workflow and processautomation robust searching and reporting and point-and-click customization

The newest member of the DevSuite DevPlan is an inno-vative project-tracking tool designed exclusively for applicationlife-cycle management DevPlan unites project tracking andissue management and incorporates configurable workflowsnotifications meeting requests and process automation

Zhou explains that in order to be effective a QA organiza-tion needs both preparation and education and that TechExcelrsquostools can help

ldquoDevTest provides all of the materials needed to create andexecute test assignments in one location so that leads and man-agers have the information they need to craft effective testsand testers have the information they need to execute themrdquohe says ldquoDevTestrsquos planning wizards are a favorite amongtest managers because they allow them to query existing datato help plan their assignments and reduce guesswork In addi-tion the built-in integration with DevTrack allows testers toexecute their test assignments and log and regress defects froma single interfacerdquo

In the near future the DevSuite will support distributeddeployment allowing global development organizations toachieve the benefits of local performance even when workingon a global projectmdashstay tuned for more about this excitingdevelopment

DevTest can help both small and large organizations get ahandle on quality assurance ldquoDevTest is useful in any develop-ment environmentrdquo Zhou says ldquoItrsquos well suited for large glob-al development organizations because it provides them with ascalable real-time view of their test projects regardless of whetherthe tests are executed by their core team by an outsourced teamor by auto-testing tools However even smaller developmentteams benefit from the ability to create manage analyze andreuse their test coveragerdquo ampamp

TechExcel Takes the Guesswork Out of Quality Assurance

a p p l i c a t i o n l i f e c y c l e m a n a g e m e n t

gold sponsor

Tieren Zhou

CEO and Chief Software Architect

1 october 2006 the artampamp science of software testing u 15

The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking

OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time

Available for Windows Web amp VSNET 20032005

800middot653middot0024

wwwaxosoftcomso f tware fo r so f tware deve lopment trade

Ship Software OnTimetrade

Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations

OnTime 2006 Small Team Edition

bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members

OnTime 2006 Professional Edition

bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members

bull For Teams of 10 to 1000 Membersbull From $149 Per User

SDTimesAd_OnTime2006indd 1 62706 14151 PM

The Fast amp Scalable Team Solution forDefect amp Issue Tracking bull Feature amp Change Tracking bull Task amp To-do List Tracking bull Helpdesk Ticket Tracking

OnTime is the market-leading project defect and feature management tool for agile software development and test teamsOnTime facilitates tracking analyzing and trending team-based software development efforts in an intuitive and powerful user interface A fully customizable UI powerful workflow process enforcements two-way email communications and custom reports combine to help software development teams ship software on-time

Available for Windows Web amp VSNET 20032005

800middot653middot0024

wwwaxosoftcomso f tware fo r so f tware deve lopment trade

Ship Software OnTimetrade

Only $495 for up to 5 Users bull Only $995 for up to 10 UsersFree Single-User Installations

OnTime 2006 Small Team Edition

bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members

OnTime 2006 Professional Edition

bull For Teams up to 10 Membersbull Free Single-User Installationsbull $495 for 5-Team Membersbull $995 for 10-Team Members

bull For Teams of 10 to 1000 Membersbull From $149 Per User

SDTimesAd_OnTime2006indd 1 62706 14151 PM

Managing the entire scope of one or more software devel-opment projects can be daunting Within the sea of

tasks features defects and test cycles among team mem-bers managers stakeholders and customers just figuringout whorsquos working on what can be a challenge Thatrsquos where

Axosoftrsquos OnTime 2006enters the picture

OnTime 2006 is adefect management sys-tem designed to help soft-ware development teamsship software on timeWith a focus on ship-ping software on timeit not only effectivelytracks and managesbugs it effectivelyaddresses the broad-er challenges andbest practices ofproject manage-ment

OnTime 2006offers maximum

flexibility for adminis-trators and users alikewith ready access from

a Windows client a Webbrowser or within Visual Studio The highly

configurable and customizable application can also be inte-grated with leading SCM packages including Perforce SCMSourceGear Vault Subversion and Visual SourceSafe

In addition to tracking defects features and tasks thesoftware governs projects with highly customizable workflowand security rules It also offers e-mail notifications and con-versation thread tracking time tracking and work loggingcustom fields and reporting audit trails and archiving

While many of Axosoftrsquos clients switch to OnTime 2006from other products utilizing its importing functionalityothers seek to upgrade from a manual process

ldquoTraditionally development tasks requests defects andother items that occur over the course of the developmentcycle have been tracked in spreadsheets providing ampleopportunity for human error and miscommunicationbetween teamsrdquo says Dan Suceava chief software architectfor Axosoft (wwwaxosoftcom)

ldquoOnTime 2006 keeps track of everything and prevents

important items from slipping through the cracksrdquo heexplains ldquoInstead of wasting developer time with trivialupdates and report requests managers can easily pull theinformation themselves OnTime 2006 frees developers frommost if not all of the process-related overhead that sur-rounds a project and it helps them focus on what they dobest building great softwarerdquo

Developers who use OnTime 2006 see a clear intuitiveview of all of their projects including all the issues defectsfeature requests milestones and tasks As they completetasks developers escalate those items to subsequent work-flow steps This can be set to automatically e-mail otherteam members when the next set of actions is ready to beperformed

OnTime 2006 allows project managers to define speci-fications workflows and security rules They can create ldquowhatifrdquo scenarios that generate predictions for completion timesworkload distribution and other milestones During the pro-jectrsquos execution project managers have access to total proj-ect visibility They know who is working on what the progressbeing made on tasks where bottlenecks are occurring defectrates and estimated completion times

After a project has been completed managers can takeadvantage of the accurate project history that continues toreside within OnTime 2006 This information can be usedfor anything from measuring various productivity rates toproviding a basis for decisions concerning future projectsThus past projects become part of a living knowledge basethat can be consulted at any time

Axosoft even offers a Customer Portal add-on for OnTime2006 that embraces customers as participants in the processof shipping software on time It provides a Web interfacetied to the OnTime database where customers can submit

bugs and other inputCustomizable security set-tings determine how muchproject information will bevisible to customers This

functionality is especially useful for consultants and ISVsduring beta-testing phases

More functionality is in the works for the productldquoOnTime 2006 is already a tool designed to enable the entiredevelopment team to ship software on timerdquo explainsSuceava ldquoWhile today it meets the needs of project man-agers developers and testers extremely well future versionswill provide further functionality for support professionalsIT directors and executivesrdquo ampamp

Axosoft Keeps DevelopmentProjects on the Fast Track

1 october 2006 the artampamp science of software testing u 17

d e f e c t t r a c k i n g

gold sponsor

Dan Suceava

Chief Software Architect

Software configuration management isnrsquot a luxurymdashforany modern software development team itrsquos a necessi-

ty But when an SCM system is as easy to use easy to admin-ister and full of productivity-enhancing features and bene-fits as Perforcersquos system itrsquoll feel like all of your developersare traveling in first class all the way

ldquoThe Perforce SCM System lets teams of local and dis-tributed developers share project files that are centrally storedand managed by the Perforce Serverrdquo says John WalkerPerforcersquos principal product consultant ldquoThe Perforce Serverhandles user requests and tracks all development activity inthe built-in Perforce database Each filersquos state informationcan be quickly discerned from any of Perforcersquos cross-platform clients This means that a user can see when a ver-sion of a particular file has been updated deleted or addedto the server Icons associated with the files indicate whethertheyrsquore currently being edited added or deleted by otherdevelopersrdquo

The Perforce SCM System(wwwperforcecom) letseach developer on your teamobtain and refresh a localprivate copy of versionedfiles by synchronizing themwith files stored in a filedepot Since all of themetadata is centralizedfile state information canbe gathered quickly andeasily Perforce efficient-ly manages both bina-ry and text files fromsource code to graph-ics to documentationThatrsquos one reasonwhy Perforce has

been adopted not onlyby pure developmentshops but also by

chip and hardwaremanufacturers that maintain

large binary assets For example Walker says the Perforce System is very pop-

ular in the game development industry where artists arerequired to create and store large numbers of large imagefiles To help all usersmdashnot just software developersmdashthePerforce software automatically renders thumbnail versions

of image files that can be viewed from within the applica-tionrsquos cross-platform graphical client P4V Regardless ofwhether yoursquore working with text or binary artifacts Perforcehandles the job with style

Perforcersquos clientserverapplication operates overany network or the In-ternet and includes itsown internal database

enginemdashsaving you money improving performance and get-ting the system up and running fast Installing the PerforceServer is simple Download the installation set from the com-panyrsquos Web site and run it Installing any one of a varietyof Perforce clients is also straightforward and the resourcesneeded to manage the installation are minimal Even largedistributed sites of 2000 or more users can be maintainedby a single administrator explains Walker

While Perforce has the basics down pat the real payoff isin the extras that the softwarersquos creators developed to makeyour own coders more efficient Perforcersquos intelligent branch-ing mechanism lets developers work on different releasebranches of a particular application in parallel Code linessupporting specific releases are clearly and visually definedin the system Once a branch is created the relationship itshares with the parent branch is tracked in the serverrsquos meta-data Since the ancestry of files is tracked between branch-es the integration history is maintained in the metadataThat makes it easy to see which changes have and havenrsquotbeen integrated between the related branchesmdashwhether yoursquoreworking in Java J2EE NET CC++ C Visual Basic orHTML

Perforce provides a basic defect-tracking system called jobsA job typically represents an enhancement request or a bugto be fixed Job definitions are customizable to support work-flow and jobs can also work with leading third-party defecttracking systems Support for Mercuryrsquos Quality Center isplanned for the next release This integration will allow usersto enter bugs in Quality Center and have them replicatedinto Perforce as jobs

ldquoThe Perforce SCM Systemrsquos high performance is the prod-uct of a streamlined architecture and closely integrated imple-mentationmdashnot expensive server or network hardwarerdquoWalker says ldquoThe Perforce Server does not require dedicat-ed hardware and client workstations never need upgradingThe systemrsquos networking capabilities arenrsquot a significant loadfor a typical LAN With little need for customization or con-figuration you can be up and running in minutesrdquo ampamp

Perforce Makes SCM FastEasy and First-Class

s o f t w a r e c o n f i g u r a t i o n m a n a g e m e n t

gold sponsor

John Walker

Principal Product Consultant

1 october 2006 the artampamp science of software testing u 19

The faster you find a security problem in the application lifecycle the better If developers catch vulnerabilities early they

can be fixed prior to pushing apps off to QA and customersFixing problems after they slip into a product can cost far morethan catching them early And of course if you donrsquot find the

defects someone else willldquoCenzic solutions provide

tremendous efficiencies andan immediate return on investmentrdquo says JohnWeinschenk Cenzicrsquospresident and CEO (wwwcenziccom) ldquoBy auto-mating the security test-ing process our cus-tomers can secure theirapplications faster andless expensively Likeperformance andfunctionality testingsecurity testingshould be automat-ed Vulnerabilitiesare an open invita-

tion to hackers AllWeb applications andinfrastructure can be

tested with Cenzic solutionsto locate security problemsrdquo

Cenzicrsquos two key security products are Cenzic Hailstorm andCenzic ClickToSecure These solutions provide the most com-prehensive and accurate tests in the industry Weinschenk saysthat while most vendors use scanning technology to focus onlyon commonly known vulnerabilities Hailstorm uses Cenzicrsquospatent-pending Stateful Assessment technology to automatepenetration testing by using a series of transactions to identifyvulnerabilities

Plus Hailstormrsquos SmartAttack Objects Library takes thesoftware beyond merely finding security holesmdashit can also helpenforce internal policies as well as bring organizations intoregulatory compliance with rules like the Gramm-Leach-BlileyAct (GLBA) SB1386 and Sarbanes-Oxley and the PaymentCard Industry (PCI) Data Security Standard and with bestpractices including SANS and OWASP in addition to manyother regulatory standards

ClickToSecure is a service that allows developers to make useof Software as a Service (SaaS) Tests are conducted remotely by

Cenzicrsquos own security expertsmdashand that means that your devel-opment team can build secure software even if it doesnrsquot haveexpertise in that area ClickToSecure accesses applicationsusing a combination of Hailstorm technology and the CenzicIntelligent Analysis (CIA) Research Lab to run assessments

In fact that CIA Lab is the foundation of the SmartAttackObjects Library Cenzic provides continuous updates based onnew vulnerabilities through the lab similar to an anti-virusmodel to help you stay ahead of the exploits and attacks Youcan use the library just as it is plus you can use CenzicrsquosSmartAttack Objects as templates that your own security archi-tects can customize for your organizationrsquos special require-ments if necessary

In addition to its software applications and services Cenzicoffers security training courses that give customers the knowl-edge and skills to use and maintain its products successfullyThe company also offers consulting services that include on-site assessment methodology and software-engineering andconsulting implementation with professional vulnerabilityconsultants who have experience in penetration testing andethical hacking

Among Cenzicrsquos many customers are Boston College DebtExchange IRIS Link and K2 Networks Boston Collegersquos devel-opment group uses Hailstorm in-house throughout its infor-mation security group to test all university Web applicationsAs they find security weaknesses developers use Hailstormreports to remediate those vulnerabilities The same group alsoaddresses its regulatory compliance issues with Hailstorm

In the case of Debt Exchange IRIS Link and K2 NetworksClickToSecure is put to work to perform security assessmentsCustomers call or fax requests to test their applications andCenzic experts test them remotely using Hailstorm in collabo-ration with the CIA Labrsquos expertise Detailed results are then

presented to the customeralong with detailed remedi-ation information

Weinschenk says thatCenzic solutions are a perfect fit for development managersand other executives Using a dashboard customers can viewapplications in the testing phase as well as the number andtypes of vulnerabilities that are found

Weinschenk adds that testing is extremely importantbecause susceptibility to intrusion can result in major recoverycosts and regulatory penalties He also reports that Cenzic hasnt tested a single application that was not vulnerable Butdonrsquot worrymdashCenzic finds the vulnerabilities before the badguys do Put Cenzic to work for you ampamp

Keeping the Bad Guys At Bay With Cenzic Solutions

1 october 2006 the artampamp science of software testing u 21

w e b s e c u r i t y

John Weinschenk

President and Chief Executive Officer

gold sponsor

Donrsquot MissOut

On Another Issue of The

Test amp QAReport

e-newsletter

Each FREE weekly issue includes original articles that interview top thought leaders in software testingand quality trends best practices

and testQA methodologiesGet must-read articles that appear

only in this e-newsletter

Sign up atwwwstpmagcomtqa

Many tools exist that help QA departments and programmerstest applications However itrsquos also crucial that senior-

level software managers have the necessary information to inter-act with executive management and accurately assess the effec-tiveness of their development teams ldquoIf addressing software qual-ity early in the development cycle is an important priority for yourcompany Stelligent can helprdquo says Burke Cox Stelligentrsquos CEO

ldquoAs the thought leaders in early testing we use commercialand open-source technologies to assess and manage softwarequality during the development and assembly of applicationsrdquoCox says ldquoThe Quality Risk Index provides an objective meas-ure of software quality that enables an entire organization toassess quality and progressrdquo

Cox believes that Stelligent (wwwstelligentcom) transformsthe way software is developed and tested By introducing com-prehensive inspection as part of a continuous process both thedevelopment team and senior management gain crucial visibil-ity into the quality risks associated with software projects This

real-time feedback lets organ-izations manage quality longbefore products are everdelivered to the QA team forevaluation

Stelligentrsquos services suchas its Kickstart Quality RiskAssessment are used byorganizations developingsoftware using managedlanguages like C orJava providing bothstatic and dynamicanalysis on the appli-cation source andcompiled code

How effective isthe Kickstart Quality

Risk Assessment Wellone customer recentlyacquired a software

product that it believedwas of high quality based on a

due-diligence assessment of the source code andend user experience However when trying to integrate the prod-uct into its larger solution it found the product had poor toler-ance to change

Thatrsquos when the customer called Stelligent whose KickstartQuality Risk Assessment identified the problems Then its

Quality Roadmap and Seven-Point Implementation Plan identified top priorities for making programmatic changes through refactoring and introduced developer testing as an active part of the continuous integration process The customer

found that Stelligentrsquos QualityRoadmap quickly solved theproblem and then its Con-tinuous Quality program kept

them solved Case closedldquoFor many organizations our Quality Risk Index is the first

objective measure of their software quality that they have everseenrdquo Cox says ldquoThis helps the software manager by providingan independent assessment of what their quality practices cur-rently achieve When working with the executive managementteam in developing schedules approving budgets and other prod-uct management activities the software manager uses Stelligentreports as a basis for discussion For example high code com-plexity with poor test coverage might make a compelling argu-ment to delay release as opposed to simply stating that the prod-uct is just not ready for prime timerdquo

Stelligentrsquos services provide a critical and objective meas-urement of your softwarersquos quality Use this intelligence tomake it difficult to enter new defects into the source repos-itory helping your developers leverage the continuous qual-ity feedback before code even reaches the QA department

Stelligent is a subsidiary of JNetDirect which makesConvergence a solution for providing real-time visibility intosoftware quality CoView which helps your team develop accu-rate and effective JUnit tests and also high-performance JDBCdrivers But long before JNetDirect acquired Stelligent it was acustomer using the Kickstart Quality Risk Assessment servicesto help build its own products Stelligentrsquos impact was so dra-matic that JNetDirect immediately saw the value of bringingStelligentrsquos services to a broader audience

ldquoDefects will always enter the build systemrdquo Cox says ldquoOurbusiness is ensuring that they cannot live there for very longQuality may not have a material impact on initial product rev-enue but it is the most significant driver of product profitabil-ity Most companies struggle with describing quality it is mixedwith subjective measures and anecdotal evidence Our customershave a quality score they can point to when setting goals andbenchmarks For organizations interested in increasing earningslowering the costs associated with poor product quality is thebest place to startrdquo

Stelligentrsquos services start with the Kickstart Quality RiskAssessmentmdashask the company how its objective measures canhelp you too improve your software quality ampamp

Stelligent Brings ObjectivityTo Quality Measurement

t e s t i n g s e r v i c e s

Burke Cox

Chief Executive Officer

1 october 2006 the artampamp science of software testing u 23

silver sponsor