Solve the Mystery ofSuccessful End-of-Project

Retrospectives

February 2007 $9.95 www.StickyMinds.com

The Print Companion to

THE MAGIC NUMBER

Three techniques for testing with databases

COMPLEX CODE?

Get back to basics

The Case of theMissing

Fingerprint

www.StickyMinds.com FEBRUARY 2007 BETTER SOFTWARE 39

The Last Word

A Box of a Different Colorby Matthew Heusser

A short time ago I had to prepare a testingcourse for software developers whowere responsible for both unit testing andsystem testing and programmed in a relatively unpopular programming languagebased loosely on Pascal. The information Ifound when searching for course materialspointed back to Java and C++, but thedevelopment staff with which I was work-ing did not know or use those languages.In fact, the best material I could findwas from a Cem Kaner course called“Black Box Software Testing,” or BBST(see the StickyNotes for a link).

I posted on a software discussionlist, actively askingpeople what theythought of usingBBST materials forsoftware develop-ers and solicitingrecommendationsfor open sourcematerials. The an-swers I receivedwere consistent:“No, no, no. Youcan’t teach acourse in black box testing; get somematerial on glass box testing,” and thenthey would recommend a course thatwas in Java or C++. Occasionally some-one would recommend a book or classand say something like, “I have heardgood things about this but haven’t readit”—and it would wind up being in Javaor C++. The existing material simplywould not be helpful to the develop-ment staff, and I would keep comingback to BBST.

I next asked why black box testingwas inappropriate. The typical responsewas “It’s black box, genius. That’s notthe right kind of testing.” Others wouldsay, “Well, black box testing is for peoplewho can’t see the code—it’s not reallyfor the developers.”

Both answers were unsatisfying tome. They were definitions of the term,not reasons to reject the course material.

right out of the training materials andthe developers already have the skills onthe left, then the differences betweenblack box and glass box testing (or thevalue of distinguishing the difference)greatly decreases.

If the difference between the two isnot valuable, then why are we splittingthem up? On this project, splitting hairsover the color of the box was actuallyharmful. It caused me to shut out poten-tially valuable learning opportunitiesthat were available—even free.

Yet the valuable things in the courseare not specific techniques (like jUnit orMockObjects) but are genuine skills thatcan be applied to many areas of our lives.We could learn those skills from manysources, which is one of the traditionalarguments for a liberal arts education.When it comes to evaluating courses,conferences, and books, instead of listingspecific techniques, we might be betteroff listing skills and asking if the coursewill teach those skills. In that case, “bugadvocacy” becomes influence and “howto write a good bug report” becomeswriting skills—both of which have uni-versal value to technology workers.

Once we have the skills, the tech-niques and terminology can help. After all,our community created test terminology asa servant. Our terminology is a patternlanguage that allows us to express complexideas quickly and unambiguously. I maygrouse and complain a bit about “oracles,”

One or two people mentioned that blackbox testing doesn’t include statement orpath coverage metrics, which is true. Butit turned out that this particular customerhad no requirements for coverage to bemeasured, and the BBST course actuallyoffered just enough information aboutcoverage and coverage metrics to meetmy needs.

So I looked at the basic skills that theblack box course covered: critical thinking,general systems thinking, identifyingpatterns of software failure, issue isolation,and investigation skills. These are skillsthat any tester might want to grow, and

I found a huge overlap, regardless ofbox color (see Figure 1).

There were areas of the BBST that Iwanted to skip. Bug advocacy and howto write a good bug report become lessimportant if the person who logs thebug is the one who is going to fix it, andbugs found in unit testing might not getlogged at all. Still, it seemed foolish tothrow out the entire course when I couldjust toss out a couple of chapters.

This got me thinking about what ittakes to be a good black box tester andwhat it takes to be a good glass box tester.

The developers who would take mycourse already have the skills on the left.Because equivalence classes work just aswell on functions as they do on GUIs,the developers could stand to benefitfrom any black box material that wouldcover the skills in the middle.

So, if I can screen things from the far

CodingAbstractionCoverageDesign Patterns

Bug AdvocacyTesting DocumentationIEEE 829Bug Triage and Metrics

Systems Thinking

Critical Thinking

Failure Patterns

Techniques such asBoundaries, Decision Trees,

Equivilance...

Glass Box Skills Black Box Skills

Figure 1

40 BETTER SOFTWARE FEBRUARY 2007 www.StickyMinds.com

Display Advertising

Shae Young syoung@sqe.com

All Other Inquiries

info@bettersoftware.com

Better Software (USPS: 019-578, ISSN: 1532-3579) ispublished twelve times per year. Subscription rate is US $75 per year. A US $35 shipping charge is incurred for all non-US addresses. Payments toSoftware Quality Engineering must be made in US funds drawn from a US bank. For more information, contact info@bettersoftware.com or call (800) 450-7854. Back issues may be purchased for $15 per issue (plus shipping). Volume discounts available.

Entire contents © 2007 by Software Quality Engineering (330 Corporate Way, Suite 300, OrangePark, FL 32073), unless otherwise noted on specificarticles. The opinions expressed within the articlesand contents herein do not necessarily expressthose of the publisher (Software Quality Engineering).All rights reserved. No material in this publicationmay be reproduced in any form without permission.Reprints of individual articles available. Call for details.

Periodicals Postage paid in Orange Park, FL, and other mailing offices. POSTMASTER: Send address changes to Better Software, 330 Corporate Way, Suite 300, Orange Park, FL 32073, info@bettersoftware.com.

Index to Advertisers

American Society for Quality www.asq.org 38

Better Software Conference & EXPO www.sqe.com/BetterSoftwareConf 21

Bredex www.bredexsw.com 13

iTKO LISA SOA Testing www.iTKO.com 15

Mercury Interactive Corporation www.mercury.com 5

Mercury Interactive Corporation www.mercury.com Back Cover

PNSQC www.pnsqc.org 10

Rally Software www.rallydev.com/bsm 37

Seapine Software www.seapine.com 1

Software Planner www.SoftwarePlanner.com 40

Software Quality Engineering Training www.sqe.com/training.asp 11

Software Security Summit www.s-3con.com 29

SPI Dynamics www.spidynamics.com Inside Back Cover

STAREAST 2007 www.sqe.com/stareast 2

StickyMinds.com PowerPass www.StickyMinds.com 36

TechExcel, Inc. www.techexcel.com Inside Front Cover

“heuristics,” and “context,” but when I use the terms, peopleknow what I mean. With this glass/black box discussion, wewere becoming servants of the terminology and got theessence confused with the accident (see the StickyNotes formore on essence and accident).

Speaking of liberal arts, when I was preparing for thecourse I asked my wife what books she would recommend toteach systems thinking. Her response was simply “Aristotle.”

What color box is that? {end}

Matthew Heusser (matt@xndev.com) actively develops workingsoftware, writes and speaks about systems improvement, and is a cofounder of the Great LakesSoftware Excellence Conference (www.glsec.org).Read his blog at xndev.blogspot.com.

StickyNotes

For more on the following topics, go towww.StickyMinds.com/bettersoftware.

� Link to Cem Kaner’s BBST course� Essence and accident� Further reading