45
1
1
OOutlineutline
Introducing BDD BDD principles BDD tools
What is Concordion? Why Concordion? Concordion in use Getting along with Maven, Ant and Eclipse
BDD with Concordion Technique Common Smells
Resources
2
Introducing BDDIntroducing BDD
3
What is BDD?What is BDD?
BDD: Stands for behavior-driven development.
Was originally conceived in 2003 by Dan North as a response to TDD.
Is an evolution in the thinking behind Test-Driven Development and Acceptance Test-Driven Planning.
Aims to help focus development on the delivery of prioritized, verifiable business value by providing a common vocabulary (also referred to as a Ubiquitous Language) that spans the divide between Business and Technology.
Relies on the use of a very specific (and small) vocabulary to minimize miscommunication and to ensure that everyone – the business, developers, testers, analysts and managers – are not only on the same page but using the same words.
4
BDD principlesBDD principles
Test method names should be sentencesIt can do at least some of your documentation for you, so start to write testmethods that are real sentences. What’s more, when you write the methodname in the language of the business domain, the generated documents makesense to business users, analysts, and testers. A simple sentence template keeps test methods focusedStart test method names with the word “should.” This sentence template – Theclass should do something – means you can only define a test for the currentclass. This keeps you focused. If you find yourself writing a test whose namedoesn’t fit this template, it suggests the behavior may belong elsewhere. If aclass is doing more than one thing, take it as an indication that you shouldintroduce other classes to do some of the work.
BDD principles BDD principles (continued)(continued)
An expressive test name is helpful when a test failsIf you’re changing code and cause a test to fail, you can look at the test method nameand identify the intended behavior of the code. Typically one of three things happen:
You had introduced a bug. Solution: Fix the bug. The intended behavior is still relevant but have moved elsewhere. Solution:
Move the test and maybe change it. The behavior is no longer correct – the premise of the system have changed.
Solution: Delete the test.The latter is likely to happen on agile projects as your understanding evolves.Unfortunately, novice TDDers have an innate fear of deleting tests, as though itsomehow reduces the quality of their code.
A more subtle aspect of the word should becomes apparent when compared with themore formal alternatives of will or shall. Should implicitly allows you to challenge thepremise of the test: “Should it? Really?” This makes it easier to decide whether a
testis failing due to a bug you have introduced or simply because your previous
assumptionsabout the system’s behavior are now incorrect.
BDD principles BDD principles (continued)(continued)
“Behavior” is a more useful word than “test”BDD gives answers to some of the most vague TDD questions:
What to call your test is easy – it’s a sentence describing the next behavior in which you are interested.
How much to test becomes moot – you can only describe so much behavior in a single sentence.
When a test fails, simply work through the process described above – either you introduced a bug, the behavior moved, or the test is no longer relevant.
BDD principles BDD principles (continued)(continued)
Determine the next most important behavior
A really useful way to stay focused is to ask: What’s the next most important
thing the system doesn’t do?
This question requires you to identify the value of the features you haven’t yet
implemented and to prioritize them. It also helps you formulate the behavior
method name: The system doesn’t do X (where X is some meaningful
behavior), and X is important, which means it should do X; so your next
behavior method is simply:
public void shouldDoX() { // ... }
Now we have an answer to another TDD question, namely where to start.
BDD principles BDD principles (continued)(continued)
Requirements are behavior, too BDD provides a “ubiquitous language” for analysis
BDD ToolsBDD Tools
10
BDD Tools BDD Tools (continued)(continued)
• ASSpec - ActionScript 3• BDoc - Extracting documentation from unit
tests, supporting behaviour driven development
• BDD in Python is core module doctest• Bumblebee - Extract documentation from JUnit
tests with support for adding text, code-snippets, screenshots and more. Puts focus on the end-user.
• beanSpec - Java• cfSpec - ColdFusion• CSpec - C• dSpec - Delphi• Concordion - a Java automated testing tool for
BDD that uses plain English to describe behaviors.
• Cucumber - Plain text + Ruby. Works against Java, .NET, Ruby, Flex or any web application via Watir or Selenium.
• easyb - Groovy/Java• EasySpec - Groovy, usable in Java. Developer
also working on Perception a tool for doing Context/Specification reporting for many different tools.
• GSpec - Groovy
• Instinct - Java• JavaStubs - Java - BDD framework supporting
partial-mocking/method stubbing
• JBehave - Java• JDave - Java• JFXtras Test - JavaFX• JSpec - JavaScript• JSSpec - JavaScript• NBehave - .Net• NSpec - .Net• NSpecify - .Net• NUnit - Another implementation of BDD
framework in .Net with focus on specification testing
• PHPSpec - PHP• Pyccuracy - Behavior-driven framework in
Python.• Pyhistorian - General purpose BDD Story
Runner in Python (internal DSL, not plain-text)• RSpec - Ruby• ScalaTest - Scala• specs - Scala• spec-cpp - C++• Specter - Another implementation of BDD
framework in .Net with focus on specification readability
• StoryQ - .Net 3.5, can be integrated with NUnit to provide both specification readability and testing
• tspec - Groovy (Thai syntax)
12
What is Concordion?What is Concordion?
Key FeaturesKey Features
Concordion is an open source tool for writing automated acceptance tests in Java
Powerful, yet simple to use Concordion integrates directly with JUnit.
Highly readable tests Concordion acceptance tests are so readable they can
double up as system documentation. And, since the tests are linked to the system, you know the documentation is always up-to-date.
Separates tests from implementation Tests that include a lot of implementation detail lock you
into that implementation. Concordion helps you to document the logic and behavior of your system in a way that does not lock you in.
What's special about it?What's special about it?
Plain English specifications Rather than forcing product owners to specify requirements in a
specially structured language, Concordion lets you write them in plain English using paragraphs, tables and proper punctuation. This makes the specifications much more natural to read and write, and helps everyone to understand and agree about what a feature is supposed to do.
Always bang up-to-date Concordion specifications are active. Behind the scenes, they are
linked to the system under test and therefore do not go out-of-date. If a change is made to the system's behavior then the tests associated with the relevant specification will fail and let you know.
Complex behaviors can be decomposed Each question at the bottom of the specification links to another active
specification and so on until you have either run out of questions or the answer is "Out of Scope". In this way, a complex behavior can be broken down into small, focused and easily understood requirements.
Concordion in useConcordion in use
Specifications are written in simple HTML. Developers instrument the concrete examples in each specification with commands (e.g. "set", "execute", "assertEquals") that allow the examples to be checked against a real-life system.
The instrumentation is invisible to a browser, but is processed by a Java fixture class that accompanies the specification and acts as a buffer between the specification and the system under test. The fixture is also a JUnit test case, so it's easy to run and integrate into an automated build. The results of running the examples are exported with green and red indicating successes and failures.
Some example instrumentation:<p> When <span concordion:set="#firstName">Bob</span> logs in a greeting <span concordion:assertEquals="greetingFor(#firstName)">Hello
Bob!</span> should be displayed.</p>
Concordion commandsConcordion commands
A Concordion active specification consists of two parts: 1. a well-formed XHTML document describing the functionality2. fixture code written in Java that finds concrete examples in
the document and uses them to verify the system under test. Both files must be in the same package.
In order for the magic to happen, the document must first be instrumented with commands.
Concordion commands are specified as attributes on elements in the XHTML document. Web browsers ignore attributes that they don't understand, so these commands are effectively invisible.
The commands use a "concordion" namespace defined at the top of each document as follows:
<html xmlns:concordion="http://www.concordion.org/2007/concordion">
concordion:assertEqualsconcordion:assertEquals
<html xmlns:concordion="http://www.concordion.org/2007/concordion"> <body> <p concordion:assertEquals="getGreeting()">Hello World!</p> </body></html>
package example;
import org.concordion.integration.junit4.ConcordionRunner;
import org.junit.runner.RunWith;
@RunWith(ConcordionRunner.class)
public class HelloWorldTest {
public String getGreeting() {
return "Hello World!";
}
}
By default, Concordion outputs to the directory specified by the systemproperty java.io.tmpdir.
concordion:setconcordion:set
<html xmlns:concordion="http://www.concordion.org/2007/concordion">
<body>
<p>
The greeting for user <span concordion:set="#firstName">Bob</span>
will be:
<span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span>
</p>
</body>
</html>
package example;
import org.concordion.integration.junit4.ConcordionRunner;
import org.junit.runner.RunWith;
@RunWith(ConcordionRunner.class)
public class HelloWorldTest {
public String greetingFor(String firstName) {
return "Hello " + firstName + "!";
}
}
concordion:executeconcordion:execute
The execute command has three main uses:1. Executing an instruction with a "void" result.2. Executing an instruction with an object result (to allow multiple
properties of the object to be checked).3. Handling unusual sentence structures.
concordion:executeconcordion:execute. Part 1. Part 1
Executing an instruction with a void result It can occasionally be useful to execute an instruction that sets up some
system state. Every time you do this, however, alarm bells should ring in your head and you should question yourself to make sure that you are not inadvertently writing a script instead of a specification. E.g. a call to clearDatabase() would be a blatant misuse.
As a rule of thumb, methods with a void result called from an execute should start with the word set or setUp. E.g. setUpUser(#username).
<html xmlns:concordion="http://www.concordion.org/2007/concordion">
<body>
<p>
If the time is
<span concordion:execute="setCurrentTime(#TEXT)">09:00AM</span>
then the greeting will say:
<span concordion:assertEquals="getGreeting()">Good Morning World!</span>
</p>
</body>
</html>
#TEXT special variable contains the text of the current element.
concordion:executeconcordion:execute. Part 2. Part 2
Executing an instruction with an object resultSometimes you need to check more than one result of a behavior. For example, here we
want to
check that both the first name and the last name are correctly extracted from the full name:
<html xmlns:concordion="http://www.concordion.org/2007/concordion">
<body>
<div class="example">
<p>
The full name
<span concordion:execute="#result = split(#TEXT)">John Smith</span>
will be broken into first name
<span concordion:assertEquals="#result.firstName">John</span>
and last name
<span concordion:assertEquals="#result.lastName">Smith</span>.
</p>
</div>
</body>
</html>
concordion:executeconcordion:execute. Part 2. Part 2 (continued)(continued)
package example;
import org.concordion.integration.junit4.ConcordionRunner;
import org.junit.runner.RunWith;
@RunWith(ConcordionRunner.class)
public class SplittingNamesTest {
public Result split(String fullName) {
Result result = new Result();
String[] words = fullName.split(" ");
result.firstName = words[0];
result.lastName = words[1];
return result;
}
class Result {
public String firstName;
public String lastName;
}
}
concordion:executeconcordion:execute. Part 3. Part 3
Handling unusual sentence structuresOne of the great things about Concordion is that when you're writing the specifications you do nothave to worry about how you're going to instrument it. You can just concentrate on making thedocument as readable as possible.
<p> Upon login, the greeting for user <span concordion:set="#firstName">Bob</span> will be: <span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span></p>
vs<p concordion:execute="#greeting = greetingFor(#firstName)"> The greeting "<span concordion:assertEquals="#greeting">Hello Bob!</span>" should be given to user <span concordion:set="#firstName">Bob</span> when he logs in.</p>
The execute command is designed to process commands on its child elements in aspecial order. First of all it processes any child set commands then it runs its owncommand, then any child execute commands and finally any child assertEqualscommands.
concordion:execute on a <table>concordion:execute on a <table>
When you want to show several examples of a behavior, repeating the same sentence structure overand over again probably isn't going to be very nice to read. It would be better to use a table.
<table concordion:execute="#result = split(#fullName)"> <tr> <th concordion:set="#fullName">Full Name</th> <th concordion:assertEquals="#result.firstName">First Name</th> <th concordion:assertEquals="#result.lastName">Last Name</th> </tr> <tr> <td>John Smith</td> <td>John</td> <td>Smith</td> </tr> <tr> <td>David Peterson</td> <td>David</td> <td>Peterson</td> </tr></table>
concordion:verifyRowsconcordion:verifyRows
Sometimes you want to check the contents of a collection of results returned from the system. In theFit Framework you might use a RowFixture. In Concordion, you use the verifyRows command.
<table concordion:execute="setUpUser(#username)"> <tr><th concordion:set="#username">Username</th></tr> <tr><td>john.lennon</td></tr> <tr><td>ringo.starr</td></tr> <tr><td>george.harrison</td></tr> <tr><td>paul.mccartney</td></tr></table>
<p>Searching for "<b concordion:set="#searchString">arr</b>" will return:</p>
<table concordion:verifyRows="#username : getSearchResultsFor(#searchString)"> <tr><th concordion:assertEquals="#username">Matching Usernames</th></tr> <tr><td>george.harrison</td></tr> <tr><td>ringo.starr</td></tr></table>
The syntax for a verifyRows command is: #loopVar : expressionWhere expression returns an Iterable object with a predictable iteration order, (e.g. a List,
LinkedHashSet or aTreeSet). And #loopVar provides access to the current object during iteration and allows the
assertEquals methodto check its value.
concordion:verifyRowsconcordion:verifyRows (continued)(continued)
The order of the items in the table being verified must match the iteration order of the items returned by theexpression. You may need to sort the items to ensure they are in a known and consistent order. In our
example, weare using alphabetical order ("george" before "ringo").
@RunWith(ConcordionRunner.class)public class PartialMatchesTest {
private Set<String> usernamesInSystem = new HashSet<String>();
public void setUpUser(String username) { usernamesInSystem.add(username); }
public Iterable<String> getSearchResultsFor(String searchString) { SortedSet<String> matches = new TreeSet<String>(); for (String username : usernamesInSystem) { if (username.contains(searchString)) { matches.add(username); } } return matches; }}
Getting along with Maven, Ant and EclipseGetting along with Maven, Ant and Eclipse
The simplest configuration to run Concordion tests with maven.
<build> <plugins> <plugin> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.5</source> <target>1.5</target> </configuration> </plugin> <plugin> <artifactId>maven-surefire-plugin</artifactId> <configuration> <systemProperties> <property> <name>concordion.output.dir</name> <value>target/concordion</value> </property> </systemProperties> </configuration> </plugin> </plugins></build>
Getting along with Maven, Ant and Eclipse Getting along with Maven, Ant and Eclipse (continued)(continued)
The simplest configuration to run Concordion tests with Ant.Since Concordion is built on top of JUnit it takes no additional effort to run Concordion tests with JUnit.
The same istrue for the Eclipse IDE as well as any IDE that supports JUnit.
<junit fork="yes" forkmode="once" printsummary="yes" haltonfailure="yes" showoutput="yes"> <jvmarg value="-Dconcordion.output.dir=build/concordion-output"/>
<classpath> <path refid="compile.classpath"/> <pathelement location="build/classes"/> </classpath>
<formatter type="plain" />
<batchtest todir="build/test-output"> <fileset dir="specs"> <include name="**/*Test.java"/> <exclude name="**/Abstract*"/> </fileset> </batchtest></junit>
BDD with ConcordionBDD with Concordion
29
TechniqueTechnique
Write specifications, not scriptsWrite specifications, not scripts
Scripts over-specify Test scripts are a list of instructions to be followed. For
example: Clear database Load database from "sample-data.sql" Start webserver Open URL: http://localhost:8080/myapp Enter username: admin Enter password: admin1 Click the "Login" button Click the "User Administration" link Click the "Create User" button: Enter name: John Smith Enter username: john Enter password: john99 Click the "OK" button Click the "Logout" link Enter username: john Enter password: john99 Click the "Login" button Check page contains text: Hello John!
Write specifications, not scriptsWrite specifications, not scripts (continued)(continued)
Concealed inside the script is a behavior that the test is trying to demonstrate. But because the requirement is not explicit, it's hard to know what it is (e.g. "Can we change this link to a button, or is the link part of the requirement?").
Scripts also tend to suffer badly from duplication. If we introduce an extra step into the authentication process then all the scripts that mention logging-in will need modification. This is exactly the kind of duplication that programming languages are designed to address and why plain English is not a good language for scripting.
Write specifications, not scriptsWrite specifications, not scripts (continued)(continued)
Specifications give you freedom Specifications tell you the requirements explicitly. They are
written at a higher level of abstraction to test scripts. For example:When John logs in, a greeting "Hello John!" is displayed.
This can be implemented in multiple ways. All of the details about how to test the requirement are hidden inside the fixture code where they can be refactored as the system evolves.
Specifications should be stableSpecifications should be stable
The specifications themselves should rarely change. Even on agile projects, though new behaviour is added frequently, existing behaviour is normally maintained from iteration to iteration.
Evolve a domain-specific languageEvolve a domain-specific language
As you write the fixture code, and refactor it to remove duplication,
you'll find you gradually build up a scripting API – a domain-specific
language (DSL) – that lets you manipulate the system under test.Eventually, the fixture code will become very stable too.
Isolate behavioursIsolate behaviours
Each active specification should have a narrow focus and cover a single behavior as independently as possible from other behaviors. For example, one specification might describe how text searches are case-insensitive, another will describe how date searches work, and another will describe how search results are to be presented (what data is displayed). The idea is to keep each specification very simple and to avoid overlaps, so that we can change the specification about the way search results are presented without having to make changes to the other specifications.
If you want to test a combination of behaviors, write a separate specification of the behavior for the combination. But always write the specifications for the individual behaviors first. When you combine behaviors there is a penalty in terms of complexity – of both the specification and the fixture code to support it.
Think "Given-When-Then"Think "Given-When-Then"
This is an excellent way of structuring the concrete examples in the specifications, and getting into the mindset of specifying instead of scripting.
Given (some context)When (something happens)then (some behavior)Example:Given a user called John;When John logs in,then a greeting is displayed saying "Hello John!".
Once you've written or thought about your behavior using the Given-When-Thentemplate, you can reword the sentence to make it less clunky. For example, in this
casewe can deduce that John is a user from the phrase "John logs in", so we can drop
the"Given" part and write something more readable like this:
When John logs in, a greeting "Hello John!" is displayed.
Common SmellsCommon Smells
Existing specifications are often changedExisting specifications are often changed
You're tied to an implementation If you find you're having to change the contents of
specifications/instrumentation, on a regular basis, this is a strong indicator that the specifications are too closely coupled to the implementation.
So, pretend there are multiple implementations The solution is to describe the underlying behavior in a more
abstract way. It might help to imagine that there are several different implementations - a web application, a Swing application, and a command-line executable, for example. Only describe behavior that is required so that you leave as many options open as possible for alternative implementations.
Lots of "execute" commandsLots of "execute" commands
You're writing a script If you have lots of "execute" commands you are probably
writing a script instead of a specification. So, hide the scripting in the fixture
Avoid describing the steps of how to test the behavior and simply state the context (as a sentence), and the behavior you expect to see. Don't explain how to get into that context, don't explain how to perform the operation and don't explain how to extract the results. All of these things are implementation-specific and should be hidden in the fixture code.
Complicated instrumentationComplicated instrumentation
You're testing too much in one go As a rule of thumb, well-written fixtures should have no more
than three public methods and no method should have more than one or two parameters. Complicated instrumentation is usually a sign of trying to test too many things at once.
So, decompose the behavior Focus on one tiny behavior at a time. Break the behavior into
smaller and smaller pieces until you cannot break it down any further. Then write separate specifications for each piece.
Complicated fixture codeComplicated fixture code
Your fixture code is verbose and hard to follow All the scripting should be done in the fixture code, but this can
make the code seem complicated. So, create classes to help with the scripting
Begin to create objects to help with the scripting activity - push the implementation details into them. Eventually you'll create a little domain specific language (DSL).
Examples all have the same structureExamples all have the same structure
Your examples are too generic If the examples in your specifications
look very similar (i.e. they have the same kind of context set-up, the same kind of checks etc.) this is a strong sign that the examples are too generic.
So, focus the examples more carefully The examples should demonstrate the
particular behavior you are describing and should not include any irrelevant details. Push everything you can into the fixture code.
The layout above is "one size fits all". It contains
lots of context that is not relevant to the
behavior we want to show.
Focusing the examples improves their clarity an
reduces duplication.
ResourcesResources
Useful linksUseful links
Dan North's article introducing BDD: http://dannorth.net/introducing-bdd Introduction to Behavior Driven Development: http://behavior-driven.org/ In pursuit of code quality: Adventures in behavior-driven development by
Andrew Glover: http://www.ibm.com/developerworks/java/library/j-cq09187/index.html
The RSpec Book: Behaviour Driven Development with RSpec, Cucumber, and Friends: http://www.pragprog.com/titles/achbd/the-rspec-book
Concordion site: http://www.concordion.org/ Concordion page at google code: http://
code.google.com/p/concordion/w/list
