Date post: | 17-Jul-2015 |
Category: |
Documents |
Upload: | giovanni-scerra |
View: | 63 times |
Download: | 0 times |
GettingStarted
WithTesting
Topics:
� Bugs
� Tests
� Unit Tests
� Testability
� Isolation
� Common Mistakes
ABug’sLife
1 X
500 X
DEV PRODUCTION QA DEV TEST LAB
The cost of a bug can increase 500 times from
development to the production release.
DESIGN
0.01
X
5 X
50 X
IneffectivePrevention
� We should code fewer bugs
� We need a tool that finds all bugs
� We should hire more QA people
� We should have (manual) test plans
� We should refactor (without tests)
� There is nothing to do about it
Oops…
EffectivePrevention
Good Design (complexity & coupling)
High-Level technical documentation
Code Reviews – Fagan, Over The Shoulder, Remote
Pair Programming
Use logs, error logs, e-mail notification, etc.
Write Tests (especially Unit Tests)
BenefitsofwritingUnitTests
1. Bugs are found (much) earlier.
2. Safer changes/refactoring (regression).
3. Up-to-date technical documentation and
examples.
4. Better low level architecture.
Idonotwritetestsbecause…
� I do not know how.
� It is not my job. The QA people are paid to do that.
� It is not my job. Let the customer figure what’s wrong.
� I work with untestable legacy code.
� I only develop UI.
� It makes it difficult to refactor the code.
� Requirements are not clear.
� I can test everything manually.
� There is no time for that.
AnatomyofaUnitTest
Set up the needed
dependencies/para
Select a story (class
method + scenario)
Instantiate the class
Exercise/Stimulus
Run the method
Assert the expected
behavior
[TestClass]
public class CalculatorTest {
[TestMethod]
public void ShouldAddTwoPositiveIntegerNumbers() {
int addendum1 = 1;
int addendum2 = 2;
Calculator calc = new Calculator();
int sum = calc.Add(addendum1, addendum2);
Assert.AreEqual(sum, 3);
}
}
On-FieldMission:
TestingTheOrderProcessor
Your Mission, should you choose to accept
it, is to unit test ValidateOrderTotal.
INPUT:
A customer id
OUTPUT:
True is the customer order for next delivery exists and the total
amount is > 0. False otherwise.
OrderProcessor processor = OrderProcessor.GetInstance();
bool isValidOrderTotal = processor
.ValidateOrderTotal("00000345-00000-000056");
MissionImpossible?
Mission log:
Attempt # Problem Solution
1 File Not Found Need configuration file
2 Invalid Database Set valid connection string
3 Invalid Ross Customer Id Set valid Ross Customer ID
4 Folder not found The export folder must exists and have
permissions
5 SMPT error Update SMPT configuration with valid server,
credential and set notification email
6 Order Processor Stuck Exception Delete marker file
7 Customer does not have order for next del.
date
Create order in DB for customer
8 What about the following delivery? …
… $#%k! …
… … …$#%k! $#%k!
[TestMethod]
public void CheckCustomerWithValidOrderForNextDeliveryDate() {
OrderProcessor processor = OrderProcessor.GetInstance();
bool IsValidOrderTotal = processor.ValidateOrderTotal("00000345-00000-000056");
Assert.IsTrue(IsValidOrderTotal, "...some error message..."); }
Bro!Seriously?!
There should be only one reason for a unit test to fail…
Do you have the configuration file?
What is the database connection
string?
Did you set up the export folder?
Did you set up a valid SMTP server?
Did you delete the marker file?
Will the test work next week? Did you set up the Ross Customer
ID correctly?
Does the customer key exist?
Does the customer have an order
for the next delivery?
Does the customer have an order
for the next delivery?
Did you set up a customer with a
valid order total?
Did you set up a customer with an
invalid order total?
Did you set up a customer with a
valid order total?
Did you set up the maximum
waiting time?
Corpus
Delicti
public class OrderProcessor {
private static OrderProcessor _instance = new OrderProcessor();
private bool _isMarkerFilePresent = false; private int _rossCustomerId = 0; private string _exportPath = string.Empty;
private OrderProcessor() { _exportPath = ConfigurationSettings.AppSettings["OrderExportPath"];
_isMarkerFilePresent = IOHelper.CheckMarkerFile(this._exportPath); _rossCustomerId = Convert.ToInt32(ConfigurationSettings.AppSettings["RossCustomerID"]);
int maxSecs = Convert.ToInt32(ConfigurationSettings.AppSettings["MaxSecsToWait"]); int secsWaited = 0; while (_isMarkerFilePresent) {
Thread.Sleep(1000); secsWaited += 1000; if (secsWaited > maxSecs) {
string email = DBService.GetRossCustomerEmail(_rossCustomerId); EmailManager.SendEmail(email, "Order Processor is stuck! Check the marker file!"); throw new Exception("Order Processor is stuck! Check the marker file!");
} }
IOUtility.CreateMarkerFile(_exportPath, "OrderProcessor.MRK"); }
public static OrderProcessor GetInstance() { return _instance; }
public void ExportOrder(string customerId) { … }
public bool ValidateOrderTotal(string customerId) { decimal orderTotal = 0; Order order = DBService.GetOrder(customerId, DateTime.Now);
if (order != null) { foreach (OrderDetail orderDetail in order.Details) { orderTotal += orderDetail.TotalAmount;
} }
return orderTotal>0; }
}
TheThreeFactors
Easy-to-write
meaningful
automated tests
Consistent
behavior across
multiple
deployments
Introspection
Testablevs.Untestable
Dependency Injection/Transparency
Interfaces/Factories/IOC
Law Of Demeter
Seams
Single Responsibility/Layered Architecture
Composition
Simple Constructors
Idempotence
Commutativity
Implicit Dependencies
New Operator
Service Locators
Static
Mixed Concerns
Deep Inheritance
Complex Constructors
Singletons, DateTime.Now/Random
Implicit Order (hints: Init, Pre, Post, Cleanup, Load)
ABetterPointofView
Writing tests requires the code to be
testable.
Refactoring code for testability
requires time, skills and experience.
Before writing code, answer this question:
How would you like to test it?
TestFirst
Tests First = testability + small, well-isolated components
Focus on the API first and implementation later
Where did the database go?
Where is the marker file?
What happened to the configuration?
[TestMethod, TestCategory("Order Validation")]
public void OrderWithTotalAmountGreaterThanZeroShouldBeValid() {
Order order = CreateFakeOrder(totalAmount: 10.0);
IOrderValidation orderValidation = new OrderValidation(order);
bool isValidOrderTotal = orderValidation.IsValidTotal();
Assert.IsTrue(isValidOrderTotal,
"Total order amount > 0 should be valid");
}
[TestMethod, TestCategory("Order Validation")]
public void OrderWithTotalAmountZeroShouldBeInvalid() {
Order order = CreateFakeOrder(totalAmount: 0.0);
IOrderValidation orderValidation = new OrderValidation(order);
bool isValidOrderTotal = orderValidation.IsValidTotal();
Assert.IsFalse(isValidOrderTotal,
"Total order amount = 0 should be invalid");
}
[TestMethod, TestCategory("Order Validation")]
public void EmptyOrderShouldBeInvalid() {
Order order = Order.Empty;
IOrderValidation orderValidation = new OrderValidation(order);
bool isValidOrderTotal = orderValidation.IsValidTotal();
Assert.IsFalse(isValidOrderTotal,
"Empty Order should be invalid");
}
public class OrderValidation : IOrderValidation {
Order _order = Order.Empty;
public OrderValidation(Order order) {
_order = order;
}
public bool IsValidTotal() {
decimal orderTotal = 0M;
if (!_order.IsEmpty()) {
foreach (OrderDetail orderDetail in _order.Details) {
orderTotal += orderDetail.DetailAmount;
}
}
return orderTotal > 0;
}
}
Controllingthe
TestEnvironment
A good unit test is like a good scientific
experiment: it isolates as many variables as
possible (these are called control variables)
and then validates or rejects a specific
hypothesis about what happens when the one variable (the
independent variable) changes.
Test Code System Under
Test (SUT)
Dep. 1
Dep. 2
Dep. N ISOLATED TEST
SEAMS
TestDoubles
Do you need a real plane?
� We do not have a real plane available yet
� It takes time and it is expensive to prepare a real
plane (checkup, gas, etc.)
� Too dangerous, the plane may crash
� If there is an issue it is hard to tell if the problem
was the pilot or the plane
Dummy
Dummy objects are passed around but never
actually used. Usually they are just used to fill
parameter lists.
public class DummyPlane implements IPlane {
public DummyPlane() {
// Real simple; nothing to initialize!
}
public int getAltitude() {
throw new RuntimeException("This should never be called!");
}
public int getSpeed() {
throw new RuntimeException("This should never be called!");
}
}
Stub/Spy
Stubs provide canned answers to calls
made during the test, usually not
responding at all to anything outside what's programmed in for
the test. Spies can record some information based on how
they were called
public class PlaneStub implements IPlane {
public PlaneStub () {
// Real simple; nothing to initialize!
}
public int getAltitude () {
log.Write(“getAltitude was called”);
return 300;
}
public int getSpeed () {
throw new RuntimeException("This should never be called!");
}
}
Fake
Fake objects actually have working
implementations, but usually take some shortcut which makes
them not suitable for production (an In Memory Database is a
good example).
public class FakePlane implements IPlane {
…
public FakePlane (FlightStatus status) {
_flightStatus = status;
}
public int getAltitude () {
return CalculateAltitude(_flightStatus);
}
public int getSpeed () {
return CalculateSpeed(_flightStatus);
}
}
Mock
Mocks are pre-programmed with expectations
which form a specification of the calls they are
expected to receive. They can throw an exception if they
receive a call they don't expect and are checked during
verification to ensure they got all the calls they were
expecting.
planeControl = MockControl.createControl(Plane.class);
planeMock = (IPlane) planeControl.getMock();
public void testPilotCheckAltitude() {
Pilot pilot = new Pilot();
planeMock.getAltitude();
planeMock.setReturnValue(300);
planeMock.replay();
pilot.Flight((IPlane) planeMock);
planeControl.verify();
}
Whatisit?
Microsoft Fakes is a code isolation framework that can help you isolate code for
testing by replacing other parts of the application with stubs or shims. It allows
you to test parts of your solution even if other parts of your app haven’t been
implemented or aren’t working yet.
Stubs
[TestMethod]
public void
StubAccountOpenPostsInitialBalanceCreditTransaction()
{
// Arrange:
int callCount = 0;
StubITransactionManager stubTM = new StubITransactionManager
{
PostTransactionDecimalTransactionType = (amount, type) =>
{
if (amount == 10m && type == TransactionType.Credit)
{
callCount++;
}
}
};
Account cut = new Account(stubTM);
// Act
cut.Open(10m);
// Assert
Assert.AreEqual<int>(1, callCount);
}
Shims(MonkeyPatching)
Shims are a feature of Microsoft Fakes that allows creating
unit tests for code that would otherwise not be testable in
isolation. In contrast to Stubs, Shims do not require the
code to be tested be designed in a specific way.
[TestMethod]
public void DateTimeTes()
{
using (ShimsContext.Create())
{
// Arrange:
System.Fakes.ShimDateTime.NowGet = () => { return new
DateTime(2016, 2, 29); };
// Act
int result = MyCode.DoSomethingSpecialOnALeapYear();
// Assert
Assert.AreEqual(100, result);
}
}
All-In-One
This is by far the most common mistake among beginners.
It comes in two flavors:
1. Unrelated features tested in the same test class/method
2. Single feature tested across multiple layers Consequences: when the test fails, it is hard to point out at
what has failed
LongParking
Tests are written correctly but they are executed rarely
and not maintained Consequences: obsolete tests do not help preventing
bugs. They could mislead developers who read them to
try to understand the program
Whereisthenegative?
Other common mistake, tests are checking only what
the application should do (positive cases) and not
what it should not do (negative cases). Edge cases
and expected exception are also typically ignored by
beginners. Consequences: poor coverage due to untested scenarios
BadNames
Tests methods have names that are too
generic or meaningless. E.g. “reportTest”
or “authenticationTest” Consequences: when the test fails, it is
not immediately obvious what has failed.
They could mislead developers who read
them to try to understand the program
DonotsuccumbtotheDarkSide
WriteUnitTests!
Further Reading
Guide: Writing Testable Code
Exploring The Continuum of Test Doubles
Better Unit Testing with Microsoft Fakes
Videos Introduction to Unit Testing in .NET
http://dotnet.dzone.com/articles/introduction-unit-testing-net
The Clean Code Talks -- Unit Testing
http://www.youtube.com/watch?v=wEhu57pih5w
GTAC 2010: Lessons Learned from Testability Failures
http://www.youtube.com/watch?v=4CFj5thxYQA