Oh you test?cool test bro

Paul Blundell

@blundell_apps blog.blundellapps.com github.com/blundell

AutoTrader

- Define different types of testing- Clarify testing & naming- Benefits of each type of testing- Codez available to help you- Some architecture hints & tips

Abstract

Test Types

- Unit Tests- Integration Tests- Functional Tests- Acceptance Tests- Mutation Tests- Vendor Tests- Fuzz Tests

- Test a single block of code / function- Test independently from collaborators- Only test against interface definitions- New behaviour : new unit test

Unit Tests - What

@Testpublic void testGetLastPathSegmentReturnsFilename() { SecureUrl secureUrl = new SecureUrl( "http://test.com/some/random/url/filename.mp4");

String lastPathSegment = secureUrl.getLastPathSegment();

assertThat(lastPathSegment).isEqualTo("filename.mp4");}

Unit Tests - Example

- Daily development work- Allow for continuous refactoring- Test Driven Development- Documentation for legacy developers

Unit Tests - When

Unit Tests - Where

- All building blocks- All levels of an application architecture- Help pinpoint a place of failure

- Highlight UI issues- Guarantee combined system integrity- Prove integration of components

Unit Tests - Won’t

- Interaction between two or more classes- Use real objects- Can use threads- Can access database

Integration Tests - What

@Test(expected = IllegalStateException.class)public void testFindFragmentWithUnknownTagResourceIdThrowsError() { Activity activity = Robolectric.buildActivity(Activity.class).get(); FragmentManager fM = activity.getFragmentManager(); Resources resources = activity.getResources(); TaskFragmentFinder finder = new TaskFragmentFinder(resources);

finder.findTaskFragment(fM, UNKNOWN_TASK_ID);}

Integration Tests - Example

- Ensure parity of collaboration between objects- Testing expected changes - db schema- Testing environment integration

Integration Tests - When

- All building blocks- Interactions between layers- Help pinpoint integration issues

Integration Tests - Where

- Highlight UI issues- Show specific code block of failure- Give instant feedback- Hard to diagnose some failures- Full system confidence

Integration Tests - Won’t

- Whole system, nearly end to end- Don’t care about intermediary steps- Slower to run- Mostly controller driven

Functional Tests - What

public void testShowsFilmInformationAfterApiCallFinished() { String expectedTitle = "Swimming Pool";

startFilmActivity();

onData(is(instanceOf(Film.class))) .atPosition(FIRST) .onChildView(withId(R.id.film_text_view_title)) .check(matches(allOf(isDisplayed(), withText(equalToIgnoringCase(expectedTitle)))));}

Functional Tests - Example

- Evaluate expectation of sum of the parts- Confirming feature completion- TDD Keep you focused- Confidence in features of system

Functional Tests - When

- Span whole architecture- Touching live systems- Above integrated components

Functional Tests - Where

- Show specific broken units of code- Lack of knowledge of the details- Give quick feedback for specific problems

Functional Tests - Won’t

- Black box tests- From user perspective- Specialised form of functional tests- Model the final complete system

Acceptance Tests - What

public void testRotationInBrowseScreenMaintainsActionBar() { swipeToBrowseScreen();

Activity activity = rotate(this);

assertActionBarOpen(activity);}

private void assertActionBarOpen(Activity activity) { assertTrue("Expected ActionBar open but was closed.", activity.getActionBar().isShowing());}

Acceptance Tests - Example

- User interface is integrated- TDD feedback loop- Capturing differences across devices- Screenshots for greater feedback

Acceptance Tests - When

- Span whole architecture- Touching live systems- Above integrated components

Acceptance Tests - Where

- Run very fast- No feedback upon cause (just symptoms)- Help day-to-day incremental improvements

Acceptance Tests - Won’t

- Mutate the state of your code- Insert faults in your software- Unit tests are ran- Tests fail “mutant is killed” thumbs up- Tests pass “mutant survived” thumbs down

Mutation Tests - What

Mutation Tests - Example

if (a == b) { // do something}

will be mutated to

if (a != b) { // do something}

Different Mutators:

if (a == b) { // do something}

will be mutated to

if (true) { // do something}

public int method(int i) { i++; return i;}

will be mutated to

public int method(int i) { i--; return i;}

- Find holes in your test suite- Depends on % code coverage- Confidence in your unit tests- Complexity of your problem is high

Mutation Tests - When

- Outside of your code- Around your test suite- Not written yourself but configured- Configuration is key

Mutation Tests - Where

- Give any confidence in working software- Show collaboration between objects- Prove if the application actually works- Help at all without unit tests

Mutation Tests - Won’t

- can be unit, integration, functional- learn how 3rd party libraries work- confirming understanding- safety net for outside changes

Vendor Tests - What

@Testpublic void setAndGetActiveSessionAreTrustworthy() { Session.setActiveSession(mockFacebookSession);

FacebookSession session = Session.getActiveSession();

assertThat(mockFacebookSession).isEqualTo(session);}

Vendor Tests - Example

- Incorporating 3rd party library- Not beneficial to add retrospectively- Replacing libraries with confidence- Wanting to ‘in house’ a library feature

Vendor Tests - When

- Around your 3rd party libraries- One off test- Suite inside unit tests

Vendor Tests - Where

- Confirm your domain code- Replace unit, integration or acceptance tests- Adds overhead to development

Vendor Tests - Won’t

- Feeding your software random data- Wait to see what breaks- It is not logical- Android = Application Exerciser Monkey- Combinatory with other tools

Fuzz Tests - What

adb shell monkey -p com.your.package.name -v 50000

Fuzz Tests - Example

// Allowing start of Intent { act=mubi.intent.action.ON_BOARD cmp=com.mubi/.onboard.OnboardActivity } in package com.mubi:Sending Touch (ACTION_DOWN): 0:(1091.0,659.0):Sending Touch (ACTION_UP): 0:(1085.1356,667.17145):Sending Touch (ACTION_DOWN): 0:(467.0,404.0):Sending Touch (ACTION_UP): 0:(472.5769,398.45746)// CRASH: com.mubi (pid 754)// Short Msg: java.lang.IllegalStateException// Long Msg: java.lang.IllegalStateException: Fragment WatchFragment{413d0d98} is not currently in the FragmentManager// Build Label: google/nakasi/grouper:4.1.1/JRO03H/405518:user/release-keys// Build Changelist: 405518// Build Time: 1364293068000// java.lang.IllegalStateException: Fragment WatchFragment{413d0d98} is not currently in the FragmentManager// at android.app.FragmentManagerImpl.saveFragmentInstanceState(FragmentManager.java:586)// at android.support.v13.app.FragmentStatePagerAdapter.destroyItem(FragmentStatePagerAdapter.java:140)// at android.support.v4.view.ViewPager.setAdapter(ViewPager.java:413)// at com.mubi.onboard.OnboardActivity.updateViewPager(OnboardActivity.java:139)// at com.mubi.onboard.OnboardActivity.onRetrieved(OnboardActivity.java:132)// at com.mubi.onboard.OnboardTaskFragment$3.run(OnboardTaskFragment.java:77)// at android.os.Handler.handleCallback(Handler.java:725)// at android.os.Handler.dispatchMessage(Handler.java:92)// at android.os.Looper.loop(Looper.java:137)// at android.app.ActivityThread.main(ActivityThread.java:5195)// at java.lang.reflect.Method.invokeNative(Native Method)// at java.lang.reflect.Method.invoke(Method.java:511)// at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:795)// at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:562)// at com.android.internal.os.ZygoteInit.main(Native Method)// at dalvik.system.NativeStart.main(Native Method)//** Monkey aborted due to error.Events injected: 4191

- From the beginning- Thin slice (UI)- Continuous integration, faster feedback- Identifying differences across Android versions- Highlight refactoring effort

Fuzz Tests - When

- Black box testing- On top of your application (Android)- Custom fuzz testing - input level - Continuous integration

Fuzz Tests - Where

- Prove application is running correctly- Only reports crashes- Highlights quality rather than bugs- Not a replacement for unit, integration etc

Fuzz Tests - Won’t

Helpers / Libraries

- Brief overview- What test types it benefits- Any positives- Any negatives

Mockito

//mock creationList mockedList = mock(List.class);//using mock object - doesn’t throw exceptionsmockedList.add(“one”);//selective & explicit verification verify(mockList).add(“one”);

Mockito is a mocking framework that tastes really good. It lets you write beautiful tests with clean & simple API. Mockito doesn't give you a hangover because the tests are very readable and they

produce clean verification errors.

- Unit Tests : mocking dependencies- Integration Tests : questionable use

FestFEST is a collection of libraries, whose mission is to simplify software testing.

No more confusion about the order of the "expected" and "actual" values. Our assertions are very readable as well: they read very close to plain English, making it easier for non-technical people

to read test code.Regular JUnit:assertEquals(View.GONE, view.getVisibility());

Regular FEST:assertThat(view.getVisibility()).isEqualTo(View.GONE);

FEST Android:assertThat(view).isGone(); - Unit Tests

- Integration Tests- Functional Tests- Acceptance Tests : anywhere you do assertions improving readability

“Robolectric is a unit test framework that de-fangs the Android SDK jar so you can test-drive the development of your Android app. Tests run inside the JVM on your workstation in seconds. “

Robolectric

@RunWith(RobolectricTestRunner.class)public class ViewingQueryTaskTest {

@Test public void testAlwaysClosesCursor() { viewingQueryTask.run();

verify(mockCursor).close(); }}

- Unit tests : for day to day activity including TDD - Integration tests : to handle Android - Positive : speed vs on device tests - Negative : the rabbit hole

“Robotium is an Android test automation framework. Robotium makes it easy to write powerful and robust automatic black-box UI tests. With the support of Robotium, test case developers can

write functional and user acceptance test scenarios, spanning multiple Android activities.“

Robotium

- Functional tests : check when you click a button the shared preferences are updated - Acceptance tests : ensure features have been completed and take screenshots for feedback

public class EditorTest extends ActivityInstrumentationTestCase2<EditorActivity> { private Solo solo;

public void setUp() throws Exception { solo = new Solo(getInstrumentation(), getActivity()); }

public void testFileExtensionIsInMenu() { solo.sendKey(Solo.MENU); solo.clickOnText("More"); solo.clickOnText("Preferences"); solo.clickOnText("Edit File Extensions"); Assert.assertTrue(solo.searchText("rtf")); }}

Use Espresso to write concise, beautiful, and reliable Android UI tests

Espresso tests state expectations, interactions, and assertions clearly without the distraction of boilerplate content, custom infrastructure, or messy implementation details getting in the way.

Espresso

public void testSayHello() { onView(withId(R.id.name_field).perform(typeText("Dave")); onView(withId(R.id.greet_button)).perform(click()); onView(withText("Hello Dave!")).check(matches(isDisplayed()));}

- Functional tests : check when you click a button the shared preferences are updated - Acceptance tests : ensure features have been completed, doesn’t support screenshots as an API but you can still do it

http://tiny.cc/disableAnim

http://tiny.cc/espressoScreenshot

The Monkey is a program that runs on your emulator or device and generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events. You can use the Monkey to stress-test applications that you are developing, in a random

yet repeatable manner.

UI Exerciser Monkey

adb shell monkey -p com.your.package.name -v 50000

- Fuzz tests : as already discussed allows you to test input events

Don’t get the Application Exerciser Monkey & the Monkey Runner mixed up.

App Ex Monkey – Fuzz TestingMonkey Runner – Android device control using Python scripts

The uiautomator testing framework lets you test your user interface (UI) efficiently by creating automated functional UI testcases that can be run against your app on one or more

devices.

UI Automator

extend UIAutomatorTestCase

UIObject & UISelector

Min SDK 16 Jelly Bean

adb shell uiautomator runtest MyTests.jar -c com.example.MyApp

Tests are in the jar, and the package is the package of the app under test.

- Acceptance tests : acts just like a user.

Architecture Empower / Encumber

- Allows for separation of concerns- Java only module- Directed testing- Encapsulating change- Reuse

Modules / Modularity Empowers

- Readable & flexible CI builds - Flavors, build types- Swapping out whole components- Greater range of testing flexibility

Gradle Empowers

- Test reliability- Test speed- Data integrity

Mock Services Empower

- Hexagonal architecture- Insulates system from outside dependencies- Empowers testing- Disconnect from platform requirements

Ports & Adapters Empower

S.O.L.I.D Empowers

- Quality of your code reflects quality of your tests- Object Oriented programming lends itself to quality testing- TDD- Static methods are evil

Low Code Quality Encumbers

- Framework lock in- Google Play services- Android code examples- Activity life cycle

Android (frameworks) Encumber

- Believe in testing- Believe in clean code- Software craftsmanship- TDD- Be energetic- Enjoy it

YOU Empower || Encumber

Monkey RunnerSpoonJmockHexagonal ArchitectureContract Tests

Others

Groovy TestsStatic analysis toolsCharlescURLPostman

Google sizes (small medium large)

We are hiring developers

Online Blogs

Martin Fowler http://martinfowler.com/tags/testing.html

JB Rainsberger http://blog.thecodewhisperer.com/

Kevin Rutherford http://silkandspinach.net/

Robert Martin https://sites.google.com/site/unclebobconsultingllc/blogs-by-robert-martin

References

More InfoAndroid Fuzz Tests http://developer.android.com/tools/help/monkey.htmlHardcore Fuzz Tests http://www.ibm.com/developerworks/library/j-fuzztest/index.htmlVendor Tests https://www.youtube.com/watch?v=47nuBTRB51c#t=23m34sMutation Tests http://pitest.org/Google Testing Opinion http://googletesting.blogspot.co.uk/2010/12/test-sizes.htmlMockito https://code.google.com/p/mockito/Mocking Frameworks http://blogs.telerik.com/skimedic/posts/13-07-23/top-5-reasons-to-use-a-mocking-frameworkAndroid Fest https://github.com/square/fest-androidRobolectric http://robolectric.org/Robotium https://code.google.com/p/robotium/Espresso https://code.google.com/p/android-test-kit/wiki/EspressoUI Automator http://developer.android.com/tools/help/uiautomator/index.htmlSOLID Principles http://www.its-on-the-internet-so-it-must-be-true.com/2012/06/solid-principles.html

Paul Blundell

Questions?

@blundell_apps