113
@AlexeyBuzdin
GDGRiga.lv JUG.lv
RigaDevDay.lvCitadele.lv
What does Android need?
UI Tests
Integration Tests
Unit Tests
UI Tests
Integration Tests
Unit Tests
Effort
Effort Cost
UI Tests
Integration Tests
Unit Tests
UI Tests
Integration Tests
Unit Tests
Effort Cost
Unit Test๏ Uses simple JUnit ๏ Runs on JVM, not on a device ๏ Lightning fast (5k test in 10
seconds) ๏ Needs Android SDK Stubs
static protected void markConflicting(ArrayList<ScheduleItem> items) { for (int i=0; i<items.size(); i++) { ScheduleItem item = items.get(i); // Notice that we only care about sessions when checking conflicts. if (item.type == ScheduleItem.SESSION) for (int j=i+1; j<items.size(); j++) { ScheduleItem other = items.get(j); if (item.type == ScheduleItem.SESSION) { if (intersect(other, item, true)) { other.flags |= ScheduleItem.FLAG_CONFLICTS_WITH_PREVIOUS; item.flags |= ScheduleItem.FLAG_CONFLICTS_WITH_NEXT; } else { // we assume the list is ordered by starttime break; } } } } } https://github.com/google/iosched
@Override public void displayData(final SessionFeedbackModel model, final SessionFeedbackQueryEnum query) { switch (query) { case SESSION: mTitle.setText(model.getSessionTitle()); if (!TextUtils.isEmpty(model.getSessionSpeakers())) { mSpeakers.setText(model.getSessionSpeakers()); } else { mSpeakers.setVisibility(View.GONE); } AnalyticsHelper.sendScreenView("Feedback: " + model.getSessionTitle()); break; } }
https://github.com/google/iosched
java.lang.RuntimeException: Method setText in android.widget.TextView not mocked. See http://g.co/androidstudio/not-mocked for details.
at android.widget.TextView.setText(TextView.java) at lv.buzdin.alexey.MainActivityPresenterTest.test(MainActivityPresenterTest.java:39)
@Test public void name() throws Exception { TextView textView = new TextView(null); textView.setText("hello"); }
android { ….
testOptions { unitTests.returnDefaultValues = true } }
Toast.makeText(null, "", Toast.LENGTH_SHORT).show();
Toast.makeText(null, "", Toast.LENGTH_SHORT).show();
Unmockable*
Testable architecture๏ Use MV* Patterns ๏ Dependency Injection is a must ๏ Try to use POJOs where possible ๏ Wrap static Android classes to Services
(Log, Toast, etc) ๏ Avoid highly coupling to android classes in *
code
Model - View - Whatever(Activity/Fragment)
๏ Activity is polluted. (Lifecycle Logic, LayoutInflater, static calls, etc)
๏ We need a POJO
@Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); BaseApplication.inject(this); setContentView(R.layout.activity_screen);
presenter.initPresenter(this); presenter.initNavigationDrawer(); presenter.openScheduleScreen();
if (presenter.firstApplicationStart()) { presenter.openNavigationDrawer(); } }
public class MainActivityPresenter { … public void initPresenter(ActionBarActivity activity) { this.activity = activity; ButterKnife.inject(this, activity); }
public void initNavigationDrawer() { activity.setSupportActionBar(toolbar); drawerToggle = new ActionBarDrawerToggle(activity, drawerLayout, R.string.app_name, R.string.app_name) { @Override public void onDrawerOpened(View drawerView) { super.onDrawerOpened(drawerView); listView.invalidateViews(); //Refresh counter for bookmarks } }; drawerToggle.setDrawerIndicatorEnabled(true); drawerLayout.setDrawerListener(drawerToggle); activity.getSupportActionBar().setDisplayHomeAsUpEnabled(true); activity.getSupportActionBar().setHomeButtonEnabled(true); listView.setAdapter(navigationAdapter); } }
Testable architecture๏ Use MV* Patterns ๏ Dependency Injection is a must ๏ Try to use POJOs where possible ๏ Wrap static Android classes to Services
(Log, Toast, etc) ๏ Avoid highly coupling to android classes in *
code
Dagger 2The fastest Java DI Framework!
https://google.github.io/dagger/
@Singleton public class MainActivityPresenter {
@Inject SocialNetworkNavigationService socialsService; @Inject SharedPrefsService preferences; @Inject NavigationAdapter navigationAdapter; … public boolean firstApplicationStart() { boolean subsequentStart = preferences.getBool(PreferencesConstants.SUBSEQUENT_START); if (!subsequentStart) { preferences.setBool(PreferencesConstants.SUBSEQUENT_START, true); return true; } return false; } }
Testable architecture๏ Use MV* Patterns ๏ Dependency Injection is a must ๏ Try to use POJOs where possible ๏ Wrap static Android classes to Services
(Log, Toast, etc) ๏ Avoid highly coupling to android classes in *
code
public class SharedPrefsService {
@Inject public Context context;
private SharedPreferences getPrefs() { return PreferenceManager.getDefaultSharedPreferences(context); }
public boolean getBool(String key) { return getPrefs().getBoolean(key, false); }
public void setBool(String key, boolean value) { getPrefs().edit().putBoolean(key, value).commit(); } }
Testable architecture๏ Use MV* Patterns ๏ Dependency Injection is a must ๏ Try to use POJOs where possible ๏ Wrap static Android classes to Services
(Log, Toast, etc) ๏ Avoid highly coupling to android classes in *
code
Removes View dependency for Whatever
Button b = (Button)findViewById(R.id.button); b.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { b.setBackgroundColor(Red); } });
@InjectView(R.id.button) Button b;
@OnClick(R.id.button) public void onClick(View v) {
b.setBackgroundColor(Red); }
Removes View dependency for Whatever
RxBus rxBus = new RxBus(); @InjectView(R.id.button) Button b; Observable<Void> clicks = RxView.clicks(b);
public init() { clicks.subscribe(aVoid -> { rxBus.send(new Click()); }); rxBus.toObserverable()
.filter(e -> e instanceof Click)
.subscribe(e -> { b.setBackgroundColor(Red); }); }
https://github.com/JakeWharton/RxBinding
Testable architecture with Rx
@Test public void test() throws Exception { MyFragment fragment = new MyFragment(); fragment.b = new Button(null) { @Override public void setBackgroundColor(int color) { assertEquals(color, Red); } }; fragment.clicks = Observable.just(null); fragment.init(); }
Mockito + PowerMock
Mocking, Spying
Object o = mock(Object.class); doReturn(true).when(o).equals(any());
Object o = spy(“Hi”); doReturn(true).when(o).equals(any()); o.hashCode() -> is real
@RunWith(MockitoJUnitRunner.class)public class MyClassTest {
@InjectMocks MyFragment fragment; @Mock Button b;
@Test public void test() throws Exception { fragment.clicks = Observable.just(null); fragment.init(); verify(b).setBackgroundColor(Red); } }
Toast.makeText(null, "", Toast.LENGTH_SHORT).show();
Unmockable*
Toast.makeText(null, "", Toast.LENGTH_SHORT).show();
Unmockable*
@RunWith(PowerMockRunner.class)@PrepareForTest( { Toast.class })public class PowerMockExample {
@Test public void testPowerMock() throws Exception { Toast mock = mock(Toast.class); mockStatic(Toast.class); when(Toast.makeText(any(), anyString(), anyInt())).thenReturn(mock);
Toast.makeText(null, "1", Toast.LENGTH_SHORT).show(); } }
JUnit Extra FeaturesHelpful for Android Developers
JUnit Lifecyclepublic class RunnerTest { @BeforeClass public static void beforeClass() { out.println("Before Class");} @Before public void before() { out.println("Before");} @Test public void test() { out.println("Test"); } @After public void after() { out.println("After"); } @AfterClass public static void afterClass() { out.println("After Class"); } }
JUnit Lifecyclepublic class RunnerTest { @BeforeClass public static void beforeClass() { out.println("Before Class");} @Before public void before() { out.println("Before");} @Test public void test() { out.println("Test"); } @After public void after() { out.println("After"); } @AfterClass public static void afterClass() { out.println("After Class"); } }Before Class Before Test After After Class Process finished with exit code 0
Custom Runner:BlockJUnit4ClassRunner
or Runner
public class CustomRunner extends BlockJUnit4ClassRunner{
public static void runnerBefore() { System.out.println("Runner Before");} public static void runnerBeforeClass() { System.out.println("Runner Before Class"); } public static void runnerAfter() { System.out.println("Runner After");} public static void runnerAfterClass() { System.out.println("Runner After Class"); }
@Override protected Statement withBefores(FrameworkMethod method, Object t, Statement st) { List<FrameworkMethod> list = getFrameworkMethods("runnerBefore"); return new RunBefores(super.withBefores(method, t, st), list, t); } …. private List<FrameworkMethod> getFrameworkMethods(String methodName) { try { Method runnerBefore = getClass().getDeclaredMethod(methodName); return Collections.singletonList(new FrameworkMethod(runnerBefore)); } catch (Exception e) { throw new RuntimeException(e); } } }
@RunWith(CustomRunner.class)public class RunnerTest { …. }
Runner Before Class Before Class Runner Before Before Test After Runner After After Class Runner After Class Process finished with exit code 0
@RunWith(CustomRunner.class)public class RunnerTest { …. }
Runner Before Class Before Class Runner Before Before Test After Runner After After Class Runner After Class Process finished with exit code 0
AndroidJUnitRunner + MockitoJUnitRunner +Parametrized
JUnit Rules
•Rules allow flexible addition of the behaviour of each test method in a test class
•Base Rules Provided in JUnit:
Temporary Folder Rule; ExternalResource Rule; ErrorCollector Rule; TestName Rule; Timeout Rule;
RuleChain
public class CustomRule implements TestRule { private boolean classRule; public CustomRule(boolean classRule) { this.classRule = classRule; }
@Override public Statement apply(Statement base, Description description) { return new Statement() { @Override public void evaluate() throws Throwable { System.out.println(classRule ? "Class Rule Before" : "Rule Before"); try { base.evaluate(); } finally { System.out.println(classRule ? "Class Rule After" : "Rule After”); } } }; } }
@RunWith(CustomRunner.class)public class RunnerTest {
@ClassRule public static CustomRule classRule = new CustomRule(true); @Rule public CustomRule rule = new CustomRule(false);
@BeforeClass public static void beforeClass() { out.println("Before Class");} @Before public void before() { out.println("Before");} @Test public void test() { out.println("Test"); } @After public void after() { out.println("After"); } @AfterClass public static void afterClass() { out.println("After Class"); } }
Class Rule Before Runner Before Class Before Class Rule Before Runner Before Before Test After Runner After Rule After After Class Runner After Class Class Rule After
Class Rule Before Runner Before Class Before Class Rule Before Runner Before Before Test After Runner After Rule After After Class Runner After Class Class Rule After
public static class UseRuleChain { @Rule public RuleChain chain= RuleChain .outerRule(new LoggingRule("outer rule") .around(new LoggingRule("middle rule") .around(new LoggingRule("inner rule");
@Test public void example() { assertTrue(true); } }
starting outer rule starting middle rule starting inner rule finished inner rule finished middle rule finished outer rule
Rules > Runners
Test Specification1. Preparation 2. Testable Action 3. Assertion
Test Specification1. Preparation 2. Testable Action 3. Assertion
https://github.com/hamcrest/JavaHamcrest
assertThat(T object, Matcher<T> matcher)
@Test public void testValidIPAddress() throws InvalidIPAddressException { IPAddress addr = new IPAddress("127.0.0.1"); byte[] octets = addr.getOctets();
assertTrue(octets[0] == 127); assertTrue(octets[1] == 0); assertTrue(octets[2] == 0); assertTrue(octets[3] == 1); }
Bad Test
Examples•assertThat(2, is(2))
•assertThat(“s”, is(nullValue()))
•assertThat(“s”, is(new String(“s”)))
•assertThat(“s”, equalTo(new String(“s”)))
•assertThat(“s”, not(equalTo(“d”)))
•assertThat(“s”, instanceOf(String.class))
http://hamcrest.org/JavaHamcrest/javadoc/1.3/
List matcher
•assertThat(ids, hasItem(10))
•assertThat(ids, contains(5, 8))
•assertThat(ids, containsInAnyOrder(5, 8))
•assertThat(ids, everyItem(greaterThan(3)))
AllOf AnyOf
assertThat(name, anyOf(startsWith(“A”), endsWith(“B”)))
assertThat(ids, allOf(
hasSize(5),
hasItem(10),
everyItem(greaterThan(3))
))
Error messagesassertThat("s", is(nullValue()))
assertThat(true, is(false))
Error messages
assertThat(1, is(allOf(not(1), not(2), not(10))))
Custom matchersprivate Matcher<Foo> hasNumber(final int i) { return new TypeSafeMatcher<Foo>() { @Override public void describeTo(final Description description) { description.appendText("getNumber should return ").appendValue(i); } @Override protected void describeMismatchSafely(final Foo item, final Description mismatchDescription) { mismatchDescription.appendText(" was ").appendValue(item.getNumber()); } @Override protected boolean matchesSafely(final Foo item) { return i == item.getNumber(); } }; }
hamcrest-rxMatcher<TestSubscriber<T>> hasValues(final Matcher<? super List<T>> eventsMatcher) Matcher<TestSubscriber<T>> hasOnlyValues(final Matcher<? super List<T>> values) Matcher<TestSubscriber<T>> hasOnlyValue(final Matcher<? super T> valueMatcher) Matcher<TestSubscriber<T>> hasNoValues() Matcher<TestSubscriber<T>> hasErrors(final Matcher<? super List<Throwable>> err) Matcher<TestSubscriber<T>> hasNoErrors() Matcher<TestSubscriber<T>> hasOnlyErrors(final Matcher<? super List<Throwable>> err)
https://github.com/zalando-incubator/undertaking/blob/master/src/test/java/org/zalando/undertaking/test/rx/hamcrest/TestSubscriberMatchers.java
https://github.com/hertzsprung/hamcrest-json
assertThat( "{\"age\":43, \"friend_ids\":[16, 52, 23]}", sameJSONAs("{\"friend_ids\":[52, 23, 16]}") .allowingExtraUnexpectedFields() .allowingAnyArrayOrdering());
hamcrest-json
Parameterised tests
Parameterised with Name
Square Burst
https://github.com/square/burst
public enum Soda { PEPSI, COKE }public enum Sets { HASH_SET() { @Override public <T> Set<T> create() { return new HashSet<T>(); } }, LINKED_HASH_SET() { … }, TREE_SET() { … } public abstract <T> Set<T> create(); }
https://github.com/square/burst
@RunWith(BurstJUnit4.class) public class DrinkSodaTest { @Burst Soda soda; @Burst Sets sets; @Test public void drinkFavoriteSodas(Soda soda) { // TODO Test drink method with 'soda'... } }
Square Burst
https://github.com/square/burst
@RunWith(BurstJUnit4.class) public class DrinkSodaTest { private final Set<Soda> favorites; public DrinkSodaTest(Sets sets) { favorites = sets.create(); } @Test public void trackFavorites() { // TODO … } @Test public void drinkFavoriteSodas(Soda soda) { //TODO … } }
Square Burst
JUnitParams
https://github.com/Pragmatists/JUnitParams
@RunWith(JUnitParamsRunner.class)public class PersonTest {
@Test @Parameters({"17, false”, "22, true" }) public void personIsAdult(int age, boolean valid) throws Exception { assertThat(new Person(age).isAdult(), is(valid)); }
}
EnclosedProvides a way to use multiple JUnit Runners in a test class
http://junit.org/junit5/
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage
import static org.junit.jupiter.api.Assertions.assertEquals; import org.junit.jupiter.api.Test;
class FirstJUnit5Tests { @Test void myFirstTest() { assertEquals(2, 1 + 1); } }
JUnit 5 Features
Grouped Assertions
@Test void groupedAssertions() { // In a grouped assertion all assertions are executed, and any // failures will be reported together. assertAll("address", () -> assertEquals("John", address.getFirstName()), () -> assertEquals("User", address.getLastName()) ); }
Throwable Assertions
@Test void exceptionTesting() { Throwable exception = assertThrows(IllegalArgumentException.class, () -> { throw new IllegalArgumentException("a message"); }); assertEquals("a message", exception.getMessage()); }
No assertThat() import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.MatcherAssert.assertThat;
import org.junit.jupiter.api.Test;
class HamcrestAssertionDemo {
@Test void assertWithHamcrestMatcher() { assertThat(2 + 1, is(equalTo(3))); } }
No Runners or Rules > Extensions@ExtendWith(MockitoExtension.class)class MyMockitoTest {
@BeforeEach void init(@Mock Person person) { when(person.getName()).thenReturn("Dilbert"); }
@Test void simpleTestWithInjectedMock(@Mock Person person) { assertEquals("Dilbert", person.getName()); } }
Dynamic Tests
class DynamicTestsDemo { @TestFactory Collection<DynamicTest> dynamicTestsFromCollection() { return Arrays.asList( dynamicTest("1st dynamic test", () -> assertTrue(true)), dynamicTest("2nd dynamic test", () -> assertEquals(4, 2 * 2)) ); }}
Collection, Iterable, Iterator, Stream
in Android
• Not coming soon
• Would require a rewrite for all testing libraries
• Would require Android Studio support
• Would simplify the Extension model
https://github.com/junit-team/junit5/issues/204
Integration Tests
๏ Uses Robolectric framework ๏ Runs on JVM with Shadow Android SDK ๏ Has access to Context and all Android
peripheral
Robolectric@RunWith(RobolectricTestRunner.class)public class MyActivityTest {
@Test public void clickingButton_shouldChangeResultsViewText() throws Exception { MyActivity activity = Robolectric.setupActivity(MyActivity.class);
Button button = (Button) activity.findViewById(R.id.button); TextView results = (TextView) activity.findViewById(R.id.results);
button.performClick(); assertThat(results.getText().toString()).isEqualTo("Robolectric Rocks!"); } }
ActivityController controller = Robolectric.buildActivity(MyAwesomeActivity.class).create().start(); Activity activity = controller.get(); // assert that something hasn't happened activityController.resume(); // assert it happened!
Activity activity = Robolectric.buildActivity(MyAwesomeActivity.class) .create().start().resume().visible().get();
Activity Lifecycle
public class SharedPrefsService {
@Inject public Context context;
private SharedPreferences getPrefs() { return PreferenceManager.getDefaultSharedPreferences(context); }
public boolean getBool(String key) { return getPrefs().getBoolean(key, false); }
public void setBool(String key, boolean value) { getPrefs().edit().putBoolean(key, value).commit(); } }
Robolectric๏ Life saver when complex Android SDK
calls should be tested ๏ Slow compared to Unit Tests ๏Not up to date to the latest SDKs
(API 24 not supported yet)
UI Tests
๏ Runs on actual Android Device ๏ Slower the Unit tests ๏ Brittle and dependant on
device health
public void testRecorded() throws Exception { if (solo.waitForText("Hello!")) { solo.clickOnView(solo.findViewById("R.id.sign_in")); solo.enterText((EditText) solo.findViewById("R.id.login_username"),"username"); solo.enterText((EditText) solo.findViewById("R.id.login_password"),"password"); solo.clickOnView(solo.findViewById("R.id.login_login")); solo.waitForActivity("HomeTabActivity"); } solo.clickOnView(solo.findViewById("R.id.menu_compose_tweet") ); solo.enterText((EditText) solo.findViewById(“R.id.edit"), "Testdroid"); solo.clickOnView(solo.findViewById("R.id.composer_post")); }
Robotium
Android Espresso
@Test public void multiActivityTest() { onView(withId(R.id.date)) .perform(click()); // Loads another activity riiiight here onView(allOf(withId(R.id.date_expanded), withText("SomeRandomDate"))) .check(matches(isDisplayed())) .perform(click()); // Yay! No waiting!}
Espresso comes with Hamcrest integration
@Test public void dateTest() { onView(withId(R.id.date)) .check(matches(withText("2014-10-15"))); }
Robotium vs Espresso
• Espresso faster •Robotium has bigger SDK coverage • Espresso has built in wait mechanism that is optimised for android lifecycle
http://www.stevenmarkford.com/android-espresso-vs-robotium-benchmarks/
Looking into Cross-platform UI Test Automation?
Cross platform tests
If you have dedicated QA team and product on multiple platforms - go Calabash or Appium
• More flacky tests• Less performant speed• Some test reuse• Easier for QA
BFF
BFF
DB
DB
DB
AMQP
How to run UI test
•Don’t initialize run-time dependencies (event tracking, analytics, long-init things like payment solutions) •Don’t hit up real backend, mock out responses • Insert appropriate test data before test starts running
How to mock a Server
- Mock Server through DI - Mock HTTP Server instance on Device - Dev instance of Server
DI: Create a custom Test Runner
public class MockTestRunner extends AndroidJUnitRunner { @Override public Application newApplication(ClassLoader cl, String className, Context ctx) throws InstantiationException, IllegalAccessException, ClassNotFoundException { return super.newApplication(cl, MockDemoApplication.class.getName(), ctx); }}
DI: Create a custom Test Application
public class MockDemoApplication extends DemoApplication { @Override protected DemoComponent createComponent() { return DaggerMainActivityTest_TestComponent.builder().build(); }}
MockServer on Device: AndroidAsync
https://github.com/koush/AndroidAsync
AsyncHttpServer server = new AsyncHttpServer(); server.get("/", new HttpServerRequestCallback() { @Override public void onRequest(AsyncHttpServerRequest request, AsyncHttpServerResponse response) { response.send("Hello!!!"); } }); server.listen(5000);
How to run UI tests on CI?
Android Jenkins Plugin
https://wiki.jenkins-ci.org/display/JENKINS/Android+Emulator+Plugin
Multi-configuration (matrix) job
Android Jenkins Plugin
https://github.com/Genymobile/genymotion-gradle-plugin
1. genymotion { 2. devices { 3. nexus5 { 4. template "Google Nexus 5 - 4.4.4 - API 19 - 1080x1920" 5. } 6. } 7. }
https://medium.com/@Genymotion/android-os-now-available-as-an-amazon-machine-image-72748130436b#.njabkxnih
https://github.com/square/spoonhttps://github.com/stanfy/spoon-gradle-plugin
Conclusion๏ Unit tests are cheap, make them your first
frontier ๏ Adapt code to make it more testable ๏ Structure tests with @Rules and Hamcrest
Matchers ๏ Mockito + Powermock will help to mock
Android else Robolectric will
Conclusion๏ UI tests are harder to write and maintain ๏ If you have a dedicated mobile QA team
think of cross-platform tests ๏ For UI tests have a config for Mocked server
and other integration points ๏ Configure either emulator startup or device
farm on your CI
TESTS ARE MADE TO MAKE YOU FEEL SECURE!
LIKE A LOVELY HUG ♥
Q&AThank You!
@AlexeyBuzdinFollow me at
