Alexey Buzdin "Maslow's Pyramid of Android Testing"

Post on 22-Jan-2017

119 views 0 download

transcript

@AlexeyBuzdin

GDGRiga.lv JUG.lv

RigaDevDay.lvCitadele.lv

What does Android need?

UI Tests

Integration Tests

Unit Tests

UI Tests

Integration Tests

Unit Tests

Effort

Effort Cost

UI Tests

Integration Tests

Unit Tests

UI Tests

Integration Tests

Unit Tests

Effort Cost

Unit Test๏ Uses simple JUnit ๏ Runs on JVM, not on a device ๏ Lightning fast (5k test in 10

seconds) ๏ Needs Android SDK Stubs

static protected void markConflicting(ArrayList<ScheduleItem> items) { for (int i=0; i<items.size(); i++) { ScheduleItem item = items.get(i); // Notice that we only care about sessions when checking conflicts. if (item.type == ScheduleItem.SESSION) for (int j=i+1; j<items.size(); j++) { ScheduleItem other = items.get(j); if (item.type == ScheduleItem.SESSION) { if (intersect(other, item, true)) { other.flags |= ScheduleItem.FLAG_CONFLICTS_WITH_PREVIOUS; item.flags |= ScheduleItem.FLAG_CONFLICTS_WITH_NEXT; } else { // we assume the list is ordered by starttime break; } } } } } https://github.com/google/iosched

@Override public void displayData(final SessionFeedbackModel model, final SessionFeedbackQueryEnum query) { switch (query) { case SESSION: mTitle.setText(model.getSessionTitle()); if (!TextUtils.isEmpty(model.getSessionSpeakers())) { mSpeakers.setText(model.getSessionSpeakers()); } else { mSpeakers.setVisibility(View.GONE); } AnalyticsHelper.sendScreenView("Feedback: " + model.getSessionTitle()); break; } }

https://github.com/google/iosched

java.lang.RuntimeException: Method setText in android.widget.TextView not mocked. See http://g.co/androidstudio/not-mocked for details.

at android.widget.TextView.setText(TextView.java) at lv.buzdin.alexey.MainActivityPresenterTest.test(MainActivityPresenterTest.java:39)

@Test public void name() throws Exception { TextView textView = new TextView(null); textView.setText("hello"); }

android { ….

testOptions { unitTests.returnDefaultValues = true } }

Toast.makeText(null, "", Toast.LENGTH_SHORT).show();

Toast.makeText(null, "", Toast.LENGTH_SHORT).show();

Unmockable*

Testable architecture๏ Use MV* Patterns ๏ Dependency Injection is a must ๏ Try to use POJOs where possible ๏ Wrap static Android classes to Services

(Log, Toast, etc) ๏ Avoid highly coupling to android classes in *

code

Model - View - Whatever(Activity/Fragment)

๏ Activity is polluted. (Lifecycle Logic, LayoutInflater, static calls, etc)

๏ We need a POJO

@Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); BaseApplication.inject(this); setContentView(R.layout.activity_screen);

presenter.initPresenter(this); presenter.initNavigationDrawer(); presenter.openScheduleScreen();

if (presenter.firstApplicationStart()) { presenter.openNavigationDrawer(); } }

public class MainActivityPresenter { … public void initPresenter(ActionBarActivity activity) { this.activity = activity; ButterKnife.inject(this, activity); }

public void initNavigationDrawer() { activity.setSupportActionBar(toolbar); drawerToggle = new ActionBarDrawerToggle(activity, drawerLayout, R.string.app_name, R.string.app_name) { @Override public void onDrawerOpened(View drawerView) { super.onDrawerOpened(drawerView); listView.invalidateViews(); //Refresh counter for bookmarks } }; drawerToggle.setDrawerIndicatorEnabled(true); drawerLayout.setDrawerListener(drawerToggle); activity.getSupportActionBar().setDisplayHomeAsUpEnabled(true); activity.getSupportActionBar().setHomeButtonEnabled(true); listView.setAdapter(navigationAdapter); } }

Testable architecture๏ Use MV* Patterns ๏ Dependency Injection is a must ๏ Try to use POJOs where possible ๏ Wrap static Android classes to Services

(Log, Toast, etc) ๏ Avoid highly coupling to android classes in *

code

Dagger 2The fastest Java DI Framework!

https://google.github.io/dagger/

@Singleton public class MainActivityPresenter {

@Inject SocialNetworkNavigationService socialsService; @Inject SharedPrefsService preferences; @Inject NavigationAdapter navigationAdapter; … public boolean firstApplicationStart() { boolean subsequentStart = preferences.getBool(PreferencesConstants.SUBSEQUENT_START); if (!subsequentStart) { preferences.setBool(PreferencesConstants.SUBSEQUENT_START, true); return true; } return false; } }

Testable architecture๏ Use MV* Patterns ๏ Dependency Injection is a must ๏ Try to use POJOs where possible ๏ Wrap static Android classes to Services

(Log, Toast, etc) ๏ Avoid highly coupling to android classes in *

code

public class SharedPrefsService {

@Inject public Context context;

private SharedPreferences getPrefs() { return PreferenceManager.getDefaultSharedPreferences(context); }

public boolean getBool(String key) { return getPrefs().getBoolean(key, false); }

public void setBool(String key, boolean value) { getPrefs().edit().putBoolean(key, value).commit(); } }

Testable architecture๏ Use MV* Patterns ๏ Dependency Injection is a must ๏ Try to use POJOs where possible ๏ Wrap static Android classes to Services

(Log, Toast, etc) ๏ Avoid highly coupling to android classes in *

code

Removes View dependency for Whatever

Button b = (Button)findViewById(R.id.button); b.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { b.setBackgroundColor(Red); } });

@InjectView(R.id.button) Button b;

@OnClick(R.id.button) public void onClick(View v) {

b.setBackgroundColor(Red); }

Removes View dependency for Whatever

RxBus rxBus = new RxBus(); @InjectView(R.id.button) Button b; Observable<Void> clicks = RxView.clicks(b);

public init() { clicks.subscribe(aVoid -> { rxBus.send(new Click()); }); rxBus.toObserverable()

.filter(e -> e instanceof Click)

.subscribe(e -> { b.setBackgroundColor(Red); }); }

https://github.com/JakeWharton/RxBinding

Testable architecture with Rx

@Test public void test() throws Exception { MyFragment fragment = new MyFragment(); fragment.b = new Button(null) { @Override public void setBackgroundColor(int color) { assertEquals(color, Red); } }; fragment.clicks = Observable.just(null); fragment.init(); }

Mockito + PowerMock

Mocking, Spying

Object o = mock(Object.class); doReturn(true).when(o).equals(any());

Object o = spy(“Hi”); doReturn(true).when(o).equals(any()); o.hashCode() -> is real

@RunWith(MockitoJUnitRunner.class)public class MyClassTest {

@InjectMocks MyFragment fragment; @Mock Button b;

@Test public void test() throws Exception { fragment.clicks = Observable.just(null); fragment.init(); verify(b).setBackgroundColor(Red); } }

Toast.makeText(null, "", Toast.LENGTH_SHORT).show();

Unmockable*

Toast.makeText(null, "", Toast.LENGTH_SHORT).show();

Unmockable*

@RunWith(PowerMockRunner.class)@PrepareForTest( { Toast.class })public class PowerMockExample {

@Test public void testPowerMock() throws Exception { Toast mock = mock(Toast.class); mockStatic(Toast.class); when(Toast.makeText(any(), anyString(), anyInt())).thenReturn(mock);

Toast.makeText(null, "1", Toast.LENGTH_SHORT).show(); } }

JUnit Extra FeaturesHelpful for Android Developers

JUnit Lifecyclepublic class RunnerTest { @BeforeClass public static void beforeClass() { out.println("Before Class");} @Before public void before() { out.println("Before");} @Test public void test() { out.println("Test"); } @After public void after() { out.println("After"); } @AfterClass public static void afterClass() { out.println("After Class"); } }

JUnit Lifecyclepublic class RunnerTest { @BeforeClass public static void beforeClass() { out.println("Before Class");} @Before public void before() { out.println("Before");} @Test public void test() { out.println("Test"); } @After public void after() { out.println("After"); } @AfterClass public static void afterClass() { out.println("After Class"); } }Before Class Before Test After After Class Process finished with exit code 0

Custom Runner:BlockJUnit4ClassRunner

or Runner

public class CustomRunner extends BlockJUnit4ClassRunner{

public static void runnerBefore() { System.out.println("Runner Before");} public static void runnerBeforeClass() { System.out.println("Runner Before Class"); } public static void runnerAfter() { System.out.println("Runner After");} public static void runnerAfterClass() { System.out.println("Runner After Class"); }

@Override protected Statement withBefores(FrameworkMethod method, Object t, Statement st) { List<FrameworkMethod> list = getFrameworkMethods("runnerBefore"); return new RunBefores(super.withBefores(method, t, st), list, t); } …. private List<FrameworkMethod> getFrameworkMethods(String methodName) { try { Method runnerBefore = getClass().getDeclaredMethod(methodName); return Collections.singletonList(new FrameworkMethod(runnerBefore)); } catch (Exception e) { throw new RuntimeException(e); } } }

@RunWith(CustomRunner.class)public class RunnerTest { …. }

Runner Before Class Before Class Runner Before Before Test After Runner After After Class Runner After Class Process finished with exit code 0

@RunWith(CustomRunner.class)public class RunnerTest { …. }

Runner Before Class Before Class Runner Before Before Test After Runner After After Class Runner After Class Process finished with exit code 0

AndroidJUnitRunner + MockitoJUnitRunner +Parametrized

JUnit Rules

•Rules allow flexible addition of the behaviour of each test method in a test class

•Base Rules Provided in JUnit:

Temporary Folder Rule; ExternalResource Rule; ErrorCollector Rule; TestName Rule; Timeout Rule;

RuleChain

public class CustomRule implements TestRule { private boolean classRule; public CustomRule(boolean classRule) { this.classRule = classRule; }

@Override public Statement apply(Statement base, Description description) { return new Statement() { @Override public void evaluate() throws Throwable { System.out.println(classRule ? "Class Rule Before" : "Rule Before"); try { base.evaluate(); } finally { System.out.println(classRule ? "Class Rule After" : "Rule After”); } } }; } }

@RunWith(CustomRunner.class)public class RunnerTest {

@ClassRule public static CustomRule classRule = new CustomRule(true); @Rule public CustomRule rule = new CustomRule(false);

@BeforeClass public static void beforeClass() { out.println("Before Class");} @Before public void before() { out.println("Before");} @Test public void test() { out.println("Test"); } @After public void after() { out.println("After"); } @AfterClass public static void afterClass() { out.println("After Class"); } }

Class Rule Before Runner Before Class Before Class Rule Before Runner Before Before Test After Runner After Rule After After Class Runner After Class Class Rule After

Class Rule Before Runner Before Class Before Class Rule Before Runner Before Before Test After Runner After Rule After After Class Runner After Class Class Rule After

public static class UseRuleChain { @Rule public RuleChain chain= RuleChain .outerRule(new LoggingRule("outer rule") .around(new LoggingRule("middle rule") .around(new LoggingRule("inner rule");

@Test public void example() { assertTrue(true); } }

starting outer rule starting middle rule starting inner rule finished inner rule finished middle rule finished outer rule

Rules > Runners

Test Specification1. Preparation 2. Testable Action 3. Assertion

Test Specification1. Preparation 2. Testable Action 3. Assertion

https://github.com/hamcrest/JavaHamcrest

assertThat(T object, Matcher<T> matcher)

@Test public void testValidIPAddress() throws InvalidIPAddressException { IPAddress addr = new IPAddress("127.0.0.1"); byte[] octets = addr.getOctets();

assertTrue(octets[0] == 127); assertTrue(octets[1] == 0); assertTrue(octets[2] == 0); assertTrue(octets[3] == 1); }

Bad Test

Examples•assertThat(2, is(2))

•assertThat(“s”, is(nullValue()))

•assertThat(“s”, is(new String(“s”)))

•assertThat(“s”, equalTo(new String(“s”)))

•assertThat(“s”, not(equalTo(“d”)))

•assertThat(“s”, instanceOf(String.class))

http://hamcrest.org/JavaHamcrest/javadoc/1.3/

List matcher

•assertThat(ids, hasItem(10))

•assertThat(ids, contains(5, 8))

•assertThat(ids, containsInAnyOrder(5, 8))

•assertThat(ids, everyItem(greaterThan(3)))

AllOf AnyOf

assertThat(name, anyOf(startsWith(“A”), endsWith(“B”)))

assertThat(ids, allOf(

hasSize(5),

hasItem(10),

everyItem(greaterThan(3))

))

Error messagesassertThat("s", is(nullValue()))

assertThat(true, is(false))

Error messages

assertThat(1, is(allOf(not(1), not(2), not(10))))

Custom matchersprivate Matcher<Foo> hasNumber(final int i) { return new TypeSafeMatcher<Foo>() { @Override public void describeTo(final Description description) { description.appendText("getNumber should return ").appendValue(i); } @Override protected void describeMismatchSafely(final Foo item, final Description mismatchDescription) { mismatchDescription.appendText(" was ").appendValue(item.getNumber()); } @Override protected boolean matchesSafely(final Foo item) { return i == item.getNumber(); } }; }

hamcrest-rxMatcher<TestSubscriber<T>> hasValues(final Matcher<? super List<T>> eventsMatcher) Matcher<TestSubscriber<T>> hasOnlyValues(final Matcher<? super List<T>> values) Matcher<TestSubscriber<T>> hasOnlyValue(final Matcher<? super T> valueMatcher) Matcher<TestSubscriber<T>> hasNoValues() Matcher<TestSubscriber<T>> hasErrors(final Matcher<? super List<Throwable>> err) Matcher<TestSubscriber<T>> hasNoErrors() Matcher<TestSubscriber<T>> hasOnlyErrors(final Matcher<? super List<Throwable>> err)

https://github.com/zalando-incubator/undertaking/blob/master/src/test/java/org/zalando/undertaking/test/rx/hamcrest/TestSubscriberMatchers.java

https://github.com/hertzsprung/hamcrest-json

assertThat( "{\"age\":43, \"friend_ids\":[16, 52, 23]}", sameJSONAs("{\"friend_ids\":[52, 23, 16]}") .allowingExtraUnexpectedFields() .allowingAnyArrayOrdering());

hamcrest-json

Parameterised tests

Parameterised with Name

Square Burst

https://github.com/square/burst

public enum Soda { PEPSI, COKE }public enum Sets { HASH_SET() { @Override public <T> Set<T> create() { return new HashSet<T>(); } }, LINKED_HASH_SET() { … }, TREE_SET() { … } public abstract <T> Set<T> create(); }

https://github.com/square/burst

@RunWith(BurstJUnit4.class) public class DrinkSodaTest { @Burst Soda soda; @Burst Sets sets; @Test public void drinkFavoriteSodas(Soda soda) { // TODO Test drink method with 'soda'... } }

Square Burst

https://github.com/square/burst

@RunWith(BurstJUnit4.class) public class DrinkSodaTest { private final Set<Soda> favorites; public DrinkSodaTest(Sets sets) { favorites = sets.create(); } @Test public void trackFavorites() { // TODO … } @Test public void drinkFavoriteSodas(Soda soda) { //TODO … } }

Square Burst

JUnitParams

https://github.com/Pragmatists/JUnitParams

@RunWith(JUnitParamsRunner.class)public class PersonTest {

@Test @Parameters({"17, false”, "22, true" }) public void personIsAdult(int age, boolean valid) throws Exception { assertThat(new Person(age).isAdult(), is(valid)); }

}

EnclosedProvides a way to use multiple JUnit Runners in a test class

https://www.indiegogo.com/projects/junit-lambda

http://junit.org/junit5/

JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

import static org.junit.jupiter.api.Assertions.assertEquals; import org.junit.jupiter.api.Test;

class FirstJUnit5Tests { @Test void myFirstTest() { assertEquals(2, 1 + 1); } }

JUnit 5 Features

Grouped Assertions

@Test void groupedAssertions() { // In a grouped assertion all assertions are executed, and any // failures will be reported together. assertAll("address", () -> assertEquals("John", address.getFirstName()), () -> assertEquals("User", address.getLastName()) ); }

Throwable Assertions

@Test void exceptionTesting() { Throwable exception = assertThrows(IllegalArgumentException.class, () -> { throw new IllegalArgumentException("a message"); }); assertEquals("a message", exception.getMessage()); }

No assertThat() import static org.hamcrest.CoreMatchers.equalTo; import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.MatcherAssert.assertThat;

import org.junit.jupiter.api.Test;

class HamcrestAssertionDemo {

@Test void assertWithHamcrestMatcher() { assertThat(2 + 1, is(equalTo(3))); } }

No Runners or Rules > Extensions@ExtendWith(MockitoExtension.class)class MyMockitoTest {

@BeforeEach void init(@Mock Person person) { when(person.getName()).thenReturn("Dilbert"); }

@Test void simpleTestWithInjectedMock(@Mock Person person) { assertEquals("Dilbert", person.getName()); } }

Dynamic Tests

class DynamicTestsDemo { @TestFactory Collection<DynamicTest> dynamicTestsFromCollection() { return Arrays.asList( dynamicTest("1st dynamic test", () -> assertTrue(true)), dynamicTest("2nd dynamic test", () -> assertEquals(4, 2 * 2)) ); }}

Collection, Iterable, Iterator, Stream

in Android

• Not coming soon

• Would require a rewrite for all testing libraries

• Would require Android Studio support

• Would simplify the Extension model

https://github.com/junit-team/junit5/issues/204

Integration Tests

๏ Uses Robolectric framework ๏ Runs on JVM with Shadow Android SDK ๏ Has access to Context and all Android

peripheral

Robolectric@RunWith(RobolectricTestRunner.class)public class MyActivityTest {

@Test public void clickingButton_shouldChangeResultsViewText() throws Exception { MyActivity activity = Robolectric.setupActivity(MyActivity.class);

Button button = (Button) activity.findViewById(R.id.button); TextView results = (TextView) activity.findViewById(R.id.results);

button.performClick(); assertThat(results.getText().toString()).isEqualTo("Robolectric Rocks!"); } }

ActivityController controller = Robolectric.buildActivity(MyAwesomeActivity.class).create().start(); Activity activity = controller.get(); // assert that something hasn't happened activityController.resume(); // assert it happened!

Activity activity = Robolectric.buildActivity(MyAwesomeActivity.class) .create().start().resume().visible().get();

Activity Lifecycle

public class SharedPrefsService {

@Inject public Context context;

private SharedPreferences getPrefs() { return PreferenceManager.getDefaultSharedPreferences(context); }

public boolean getBool(String key) { return getPrefs().getBoolean(key, false); }

public void setBool(String key, boolean value) { getPrefs().edit().putBoolean(key, value).commit(); } }

Robolectric๏ Life saver when complex Android SDK

calls should be tested ๏ Slow compared to Unit Tests ๏Not up to date to the latest SDKs

(API 24 not supported yet)

UI Tests

๏ Runs on actual Android Device ๏ Slower the Unit tests ๏ Brittle and dependant on

device health

public void testRecorded() throws Exception { if (solo.waitForText("Hello!")) { solo.clickOnView(solo.findViewById("R.id.sign_in")); solo.enterText((EditText) solo.findViewById("R.id.login_username"),"username"); solo.enterText((EditText) solo.findViewById("R.id.login_password"),"password"); solo.clickOnView(solo.findViewById("R.id.login_login")); solo.waitForActivity("HomeTabActivity"); } solo.clickOnView(solo.findViewById("R.id.menu_compose_tweet") ); solo.enterText((EditText) solo.findViewById(“R.id.edit"), "Testdroid"); solo.clickOnView(solo.findViewById("R.id.composer_post")); }

Robotium

Android Espresso

@Test public void multiActivityTest() { onView(withId(R.id.date)) .perform(click()); // Loads another activity riiiight here onView(allOf(withId(R.id.date_expanded), withText("SomeRandomDate"))) .check(matches(isDisplayed())) .perform(click()); // Yay! No waiting!}

Espresso comes with Hamcrest integration

@Test public void dateTest() { onView(withId(R.id.date)) .check(matches(withText("2014-10-15"))); }

Robotium vs Espresso

• Espresso faster •Robotium has bigger SDK coverage • Espresso has built in wait mechanism that is optimised for android lifecycle

http://www.stevenmarkford.com/android-espresso-vs-robotium-benchmarks/

Looking into Cross-platform UI Test Automation?

https://www.youtube.com/watch?v=iwvueGzwVyk

Cross platform tests

If you have dedicated QA team and product on multiple platforms - go Calabash or Appium

• More flacky tests• Less performant speed• Some test reuse• Easier for QA

BFF

BFF

DB

DB

DB

AMQP

How to run UI test

•Don’t initialize run-time dependencies (event tracking, analytics, long-init things like payment solutions) •Don’t hit up real backend, mock out responses • Insert appropriate test data before test starts running

How to mock a Server

- Mock Server through DI - Mock HTTP Server instance on Device - Dev instance of Server

DI: Create a custom Test Runner

public class MockTestRunner extends AndroidJUnitRunner { @Override public Application newApplication(ClassLoader cl, String className, Context ctx) throws InstantiationException, IllegalAccessException, ClassNotFoundException { return super.newApplication(cl, MockDemoApplication.class.getName(), ctx); }}

DI: Create a custom Test Application

public class MockDemoApplication extends DemoApplication { @Override protected DemoComponent createComponent() { return DaggerMainActivityTest_TestComponent.builder().build(); }}

MockServer on Device: AndroidAsync

https://github.com/koush/AndroidAsync

AsyncHttpServer server = new AsyncHttpServer(); server.get("/", new HttpServerRequestCallback() { @Override public void onRequest(AsyncHttpServerRequest request, AsyncHttpServerResponse response) { response.send("Hello!!!"); } }); server.listen(5000);

How to run UI tests on CI?

Android Jenkins Plugin

https://wiki.jenkins-ci.org/display/JENKINS/Android+Emulator+Plugin

Multi-configuration (matrix) job

Android Jenkins Plugin

https://github.com/Genymobile/genymotion-gradle-plugin

1. genymotion { 2. devices { 3. nexus5 { 4. template "Google Nexus 5 - 4.4.4 - API 19 - 1080x1920" 5. } 6. } 7. }

https://medium.com/@Genymotion/android-os-now-available-as-an-amazon-machine-image-72748130436b#.njabkxnih

https://github.com/square/spoonhttps://github.com/stanfy/spoon-gradle-plugin

http://openstf.io/

Conclusion๏ Unit tests are cheap, make them your first

frontier ๏ Adapt code to make it more testable ๏ Structure tests with @Rules and Hamcrest

Matchers ๏ Mockito + Powermock will help to mock

Android else Robolectric will

Conclusion๏ UI tests are harder to write and maintain ๏ If you have a dedicated mobile QA team

think of cross-platform tests ๏ For UI tests have a config for Mocked server

and other integration points ๏ Configure either emulator startup or device

farm on your CI

TESTS ARE MADE TO MAKE YOU FEEL SECURE!

LIKE A LOVELY HUG ♥

Q&AThank You!

@AlexeyBuzdinFollow me at