1. Pitfalls and tips with browser based testIng

2. My Background

• several projects with functional testing

• consulted at organizations to introduce functional testing and builds

• heavily involved in trying to improve effectiveness and reduce maintenance

• developed a framework over several projects that made use of my learnings

3. Common uses of functional tests

• smoke/sanity tests for most valuable and risky features

• black box/regression tests

• post deployment checks and rollbacks

• multi browser testing

4. what do we want from tests?

• correctness (when my tests work, my application is not broken)

• robustness (when my tests break, my application is broken)

5. what do we focus on?

• whats valuable? (it matters to us that people can do this)

• whats risky? (if this breaks, it really affects us)

6. so whats the problem?

7. five problems

8. 1. slow tests

9. Because

10. uses the entire stack

11. low spec hardware

12. tons of setup and teardown

13. talking to the browser is slow

14. so

15. very long feedback loop

16. can delay deployments

17. 2. Brittle tests

18. UI changes a lot

19. tests over specify

20. 3. Random failures

21. race conditions

22. unreliable dependencies

23. 4. tests are hard to debug

24. too many tests break

25. error messages tell you nothing

26. 5. test code becomes unmaintainable

27. insufficient commitment to testing

28. inadequate understanding of testing

29. what happens

• there is an initial stream of tests

• tests break unnecessarily

• tests become hard to maintain

• team loses confidence and motivation

• some teams even abandon testing via the UI

30. tips and tricks

31. Treat test code like real code like real code

• speed

• maintainability

32. A lot of the techniques we use with real code work with test code

33. similar levels of commitment

34. test creation/maintenance can be part of the dev cycle

35. Use Ruby

• maintainability

• development time

36. use a headless browser when possible

• speed

37. Use selenium RC (if you use selenium)

• avoid brittleness

• maintainability

38. Dont record and replay (if you use selenium)

• avoid brittleness

• maintainability

• development time

39. Introduce an Abstraction layer

• speed

• maintainability

40. Categorize/tag tests

• speed

• maintainability

41. Use the page object pattern

• maintainability

42. Parallelize

• speed

43. separate data for tests

• speed

• avoid random failures

44. clump data into common location

• maintainability

45. Push setup and teardown out of tests

• speed

• avoid brittleness

• development time

• debuggability

46. Only exercise parts of the application you want to test

• speed

• avoid brittleness

• debuggability

47. Build stub services

• speed

• avoid random failures

48. dont over specify

• avoid brittleness

• debuggability

49. run on better hardware

• speed

50. slice vertically

• debuggability

51. Use the right locator

• maintainability

• avoid brittleness

• speed

52. learn xpath/css selectors

• avoid brittleness

• maintainability

53. fail later

• speed

54. use polls over sleep

• speed

• avoid random failures

55. use existing waits of selenium

• speed

• avoid random failures

• avoid brittleness

• avoid random

56. Dont use assertions to identify transitions

• avoid brittleness

• debuggability

• avoid brittleness

• avoid random

57. screenshots and html dumps

• debuggability

58. xpath/css selector tester

• development time

59. firebug or w/e

• development time

• maintainability

60. talk to me if youre testing tabular data

61. Im always available for questions

62. Questions?

63. thank you