Post on 06-May-2015
description
transcript
Browser-level testing
Martin KleppmannGo Test It
Email: martin@go-test.itBlog: yes-no-cancel.co.uk
Twitter: @martinkl
“But I have unit tests!”
?
Array literals?
var x = [1,];
x.length == 1 // Firefox 3.5x.length == 2 // IE 7
Uninitialised vars?
<div id="hello"></div>
typeof(hello) == 'undefined' // Firefox 3.5typeof(hello) == 'object' // IE 7
Attempt #1:Simulate events in JavaScript.
(Selenium, Windmill, Sahi)
Selenium example.com
Attempt #1: Simulate events in JavaScript.
Seleniumclick
example.com
Attempt #1: Simulate events in JavaScript.
Selenium
http://test.example.comhttp://localhost:8080
clickexample.com
Attempt #1: Simulate events in JavaScript.
Selenium
http://test.example.comhttp://localhost:8080
clickexample.com
Cross-domain security policy
Attempt #1: Simulate events in JavaScript.
Selenium example.com
Attempt #1: Simulate events in JavaScript.
Selenium example.com
test.example.com proxy
Attempt #1: Simulate events in JavaScript.
Selenium example.com
test.example.com proxy/selenium-server/ /*
Attempt #1: Simulate events in JavaScript.
clickSelenium example.com
test.example.com proxy/selenium-server/ /*
Attempt #1: Simulate events in JavaScript.
Limitations
• Redirect to other domain or https
• Outside of DOM: pop-ups, Flash, ...
• Slow
Attempt #2:Browser-specific automation
APIs.
(WebDriver, Watir)
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
Watir
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
Watir
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
Watir
Selenium
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
Watir
Selenium
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
Selenium
WebDriver
Watir
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
Selenium WebDriver Watir
Selenium WebDriver Watir
HTA/Proxy ✔ COM ✔ COM
Ext/Proxy ✔ Extension ✔ Extension
Proxy ✘ ✔ AppleScript
Proxy ✔ Extension ✔ V8 Debug
Proxy Dragonfly? ✘
✘ ✔ ✔ Celerity
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
Watir
require 'watir'Watir::Browser.default = 'firefox'
describe 'Google' do before(:each) { @browser = Watir::Browser.new } after(:each) { @browser.close }
it 'should return search results for "hello world"' do @browser.goto "http://www.google.co.uk" @browser.text_field(:name, "q").set("hello world") @browser.button(:name, "btnG").click @browser.contains_text( "Hello world program - Wikipedia").should be_true endend
WebDriver
require 'selenium-webdriver'
describe 'Google' do before(:each) { @browser = Selenium::WebDriver.for :firefox } after(:each) { @browser.quit }
it 'should return search results for "hello world"' do @browser.navigate.to "http://www.google.co.uk" @browser.find_element(:name, "q").send_keys("hello world") @browser.find_element(:name, "btnG").submit @browser.find_element(:partial_link_text, "Hello world program - Wikipedia") endend
Selenium
gem 'selenium-client'; require 'selenium/client'
describe 'Google' do before(:each) { @browser = Selenium::Client::Driver.new :browser => "*firefox", :url => "http://www.google.co.uk" @browser.start } after(:each) { @browser.stop }
it 'should return search results for "hello world"' do @browser.open "/" @browser.type "q", "hello world" @browser.click "btnG", :wait_for => :text, :text => 'Results' @browser.is_text_present("Hello world program - Wikipedia"). should be_true endend
Go Test It (≈Selenium)
require 'gotest'
gotest "http://www.google.co.uk" do open "/" type "q", "hello world" click "btnG" verify_text_present "Hello world program - Wikipedia"end
Cucumber
Feature: Google search In order to learn something new As an enthusiast for obscure programming languages I want to find "Hello world" programs on Google
Scenario: Search for "hello world" Given that I am on google.co.uk When I enter "hello world" into the search box And I submit the form Then I should be shown a list of results And the page should contain "Hello world program - Wikipedia"
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
Watir
Selenium
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
Watir
Selenium
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
Watir
Selenium
IE
Firefox
Safari
Chrome
Opera
HTMLUnit
WebDriver
Watir
Selenium
The future?
Watir API
Selenium API
Ok.
Fundamental goal:
Robust tests
Choose your locators carefully
Choose your locators carefully
id = searchInputname = search
xpath = //form[@id='searchform']/input[2]
link = EU makes 7bn euro climate pledge
link = EU makes 7bn euro climate pledge
xpath = //td[@class='text']/div[1]/a
link = EU makes 7bn euro climate pledge
css = a.tsh
xpath = //td[@class='text']/div[1]/a
People are complicated
⇒Frontend tests are
messy
2 approaches:
Disposable tests(record/playback, quick&dirty)
Domain-specific abstractions(carefully crafted)
Which approach will
you take?
Thank you!
Martin KleppmannGo Test It
Email: martin@go-test.itBlog: yes-no-cancel.co.uk
Twitter: @martinkl
Fundamental goal:
Parallel tests
Don’t assume a clean database
Don’t assume a clean database
Unit tests:
Load fixtures
Given...
When...
Then...
Reset database
Don’t assume a clean database
Unit tests:
Load fixtures
Given...
When...
Then...
Reset database
Functional tests:
Set up example data
Do stuff
Checkoutcome
Do stuff
Checkoutcome
Tolerate partially run tests
Tolerate partially run tests
(Assume there is no frob)
Create frob
Check frob was created
Delete frob
Tolerate partially run tests
(Assume there is no frob)
Create frob
Check frob was created
Delete frob
Tolerate partially run tests
(Assume there is no frob)
Create frob
Check frob was created
Delete frob
Boom!
Bad:
Tolerate partially run tests
(Assume there is no frob)
Create frob
Check frob was created
Delete frob
Boom!
Bad:
Delete frob if it exists
Create frob
Check frob was created
Delete frob (optional)
Better:
Avoid race conditions: isolation
Avoid race conditions: isolation
Delete frob if it exists
Create frob
Check frob was created
Delete frob if it exists
Create frob
Check frob was created
Avoid race conditions: isolation
Delete frob if it exists
Create frob
Check frob was created
Delete frob if it exists
Create frob
Check frob was created
Log in as User A Log in as User B
Avoid race conditions: isolation
Delete frob if it exists
Create frob
Check frob was created
Delete frob if it exists
Create frob
Check frob was created
Log in as User A Log in as User B
User A and User B: independent & not used concurrently
Make test runs unique
Make test runs unique
Make test runs unique
Take care with external services
Thank you!
Martin KleppmannGo Test It
Email: martin@go-test.itBlog: yes-no-cancel.co.uk
Twitter: @martinkl