Browser-level testing

Browser-level testing
Martin Kleppmann
Go Test It
Email: martin@go-test.it
Blog: yes-no-cancel.co.uk
Twitter: @martinkl

“But I have unit
tests!”

Array literals?

var x = [1,];

x.length == 1 // Firefox 3.5
x.length == 2 // IE 7

Uninitialised vars?

<div id="hello"></div>

typeof(hello) == 'undefined' // Firefox 3.5
typeof(hello) == 'object' // IE 7

Attempt #1:
Simulate events in JavaScript.

(Selenium, Windmill, Sahi)

Attempt #1: Simulate events in JavaScript.

Selenium example.com


click


click

http://localhost:8080 http://test.example.com

Cross-domain
security policy

click

http://localhost:8080 http://test.example.com



test.example.com proxy



/selenium-server/ /*


click

/selenium-server/ /*

Limitations

• Redirect to other domain or https
• Outside of DOM: pop-ups, Flash, ...
• Slow

Attempt #2:
Browser-speciﬁc automation
APIs.
(WebDriver, Watir)

IE

Firefox

Safari

Chrome

Opera

HTMLUnit

IE

Firefox

Safari
WebDriver
Chrome

Opera

HTMLUnit

IE

Firefox

Safari
WebDriver
Chrome

Opera
Watir

HTMLUnit

IE

Selenium
Firefox

Safari
WebDriver
Chrome

Opera
Watir

HTMLUnit

Selenium WebDriver Watir

IE

Firefox

Safari

Chrome

Opera

HTMLUnit

Selenium WebDriver Watir

IE HTA/Proxy ✔ COM ✔ COM

Firefox Ext/Proxy ✔ Extension ✔ Extension

Safari Proxy ✘ ✔ AppleScript

Chrome Proxy ✔ Extension ✔ V8 Debug

Opera Proxy Dragonﬂy? ✘

HTMLUnit ✘ ✔ ✔ Celerity

Watir
require 'watir'
Watir::Browser.default = 'firefox'

describe 'Google' do
before(:each) { @browser = Watir::Browser.new }
after(:each) { @browser.close }

it 'should return search results for "hello world"' do
@browser.goto "http://www.google.co.uk"
@browser.text_field(:name, "q").set("hello world")
@browser.button(:name, "btnG").click
@browser.contains_text(
"Hello world program - Wikipedia").should be_true
end
end

WebDriver
require 'selenium-webdriver'

before(:each) { @browser = Selenium::WebDriver.for :firefox }
after(:each) { @browser.quit }

@browser.navigate.to "http://www.google.co.uk"
@browser.find_element(:name, "q").send_keys("hello world")
@browser.find_element(:name, "btnG").submit
@browser.find_element(:partial_link_text,
"Hello world program - Wikipedia")
end
end

Selenium
gem 'selenium-client'; require 'selenium/client'

before(:each) {
@browser = Selenium::Client::Driver.new :browser => "*firefox",
:url => "http://www.google.co.uk"
@browser.start
}
after(:each) { @browser.stop }

@browser.open "/"
@browser.type "q", "hello world"
@browser.click "btnG", :wait_for => :text, :text => 'Results'
@browser.is_text_present("Hello world program - Wikipedia").
should be_true
end
end

Go Test It (≈Selenium)
require 'gotest'

gotest "http://www.google.co.uk" do
open "/"
type "q", "hello world"
click "btnG"
verify_text_present "Hello world program - Wikipedia"
end

Cucumber
Feature: Google search
In order to learn something new
As an enthusiast for obscure programming languages
I want to find "Hello world" programs on Google

Scenario: Search for "hello world"
Given that I am on google.co.uk
When I enter "hello world" into the search box
And I submit the form
Then I should be shown a list of results
And the page should contain "Hello world program -
Wikipedia"

The future? IE

Selenium
Selenium API
Firefox

Safari
WebDriver
Chrome

Opera
Watir
Watir API

HTMLUnit

Fundamental goal:
Robust tests

Choose your locators
carefully

Choose your locators
carefully
id = searchInput
name = search
xpath = //form[@id='searchform']/input[2]

link = EU makes 7bn euro climate pledge


xpath = //td[@class='text']/div[1]/a


xpath = //td[@class='text']/div[1]/a

css = a.tsh

People are complicated

Frontend tests are
messy

2 approaches:

Disposable tests
(record/playback, quick&dirty)

Domain-speciﬁc abstractions
(carefully crafted)

Which
approach will
you take?

Thank you!
Martin Kleppmann
Go Test It
Email: martin@go-test.it
Blog: yes-no-cancel.co.uk
Twitter: @martinkl

Fundamental goal:
Parallel tests

Don’t assume a clean database

Unit tests:

Load ﬁxtures

Given...

When...

Then...

Reset database

Unit tests: Functional tests:

Load ﬁxtures Set up example data

Given... Do stuff Do stuff

When...
Check Check
Then... outcome outcome

Reset database

Tolerate partially run tests

(Assume there is no frob)

Create frob

Check frob was created

Delete frob

Bad:

Boom!
(Assume there is no frob)

Create frob

Check frob was created

Delete frob

Bad: Better:

Boom!
(Assume there is no frob) Delete frob if it exists

Create frob Create frob

Check frob was created Check frob was created

Delete frob Delete frob (optional)

Avoid race conditions: isolation


Delete frob if it exists Delete frob if it exists



Log in as User A Log in as User B




Log in as User A Log in as User B




User A and User B: independent & not used concurrently

Take care with external services

Browser-level testing

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Browser-level testing (20)

Recently uploaded (20)

Browser-level testing