TaaS - Test Automation as a Service... · Selenium Grid Node 1 on 10.1 Node 2 on 10.2 selenium node...

transcript

TaaS - Test Automation as a Service

Oren Rubin

What have the cloud done for us lately?

Testim.io CEO

Applitools Director of R&D

WixWeb Architect

Cadence Compiler Engineer

IBM Cloud Engineer

Google Developer ExpertOrenRubin

oren@testim.io

Mentor Google Launchpad

External LecturerTechnion University

Meetup Co-organizerSelenium-IL, GDG, Ember

Testing Theory

La järɡən

Testing Theory

La järɡən

● Scope

○ Unit○ Integration○ End to End

● Type

○ Functional○ Visual○ Performance○ Load○ Stress

● Accuracy

○ Smoke○ Acceptance○ Coverage

Black Box

Check against an API

White Box

Look inside

BeholdThe Interweb!

So simple..

So simple.. Not!

E2E Path of Requests and Cache Hierarchy

BeholdThe Interweb!

FRONT END BACK END

Load/Stress Testing

Why linear? Why not hard start/stop?

CommonScenario

Metrics: CPU / Memory / Response

CommonScenario

Response Metrics: Throughput, Average, Median, Deviation

CommonScenario

Testing the Backend

Number of Threads | Ramp-Up | Loop Count

JMeter

JMeter in the Cloud

JMeterIn The Cloud

||Equivalent

Why Front-End/E2E

Matters?

Depends on the app

Front EndMatters

More Than Ever

Who Moved My Code?

FRONT ENDBACK END FRONT ENDBACK END

1997 2017

How to test Front End?

Meet Selenium

Many Languages

Many OS

BrowsersAll major browsers

On all major (desktop) operating systems

Can anyone suggest a good architecture?

HTTP as a universal language

?TheWire

Protocol

Binding(SDK) Driver

One (W3C) Standard to rule them all!

https://code.google.com/p/selenium/wiki/JsonWireProtocol http://www.w3.org/TR/webdriver/

Selenium != Selenium IDE

Meet Selenium's Hello World

Selenium

Meet Selenium IDE

Why Record/Playback

Unreliable, Unreadable, Uneditable

Use Machine Learning to find elements

MeetTestim.io

Selenium Grid

new Chrome(); chromedriver.exe

Browser

HTTP Request

localhost:8989/click

driver = new Chrome();...submitBtn.click();

driver (Binding)

What We Want

new Chrome();

chromedriver.exe

Browser

HTTP Request

localhost:8989/click

driver = new Chrome();...submitBtn.click();

driver (Binding)

Machine 1 Machine 2

java -jar selenium.jar -port 4444 -role hub

selenium hub

Node 0 on 10.0

Selenium Grid

Node 1 on 10.1

Node 2 on 10.2

selenium node

java -jar selenium.jar -role webdriver -browser firefox -hubHost 10.0 –port 8989

selenium node

java -jar selenium.jar -role webdriver -browser chrome -hubHost 10.0 –port 8989

selenium hub

Node 0 on 10.0

chromedriver.exe

firefox-webdriver.dpi

Selenium Grid

driver (Binding)

new Chrome();

BrowserHTTP Request

10.0:4444/chrome

hub = new URL("10.0:4444/wd/hub")cap = new Capabilities ("chrome", "v3", "windows")driver = new RemoteWebDriver( hub, cap );...

my computer / CI server Node 2 on 10.2

selenium node

java -jar selenium.jar -role webdriver -hubHost 10.1 –port 8989

selenium node

selenium hub

Node 0 on 10.0

HTTP Request

10.2:8989/chrome

chromedriver.exe

firefox-webdriver.dpi

Node 1 on 10.1

new Chrome();

BrowserHTTP Request

10.0:4444/click

driver = ...

submitBtn = driver.find(By.id("submit"));submitBtn.click();

driver

my computer / CI server Node 2 on 10.2

selenium node

selenium hub

Node 0 on 10.0

HTTP Request

10.2:8989/click

chromedriver.exe

firefox-webdriver.dpiSelenium Grid

You need to get the drivers :(

?TheWire

Protocol

Binding(SDK) Driver

Protractor to the rescue

> npm install -g > webdriver-manager

> webdriver-manager update

> webdriver-manager start

Don't you lovedocker?

$ docker run -d -P --name selenium-hub selenium/hub:3.4.0-einsteinium

$ docker run -d --link selenium-hub:hub selenium/node-chrome:3.4.0-einsteinium$ docker run -d --link selenium-hub:hub selenium/node-firefox:3.4.0-einsteinium

Load Balance

Selenium Grid works well for only 25 nodes

https://github.com/seleniumkit/gridrouter

Selenium in the cloud

Browsers in the cloud

SaaS Open Source

Visual Validation

Can you spot the differences?

1 image is worth a 1000 assertions

Pixels Matters

BTW, CSS is Crosscutting

Use Machine Learning to find elements

MeetTestim.io

Thank you :)

oren@testim.io | @shexman | linkedin

Oren Rubin

TaaS - Test Automation as a Service... · Selenium Grid Node 1 on 10.1 Node 2 on 10.2 selenium node...

Documents