PerformanceTes-ngusingSelenium CITS5501 Software Testing and Quality Assurance

2017, Semester 1 Material from Xiangyu Zhang, Shari Pfleeger and Joann Atlee

Outline

•  Performance Testing •  Uniqueness of web app testing

–  Heterogonous system –  Dynamic pages –  Performance testing –  Security testing

•  Selenium WebDriver

9.3 Performance Tests Purpose and Roles

•  Used to examine –  the calculation –  the speed of response –  the accuracy of the result –  the accessibility of the data

•  Designed and administrated by the test team

9.3 Performance Tests Types of Performance Tests

•  Stress tests •  Volume tests •  Configuration tests •  Compatibility tests •  Regression tests •  Security tests •  Timing tests

•  Environmental tests •  Quality tests •  Recovery tests •  Maintenance tests •  Documentation tests •  Human factors

(usability) tests

9.4 Reliability, Availability, and Maintainability

Definition

•  Software reliability: operating without failure under given condition for a given time interval

•  Software availability: operating successfully according to specification at a given point in time

•  Software maintainability: for a given condition of use, a maintenance activity can be carried out within stated time interval, procedures and resources

9.4 Reliability, Availability, and Maintainability

Different Level of Failure Severity

•  Catastrophic: causes death or system loss •  Critical: causes severe injury or major system

damage •  Marginal: causes minor injury or minor system

damage •  Minor: causes no injury or system damage

9.5 Acceptance Tests Purpose and Roles

•  Enable the customers and users to determine if the built system meets their needs and expectations

•  Written, conducted and evaluated by the customers

9.5 Acceptance Tests Types of Acceptance Tests

•  Pilot test: install on experimental basis •  Alpha test: in-house test •  Beta test: customer pilot •  Parallel testing: new system operates

in parallel with old system

Web application architecture

•  Heterogeneous system –  Front end

•  Browser: IE, Firefox, Chrome, Safari… –  Server side

•  Application Server •  Database Server •  File System •  ……

Heterogeneous system

•  Front end –  HTML, JavaScript, Adobe Flash……

HTML

JavaScript

PageinBrowser

Sourcebehind

Uniqueness 1: Heterogeneous system

•  Server side –  Can be written in PHP, Java, C#... –  Communicate with Database server in SQL

PHPScript

SQL

HTML

SQL

PHP

Heterogeneous System

•  Need to test all sub-components –  Anything could go wrong…

•  However, only front end is accessible for testing –  Can not directly test the Server code and SQL –  Have to drive the test execution

•  Frontend –  HTML: Malformed HTML page? –  JavaScript: Runtime Errors?

•  Server script –  PHP, Java…: Runtime Errors? –  SQL: Malformed SQL query string?

Test from the front end

•  Pros –  Hide the complexity of the backend –  Uniform interface –  Can use a robot to automate test execution

q Cons –  The front end is not trustable

•  Crafted malicious requests

Selenium

•  A tool set that automates web app testing across platforms •  Can simulate user interactions in browser •  Two components

–  Selenium IDE –  Selenium WebDriver (aka. Selenium 2)

Selenium IDE

•  Firefoxextension

•  Easyrecordandreplay

•  Debugandsetbreakpoints

•  SavetestsinHTML,WebDriverandotherformats.

Selenium IDE test cases

•  Selenium saves all information in an HTML table format

•  Each record consists of:

–  Command – tells Selenium what to do (e.g. “open”, “type”, “click”, “verifyText”)

–  Target – tells Selenium which HTML element a command refers to (e.g. textbox, header, table)

–  Value – used for any command that might need a value of some kind (e.g. type something into a textbox)

How to record/replay with Selenium IDE

1.  Start recording in Selenium IDE

2.  Execute scenario on running web application

3.  Stop recording in Selenium IDE

4.  Verify / Add assertions

5.  Replay the test.

Selenium IDE Demo……

Bad things of testing from the front end •  The front end is not trustable

–  Front end code can be accessed by anybody –  Malicious users can infer the input parameters –  Crafted requests!

•  Demo –  Front end limits the length of the input values –  Front end limits the content of the input values –  Front end limits the combination of the input values

Uniqueness 2: Dynamic pages

•  Client page could be dynamic –  It can change itself at runtime –  HTML can be modified by JavaScript –  JavaScript can modify itself –  Demo

•  Server script could be dynamic –  Client pages are constructed at runtime –  The same server script can produce completely

different client pages –  Demo

•  SchoolMate

Uniqueness 3: Performance

•  Performance is crucial to the success of a web app –  Recall the experience to register for a class in the first

days of the semester…

•  Performance testing evaluates system performance under normal and heavy usage –  Volume testing

•  For expected concurrent number of users –  Stress testing

•  To understand the upper limits of capacity

•  Performance testing can be automated

Uniqueness 4: Security

•  Web app usually deals with sensitive info, e.g. –  Credit card number –  SSN –  Billing / Shipping address

•  Security is the biggest concern

•  Security testing should simulate possible attacks

Uniqueness 4: Security

•  SQL Injection –  The untrusted input is used to construct dynamic

SQL queries. –  E.g, update my own password

$str="UPDATEusersSETpassword=\”“.$_POST['newPass’].“\”WHEREusername=\”“.$_POST['username'].“\””;mysql_query($str);

$_POST['newPass’]=pass,$_POST['username']=meQueryString:UPDATEusersSETpassword=“pass”WHEREusername=“me”

$_POST['newPass’]=pass,$_POST['username']=“OR1=1--QueryString:UPDATEusersSETpassword=“pass”WHEREusername=“”OR1=1--”

NormalCase

A9ack

PHPScript

Uniqueness 4: Security

•  Cross Site Scripting (XSS) –  The untrusted input is used to construct dynamic

HTML pages. –  The malicious JS injected executes in victim’s browser –  The malicious JS can steal sensitive info

–  Demo

•  Solution: Never trust user inputs •  Design test cases to simulate attacks

Outline

•  Uniqueness of web app testing –  Heterogonous system –  Dynamic pages –  Performance testing –  Security testing

•  Selenium WebDriver

Selenium WebDriver (Selenium 2)

•  Selenium-WebDriver –  Write a program to control the test execution of a web app –  More flexible and powerful than IDE

•  Selenium-WebDriver supports multiple browsers in multiple platforms –  Google Chrome 12.0.712.0+ –  Internet Explorer 6+ –  Firefox 3.0+ –  Opera 11.5+ –  Android – 2.3+ for phones and tablets –  iOS 3+ for phones –  iOS 3.2+ for tablets

Selenium WebDriver

•  WebDriver is designed to providing a simpler and uniform programming interface –  Same WebDriver script runs for different platforms

•  Support multiple programming languages: –  Java, C#, Python, Ruby, PHP, Perl…

•  It’s efficient –  WebDriver leverages each browser’s native support

for automation.

How to use Selenium WebDriver

(1) Go to a page

(2) Locate an element (3) Do something with that element

...... (i) Locate an element

(i+1) Do something with that element (i+2) Verify / Assert the result

Demo: Verify page title publicsta=cvoidmain(String[]args){//CreateanewinstanceoftheFirefoxdriverWebDriverdriver=newFirefoxDriver();//(1)Gotoapagedriver.get("hap://www.google.com");//(2)LocateanelementWebElementelement=driver.findElement(By.name("q"));//(3-1)Entersomethingtosearchforelement.sendKeys("PurdueUniveristy");//(3-2)Nowsubmittheform.WebDriverwillfindtheformforusfromtheelementelement.submit();//(3-3)Waitupto10secondsforacondi-onWebDriverWaitwai-ng=newWebDriverWait(driver,10);wai-ng.un-l(ExpectedCondi-ons.presenceOfElementLocated(By.id("pnnext")));//(4)Checkthe-tleofthepageif(driver.getTitle().equals("purdueuniveristy-GoogleSearch"))System.out.println("PASS");elseSystem.err.println("FAIL");//Closethebrowserdriver.quit();}

How to locate an element

•  By id –  HTML:<div id="coolestWidgetEvah">...</div> –  WebDriver: driver.findElement( By.id("coolestWidgetEvah") );

•  By name –  HTML: <input name="cheese" type="text"/> –  WebDriver:driver.findElement( By.name("cheese") );

•  By Xpath –  HTML <html> <input type="text" name="example" /> <input type="text" name="other" /> </html>

–  WebDriver:driver.findElements( By.xpath("//input") ); –  Thereareplug-insforfirefox/chrometoautoma=callydisplaytheXpath

Timing issue

•  There are delays between submitting a request and receiving the response

•  We can wait until the response page is loaded

•  Robot doesn’t know! •  In WebDriver, sometimes it doesn’t work if

–  Submit a request –  Verify the response immediately

•  Solution: –  Simulate the wait. Wait until some HTML object

appears

Summary: What Selenium can do

•  A solution for the automated testing –  Simulate user actions –  Functional testing

•  One could even program BVA in a test script •  Create regression tests to verify functionality and user

acceptance.

–  Browser compatibility testing •  The same script can run on any Selenium platform

–  Volume testing –  Stress testing