Why Test automation guys should have hacker mindset?!

Why Test automation guys should have hacker

mindset?!

By Peter Kartashov @2011 http://at4qa.blogspot.com

a talk about being test automation engineer

• lazy one who got tired doing testing manually• crazy about automating everything• tester who wants to know and manipulate apps inside of that interfaces• looks for new opportunities to break apps• relies on testability

A few silly observations about Good test automation engineer:


• lazy one who got tired doing testing manually• crazy about automating everything• tester who wants to know and manipulate apps inside of that interfaces• looks for new opportunities to break apps• relies on testability

Great test automation engineer is same as Good one

+ plus +• experience, • effectiveness• flexibility


Challenges (technical)

1. You will never have stable app and interfaces2. You will have different ways to automate (interfaces,

approaches, frameworks)3. You will have to either reuse or grow new own framework4. You will never have smooth path5. Be ready to use these words in your vocabulary:

“workaround”, “hack”, “backdoor”, “crutch”…

If you have neither these symptoms – There is a good chance you are doing either wrong or trivial test automation


Challenges (technical)

Technically making enterprise wide automation implies thinking over solid architectural solution that involves:• Infrastructure• Integration with other systems and practices (CI, test

management, version control, etc)• Automation framework• Persistent storage test data, logs and historical information


Challenges (organizational)

1. Struggle to find out right scope, resources and solution (notorious ROI)

2. You will have to prove that you are useful (not money wasters)3. Some people will consider you as neither tester nor developer4. If a group of automation engineers – a right person should

manage and drive this team (test automation lead with background). If you looked after by test manager or manual test engineer only – personally I don’t envy you.

If you have neither these symptoms – There is a good chance you are doing either wrong automation or you are lucky


Testability

Testability is a property allowing you to manipulate with your software project by calling its interfaces.

Some most popular Issues: Some interfaces will be open, some will be closed. Some interfaces will be stable, some will be freaky to any code commit It is normal to have a choice on which level to automate your app. As functional and E2E testing is pretty complex, you will likely utilize different levels of testing. Another challenge related to testability is cross-platform (cross-browser) implementation


Testability: examples of mixed implementationAPI level testing is utilized to manipulate system components internally, transmit data and trigger events. Verification of results is running on API-level, then verification on GUI as double check

Integration between 2 components is verifying in module integration level (API). Integration with GUI layer is checking in GUI as checkpoints.

Change data in database (SQL scripting); checking results in API as checkpoint.

Vice versa: change data using UI, validate on database side (SQL querying). Additional validation can be incorporated on API level

Check Web service I/O, check queue object, check POST/GET Header and Body


Hacks, workarounds and tools

You will never have one tool or/and language covering all your needs.

So that you have to be creative and all-round to

resolve blockers quickly


Examples: Reflection


Reflection allows you to work with native code of running application in runtime, i.e. you can intercept and evaluate currently running code on machine (like attaching to currently running COM object)Java and .Net platforms allows to enable reflection so that testing framework “understands” running code of compiled application.

import java.lang.reflect.*; public class DumpMethods { public static void main(String args[]) { try { Class c = Class.forName(args[0]); Method m[] = c.getDeclaredMethods(); for (int i = 0; i < m.length; i++) System.out.println(m[i].toString()); } catch (Throwable e) { System.err.println(e); } } }

*Don’t use reflection in production code – main concerns are performance degradation and security

Examples: OS manipulation


Use internal Scripting capabilities: WMI, PowerShell, bash, Win Registry

E.g. VB Script utilizing WMI object to monitor CPU usage on hosts:

Examples: using tools and libs


Firebug to watch DOM rendering, scripting errors and memory leaking

… and evaluate scripts. I recommend using Java script to traverse, manipulate and hack over Web



To simplify your Java Script injection – I recommend utilizing jQuery



WinDbg to get memory dump, call critical section, emulate stack overflaw…

Core skills of good test automation engineer


+ Knowledge of internal mechanisms of interacted objects (e.g. DOM model, COM object, MFC, Pure Java). Basically you have to be able to invent efficient object recognition model with high robustness on intermediate changes and proper control identification (avoiding concurrent identification) + Knowledge of test automation tool(s) itself + Familiarity with programming language and scripting + Knowledge of development practices, principles (like single responsibility Open/Closed, reusability, OOP, procedural approach…) and common design patterns (singleton, decorator, factory, wrapper, etc)+ Regular expressions. That’s a very powerful tool for us. Must to know concepts and how to use. Ideally you are ready to write any expression at fly. + Knowledge of performance testing, modeling and identification of bottlenecks (for load testing)+ Knowledge of theory and practices on relational databases.+ Good knowledge of one on-demand RDBMS, querying and performance analysis+ XML – based technologies and service model (XML, XSLT, XSD, DTD, WSDL)+ Experience in version control systems+ Experience in tuning application/web/DB servers+ Broad knowledge of operation systems (server and consumers solutions) as advanced user or as administrator.


In general, Test automation engineer should quickly solve technical challenges utilizing right tools and libraries.

Finally, the effort on building automation has to be empowered by testing aptitude and passion to break software with keen tests.

Aren’t all these slides about Hackers?

Thank you!

Questions?


Date post:	18-Nov-2014
Category:	Technology
Upload:	peter-ka
View:	5,414 times
Download:	2 times

Why Test automation guys should have hacker mindset?!

Technology