Designing Reusable Frameworks for Test Automation

Designing Reusable Frameworks for Test AutomationPresenter: Byron GoodmanNational Director of Quality AssuranceNeudesicOverviewSome thoughts on automationSuccessful test automationComponents of a frameworkSome coding standardsStandard routines

Automated Testing: Why We Love ItLet the machines do the testsEasy to create alternate scenarios by leveraging existing testsUsing tools can create more interest in testing activitiesPotential to reduce testing cycle timeEffective at catching regression bugsAutomated Testing: Why We Hate ItTests break too often!Pass or fail results may not be reliableTools are expensive, although there are some good open source solutionsTools may oversimplify or overcomplicate the degree of effort required to test applicationsHave you Experienced Any of These Symptoms?More time spent working on automated tests than time needed to do the manual testingSporadic interest in implementing automated testingStatements by others that automated testing is: a sham, ineffective, waste of timeShelving of test automation in the interest of project successWhat is Successful Test Automation?As with all project related work, the time spent creating automated testing must offset this effort by the return of value. This is demonstrated in various ways:Less time expended during the testing cycle than if manual testing is the only solutionMore thorough and consistent regression coverageReduce human error and boredomFrees up people to test new functionality while computers test existing functionality

Basics of BeginningStrategy start simple and work towards a more complete and complex solutionTiming start using tests as soon as they are written, Do not wait for a completed suiteBuy-in Demonstrate to others that each test does exactly what it is intended to so that there is confidence in resultsTraceability Each automated test must trace back to a specific automated testStart SomewhereGenerally speaking, some automated testing is better than no automated testingSelect the most simple of tests to automateBegin by using record and playback to minimize time investment but to show that the tool does prove to be effectiveRecord and PlaybackFor years, I have insisted that no tests will be created using record and playbackI will admit that there is a place for R&PVery light use to prove out the tool and to minimize the reluctance people may have to use complicated technologyHowever, R&P is does not have much gain in efficiencies over manual testingTesting FrameworkMany of us have experience with testing frameworksOther names DLLs, namespace, test libraryIts just a fancy name for code that is used to automate testsA testing framework is a set of routines that are reused by tests that will speed up test development while increasing test reliabilityTesting Framework GoalsNeeds to be recognized to have the same rigor as any development effortAdds value to the overall test effortImplements complex and redundant routines for tests to leverageMinimizes the impact of a changing application to the tests, which reduces test maintenance

Layers & DesignEach layer is developed to promote portability and reusabilityThe more generic a layer, the more reusable it needs to bei.e. the Generic Routines should be able to work for everyone all the time regardless of the environments and AUTsGeneral Data Flow

Major Components of a FrameworkCommon routines that do not interact with any applicationGeneric UI layer, non-technology-specific manipulation routinesTechnology-specific layer routinesApplication-specific layer routinesApplication-specific Scenario/transactional layer routinesTests layerCommon routines that do not interact with any applicationNot related to the application or an interfaceBuild more capability and utility that the test tool may not haveFunctionality that a tester may need for any reasonIf written properly, these routines will work with test tools that use the same programming languageGeneric UI layer routinesSome control types, like grids, that are generic and rather independent from the technology that they are written inThis is a good place to create additional or alternate logging

Technology-specific layer routinesFor handling specific technologies i.e. Browsers, WinForms and WPF. Custom controls will be wrapped with calls that are identical to similar standard controls Create interfaces for new versions of technology if the tool vendor does not yet have the new technology versions integrated

Application-specific layer routinesRoutines that are specific to the AUTRepository for UI mappingsDefines a handler for every individual form/pageAUT service interfaces defined and wrappedServes the calls needed for tests to interact with the AUTApplication-specific Scenario/transactional layerUse this as a distinguishable layer if the routines for scenarios are complex and extensiveMay include multiple forms/pagesIn cases where there will not be a lot of routines, this can be combined into the Application-specific manipulation routinesTests layerThis layer will contain only testsNo test calls any other testTests call into the other layersTests should be very simple as all the complexity is implemented in the other layers General Data Flow Strictly Enforce

StandardsUsing consistent coding standards promoted reusability and maintainabilityStandards need to be documented and followed similar to what project development would doUse code reviews to enforce standards

Typical Coding StandardsNaming conventions for variables and methods/functionsUse human readable names!Limit code complexitySeparation of layersCommentsAutomation Coding StandardsDo not use the control type in the name, Address, not AddressEditNo duplication of control names for any specific window if the tools would normally allow thisConsistent naming of similar controls form to form, even if the forms label the controls differentlyAutomation Coding StandardsExecute anywhere tests need to written so that they do not rely upon a very specific environment configuration, unless that is the purpose of the testUnattended testing tests that need human interaction to execute should be segregated so that the other tests can all be run unattendedAutomation Coding StandardsNo hard coding use configuration files or similar mechanisms to store information or environment valuesIn tests, avoid using the automation tool syntaxTests should call into the other layersThis will make tests less likely to fail if the tool vendor modifies their syntaxCan make it easier to port tests to new automation toolsAutomation Coding StandardsOveruse loggingDesign for distributed testing and for concurrent execution of testsAn automated test will test just a single testNo test ever depends upon the successful actions or completion of a previous testComputers dont care if they do repetitive actionsStandard RoutinesBaseline (Initialization)NavigationEditingVerification

Tools already have their own routines defined for some of this however, you will get more flexibility and maintainability using these patterns

BaselineWill set the environment to the proper state to start a testThis state is defined for each AUT and test teamCan be simple to very complexDetermined by defining a known state that all tests can reliably execute and get consistent resultsNavigationUsed to determine if a page/form is active and ready for interactionWill navigate to the desired page/form if the steps required are not complex, or the steps will not alter the outcome of any testEvery page/form will have an unique navigation routines createdEditingEnters data into fields, pushes buttons, sets checkboxes, selects values in lists, etc.The control types should not matter when calling this routine it will figure out what the control type is during test execution and interact with the controls appropriatelyDefined as an unique routine for every individual page/formVerificationReads data from all controls typesDetermines if the values read result in a pass or failResponsible for logging resultsDefined as an unique routine for every individual page/formAreas we did not coverSolution layoutActual code examplesDealing with particularly difficult controlsConfiguration managementSkill setsSpecific tools capabilities

What we did coverBeginning automated testsTest framework goalsLayers of a frameworkCoding standardsInitial routines that every page/form requires

Questions?Thanks for your participation

Byron.goodman@neudesic.comCommon Layer

Generic UI Layer

Technology Specific Layer

Application Specific Layer

Application Test Layer

Request Flow

Response Flow

Common Layer

Generic UI Layer

Technology Specific Layer

Application Specific Layer

Application Test Layer

Request Flow

Response Flow