©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Test Automation: Agile Enablement for Data Warehousing and Business Intelligence Teams
PresentedtotheSo>wareQualityAssocia4onofDenver(SQuAD)March8,2016Denver,ColoradoLynnWinterboer,AgileAnaly4csEducatorandCoach,@AgileLynnBradEwald,DW/BISolu4onArchitect,@JBEwald
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Agenda
• Whyistestautoma4onimportantforagiledatateams?
• Whyaren’talldatateamsautoma4ngtheirteststoday?
• Whatisthepathtotestautoma4on?
• WhatdoessimpleDWtestautoma4onlooklike?
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Why is test automation important to agile data teams?
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Agile Demands Something Different Agilecallsforsmallincrementsof“poten4allyshippablecode”.ThatmeansQAisessen4alonaregular,frequentbasis.
Howcanwebesurethatsomethingwedo
here
Doesn’tbreaksomethingwedidback
here
Test Suite Test Suite Test
Suite Test Suite Test
Suite Test Suite Test
Suite
Manualtes4ngquicklybecomesinfeasible.Thetestsuitegrowslargereachitera4on.
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Testing is Central to Agile
AgileBIdevelopmentisdrivenbytests
Acceptancecriteriaarethedefini4onof“done”
Passingtestsarethemeasureof“done”
Regressiontestsarethemeasureof“s4lldone”
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Agile Teams Understand…
…thattestautoma4onis akeytechnicalenabler
to“beingagile”
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Why aren’t all data teams automating their tests
today?
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
DW/BI Test Automation Challenges Dataindustrydoesnotfocusontes4ngasmuchasotherdisciplines:
Educa4onTeamMembers
Skills&Discipline
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
DW/BI Test Automation Challenges Exis4ngtoolsarenoteasilyadaptedtothedataworld:
DevelopmentLanguages
WebandMobileFocus
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
DW/BI Test Automation Challenges Goodtestdatasetsarehardtocomeby:
Largevolumes(onsmallenvironments)
SharedtestenvironmentsDatasensi4vity
Smallsetsneedextraplanninganddesign
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
What is the path to test automation?
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Agile Testing Perspectives
HighQualityDW/BISystem
3) Tools & Automa/on
1) A2tudes & Culture
2) Skills & Prac/ces
Tes4ngisaTeamSportLearningCultureBuildQualityIn
Tes4ngSkillsTestDesign,Organiza4on&MaintenanceRepeatableTestsFeedbackLoopsBusinessDomain
TestEnvironmentBuildAutoma4onTestDataSetsTestAutoma4onTool
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Test Automation Pyramid
AppTests
AcceptanceTests
Unit&ComponentTests
ManualTests
Lisa Crispin & Janet Gregory, Agile Tes6ng: A Prac6cal Guide for Testers and Agile Teams
Pushasmanytestsaspossibletothislayer
Writetheseinbusinessdomainlanguage
BrifleteststhatareaffectedbyUIchanges
Subjec4veuserfeedbackisessen4al
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Where to Start?
Wheredoesithurtthemost?
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Where Does It Hurt the Most? Painfultestsetup?
Deployinganewbuild?
Breaksinproduc4on?
Regressiontes4ng?
Tes4ngnewfeatures?
Pronetomanualtesterror?
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Where to Start?
Whatisthesmallest,simplestwayyoucouldrelievethatpain?
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
What does simple data warehousing test automation
look like?
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com©KenCollier,2016,AllRightsReserved
Basic Approach
Sta4cTestData
ExpectedResults
TestQueries
loadtestdata
compareresults
executetests
Componentundertest
SourceSchema
TargetSchema
ActualResults
TestData
TestRunner
Setup Execute Test Report CleanUp
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Demonstration
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Testing Framework Func4onality• Screensfordescribingandcrea4ngtestcases• Proceduresthatexecutetestsandcreateexpected
results• BIReportsandDashboardsformonitoringComponents• SQLServerandRelatedModel• Microso>Access(inputscreens)• SQLServerStoredProcedures(testexecu4on)• PowerShellscripts(batchexecu4on)• MSPowerBIforRepor4ng
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
What Constitutes a Test?
ActualResult(orDataset)
ExpectedResult(orDataset)
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Testing Demo
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Session Intent
1)Why: TestAutoma9oniskeytoDW/BIagilesuccess
2)How: Automateasolidtes9ngprac9ce;don’tautomatechaos
3)Show: DW/BItestautoma9onisnotthathard
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
Thank You!
Lynn Winterboer Agile Analy/cs Educator & Coach
www.lynnwinterboer.com [email protected]
@agilelynn
Brad Ewald DW/BI Solu/on Architect [email protected]
@jbewald
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
DW/BI Test Automation Tools Tool Descrip,on
DBFithfp://dbfit.github.io/dbfit/
Opensourcedatabasetes4ngtool
iCEDQhfp://icedq.com
AunifiedplajormforETLtes4nganddatamigra4ontes4ng
Analy,xDataServiceshfp://analy4xds.com
EnterprisedatamappingandETLcodeautoma4on,includingnewautoma4cgenera4onofreusabletestcasesassociatedwithSourcetoTargetMappings
WhereScapehfps://www.wherescape.com
Datawarehouseautoma4onso>ware,includingatestharnessframeworkwithuser-configurableprofilingmethods.
Tricen,sToscahfp://www.tricen4s.com/tricen4s-tosca-testsuite/
DW-friendlytestautoma4ontool
©2016byWinterboerAgileAnaly4cs�www.WinterboerAgileAnaly4cs.com
DW/BI Test Automation References Reference Descrip,on
AgileAnaly*cs:AValue-DrivenApproachtoBusinessIntelligenceandDataWarehousingbyDr.KenCollier
Chapter7:Test-DrivenDataWarehouseDevelopment
AgileDataWarehousingfortheEnterprise:AGuideforSolu9onArchitectsandProjectLeadersByRalphHughes
PartV:AgileEDWQualityManagementPlanning
AutomatedDataWarehouseTes*ng:Beginner’sStep-by-StepGuidebyG.Suden
Java-basedtestautoma4onforsimpledatawarehousingtests
AgileTes*ng:APrac9calGuideforTestersandAgileTeamsbyLisaCrispinandJanetGregory
Greatintroduc4ontoagiletes4ngforanytypeofdevelopment
MoreAgileTes*ng:LearningJourneysfortheWholeTeambyLisaCrispinandJanetGregory
Includesseveralchaptersontestautoma4onaswellasonededicatedtoagiletes4ngforDW/BI