DevelopmentStrategies,FunctionDesignBBM101- Introduction toProgramming I

Hacettepe UniversityFall2015

FuatAkal,AykutErdem,Erkut Erdem,Vahid Garousi

Today

• Howtodevelopaprogram– Programdevelopmentstrategy

• MoreonTestingandDebugging– Testinganddebugging– Blackboxtesting

– Glassboxtesting– Integrationtestingandunittesting

– Debuggingapproaches

2

Today

• Howtodevelopaprogram– Programdevelopmentstrategy

• MoreonTestingandDebugging– Testinganddebugging– Blackboxtesting

– Glassboxtesting– Integrationtestingandunittesting

– Debuggingapproaches

Slides basedonmaterialpreparedbyRuthAnderson,MichaelErnstandBillHoweinthecourseCSE140University ofWashington

3

Programdevelopmentmethodology:Englishfirst,thenPython

1. Definetheproblem2. Decideuponanalgorithm3. Translateitintocode

Trytodothesestepsinorder

4

Programdevelopmentmethodology:Englishfirst,thenPython

1. DefinetheproblemA. WriteanEnglishdescriptionoftheinputandoutput

forthewholeprogram.(Donotgivedetailsabouthowyouwillcomputetheoutput.)

B. Createtestcasesforthewholeprogram• Inputand expectedoutput

2. Decideuponanalgorithm3. Translateitintocode

Trytodothesestepsinorder

5

Programdevelopmentmethodology:Englishfirst,thenPython

1. Definetheproblem2. Decideuponanalgorithm

A. ImplementitinEnglish• Writetherecipeor step-by-stepinstructions

B. Testitusingpaperandpencil• Usesmallbutnottrivialtestcases• Playcomputer,animatingthealgorithm• Beintrospective

– Noticewhatyoureallydo– Maybemoreorlessthanwhatyouwrotedown– Makethealgorithmmoreprecise

3. Translateitintocode

Trytodothesestepsinorder

6

Programdevelopmentmethodology:Englishfirst,thenPython

1. Definetheproblem2. Decideuponanalgorithm3. Translateitintocode

A. ImplementitinPython• Decomposeitintologicalunits(functions)• Foreachfunction:

– Nameit(importantanddifficult!)– Writeitsdocumentationstring(itsspecification)– Writetests– Writeitscode– Testthefunction

B. Testthewholeprogram

Trytodothesestepsinorder

7

Programdevelopmentmethodology:Englishfirst,thenPython

1. Definetheproblem2. Decideuponanalgorithm3. Translateitintocode

Trytodothesestepsinorder– It’sOK(evencommon)tobackuptoapreviousstep

whenyounoticeaproblem– Youareincrementallylearningabouttheproblem,

thealgorithm,andthecode– “Iterativedevelopment”

8

TheWishfulThinkingapproachtoimplementingafunction

• Ifyouarenotsurehowtoimplementonepartofyourfunction,defineahelperfunctionthatdoesthattask– “IwishIknewhowtodotaskX”

– Giveitanameandassumethatitworks– Goaheadandcompletetheimplementationofyourfunction,using thehelperfunction(andassumingitworks)

– Later,implementthehelperfunction– Thehelperfunctionshouldhaveasimpler/smallertask

9

TheWishfulThinkingapproachtoimplementingafunction

• Canyoutesttheoriginalfunction?– Yes,byusingastub forthehelperfunction– Oftenalookuptable:worksforonly5inputs,crashesotherwise,ormaybejustreturnsthesamevalueeverytime

10

Whyfunctions?Thereareseveralreasons:• Creatinganewfunctiongivesyouanopportunitytonameagroup

ofstatements,whichmakesyourprogrameasiertoreadanddebug.

• Functionscanmakeaprogramsmaller byeliminatingrepetitivecode.Later,ifyoumakeachange,youonlyhavetomakeitinoneplace.

• Dividingalongprogramintofunctionsallowsyoutodebugthepartsoneatatimeandthenassemblethemintoaworkingwhole.

• Well-designedfunctionsareoftenusefulformanyprograms.Onceyouwriteanddebugone,youcanreuseit.

11

Today

• Howtodevelopaprogram– Programdevelopmentstrategy

• MoreonTestingandDebugging– Testinganddebugging– Blackboxtesting

– Glassboxtesting– Integrationtestingandunittesting

– Debuggingapproaches

Slides basedonmaterialpreparedbyE.Grimson, J.Guttag andC.Terman inMITx 6.00.1x 12

TestingandDebugging

• Wouldbegreatifourcodealwaysworkedproperlythefirsttimewerunit!

• Butlifeain’t perfect,soweneed:– Testingmethods

• Waysoftryingcodeonexamplestodetermineifrunningcorrectly

– Debuggingmethods• Waysoffixingaprogramthatyouknowdoesnotworkasintended

13

Whenshouldyoutestanddebug?

• Designyourcodeforeaseoftestinganddebugging– Breakprogramintocomponentsthatcanbetestedanddebuggedindependently

– Documentconstraintsonmodules• Expectationsoninputs,onoutputs• Evenifcodedoesnotenforceconstraints,valuablefordebuggingtohavedescription

– Documentassumptionsbehindcodedesign

14

Whenareyoureadytotest?

• Ensurethatcodewillactuallyrun– Removesyntaxerrors– Removestaticsemanticerrors

– BothofthesearetypicallyhandledbythePythoninterpreter

• Haveasetofexpectedresults(i.e.input- outputpairings)ready

15

Testing

• Goal:– Showthatbugsexist– Wouldbegreattoprovecodeisbugfree,butgenerallyhard

• Usuallycan’trunonallpossibleinputstocheck• Formalmethodssometimeshelp,butusuallyonlyonsimplercode

“Program testing can be used to show the presence ofbugs, but never to show their absence!”

– Edsger Dijkstra

16

Testsuite

• Wanttofindacollectionofinputsthathashighlikelihoodofrevealingbugs,yetisefficient

– Partitionspaceofinputs intosubsetsthatprovideequivalentinformationaboutcorrectness

• Partitiondividesasetintogroupofsubsetssuchthateachelementofsetisinexactlyonesubset

– Constructtestsuite thatcontainsoneinputfromeachelementofpartition

– Runtestsuite

17

Exampleofpartitiondef isBigger(x, y):

“““Assumes x and y are intsreturns True if x is less than yelse False”””

• Inputspaceisallpairsofintegers• Possiblepartition

– xpositive,ypositive– xnegative,ynegative– xpositive,ynegative– xnegative,ypositive– x=0,y=0– x=0,y!=0– x!=0,y=0

18

Whythispartition?

• Lotsofotherchoices– E.g.,xprime,ynot;yprime,xnot;bothprime;bothnot

• Spaceofinputsoftenhavenaturalboundaries– Integersarepositive,negativeorzero

– Fromthisperspective,have9subsets•Splitx=0,y!=0intox=0,ypositiveandx=0,ynegative•Sameforx!=0,y=0

19

Partitioning

• Whatifnonaturalpartitiontoinputspace?

– Randomtesting– probabilitythatcodeiscorrectincreaseswithnumberoftrials;butshouldbeabletousecodetodobetter

– Useheuristicsbasedonexploringpathsthroughthespecifications– black-boxtesting

– Useheuristicsbasedonexploringpathsthroughthecode– glass-boxtesting

20

Black-boxtesting

• Testsuitedesignedwithoutlookingatcode– Canbedonebysomeoneotherthanimplementer– Willavoidinherentbiasesofimplementer,exposingpotentialbugsmoreeasily

– Testingdesignedwithoutknowledgeofimplementation,thuscanbereusedevenifimplementationchanged

21

Pathsthroughaspecificationdef sqrt(x, eps):

“““Assumes x, eps floatsx >= 0eps > 0

returns res such thatx-eps <= res*res <= x+eps”””

• Pathsthroughspecification:– x=0– x>0

• Butclearlynotenough

22

Pathsthroughaspecification

• Alsogoodtoconsiderboundarycases– Forlists:emptylist,singletonlist,manyelementlist– Fornumbers,verysmall,verylarge,“typical”

23

Example• Foroursqrt case,trythese:– Firstfouraretypical

• Perfectsquare• Irrationalsquareroot• Examplelessthan1

– Lastfivetestextremes• Ifbug,mightbecode,ormightbespec(e.g.don’ttrytofindrootifeps tiny)

x eps

0.0 0.000125.0 0.0001.05 0.00012.0 0.00012.0 1.0/2.0**64.01.0/2.0**64.0 1.0/2.0**64.02.0**64.0 1.0/2.0**64.01.0/2.0**64.0 2.0**64.02.0**64.0 2.0**64.0

24

Glass-boxTesting

• Usecodedirectlytoguidedesignoftestcases

• Glass-boxtestsuiteispath-completeifeverypotentialpaththroughthecodeistestedatleastonce– Notalwayspossibleifloopcanbeexercisedarbitrarytimes,orrecursioncanbearbitrarilydeep

• Evenpath-completesuitecanmissabug,dependingonchoiceofexamples

25

Exampledef abs(x):

“““Assumes x is an intreturns x if x>=0 and –x otherwise”””

if x < -1:return –x

else:return x

• Testsuiteof{-2,2}willbepathcomplete• Butwillmissabs(-1) whichincorrectlyreturns-1– Testingboundarycasesandtypicalcaseswouldcatchthis{-2-1,2}

26

Rulesofthumbforglass-boxtesting

• Exercisebothbranchesofallifstatements

• Ensureeachexceptclauseisexecuted• Foreachforloop,havetestswhere:

– Loopisnotentered– Bodyofloopexecutedexactlyonce– Bodyofloopexecutedmorethanonce

• Foreachwhileloop,– Samecasesasforloops– Casesthatcatchallwaystoexitloop

• Forrecursivefunctions,testwithnorecursivecalls,onerecursivecall,andmorethanonerecursivecall

27

Conductingtests

• Startwithunittesting– Checkthateachmodule(e.g.function)workscorrectly

• Movetointegrationtesting– Checkthatsystemaswholeworkscorrectly

• Cyclebetweenthesephases

28

TestDriversandStubs

• Driversarecodethat– Setupenvironmentneededtoruncode– Invokecodeonpredefinedsequenceofinputs– Saveresults,and– Report

• Driverssimulatepartsofprogramthatuseunitbeingtested

• Stubssimulatepartsofprogramusedbyunitbeingtested– Allowyoutotestunitsthatdependonsoftwarenotyetwritten

29

Goodtestingpractice

• Startwithunittesting

• Movetointegrationtesting• Aftercodeiscorrected,besuretodoregressiontesting:– Checkthatprogramstillpassesallthetestsitusedtopass,i.e.,thatyourcodefixhasn’tbrokensomethingthatusedtowork

30

“Most people, if you describe a train of events to them, will tell you what the resultwould be. They can put those events together in their minds, and argue from them thatsomething will come to pass. There are few people, however, who, if you told them aresult, would be able to evolve from their own inner consciousness what the steps werewhich led up to that result. This power is what I mean when I talk of reasoningbackwards, or analytically.“ -- Sherlock Holmes (A Study in Scarlet, by Sir Arthur Conan Doyle)

Debugging

31

Runtimebugs

• Overtvs.covert:– Overthasanobviousmanifestation– codecrashesorrunsforever

– Coverthasnoobviousmanifestation– codereturnsavalue,whichmaybeincorrectbuthardtodetermine

• Persistentvs.intermittent:– Persistentoccurseverytimecodeisrun– Intermittentonlyoccurssometimes,evenifrunonsameinput

32

Categoriesofbugs

• Overtandpersistent– Obvioustodetect– Goodprogrammersusedefensiveprogrammingtotrytoensurethatiferrorismade,bugwillfallintothiscategory

• Overtandintermittent– Morefrustrating,canbehardertodebug,butifconditionsthatpromptbugcanbereproduced,canbehandled

• Covert– Highlydangerous,asusersmaynotrealizeanswersareincorrectuntilcodehasbeenrunforlongperiod

33

Debuggingskills

• Treatasasearchproblem:lookingforexplanationforincorrectbehavior– Studyavailabledata– bothcorrecttestcasesandincorrectones

– Formanhypothesisconsistentwiththedata

– Designandrunarepeatableexperimentwithpotentialtorefutethehypothesis

– Keeprecordofexperimentsperformed:usenarrowrangeofhypotheses

34

Debuggingassearch

• Wanttonarrowdownspaceofpossiblesourcesoferror

• Designexperimentsthatexposeintermediatestagesofcomputation(useprint statements!),anduseresultstofurthernarrowsearch

• Binarysearchcanbeapowerfultoolforthis

35

Somepragmatichints

• Lookfortheusualsuspects

• Askwhythecodeisdoingwhatitis,notwhyitisnotdoingwhatyouwant

• Thebugisprobablynotwhereyouthinkitis– eliminatelocations

• Explaintheproblemtosomeoneelse

• Don’tbelievethedocumentation• Takeabreakandcomebacktothebuglater

36