Comparativeevaluationofparametricdesignsystemsforteachingdesigncomputation

RobertAishandSeanHanna

TheBartlettSchoolofArchitecture,UniversityCollegeLondon,GowerStreet,LondonWC1E6BT,UK

Note:thispaperhassubsequentlybeingpublishedinDesignStudiesasAish,R.,&Hanna,S.,Comparativeevaluationofparametricdesignsystemsforteachingdesigncomputation,DesignStudies(2017),http://dx.doi.org/10.1016/j.destud.2017.05.002

Abstract:Threeparametricdesignsystemsweretestedbytheauthorstoassesstheirsuitabilityforundergraduateteaching.Weusedcriteriatakenfromthe‘cognitivedimensions’literatureandanexerciseoftypicalgeometricoperationsinascendingorderofcomplexity.Foreachsystemthecognitivebarriersassociatedwiththesequenceofoperationswereplottedtocreatea‘learningcurve’.Differentparametricsystemspresenteddistinctlydifferentlearningcurves.Thetestexercisehadtobecompletedinitsentiretytoassessthepotentialchallengeswhichstudentswithdifferenteducationallevels,skillsandabilitiesmightencounter,soasingleexpertuserconductedthetests.Thisresearchisintendedtodevelopmethods,bothdesignexercisesandevaluativecriteriathatcouldbeusedinfutureempiricalstudies.

Keywords:architecturaldesign,designeducation,human-computerinteraction,parametricdesign,evaluation

Digitalmediaandworkingmethodsareconsideredtohaveapronouncedinfluenceondesignthinking(Oxman,2008),thereforeunderstandingthewayparametricsystemssupportparametricdesignthinkingisofcriticalimportanceforbothstudentsandeducators.

Studentswilldeveloptheirparametricdesignabilitythroughtheuseoftheseapplications.Indeed,thewaytheselectedsystempresentsitsfunctionalitymaywellbetakenbystudentsasthedefinitionofparametricdesign.Thereforetheinfluenceofparametricdesignsystemsonthestudentsandresponsibilitywhichgoeswiththismeansthatitisessentialthattheavailableapplicationsaresystematicallyevaluated.Oftenthechoiceofparametricsystemisinfluencedbyotherextraneousfactorssuchasthe‘platform’ortheapplicationsoftwareassociatedwiththeparametricsystem,wheretheplatformorapplicationmighthavealreadybeenselectedbytheuser’sinstitution.Similarlystudentswithpartialknowledgeofparametricdesignmaybeinfluencedtoselecttoolswithwhichtheyarealreadyfamiliarevenifthesesystemsmaynotbebestsuitedtolaterlearningstages.Inthisstudywehavedeliberatelyexcludedtheseextraneousfactorsandfocussedexclusivelyonasystematicevaluationofthedifferentparametricdesignsystems.

Learningratesmaydifferbetweenstudents.Differentparametricsoftwaremaybemoreorlesssuitedtodifferentparametricmodellingtasksandtodifferentstudents.Allthesedifferencesinteract.Whileitmaybepossibletoobservethistypeofparametricdesignlearninginformallyinaclassroomsetting,itwillbechallengingtodesigncontrolledempiricaltestsforstudentslearningtouseparametricdesignsoftwareforthefirsttime,sufficienttoprovideastatisticallyvaliddescriptionofthelearningprogress.Additionally,therearetheconsiderablechallengesinherentincoordinatingsufficientresourcesandappropriatestudentvolunteerstomakeinvestigationofnontrivialskilllearningpractical.Further,suchobservationsmaynotdirectlyexplaintheunderlyingreasonsforeaseordifficultyinlearning,whichareofinterestbothtosoftwaredesignersandeducators.Forthesereasonsweproposeanalternativeapproach,whichestablishesasetofrelevantcriteriaby

whichthesoftwarecanbeevaluated,aimingbothtolimitthesubjectivityofdifferentusersandtocorrespondtoparticularcognitivefactorswhichexplainpotentiallearningchallenges.Thisisintendedtoequipasingleuser,oftenanexpert,mostprobablywithsomeexistingbias,tomakethisevaluationwithsufficientobjectivity.Thesetwoapproaches,expertevaluationandempiricalstudies,canideallyinformoneanother,butatleastthefirstshouldbeexploredbeforethesecondanditisthefirstwhichisthetopicofthispaper.

Thepurposeofthisevaluationistoexplorethecognitiveissuesinvolvedwithparametricdesignsoftwarewhichwouldbeexperiencedbyanoviceuserratherthanthesubjectiveexperienceofstudentswithparticularbackgroundsorlevelsofskill.Thisevaluationinvolvedconstructingthesameabstractparametricgeometrymodelwiththedifferentsystemsandevaluatingthemodelbuildingprocesswithninecriteriadevelopedfromthe‘cognitivedimensions’literature.Thedesignofthetestexercise,thedevelopmentoftheevaluativecriteria,themodelbuildingactivitywiththedifferentsystemsandthereviewofthedifferentmodelbuildingprocesseswiththeevaluativecriteriahasbeendonebytheauthors.Theauthorshaveabackgroundindeveloping,usingandteachingparametricdesignanddesigncomputation.Theyalsohavesimilarlevelsofunfamiliaritywiththecurrentinterfacesandfunctionalityofthethreesystemstested.Expertsassociatedwiththethreesoftwaredeveloperswereconsultedbytheauthorstoensureequalknowledgeforeachsystem.

Inmanyapplicationsofparametricdesignforexampletoarchitecture,parametricmodellinginvolveoperationswhichcreateandusecollections.Thereforehowcollectionsarepresentedbyaparametricdesignapplicationtothedesigneriscruciallyimportant.Themodelbuildingexerciseinvolves:

1.Creatingofa2Darrayofpoints2.Creatingasurfaceusingthe2Darrayofpoints3.Creatingasetofcurvesthroughthepoints4.Creatingasetofcurvesthroughthetransposeofthearrayofpoints5.Creatingasinglecurvebymakinganarbitraryselectionofpointsfromthe2DarrayWhilethetestmodelisquiteabstractandonlyexercisesasmallsubsetofthefunctionalityoftheparametricdesignsystems,itisdifficulttoseehowthemoreadvancedfunctionalofaparametricdesignsystemcanbeharnessedwithouttheuserfirstbecomingproficientwiththisfunctionality.Thereforehowthisfunctionalityissupportedisaconvenientindicatoroftheoverallsuitabilityofthedifferentparametricsystems.

1Evaluativecriteria

Thecommonlyusedterm‘learningcurves’describescognitivechallengesoverthedurationofalearningprocess.Thisconceptofthe‘gentleslope’wasfirstintroducebyMacLean,Carter,Lovstrand,andMoran(1990)andthenfurtherdevelopedbyMyer(2002).Bothsuggestedthatinteachingcomputersciencetonovices,programminglanguagesandtoolsshouldbeselectedwhichpresenteda‘gentleslope’ofconceptsofgraduallyincreasingcomplexity.Itissuggestedthatthisapproachisalsovalidforteachingparametricdesign.[Figure1].

Whilealltheselearningcurvesmaybeidealisationstheyserveasawaytothinkabouttheoveralleducationalchallenge,thatis:howcanparametricandcomputationalconceptsbesimplifiedand

Figure1:Fivepossiblelearningcurvesmadeintuitiveforthenoviceuser,whilestillprovidingaconceptuallyvalideducationalfoundationfortheacquisitionofparametricandcomputationaldesignfundamentalsshouldthenoviceuserwishtoproceedtomoreadvancedcomputationaldesign.Myersuggestedthegentleslopeapproachmayhaveconsiderableadvantagesoverotherlearningcurves.

Thefollowingevaluativecriteriaarefeaturesofthesystemwhosepresenceorabsencemaycreatecognitiveorpracticalbarriersfornoviceusers,changingtheslopeofthecurve.TheyarebasedonGreenandBlackwell’sresearchintothe‘CognitiveDimensionsofInformationArtefacts’(1998)andonthecapabilitiesofmodellingandprogramminglanguageswhicharegenerallyconsideredtobefundamentalfortheireffectiveuse.

1.1Cognitivedimensions

‘Cognitivedimensions’describedifferentaspectsofacomputersystemwhichallowsorobligesausertothinkandactduringtheuseofasystem.Therearecomplexinterconnectionsbetweencognitivedimensionsandthecapabilitiesofthesystem.WeproposeanumberofadditionaldimensionswhichcombineorextendtheoriginalCognitiveDimensionsresearch.

1.1.1AbstractionBarrier

GreenandBlackwell(1998)offerthefollowingdefinition:‘Theabstractionbarrierisdeterminedbytheminimumnumberofnewabstractionsthatmustbemasteredbeforeusingthesystem.’Heretheemphasisisontheadditionalideasorwaysofworkingwhoserelevanceisnotcurrentlyappreciatedbytheuser,butwhichhavetobemasteredbeforethefunctionalityofinterestedcanbeaccesses.Theabstractionbecomesabarrieriftheuserisobligetounderstanditbeforetheabstraction’svaluetotheusercanbeappreciated.Theremaybevalidargumentsthatsomeabstractionbarriersareconfrontedearlyinthelearningprocessinordertominimisingdisruptionoverthecompleteprocess.

Ideallynewabstractionsshouldbediscoveredandappliedintheorderinwhichtheyappeartoberelevanttotheuser’sinterestandwherethedeltainunderstandingbetweenaknownabstractionandanunknownabstractioniswithintheuser’sabilitytocomprehend.Inaneducationalcontexttheaimisnottoavoidabstractionsbuttoavoidabstractionsbecomingabarrier.

1.1.2Semanticinterference

Thisisanadditionalcognitivedimensionwhichbuildsonthe‘clearnames’designpatternproposedbyWoodbury(2010)asacritical‘elementofparametricdesign’,asfollows:‘Goodnamesareclear;theyconveywhatyouintend.Theyaremeaningful;usuallythismeanstheyrelatetoeithertheformorfunctionofadesign.Theyareasshortastheyneedtobe(andnoshorter)’.If,asWoodburysuggests,thatusersofparametricdesignapplicationsshouldbeextoledtouse‘clearnames’thenthedesignoftheseparametricdesignapplicationsshouldalsouse‘clearnames’.

Semanticinterferenceoccurswhenthereisamismatchbetweenatermandthemeaningtobeconveyed.Aparametricdesignapplicationmayuseatermwithaparticulardomain-specificorvernacularmeaningwhichthenoviceusermaybefamiliarwithandthereforethenoviceusermightacceptthetermanditsmeaningasdefinitive.Inaneducationalcontextthissemanticsmayinterferewiththeobjectivesoftheinstructorwhomaywanttousetheestablishedconceptuallydefinedterminologywithprecisemeaning.

Semanticinterferencemayalsooccurwhenthesameterminologyhasmultiplemeaningsinthesameordifferentpartsofthesystemorwhenmultipletermsareusedforthesameconceptorfeatureofthesystem.Theoriginsofsomespecialisedterminologiesareessentiallymetaphorswhichhavebecomeestablished.Endusercomputingsystemsoftenintroduceun-establishedmetaphors(Barr,2003).Wecandescribea‘metaphortrap’asaspecialformofsemanticinterferencewhereaninappropriatemetaphorisusedbytheapplicationtodescribeanunderlyingcomputingconceptandthenaturallanguagemeaningassociatedwiththetermdoesnotdescribetheprecisemeaningorgeneralityoftheconcept,sothatthenoviceuserisunawareofthefullscopeofthefunctionalitybeingreferredtobythemetaphor.

Considerthefollowingprogression:

Generalcomputing:

Iwanttotransposethearrayofnumbers[preciseterminologydescribinganabstractoperationbeingappliedtoabstractdata]Domainspecificcomputing[retainingcomputingabstractions]:

Iwanttotransposethearrayofbeams[preciseterminologydescribinganabstractoperationappliedtodomainspecificterm:beams]Anexampleofmetaphorbaseddomainspecificcomputing[wherearraysareimplementedasatreedatastructureandoperationontreesusevernacularmetaphors,suchas‘flip’]:

Iwanttoflipthetreeofbeams[thevernacularmetaphor‘fliptree’combinedwithdomainspecificterm‘beams’resultsinahybridlanguagewhichisneitheravalidcomputationalexpressionnorunderstoodasavalidintheapplicationdomain.Notonlyisthisconfusingbutitmayalsomaskthefunctionalityoftheabstractoperation]

1.2Propertiesofparametricdesignsystems1.2.1ConsistencybetweenrepresentationsMostparametricdesignsystemoffermultiplerepresentationsincludingavisualgraphbaseddataflowrepresentation,geometricrepresentationandsometimesatextbasedprogramrepresentation.However,tohelptheuserbuildaunifiedinternalmentalmodelitisimportantthatthereisconsistencybetweenthesedifferentrepresentations.Specificallyfortheexerciseconsideredwhere,istheapparentgeometricorganisation[a2Darrayofpoints]reflectedinthelogicalstructuringofthedatainthegraphbasedvisualprogrammingenvironment?TheimportanceoftheconsistencyofmappingbetweenrepresentationsspecificallywithparametricdesignapplicationshaspreviouslybeendiscussedbyAishandWoodbury(2005)andHarding,Joyce,Shepherd,andWilliams(2012).1.2.2DiscoverabilityThisdescribeshowthefunctionalityofthesystemispresentedanddocumentedsothatitcanbediscoveredbytheuserunaided.Iftheeducationalintentistoteachstudentusersabouttheunderlyingconceptsincomputationandgeometry,thenitmightbeappropriatetouseaclassificationsystembasedonsomeclearconceptualbasis.Forexample,thegeometryfunctionalitymaybeclassifiedasanobject-orientedclasshierarchyusingthe‘dimensionality’ofdifferentgeometrytypes[0Dforpoints,1Dforcurves,2Dforsurfacesand3Dforsolids].Theconceptof‘type’andclasshierarchy(fromgeneraltospecific)allowsthenoviceusertounderstandwhatfunctionalityiscommonandwhatfunctionalityisuniquetodifferenttypesofgeometryorotherdomainspecificaspectsoftheapplication.However,withoutanyoveralllogic,thenoviceuserisforcedtoconsumeadditionalcognitiveresourcestodirectlylearntheidiosyncrasiesofthemenustructure.Thisrepresentsaninvestmentonthepartoftheusernotingenerallytransferrableknowledge(ofthelogicalclassificationofgeometryandotherparametricandcomputationalconcepts)butinthespecificsofaparticularparametricdesignsystem.Thisthencreatestwodisincentivesfortheusertomovetoadifferentparametricdesignsystem:abandoningtheinvestmentinonesystemandinvestinginlearninganothersystem.1.2.3FlexibilityOneofthekeyissuesinthedesignofacomputersystemistheflexibilityitofferstheuser.IntheoriginalCognitiveDimensionsresearch,GreenandBlackwell(1998)definethreedifferentdimensionswhichtodescribetheconsequencefortheusertoflexibility:First,doesthesystemrequiringtheusertoperformactions(andthereforetothinkaboutthoseactions)inaninappropriateorder?[PrematureCommitment].Second,doesthesystemallowtheusertomaketentativedecisionswhichcanbesubsequentlychanged?[Provisionality].Third,howdifficultisittomakethesesubsequentlychanges?[Viscosity].Overallparametricdesignapplicationsbasedonvisualdataflowprogrammingdataflowareextremelyflexiblecomparedtomodellingapplicationbasedondirectmanipulation.Whiledirectmanipulationsystemsofferhighlevelsofflexibilityduringinitialsketch,changingthesemodelsis

oftenextremelyarduousandoftenrequireallorsubstantialpartsofthemodeltobedeletedandfortheuser‘tostartover’.Theprincipleadvantageofparametricdesignsystemsmostfrequentlyreferredtobyusersisthecapabilitytorevisitandchangepreviousmodellingoperationsandtheconsequencesofthesechangesareautomaticallypropagatedthroughthemodel,withouttheuserhavingtodeleteandtomanuallyremodelling.Therearealsocriticismsthatoncebuilt,complexparametricmodelsaredifficulttochangeandthisinhibitsdesignexploration(Davis,Burry,&Burry,2011).ThesecommentsreinforceearlierconclusionsfromBurnettetal.(1995)thattherearescalingandusabilityissueswithvisualprogramming.Forexampleanodeinadataflowgraphcombines:thename,the‘type’,andthecalculationmethodusedtocreateitsvalue.Whiletypeandmethodmaybeinterdependent,inaregulartextbasedprogramminglanguagetheuserisfreetochangeanyoneoftheseaspectsindependently.Inanodebasedsystemtheseoptionsareoftennotavailableforcingtheusertocreateanewnode,thentomovetheconnectionsfromtheoldnodetothenewnodeandfinallytodeletetheoldnode.Aclearexampleof‘viscosity’.1.2.4SideeffectsThisisanaspectofthefunctionalityofthesystemweresomeminimalchangebytheuser(forexample,totheinputdata)hasawiderangingandunexpectedeffectonthebehaviourofthemodelorprogram(forexample,ontheoutput).Thisisaslightlydifferentandextendedinterpretationtothatusedincomputerscience.1.2.5WorkaroundsThisismodificationoradditionaloperationswhichtheuserisobligedtoaddtothemodelorprogram,whichfromtheuser’sperspectiveisneitherpartofthedesignintentandnorappearstobelogicallyrequired.Workaroundsmayalsoberequiredtocircumventaprevioussideeffect.Workaroundsmayrequiretheusertounderstandnewabstractionswhichareunrelatedtotheuser’scurrentinterestandthereforeworkaroundsarelikelytointroduceabstractionbarriers.Workaroundsareconsideredfragile,becausetheyaredevelopedinresponsetosomelimitationsintheoriginalsystem.Tofunctioncorrectlytheworkaroundisnowdependentonthatlimitation.Ifthesystemiscorrectedthentheworkaroundmaynolongerberequiredanditscontinuedpresencemaygivethewrongresult,whichtheusermaybepotentiallyunawareof.1.2.6ConvolutedworkflowConsiderthesituationwheretherequiredfunctionalityissupportedbytheapplication.Thefunctionalityisdocumentedandnonewabstractionsarerequiredtobelearnt.Neverthelesstocompletethetasktheuserhastoadoptsuchacomplexworkflowthatthewholeprocessappearstobecounterproductiveanddiscouragingtothepointwherethetask[andhencetheapplication]mightbeabandoned.1.2.7LivenessLivenessisaconceptborrowedfromtheperformingartstodescribethespontaneityandresponsivenessofaperformance.Livenessisalsousedasatermtocompareliveandrecordedperformanceandtheroleoflifeperformance(Auslander,2008).

Inuserorientedcomputingapplications‘Liveness’isusedtodescribethesystem’sperformanceandsupportforinteractivity.Indiscussingtheconceptof‘Liveness’,perhapstheclosestcomparisontoparametricdesignsystems[usedforarchitecturaldesign]areDigitalMusicsystems[usedforcompositionandperformance].Inthiscontext,Livenesshasbeendescribedas‘aqualityofthedesignexperiencethatindicateshoweasyitisforusertogetanimpressionoftheendproductduringintermediatestagesofdesign’(Nash&Blackwell,2014).Inthecaseof‘useroriented’computingapplicationssuchasparametricdesignsystems(asexamplesofinteractivecomputing)‘Liveness’isassociatedwithaspectsofprogramperformanceanduserinteraction,suchas:• Modelessinteraction-Istheuseraware(ornotaware)ofdistinctmodesofoperation?Theuser

maynotbeawareofsuchmodesif,forexample,thesystem’sdefaultmodeis‘continuousexecution’.

• Latency-Howquicklydoesaprogramrespondtouserevents?• Dynamics-Cantheusercontrolthe‘quality’ofprogramdynamics,forexamplebydetermining

thetrade-offsbetweenthecomplexityandcompletenessofthemodelwhenthisisbeingrecomputedinreal-time.

• Directnessofinteractions-Howdoestheuserinteractwiththesystem,forexample,indirectlybyusingakeyboardtochangethenumericvalueofinputparameters,viaancillaryanalogueinteractionsdevicessuchasslidersorbydirectmanipulationofthegeometrywithintheuser’smodel?

2Testresults

Theexercisesrequiretheapplicationofdifferentgeometricandlogicalconcepts.Thereaderisencouragedtoimaginehowtheinstructorcanmaintainacoherentnarrativeexplainingthevariousparametricandcomputationalconceptswhileatthesametimeexplainingthefunctionalityandterminologyusedbythedifferentsystemstoimplementtheseconcepts.

2.1GenerativeComponents[version08.11.09.288]

WithGenerativeComponents,thecompleteexercisewaseasilyanddirectlyachievedin10nodes[Figure2]withoutanyunusualfunctionality,terminologyorworkarounds.Theonlyproblemwasa‘discoverability’issue,associatedwithtransposingthepointarray.Thisisbecausethereisno

‘Transpose’nodeavailableinthevisualprogrammingenvironment.InsteadtheTransposeofthepointarray[Point01]isencapsulatedinanexpressionwithintheinputportofthecurve01[Figure3].

• AbstractionBarrier:Generallynoabstractionbarriers,butchallenginginonecasewhereascriptexpressionhadtobeused(fortheTransposefunction).Whilethisabilitytousescriptexpressionsmaybeextremelyusefulforanexperienceduser,itmaynotbesuitableforanoviceuser.Thisisbecauseitrequiresthenoviceusertolearnaboutscriptnotationveryearlyintheuseofthesystem.Ideally,thewholesetofexercisesshouldbecompletedusingVisualprogrammingnodesandthenonlysubsequentlyshouldscriptingbeofferedasamoreadvancedoption.

• SemanticInterference[includingmetaphortrap]:Good,therewerenoproblemswithterminology.

Figure2:ThecompletedexerciseinGenerativeComponents

Figure3:Transposingthepointarrayusingascriptedexpressioninthe‘PointSet’inputportoftheCurvenode

• Consistencybetweenrepresentations:Good,includingcrosshighlightingbetweengeneratedgeometryandgraphnode

• Discoverability[offunctionality,includinglogicalordering]:Good,themenuisclearandeasilynavigable[Figure2].Thereisanattempttoprovidetheuserwithan‘object-oriented’descriptionofthefunctionalityofthegeometrylibrary.Whenthecursorhoversovereachofthe‘toplevel’iconsrepresentingdifferentgeometrytypes,a‘fly-over’labelappearswhichdocumentsthedifferentinterfacesimplemented.However,thereisnoself-discoverabledocumentationtodescribethemethodswhicheachoftheinterfacesimplements,thereforethepotentialpedagogicadvantageofexplainingthefunctionalityofthegeometrylibraryin‘object-oriented’termsisnotcompletelyrealised.Inadditiontherewerechallengingaspects,forexamplethe‘Transpose’function[theuseofwhichwasanessentialaspectofthetasks]isnotavailableasagraphnode.Itisavailablewithinthescripteditor.Howeverthisisnotdocumentedandthereisno‘auto-completion’inthescripteditortoofferthisandothermethodstotheuser.Sothereisnowayforthenoviceusertodiscoverthisimportantfunctionality.

Figure4:The‘S’curvewascreatebyselectingpointsfromthepointarraydirectlyinthegeometrywindow.Notetheidentityofthepointselectedisdisplayedaspartoftheflyoverlabelwiththeindexingintothe2Dpointarray.Thecorrespondinggraphnodeishighlighted.Thecollectionexpressioncodefragment[inthescripteditor]isbeingbuiltfortheuserfromtheinteractionwiththegeometrymodel.Astheuserselectsafurtherpointsothecollectionexpressionisautomaticallyextendedwithanewmember

Figure5:Onceaspecifictypeofgraphnodehasbeencreated,themethod[or‘technique’]usedtoconstructthegeometrycanbechangedbyselectingfromalistofavailablemethods.Thisallowstheusertoexperimentwithdifferentmethods,withouthavingtodeletedandrecreatethenodeanditsconnections.Thisaddsconsiderablytotheeaseofchangingthemodel

• Flexibility[ofediting,minimisingreworking]:Good,thenodemethodcanbechangedwithouthavingtodeleteandrecreatethenode[Figure5].AlsothedesignofthegraphnodeallowstheusertoreferencetheXYZpropertiesofthepointnodebyaddingoptionaloutputportsratherthanbycreatingadditionalnodes[Figure6].Nonewnodehadtobeaddedtothegraphtoexposetheseproperties.

• Sideeffects:nonedetected• Workarounds:nonedetected• Convolutedworkflow:nonedetected.Thetaskcouldbecompletedwith10nodes.• Liveness[includingdirectinteractionwithgeometry]:Good:Thereistheabilitytodirectlyselect

geometry(forexample,tobuildanewcollectionbyselectinggeometryfromanexistingcollection)[Figure4].Thereisalsotheabilitytodirectlyinteractwithgeometryinthemodelandautomaticallyupdatethegraph.(Forexample:movingpointsalongtheaxesofcoordinatesystems,alongcurves,oronplanesoronsurfaces).

2.2Grasshopper[version0.9.0076]

WhileitwaspossibletocompletethetasksinGrasshopper,theexercisewasmademoredifficultbytheuseof‘datatrees’astheprincipleimplementationofcollections.Thisrequiredunderstandingtheratherunusualfunctionalityandterminologyinvolved.Itbecameapparentthatwhicheverwaythepointswerecreated,thenoviceuserwouldhavetolearnadditional‘workarounds’tocompletethetasks.

Oneapproachistocreateatree[or2Dcollection]ofpoints[Figure7].Thisismostprobablythelogicalapproachwhichmanyuserswouldtakesincethe2Dstructureofthedatacorrespondstothe2Dspatiallayoutofthepointgeometry.

Howevertocreatethesurface,theuserhastoaddanadditionaloperationto‘flatten’the2Dcollectionintoa1Dlistbecausethesurfacecreationnoderequiresa1Dlistasinput.Effectivelytheuserishavingtoprovidethisadditional‘flatten’operation(orworkaround)toundothehardcoded‘unflatten’operationwhichisbuiltintothesurfacecreationnode.Workingoutwhatishappeninginsidethesurfacenodeandthenhowtocircumventitsbuilt-in‘unflatten’functionalitywaschallenging.Analternativeapproachistocreatealist[or1Dcollection]ofpoints[Figure8].Thiscanbedirectlyinputintothesurfacecreationnode,butthenoviceusernowhastousethe‘PartitionList’workaroundtorestructurethepointsbackintoa2Dcollectioninordertocreatethecurves.

• AbstractionBarrier:Datatreesareanimportant‘abstractdatatype’whichsupportsaclearlydefinedsetoperations.Itcanalsobeusedtoimplementothercollectiontypessuchaslistsandarrays.TheissueofdatatreeisdiscussedatlengthbyRuttenincludingadiscussionaboutotherdatastructuresfoundinhighlevelprogramminglanguagesandthereuses.EffectivelytheuseofdatatreeinGrasshopperisanimplementationconveniencewhichisexposedasanend-usermetaphor.However,fromanenduser’sperspective[bothinstructorandstudent]theterminologyof‘trees’maynotbeausefulmetaphortoexplainorharnesstheconceptofarrays.ThequestionremainswhethertheintentionbehindtheuseofdatatreesinGrasshopperisdrivenbyimplementationconsiderationsorwhethertheintentionsarepedagogic(Rutten,2015).Wecametotheconclusionthattoomanycognitivecycleswouldhavetobespentexplainingtherelationshipbetweenthemetaphor[tree,branches,etc.]andtheunderlyingabstractions

[collections,arrays]andinparticularhavingtoexplainthattheunderlyingabstractionshavecharacteristicsthatgobeyondthemetaphor.Ifitisanticipatedthatthestudentswillprogressbeyondvisualprogrammingtoscriptingandprogrammingthentheywillhavetolearnaboutarraysandindexinganyway,thereforeitmightbearguedthattheseconceptsshouldbepresented‘upfront’tothestudents.Thisisnottodisputethatdatatreesisapowerfulabstractionwhichisincrediblyvaluablewhencorrectlyappliedtodatawhichisgenuinely‘tree-like’.Butinthiscontextdatatreesbecomesanunnecessaryabstractionbarrierandametaphortrapanditpresentstheuserwitharepresentationwhichisinconsistentwiththeuser’sconceptualisationofthedata.

Figure7:Creatingthepointsasatree,effectivelya2Darray

Figure8:Creatingthepointsasalist,effectivelya1Darray

• SemanticInterference[includingmetaphortrap]:Challenging:ThemostimportantissuewhenvaluatingGrasshopperistheuseofdatatreestogetherwiththerelatedterminologyusedtodescribeoperationsondatatrees,suchasBranching,Grafting,Flattening,Path,Flip,etc.Thesetermsareusedtoconstructanuncomfortablevernacularmetaphorwhichmaskstheunderlyingarrayconcepts.Thinkingabouttheneedsofthenoviceuser,itmighthavebeenpreferabletodirectlyexposetheconceptandterminologyofthe‘array’orevenaconsistent‘listoflists’concept.OtherterminologyinGrasshopperappearedmisalignedwiththeunderlyingconcepts.Forexample,theterm‘crossreferencing’inordinaryusageappliestoaninstancewithinadocumentwhichreferstorelatedinformationelsewhereinthesamedocument.However,inGrasshopperitisusedtoimplyaformofcombinatorialexpansion,whereanewoutputlistAiscreatedbycopyingtheoriginalinputlistAonceforeachmembersintheoriginalinputlistBANDanewoutputlistBiscreatedbycopyingtheoriginalinputlistBoncefortheeverymembersintheoriginalinputlistA.Theterm‘Crossreference’doesnotseemtobeanappropriatedescriptionoftheunderlyingprocess.Overall,theterminologyishighlyvernacularandmetaphoric.Theiconstakethemetaphorofthe‘tree’tonearvisualexcess.Theproblemisthattheapparentsimplicityofmetaphormaskssomecomplexfunctionalityandthereforethevalueofthemetaphorasan‘intuitivelead-in’tothisfunctionalityforthenoviceusermaybelost.

• Consistencybetweenrepresentations:Challenging:Anadditionalproblemwiththelistapproachisthatthedatastructure(a1Dlist)doesnotmatchthegeometricstructure(2Dconfigurationofpoints)[Figure8].Theuserhastheadditionalcognitiveloadofunderstandingthedifferentlogicalandspatialrepresentationsandtranslatingbetweenthetwotounderstandthecorrespondencebetweenthedataandgeometry.Becausethepointsarenowasinglelist,theuserhastoaltertheindexingtouseasingleindexratherthantherowandcolumnindicesusedwiththe2Dcollection,whenselectingpointswithwhichtocreatethe‘S’curve.

• Discoverability[offunctionality,includinglogicalordering]:Themenusarereasonablywell-structuredhowevertherearesomeawkwardclassificationsforexampleField,Grid,Plane,Point,andVectorcanonlybefoundunderVectortab.

• Flexibility[ofediting,minimisingreworking]:Historically,Grasshopperwasthefirstvisualprogrammingenvironmentwheretheuserdirectlycreatedandinteractedwiththegraph,asopposedtothegraphbeinggeneratedasaby-productofotherinteractions.Howevertherearealsochallenges.Forexampleitisnotpossibletochangethemethodusedbyanode.Tochangethemethods,thenodehastobedeleted.Alltheconnectionstothatnodearelost.Thenodehastobere-createdandtheconnectionsre-established.AlsotoinspecttheXYZcoordinatepropertiesofthepointnode,anadditionalnodehadtobeadded[Figure9].

• Sideeffects:none

• Workarounds:requiredtomitigateconvolutedworkflow,seebelow.

• Convolutedworkflow:Challenging:Asaconsequenceoftheuseofdatatrees,thereisnosingleapproachtothecreationofthepointswhichcanbeusedbothtocreatethesurfaceandthecurves.Twomethodswereexplored:themethodinFigure7required16nodesandthemethodinFigure8required13nodes.

• Liveness[includingdirectinteractionwithgeometry]:Good/Challenging:Whiletheoverallperformanceofthesystemindynamicsisgood,itwouldbepreferableiftherewasbetterintegrationwiththehostapplication,specificallygeneratedgeometryisnotlocatable,exceptif‘baked’andthenifitis‘baked’,itcannotbere-generated.Ideally,allgeometryshouldbelocatableandre-generatableandtheusershouldnotbeawareofthedistinctionbetween‘baked’and‘unbaked’geometry.

Figure9:ToinspecttheXYZpropertiesofapointnode,a‘deconstruct’nodemustbeaddedtothegraph

2.3Dynamo[Version0.8.0.950]

Note:thisanalysishasbeenretestedonVersion1.1.0.2094.Itwaspossibletocompletethemodellingtask,butthereweresignificantchallengeswiththefunctionalityandterminologyofDynamo[Figure10],asfollows:

• AbstractionBarrier:Challenging:Becauseoftheissueswiththedimensionalityofcollections,(seeSideeffectsandWorkaround,below)newabstractionshavetobeintroducedsuchas‘normaliseddepth’whichcouldotherwisebeavoided.

• SemanticInterference[includingmetaphortrap]:Challenging:Thereareoccasionswhenitmightbepreferabletouseterminologywhichismorepreciseandmoreconsistent.Forexampleinthe‘Sequence’node[Figure11],thesecondargumentis‘Amount’andthethirdargumentis‘Step’,buttheexplanation(inthe‘flyover’label)usestheword‘Space’.Dynamousestheterm‘lacing’,whiletheunderlyingDesignScriptlanguageusestheterm‘replication’[Figure12].Itmightbepreferabletouseasingleconsistentterm.Whiletheunderlyingreplicationfunctionalitycancreated‘lacingpatterns’,itcanalsocreatemorecomplexpatternthanthetermlacingmightsuggest.Lacingisessentiallyametaphorthatmaybemaskingthisextendedfunctionality.

InadditionDynamoUIusestheterm‘CrossProduct’todescribethegenerationoftheproductsetbetweendifferentinputssets[Figure12].Whiletheuseoftheterm‘crossproduct’isnotincorrect,themorewidelyacceptedtermis‘CartesianProduct’.

Figure10:Creatinga2Dpointcollection.ThisusesasequenceofXcoordinatesandasequenceofYcoordinatesandthedefaultvalue[0.0]forZ.This2Dpointcollectioncandirectlybeusedtocreatethecollectionofcurves,orasurface[notshown].Note:theZcoordinateisnotdefinedandthedefaultvalueofzeroisused

Figure11tocreatethearrayofpoints,theusermustfirstcreateanumbersequence.Itmightbepreferabletousethemorepreciseterm‘length’ratherthanthemoreambiguousterm‘amount’andtousethemorepreciseterm‘increment’ratherthantheterms‘step’and‘space’

Figure12:InDynamotocreatethe2Darrayofpoints,theuserneedstocombinethesequenceofXcoordinateswiththesequenceofYcoordinatesusingthe‘CrossProduct’option.[See‘SemanticInterferencesection]

Notonlyistheterm‘CartesianProduct’usedintheunderlyingDesignScriptlanguage,butthereisaDynamoNodecalledList.CartesianProduct.Dynamoalsoimplementsthe‘CrossProduct’vectoroperation.SoitwouldappearthatthecreationoftheproductsetisseparatelyreferredtoasaCrossProductandasaCartesianProductandthetermCrossProductisusedtodescribebothasetoperationandavectoroperation.Itmightbepreferableinaneducationcontexttohaveaone-to-onemappingbetweenterminologyandfunctionality.

• Consistencybetweenrepresentations:Challenging:InFigure13thereisamismatchbeenthe

dimensionalityofthecollectionofpoints[3D]andthevisualrepresentation[2Darray].• Discoverability[offunctionality,includinglogicalordering]:Challenging:BoththeList

functionalityandtheGeometryfunctionalityarepresentedwithinconsistentmenustructures.Forexample,the‘GetItemAtIndex’[the‘get’method]isfoundundermenu/core/List(togetherwith54othermethods),whilethematching‘insert’method[whichinsertsanitemintoalistatanindex]isfoundundermenu/BuiltIn(togetherwith33otherlistmethods)[Figure21].Also,thesameterm(Geometry)isboththemainandsub-menuname,butthismenustructuredoesnotcommunicatetheessentialfunctionalclassification:thatCurve,SurfaceandSolidareallsubclassesofGeometry,andthatLine,ArcandCircleareallsubclassesofCurve,sharingcommonpropertiesandmethods.

• Flexibility:Itisnotpossibletochangethemethodusedbyanode.Tochangethemethods,the

nodehastobedeleted.Alltheconnectionstothatnodearelost.Thenodehastobere-createdandtheconnectionsre-established.Whileasingle‘watch’nodecanbeusedtoinspecttheXYZcoordinatepropertiesofthepointnode,threeseparatenodesfortheX,YandZpropertieshadtobeadded[Figure20]inordertoaccessthesevalues.

Figure13:CreatinganarrayofpointsusingacollectionofXcoordinatesandacollectionofYcoordinates.InFigure10,novaluewasdefinedfortheZcoordinate,thedefaultvalueofzeroisusedanda2Darrayofpointswascreated.InthiscaseasingleZcoordinateisdefined[withthevaluezero],butthebehaviouraltersanda3Dcollectionofpointsiscreated.ThereisnochangeinthevalueoftheZcoordinate[whichiszero]justachangefromusingthedefaultvalueorexplicitlydefinedvalue.Thischangehascreatedanunexpectedsideeffect• Sideeffects:Challenging:Duringthecreationofthearrayofpointsanimportantunanticipated

sideeffectoccurred.InFigure10,whenthe2DpointarrayisinitiallycreatedthereisnoexplicitvaluedefinedfortheZcoordinate.Insteadthebuilt-indefaultvalue[0.0]isused.However,inFigure13,ifavalueisexplicitlyprovidedfortheZcoordinate[eventhevalue0.0whichisthesameasthedefaultvalue]thenthereplicationstrategy‘recognises’threevariablesasinputstothePointnodeandthereforecreatesa3Darrayofpoints.Thespatialconfigurationisa2D,butthedatastructureisa3Darray.Thisnotonlygivesan‘inconsistencybetweenrepresentations’(notedabove)butresultsintheCurvecreationnodehavingthewrongdimensionofinputwhichthenfails.SocomparingFigures10and13,wecanseethattheuser’sactionofsimplyconnectingtheZcoordinatenodetothePoint.ByCoordinatesnode,witheffectivelynochangeinvaluefortheZcoordinate,changesthedimensionalityoftheresultingoutput.Theestablishedreplicationstrategyinotherapplications[suchasGenerative-Components]orindeedintheunderlyingDesignScriptlanguage[withinDynamo,seeFigure15]doesnotaddanextradimensiontotheoutputcollectionifasinglevalueinputisdetected,butonlywhenacollectionisdetected.SotheregularnodesinDynamoappeartobepresentingadifferentreplicationbehaviourtothatavailableintheunderlyingDesignScriptlanguage.ThisissueisnowthesubjectofadiscussionontheDynamodiscussiongroups(Dynamoissue#6528,2016)

• Workarounds:Challenging:Tocorrectforthissideeffect,theuserhastointroduceaworkaroundtoreducethedimensionalityofthepointarraybackto2D[Figure14]usingthe‘normaliseddepth’node.Whiletheconceptthatanarraycanbe‘flattened’inherentinthe‘normaliseddepth’nodeisimportant,requiringitsuseinthiscontextpresentsanunfortunateandunnecessaryabstractionbarriertothenoviceuser.Figure16showsanalternativewayto

createthearrayofpointsusingacodeblocknodeandaDesignScriptexpressionwiththeunderlying‘Point.ByCoordinatesmethodandreplicationguides(seeFigure17).

Figure14:InDynamo,inordertocorrectforthesideeffect[inFigure13]theuserhastoadda‘workaround’,whichisto‘normalisethedepth’ofthepointarray.Butthisintroducesanadditionconcept‘normalisedepth’forthenoviceuser.Havingtounderstandthisadditionalconceptmightbecomean‘abstractionbarrier’forsomenoviceusers

Figure15InDynamotheuseraddsatransposenodeinordertocreatethealternativesetofcurves

Figure16:AnalternativewaytocreatethearrayofpointsistouseacodeblocknodeandaDesignScriptexpressionwiththeunderlying‘Point.ByCoordinatesmethodusingreplicationguidesandtocontrolthereplicationbehaviour

Figure17:Byselectingdifferentreplicationguides,eitherx,yorx,y,theusercancontrolthewaythe2Darrayofpointswillbebuiltandthereforethewaycurveswillbebuiltfromthearrayofpoints

Figure18:InDynamousingtheavailablenodes,toselectpointstocreatethe‘S’curve,theuserhastorepeatacomplextwostageselectionprocess,firstselectingthesublistfromthe2Dcollectionandthenselectingthespecificpointfromthesublist.Thereisnomulti-index[orpath]selectionnodetoselectanitemfromamulti-dimensionalcollection.13nodesarerequiredtoselect4points.

Figure19:InDynamo,andusingDesignScriptwithinacodeblocknode,itispossibletoselectpointsfromacollectionusingindices.Inthiscasetheuserhastohandconstructthiscodefragment

Figure20:ToinspecttheXYZpropertiesofapointnode,threeseparatenodesmustbeaddedtothegraph

Figure21:The‘Insert’functionfor‘List’isfoundunder:‘BuiltinFunctions/Insert’submenu[andusestheterm‘element’],whilethecorresponding‘Get’functionfor‘List’isfoundunder‘Core/List/Action/GetItemAtIndex’submenuandusestheterm‘item’

• Convolutedworkflow:seeLiveness,below.

• Liveness[includingdirectinteractionwithgeometry]:Challenging:Intheabsenceofanyinteractionwiththegeneratedgeometry,selectiontasksmaybecomeextremelyarduous.Forexample,tocreatethe‘S’curve[Figure18],theuserhastorepeatacomplextwostageselectionprocess,firstselectingthesublistfromthe2Dcollectionandthenselectingthespecificpointfromthesublist.Thereisnomulti-index[orpath]selectionnodewhichcanbeusedtoselectanitemfromamulti-dimensionalcollection.Thiseffectivelyrequiredtheusertodefine13nodesjusttoselect4inputpointsforthe‘S’curve.Analternativewaytoselectthepointsforthe‘S’curve,istouseaDesign-Scriptcollectionexpressionwithindiceswithinacodeblocknode[Figure19].However,theuserhastohandconstructthiscodefragment.ThisisessentiallyexactlythesamecodefragmentwhichisusedinGenerative-Components[Figure4],exceptthatinthecaseofGenerativeComponentsthiscodefragmentisbuiltautomaticallybytheapplicationinresponsetotheuserinteractivelyselectingpointsinthegeometrywindow.ToinspecttheXYZpropertiesofthepointnode,threeadditionalnodehadtobeaddedtothegraph[Figure20]comparedtoonenodeinGrasshopper[Figure9]andnonodesinGenerativeComponents[Figure6].

3AnalysisTheevaluationofthethreeparametricdesignsystemsisformallycomprisedofthestatedcognitivecriteriadetailedabove,whichexcludeinfluencessuchasinstitutionalpreferencesandpriorexperience,variationsintheabilityandpreferencesofthestudents.TheevaluationissummarisedinTable1andvisuallypresentedinFigure22.ThelearningcurvesinFigure22arepurelyindicativeandusethesamevisualconventionsintroducedbyMyer(2002).Thenormallearningactivityischaracterisedbytheinclinedlineandthecognitivechallengesarerepresentedbyverticallines.Theheightoftheverticallineindicatesthecognitivechallengesateachstageinthetestexercise.Howeverthisisnotmeanttobepreciselyquantifiedbecausedifferentstudentswithvaryinglevelsofinterest,abilitiesandperseverancemayreactdifferentlytothesechallenges.Thesecognitivechallengesmaybeclassifiedas:‘Absolutebarriers’

1. Therequiredfunctionalitydoesnotexist[theoreticallynoneoftheapplicationsfailedtoprovideallthefunctionalitybecauseanexperienceduserwasabletocompletethetests.However,therewereoccasionswitheachofthethreeapplicationswhereevenanexperienceduserwouldhavefailedtocompletethetestswithoutthedirectguidanceandinterventionofexpertsfromtherespectivesoftwarevendors].

2. [Discoverability]Therequiredfunctionalityexistsbutisundocumented,thereforethefunctionalitywouldnotbediscoveredbyanoviceuser[forexample:theTransposefunctioninGenerativeComponents]

‘Effectivebarriers’:theoreticallythenoviceusershouldbeabletocompletethetask.Whethertheuserovercomesorsuccumbstothesebarriersmaydependonotherindividualandcontextualfactorsinfluencingtheuser’scommitmentandperseverance.Bothfromaneducationalandpracticeperspective,itwouldbeessentialtoeliminateorreducethesebarriers.

3. [Discoverability]Therequiredfunctionalityexistsandisdocumented,butinsuchanillogicalwaythatthenoviceuserisunlikelytofind[forexample:ListmethodsinthemenustructureinDynamo]

4. [Abstractionbarrier]Therequiredfunctionalityisnotdirectlyaccessiblebutamoreexperienceusermightbeabletoinferor‘reverseengineer’thefunctionalityoutofotherfeaturesofthesystem(whichisunlikelyforanoviceuserasthiswouldpresentadditionalabstractionbarrier).[forexample:GrasshopperDataTreescouldbeusedtoemulatearrays]

5. [Sideeffects]Thefunctionalityexist,butcreatesunexpectedsideeffects(whichrequireadditionalconceptstobeunderstood,potentiallyintroducingadditionalabstractionbarriers)[forexample:undersomeunexpectedconditionsDynamoreplication(orlacing)changesthedimensionofthegeneratedcollection]

6. [Workaround]Aworkaroundtothesideeffectsexists[butrequireadditionalconceptstobeunderstood,againpotentiallyintroducingadditionalabstractionbarriers][forexample:Dynamo‘flatten’functionality,usedtoaddressthe‘dimensionality’sideeffect,mentionedabove]

7. [Convolutedworkflow]Therequiredfunctionalityexistsandisdocumentedbuttocompletethetaskrequiresaconvolutedworkflow[forexample:Dynamorequired13nodestoselect4points]

’Incorrectpedagogy’

1. [Semanticinterference]Therequiredfunctionalityexistsbutisdescribedbyinappropriatetermsthereforeleadingtoanincorrectpedagogy.Thisisparticularlyimportantinaneducationalcontext.[forexample:Dynamo’suseoftheterm‘CrossProduct’,whentheterm‘CartesianProduct’mightbemoreappropriate]

Table1:Comparativecognitivebarriersforthethreeparametricdesignsystems,forexercises1-5

Figure22:Thelearningcurvesforthethreeparametricdesignsystem

4ConclusionsTheimportantroleofparametricdesignapplicationsistopresentparametricdesignconceptstodesignersandforthedesignerstobeabletousetheseapplicationstoexpressparametricdesignthinking.Inthispaperwehaveusedastandardmodellingexerciseandasetofcriteriainspiredbythe‘cognitivedimensions’researchtoevaluateimportantconceptualandusabilityaspectsofthreeofthemainparametricdesignsystems.Ourownimmediatepurposeinthisevaluationwastohelpselectsystemswhichwouldbesuitableforundergraduateeducationinparametricdesignthinking.Thebroaderaimwastoestablishameansbywhichanacademicorindustryprofessional,typicallyanexpertuser,canevaluateandanticipatetheeasewithwhichnoviceuserwouldlearntouseaparametricdesignsystem.Thisevaluationnecessarilypre-datedanyparticularparametricdesigncourseandwascarriedoutbytheauthors.Whiletheoriginalintentionofthestudywastoselectasuitablesystemforteachingparametricdesign,thisstudyhasuncoveredsomemajorconceptualandusabilityconcernswithalltheavailableparametricdesignsystems.Manyoftheseconcernscouldeasilyberesolvedbysimplyexposingor‘repackaging’theunderlyingfunctionality,bychangestotheuserinterfacedesign,orevenbyquiteminorchangestoterminology.Itmaybenotedthatthisstudydoesnotincludeempiricalobservationofactualnoviceuse,andparticularlymoreformalexperimentswithrealstudentgroups.Thereareseveralrelatedreasonsforthis.Inpractice,novicestudentaptitudevariesconsiderably,makingreliableobservationofthebarrierstolearningdifficultwithoutquitelargestatisticalsamples.Further,adescriptionofsuchpointsofdifficultyinamodellingexercisedoesnotexplaintheunderlyingcognitivecause.Itisjustsuchanexplanationthatwehaveattemptedbygroundingthisevaluationentirelyinthecognitivedimensionsgiven.Thislackofempiricalexperimentnecessarilylimitstheconclusionsofthisstudy,inthatrealstudentexperienceswouldberequiredtoproperlytestthedetailsoftheevaluationmethod:whethertheparticularcognitivedimensionsusedbestrepresentrealstudentsamples,whetherdifferencesinbackgroundorbiasresultindifferentcurves,etc.Assuch,theresultsmustremaingenericatpresent,referringtobroaddifferencesbetweenthesoftware.Weanticipatefutureobservationsofactualstudentuseovertimewouldfunctionbothasatestoftheparticularchoiceofcognitivedimensionsbymakingqualitativeandquantitativecomparisonstotheobservedlearningcurvesshownhere,andtoallowextensionofthescopeofconclusions.Indeedifthereweretobeopportunitiesforfutureempiricalstudiesintothewayparametricdesignthinkingissupportedbyavailableparametricdesigntools,thenitishopedthatthesestudieswouldbeconcernedwithmorethanbasicusabilityissuesasreportedhere,andwouldbeabletofocusonmoreinterestingandsubstantiveconceptualandpracticalissues.Itishopedthatfeedbackfromsuchfutureempiricalstudieswillplayanimportantroleininformingfuturesoftwaredesigndecisionsanddesigneducationalstrategies.Inthisstudythemodellingexercisewaschosentopresentcommonandessentialparametricmodellingconceptsandoperationsofincreasingcomplexityparticularlyrelevanttoarchitecturaldesign.Asreportedearlier,theindicativelearningcurvesfromtheevaluationofthethreeparametricdesignsystemssuggestthreequitedifferentlearningprofiles.Insomecasesthecognitivechallengesoccuruniformlyoverthemodellingtask;inothercasesthecognitivechallengesareconcentratedattheearlierandlaterphases.Thedifferencesbetweenthesecurvescouldbeusedtoselectasuitableparametricdesignsystemforuserswhohavedifferentlevelsofskill,ortoplanteachingstrategies,ortoanticipatewherestudentsmightexperiencedifferentcognitivedifficulties.

Itisalsorecognisedthatdifferentexercisesmightresultindifferentlearningcurves.Forexample,incaseswheretheuseandconceptualunderstandingof‘collections’isnotconsideredanessentialaspectofparametricdesignthinking,thenamodellingtaskcouldbeconstructedwhichexcluded‘collections’.ThismightremovesomeofthelearningbarrierspresentintheinitialstagesoftheGrasshopperandDynamocurvesandlearningthesesystemsbynoviceusersmightbefarmorerapid.Afurtherinvestigationofhowthesecurvesmightchangewithexerciseswhichusedifferentparametricdesignconceptsmaywellshedlightonwhyparticularparametricdesignsystemsarechosenindifferentinstitutionalcontextsorbydifferentusercommunities.Therearealsoimportantconclusionstobemadeabouttheevaluativemethodsdevelopedforthisstudy,andhowthesemightinformfuturesoftwaredesign.Astheoriginalauthorsofthe‘cognitivedimension’researchsuggest(Green&Blackwell,1998),thereisaneedforpracticalusabilitytoolsbywhicheverydaysoftwaredevelopersandeducatorscanassesscognitively-relevantsystempropertiesandidentifyimportantsystemdesigntrade-offs.Therefore,weshouldconsidertheevaluationmethodsproposedhereasthestartofanopendiscussiontofurtherdevelopandrefinewaystomeasurethesuitabilityofparametricdesignsystems.AcknowledgementsTheauthorswouldliketoacknowledgetheassistanceof:MarcThomasinthereviewofGenerativeComponents.ArthurMamou-ManiinthereviewofGrasshopper.AndreasDieckmanninthereviewofDynamo.AlanBlackwellindiscussionsaboutCognitiveDimensions.TheReviewerswhoprovidedvaluablefeedbackwhichcontributedtotherefinementofthepaper.Thisresearchdidnotreceiveanyspecificgrantfromfundingagenciesinthepublic,commercial,ornot-for-profitsectors.ReferencesAish,R.,&Woodbury,R.(2005).Multi-levelinteractioninparametricdesign.InA.Butz,B.Fisher,A.Kruger,&P.Olivier(Eds.),Smartgraphics,5thInternationalSymposium.Springer.Auslander,P.(2008).Liveness:Performanceinamediatizedculture(2nded.).Routledge.Barr,P.(2003).User-interfacemetaphorsintheoryandpractice.VictoriaUniversityofWellington.http://www.pippinbarr.com/academic/Pippin_Barr_MSc_Thesis.pdfBurnett,M.M.,Baker,M.J.,Bohus,C.,Carlson,P.,Yang,S.,&VanZee,P.(1995).Scalingupvisualprogramminglanguages.Computer,28(3).ftp://ftp.cs.orst.edu/pub/burnett/Computer-scalingUp-1995.pdf.http://web.engr.oregonstate.edu/wburnett/Scaling/ScalingUp.htmlorDavis,D.,Burry,J.,&Burry,M.(2011).Theflexibilityoflogicprogramming.InC.Herr,G.Ning,R.Stanislav,&M.Schnabel(Eds.),CircuitBending,BreakingandMending:Proceedingsofthe16thInternationalConferenceonComputerAidedArchitecturalDesignResearchinAsia(pp.29e38).Australia:TheUniversityofNewcastle.http://www.danieldavis.com/the-flexibility-oflogic-programming-parametrically-regenerating-the-sagrada-familia/Dynamoissue6528.(2016).Point.ByCoordinates(X,Y,Z)inconsistentbehaviourusingcrossproductlacing#6528.https://github.com/DynamoDS/Dynamo/issues/6528Dynamo’http://dynamobim.org/