RealOptionsDecisionFrameworkforResearchandDevelopment:ACaseStudyonaSmall
CanadianHigh-TechnologyStart-Up
by
SallyJamalMattar
Thisthesisissubmittedinpartialfulfillmentoftherequirementsforthedegreeof
MasterofScience
in
EngineeringManagement
DepartmentofMechanicalEngineeringUniversityofAlberta
©SallyMattar,2017
ii
Abstract
ResearchandDevelopment(R&D)projectscanbeboth innovativeandhighlyuncertain.Allowing
for managerial flexibility and adopting real options methods and incorporating technology
readiness level scales to assess the maturity of technologies progressing through development
stages,canhelpmanagershedgetheunforeseenrisksthatariseduringthesestages.Managerscan
makedecisionsthatavoidthedownsideandcapturetheupsideoftheserisks.Thismethodologyis
adecision-making framework forresearchorganizations,wherepotentialvaluesofdecisionsand
projects across aportfolio canbe evaluated.Theproposed framework analyzes the risks as they
progressthroughthetechnologyreadinesslevelscale,andenablesR&Dmanagementtoplayactive
roles in project evaluations and justify continued spending on risky, long term projects that are
expectedtobeofhighfuturevalue,anareawheretraditionalvaluationmethodsfallshort.Acase
studyon a small Canadian technology start-up is used todiscuss the importanceof adopting the
proposedmethodology.
iii
Preface
ThisthesisisoriginalworkbySallyMattar.Thecasestudy,whichispartofChapter4ofthethesis
received research ethics approval from the University of Alberta Research Ethics Board, Project
Name“Realoptionsanalysisapplication”,Pro0073487,July20,2017.Thisworkwasnotpublished
anywhereelseanddoesnotcontaincollaborativework.
iv
Acknowledgements
IwouldliketothankmysupervisorDr.LipsettforgivingmetheopportunitytocompleteaMasters
inEngineeringManagement.Iamgratefulforallhissupportandguidancethroughoutthecourseof
mydegreeandIhaveenjoyedworkingwithhimimmensely.Itwasalwaysapleasuretochatwith
himandhealwayshadsomethingintelligentandinterestingtosay.Hewasacontinuoussupport
andwithouthim,thisprojectcouldnothavebeencompleted.
Iwould also like to thankDr. Lianne Lefsrud. Ever since taking her class in the Fall semester of
2016,shehasbeenaninspirationandrolemodel,andIaspiretobe likeheroneday.Shealways
offered support and gave valuable advice when needed. She also provided guidance on the
structureandorganizationofthisthesisandIamverygratefulforallherhelp.
v
TableofContents
Abstract.....................................................................................................................................................................................iiPreface......................................................................................................................................................................................iiiAcknowledgements.............................................................................................................................................................ivTableofContents..................................................................................................................................................................vListofFigures.....................................................................................................................................................................viiiListofTables..........................................................................................................................................................................ixChapter1Introduction.......................................................................................................................................................11.1ThesisMotivation&BusinessCase.......................................................................................................................11.2ObjectiveofThesis........................................................................................................................................................21.3ThesisStructure.............................................................................................................................................................4Chapter2ReviewoftheLiterature...............................................................................................................................52.1R&DandTechnologyDevelopment......................................................................................................................52.1.1R&DandTechnologyDevelopment...................................................................................................................62.1.2ExamplesofTechnologyDevelopmentModels.........................................................................................102.1.3TechnologyPush–MarketPull........................................................................................................................132.1.4CapabilitiesofaFirm............................................................................................................................................142.2TechnologicalMaturity............................................................................................................................................142.2.1TechnologyReadinessLevelsOverview.......................................................................................................142.2.2TRLs:CharacteristicsatEachLevel................................................................................................................162.2.3OtherTRAToolsandTechniques....................................................................................................................222.2.4LimitationsofTRLs................................................................................................................................................232.3TechnologyRiskManagement..............................................................................................................................242.3.1RisksinR&D&TechnologyDevelopment...................................................................................................242.3.2RiskManagementinR&D...................................................................................................................................302.3.3RiskManagementTools&Techniques.........................................................................................................312.4FinancialValuationMethods.................................................................................................................................332.4.1TraditionalValuationMethods.........................................................................................................................332.4.2FinancialOptionsTheoryOverview...............................................................................................................362.4.3Black-ScholesModel..............................................................................................................................................372.4.4BinomialModel........................................................................................................................................................382.5RealOptionsValuation.............................................................................................................................................412.5.1IntroductiontoRealOptions:ValueasaDecision-MakingTool........................................................41
vi
2.5.2RealOptionsValuation:FinancialOptionsvs.RealOptions................................................................432.5.3TypesofRealOptions...........................................................................................................................................452.6RealOptionsAnalysis...............................................................................................................................................482.6.1ApplyingRealOptions:TheSteps....................................................................................................................482.6.2Monte-CarloSimulations.....................................................................................................................................502.6.3RealOptionsinIndustry......................................................................................................................................51Chapter3DevelopmentoftheROADecisionFramework...............................................................................543.1FrameworkMethodology.......................................................................................................................................543.2Phase1:CollectionofData&InitialPlanning................................................................................................563.3Phase2:TechDevelopmentStages-gates&TRLAssessment................................................................573.4RisksinTechnologyDevelopment......................................................................................................................623.5Phase3:RealOptionsAnalysis.............................................................................................................................693.5.1Base-CaseDiscountedCashFlow&SensitivityAnalysis......................................................................693.5.2Monte-CarloSimulation.......................................................................................................................................703.5.3RealOptionsProblemFraming&OptionValuation................................................................................72Chapter4ApplicationtoCaseStudy:CopperstoneTechnologies................................................................764.1Phase1:DataCollection&Background...........................................................................................................764.1.1CompanyBackground:CopperstoneTechnologies.................................................................................774.1.2InterviewwithBusinessDevelopmentManager......................................................................................784.1.3InformationfromRelevantSources...............................................................................................................794.2Phase2:CurrentTRLandCriticalRisks...........................................................................................................804.2.1ARCurrentTechnologicalMaturity................................................................................................................814.2.2CriticalRiskFactors...............................................................................................................................................834.3Phase3:ROAApplication.......................................................................................................................................864.3.1ROAProblemSet-up..............................................................................................................................................864.3.2Base-caseDCF..........................................................................................................................................................874.3.3SensitivityAnalysis&Monte-CarloSimulation.........................................................................................894.3.4ROValuationatTRL4...........................................................................................................................................924.3.5ROValuationatTRL6...........................................................................................................................................974.3.6SensitivityAnalysisforExercisedOption..................................................................................................1084.3.7OtherCaseStudyConsiderations..................................................................................................................1094.4.NationalResearchCouncilofCanadaCaseStudy.....................................................................................1104.4.1NRCCaseBackground........................................................................................................................................110
vii
4.4.2RecommendationsforFutureNRCPortfolios..........................................................................................111Chapter5Conclusion.....................................................................................................................................................1135.1Conclusion...................................................................................................................................................................1135.2FrameworkLimitations.........................................................................................................................................1165.3RecommendationsforFutureResearch.........................................................................................................117References...........................................................................................................................................................................119Appendix1SupportingInformationforLiteratureReview..........................................................................1291.1DefinitionsofTRLs..................................................................................................................................................1291.2GOARiskAssessment.............................................................................................................................................1311.3PotentialRisksinR&D...........................................................................................................................................132Appendix2CaseStudySupportingInformation................................................................................................1342.1CopperstoneTeamBios.........................................................................................................................................1342.2ActivitiesTimeline...................................................................................................................................................1362.3CopperstoneTechnologiesFinancials.............................................................................................................1372.4SampleofInterviewQuestionswithBusinessDevelopmentManager............................................1372.5SummaryfromRelevantThesisDocument..................................................................................................138Appendix3RealOptionsAnalysisSupportingCalculations.........................................................................1403.1Base-CaseDCF...........................................................................................................................................................1403.2SensitivityAnalysis..................................................................................................................................................1413.3BuildingCapabilitiesOptionValuation...........................................................................................................1443.4MarketPivotOptionValuation...........................................................................................................................1453.5MonitoringServicesOptionValuation............................................................................................................1463.6SalesOptionValuation...........................................................................................................................................1473.7LicensingOptionValuation..................................................................................................................................148Appendix4EthicsApproval........................................................................................................................................149
viii
ListofFigures
Figure1.EREStagesofDevelopmentModel..............................................................................................................11
Figure2.DoD'sTechnologyStagesofDevelopment..............................................................................................11
Figure3.LeeandGartner’s.DevelopmentModel.....................................................................................................12
Figure4.Tritleetal.StagesofDevelopment..............................................................................................................13
Figure5.Technology-Push&Market-Pull....................................................................................................................13
Figure.6DoDTRAProcess.................................................................................................................................................21
Figure7.TechnologyRiskAcrossTRLsforHighTechnologies.........................................................................27
Figure8.3x3ProbabilityImpactMatrix.......................................................................................................................32
Figure9.GenericBinomialLattice..................................................................................................................................39
Figure10.ThestructureofRealOptions....................................................................................................................42
Figure11.FrameworkSetup.............................................................................................................................................56
Figure12.StagesofTechnologyDevelopmentAgainstTRLs.............................................................................57
Figure13.FrameworkSetupofStage-gates&RealOptions...............................................................................60
Figure14.GenericSADTModel........................................................................................................................................61
Figure15.CriticalRiskFactorsforR&D.......................................................................................................................62
Figure16.CriticalRiskFactorsDuringStagesofDevelopment.........................................................................64
Figure17.Riskprofileofmajorrisksduringtechnologydevelopment.........................................................66
Figure18.Resourcerequirementsduringtechnologydevelopment..............................................................67
Figure19.PossibleRealOptionsConsideredatTRL4..........................................................................................68
Figure20.Monte-CarloStochasticUncertaintyModeling....................................................................................71
Figure21.BinomialLatticeExample.............................................................................................................................73
Figure22.CSTCurrentTRLPositioning.......................................................................................................................81
Figure23.TornadoDiagramRangeofNPVResults................................................................................................91
Figure24.DecisionTreeatTRL4...................................................................................................................................93
ix
Figure25.MarketPivotLatticeValuation...................................................................................................................95
Figure26.BuildCapabilitiesLatticeValuation..........................................................................................................96
Figure27.DecisionTreeatTRL6.................................................................................................................................100
Figure28.MonitoringLatticeValuation.....................................................................................................................102
Figure29.LicensingLatticeValuation........................................................................................................................104
Figure30.SalesLatticeValuation..................................................................................................................................106
Figure31.PotentialRisks.................................................................................................................................................132
ListofTables
Table1.NASATechnologyReadinessLevels............................................................................................................15
Table2.R&D3LevelsandDescriptionsandTNVandDescriptions..................................................................20
Table3.MajorR&DRisksinLiterature.........................................................................................................................25
Table4.DCFAssumptionsvsDCFrealities.................................................................................................................34
Table5.FinancialOptionsvs.RealOptions................................................................................................................43
Table6.RisksandOptionsTypes....................................................................................................................................47
Table7.ExampleofPossibleEffectsRiskonProfit.................................................................................................85
Table8.Base-caseDCFResults.........................................................................................................................................87
Table9.DCFSensitivityAnalysisResults.....................................................................................................................90
Table10.SummaryofCommercializationOptionsProsandCons...................................................................98
Table11.TRLSoftwareDefinitions..............................................................................................................................129
Table12.TRLHardwareDefinitions............................................................................................................................130
Table13.GOATRASteps...................................................................................................................................................131
1
Chapter1Introduction
1.1ThesisMotivation&BusinessCase
There is a lack of simple tools that can be used to guide and value decision-making processes
associated with technology development, and provide managers with the flexibility to make
decisionsthatcancaptureopportunitiesduringatechnologyorresearchanddevelopment(R&D)
project.Thereisaneedforastreamlinedmethodologythatcanaiddecision-makersindevelopinga
setof rubricswhereacommonunderstandingofhowcritical riskelementscan influenceproject
success for technology andR&Dprojects at different stages. Themethods need to be adapted in
ordertomodelhowexpendituresatearlystagesofR&DcanleadtodiscoveriesthatbenefitCanada.
Researchers,managers,anddecision-makers inR&Dandtechnologyorganizationsarechallenged
withhow to evaluate thepotential business benefits of early stageprojects and assess the value
that these technologies will bring to the firm compared to the costs associated with their
development[1].R&Dandtechnologyprojectsneedtoolsthatmanagerisksandallowmanagersto
make decisions accordingly as conditions change and information is gained [2]. This is the
motivationbehindtheresearchconductedforthisthesisasthereisacurrentgapwherethereare
missing tools that reflect thedynamicnatureofR&Dandcanbeapplied for these longandrisky
projects. The work conducted during this research will work to fill those gaps by developing a
decision-frameworkthatutilizesastructuredapproachtotechnologydevelopmentandevaluation.
Understandingthepotentialvalueofatechnologyisusefulformanagersthatarechoosingbetween
projects and trying to understand the benefit of committing financial and human resources to a
project, and for the researchers that try to structure these projects tomaximize the value of the
potential technology [3]. R&D and technology projects can be long and unpredictable making it
challengingformanagerstoforecasttheeffortrequiredtodevelopthesenewtechnologies[4]and
their likelihood of success once pursued. These uncertainties may lead to high risks and could
result inproject failures [5].The currentlyusedvaluationmethods suchasdiscounted cash flow
(DCF)andnetpresentvalue(NPV)arenotsufficienttoolstousefortheseprojects.
2
These traditional methods assume there is only one possible route to achieve project goals or
targetsandplacemanagersinpassiverolesthatassumethatinitialpredictionsmadeandresource
commitmentsestablished,cannotbechangedevenifconditionsrequirethemtodoso[6].Thislack
ofmanagerial flexibility couldundervalueaproject [7] suchasR&Dor technologydevelopments
thathave longerhorizonsanddonothave immediatepayoffs.Organizationsmayoverlook these
projectsas theymayhavenegativenetpresentvalues [8],neglecting thepotential futurebenefit
and the opportunity to grow [9]. In addition, organizations who rely exclusively on financial
methods to assess projects are less successful at developing new products than those that also
consideredqualitativeaspects[3].Thesetraditionalapproachesusestandardinvestmentdecision-
makingthatsolelydependsonprofitcreationwhichisnotusefulforR&Dfirmswhoseultimategoal
maynotbetocreateprofit[10].Forexample,somegoalsmaybetocreateaproductimprovethe
environment,asopposedtomarketandselltothegeneralpublic.
For these reasons discussed, a methodology must be developed to overcome these gaps in
technology development management and valuation. The details behind the proposed approach
andresearchobjectivesarediscussedinthefollowingsection.
1.2ObjectiveofThesis
The objective of this research is to develop a framework that allowsmanagers, researchers, and
decision-makers to exercise flexibility through decision-making and value their technology
investments.Theframeworkwillcombinethetechnologyreadinesslevelscale(TRL)andastage-
gate approach thatwill be the logical pointswhere a project’s activities, requirements and risks
changeandshouldbereassessed.Thiswillsupportdecision-makingthroughaconsistentmethod
thatwillassessatechnology’smaturityandprogress.Theframeworkwillutilizearealoptions(RO)
approach by recommending a small number of options to be placed along the TRL scale where
managerial decisions can be evaluated through real options analysis (ROA) and encourage
management to make evidence-based decisions for technology development and project
continuation.
3
TheapplicationofTRLsandtheROmethodcanenablemoreconsistentdiscussionsonhowrisks
changefromonephaseofaprojecttoanother,andthecomparativevalueofdifferentdecisions,or
differentprojectsacrossaportfolio.Thisapproachcanbevaluablefororganizationsasitcanhelp
exploreopportunitiesandjustifyspendingonlongerandriskierprojectswithpotentiallyveryhigh
futurevaluestoCanada.
“Realoptions”,anextensionoffinancialoptiontheory,referstoadecisionpertainingtoatangible
asset as opposed to stock. It refers to the ability of managers to exercise flexibility through
decisions such as growth, delay, or abandonment of a project as technology, financial,market or
other conditions change [11]. This active decision-makingmethod can improve the likelihood of
scientific success and the positive commercial benefit to Canada. Real options will allow
organizations to establish key decision points during R&D stages for projects across a research
program. It is a structured analysis method for determining success factors as a technology
progresses through the development stages, and the comparative influence of making early
investmentsinR&Dactivitiesthatmitigatetechnicalandcommercialrisk.
ByincorporatingTRLsintoadecision-framework,thiscansupportthesedecision-makingprocess
duringdevelopmentandimplementation[12].Itisanapproachthatworkstoreduceriskthrough
proof-of-concept and system success [13]. Furthermore, incorporating real optionsmethods into
this framework can provide researchers and managers with a tool that allows them to better
understand risk.Realoptions canmakeup forwhat traditionalmethods lack,which isproviding
flexibility. It allowsmanagers to alter the course of uncertain projects by incorporating strategic
options,thatincreasetheoverallvalueoftheproject[9].ROcanbethoughtofasguidesthatallow
managerstomakedecisionsatdifferentstagesoftheproject,suchas,whetheraprojectshouldbe
continued or terminated [9]. The sequence of decision-making during these projects allows for
justifieddecisionsofwhentoundertakeanopportunity[14].
4
The framework will be applied to a small Canadian high-technology start-up that is currently
developingautonomousrovers (AR).TheseARsallowaccess to tailingsponds thatareotherwise
inaccessible using the current measuring equipment. The case study will examine current
technological maturity using the TRL scale and the proposed stage-gate approach and identify
critical risks that would threaten progress and the ability for the start-up to meet their
organizationalandprojectgoals.Thebehaviourofthesecriticalrisksandhowtheyareexpectedto
developandtheireffectsonprojectgoalswillbediscussed.ROwillthenbeappliedtovaluearange
ofpotentialdecisionsatkeystagesoftheproject.Theseoptionswillbecomparedanddiscussed.
Thestudywillalsobrieflyintroducegeneralconceptsrelatedtoapplyingthisframeworktoalarge
government research organization’s portfolio. Framework limitations and application challenges
willbeconsideredandrecommendationsforfutureresearchwillbediscussed.
1.3ThesisStructure
This thesis is organized into five chapters. Following the introduction, Chapter 2 is a literature
review that introduces and discusses relevant topics to R&D and technology development,
technology readiness levels and assessment methodologies, real options theory and application,
andframeworksusedinvariousindustries.Chapter3establishestheproposedframeworkandthe
approach behind how it can be applied to assess decisions and value options. The framework is
then applied to a case study in Chapter 4, and potential application to a government research
organization’sportfolio isalsoexamined.Theanalysisandresultsarediscussed.Toconclude the
work, Chapter 5 provides a summary and discusses the limitations of the framework and
recommendationsforfutureresearch.
5
Chapter2ReviewoftheLiterature
This chapter reviews the main topics related to the development of the real options decision
frameworkthatwillbedescribedinChapter3.
GeneralconceptsoftechnologydevelopmentandR&Darediscussed,alongwiththecommonstages
of development discussed in the literature. This conceptualization acts as the baseline for
discussing technology readiness levels (TRL), where general activities, common assessment
methodologies, challenges and applications in industry are outlined. Common risks in R&D and
technologyprojectsarethendefinedwithillustrativeexamples.ThissectionisaprimerforChapter
3,astheseriskswillthenbemappedalongthedevelopmentstagesandTRLscale.
Risk management concepts and tools used in the R&D context are reviewed. Finally, valuation
methodsarediscussed.Traditionalvaluationmethodsandfinancialoptiontheoriesareintroduced.
Real option types, application and industry use are discussed. The steps and important
considerationsforapplyingrealoptionswillbeexamined,withadditionaldetailprovidedduringits
applicationinChapter3andChapter4.
2.1R&DandTechnologyDevelopment
R&D and technology projects deliver a combination of new knowledge, new technology or
capability, or a platform of technologies [15]. Technology R&D efforts should improve the
performanceandreliabilityofatechnologyandthuscontributetooverall technologymaturation,
whileinvestmentsmadeateachstageshouldresultinrisksbeingreduced[16].
Organizational, portfolio and project goals must be properly prioritized in R&D programs [17].
ProperresourceallocationiscriticalinR&Ddecision-making,wheremanagersmustbalancemany
organizational short-term and long-term goals that may be weighted differently by different
stakeholders [17]. Firms need to be able to work effectively within financial and resource
constraints[18].Short-termandlong-termgoalsbothalsoneedtobeconsideredaccordingly[17].
6
The need to problem solve and quickly adapt to changing conditions in high-technology R&D is
magnifiedbythepresenceofunforeseenrisksassociatedwithtechnologyintegration,performance
levels, schedule andproject budgets [19]. Tools that help assess the impacts of projects and any
overlapwithotherprojectsincludeportfolioanalysis1anddevelopmentstage-gates[18].Stage-gate
methodsemploygateswherethetechnologyisassessedagainstasetofcriteriaanda“continue/go
orstop/kill”decisionismade[15].Stage-gatesanddevelopmentmodelsarediscussedin2.1.1and
2.1.2.
R&Dandtechnologyprojectsareoftencompromisedwhen inappropriatetoolsandprocesses(or
financial criteria) are applied tomanage them [15]. Examples of such tools are discussed in 2.4.
Because of the uniqueness of such projects, applying traditional methods to manage innovative
projectsmaycauseharm,asitcouldresultinterminatingahigh-profitpotentialproject[15].
2.1.1R&DandTechnologyDevelopment
Developinganewtechnologymayposehighriskstoanorganizationastheycarryalargeamountof
technical uncertainty and other unknowns [15]. Uncertainties in R&D are expected as these
projects have intensive activities associated with knowledge discovery, problem-solving,
overcoming failure, dealing with change and making breakthroughs [19]. New technology
developmentsareunpredictable,and it is–bydefinition - impossible toschedulea technological
breakthrough,whichmakesithardtoestimatefutureefforts[4].
R&D projects are separated into development phases, and milestones are set as a method to
determine and control project progress [20]. These phases behave as checkpoints where
organizationalorprojectgoalsarerealizedandprogress isassessed[17].A lackofspecificstage-
gatedecisionpointswithpre-setcriteriaresults in incorrectproductconcepts,wastedresources,
technologyfailureandexcessivespending[21].
1Portfolioanalysisreferstotheprocessesthatinvolveassessingandaddressingtheuncertainties
inprojects,allocatingandbalancingresourcesamongprojectswhilemeetingorganizationalgoals.
7
At theearlystagesof theproject, there isnoconcreteevidenceorknowledgeabout thepotential
successofatechnology[22],theprobabilityoftechnicalsuccessmaybequitelowastechnological
capability has not yet been recognized [23][15]. However, as a technology progresses andmore
information isobtained, theestimateofsuccessbegins to improve[22].Manyorganizationshave
implemented requirements for their R&D and high-technology projects, where specific business
plans and commercialization options are laid-out however, the challenge managers face when
dealingwiththeselongandhighlyuncertainprojects,istheirinabilitytocorrectlycompletesome
of the requirements of these processes [15]. For example, envisioning themarket landscape and
conducting a market and competitor analysis is difficult for a technology that is still at the
fundamentalresearchstage,andhasnotyetbeenfullydefined[15].
DuringtheconceptualphasesofR&D,theremaybeachangeinprojectdirectionastechnologyand
marketinformationbecomesapparent,itisimportantformanagerstohavetheflexibilitytomake
these changes2 [17]. Fundamental research phases may be heavily relying on studies, technical
literature, preliminary lab studies, economical valuations and patent surveys [18]. The applied
phaseofR&Dtypicallyinvolveslaboratoryworkthataimsatrefiningthetechnology’sfeatures,and
initialassessmentoffeasibilityandpotentialmarketforproductsandservicesembodyingthenew
technology.Thetechnologyshouldnowhaveaspecificpotentialapplicationorpurpose[18].These
early phases develop concepts and ideas, where pattern recognition3 and future scenario
development isvaluable [17]. It is important to identify technologies thatare feasibleandhavea
potentialformarketacceptance[24].
2This is thevalue in realoptions.Realoptionsaredefined in2.4. It allows thesedecisions tobe
estimatedandconsideredfromaspectsrelatedtotime,costs,resourcerequirements,etc.
3 Pattern recognition is the skill needed to be able to spot trends in data (if any) or projects.
Recognizingpatternsand trendanalysisgohand-in-hand, andmanagers canutilize suchskills to
improveprojectplanningandresourceallocationiftheychooseto.
8
During the technologydevelopmentphase, the focusbecomesondesign,prototyping,and testing
[18][14]. The ultimate goal during the technology development phase is to eventually be able to
deliveraproduct toauser [25].At thispoint, the technologymusthaveproven toaddeconomic
value [12].Thepilot testingof technologywouldhavemoved from laboratory to operational (or
relatively operational) environment [26]. This is also where scale-up activities may begin (and
continue into the next phase). During the process of scale-up, new information about risks is
realized [26].The scale-upactivities are separated intobatch sizes,where smaller sizes canbea
proof-of concept, and the larger batches test for the effects of larger scalemanufacturing on the
quality or viability of the technology [26]. There is also an inherent process that occurs during
development,which is the “technology transfer”.This iswhere thedevelopmentprocesschanges
fromtechnologyandsciencecreationtoproductcreation.Furthermore,thereisatransitionwhere
the project is transferred from scientific personnel to the commercialization andmarket experts
[27].Therefore, it is important formanagement toensure that functionalgroupsworkefficiently
withoneanother[27],andkeepabalanceoftheteamandindividualresponsibilities[17].
The decision to commercialize is usually made when uncertainties from the R&D stages are
resolved [14]. The commercializationwarrants a shift in tasks as the organization is involved in
market positioning and competition [24]. This phase may involve activities that involve
manufacturing, process and product launches done through marketing [14]. During the
commercializationphase,issuessuchasthecostofgoodssold(COGS),thesizeofmarketandsale
pricemaybecriticalissues[17].Commercializationofnewtechnologiescouldincludelicensingor
donating intellectual property that is not active (i.e. dormant) [9]. The commercialization phase
bringsnew technologies to themarket and can includeactivities suchasmanufacturing, refining
thetechnologyanddistributiontocustomers[25].Thelackofcommercialskillsandashortageof
financeswillpreventanorganizationfrombeingabletomoveforwardwithnewtechnologies[28].
CooperandKleinschmidt[21]discusstheresultsfromtheirresearchwheretheyfoundthatmany
companies dive far into later stages of development without any consideration for
commercialization, only to realize later their expectations of themarket are incorrect [21]. This
brings up an important concept that should be discussed, technology-push and market-pull
technologies.Thisisdiscussedin2.1.3.
9
ActivitiesinthestagesofdevelopmentarespecifictotheprojectbutCohenetal.[29]discussnine
dimensions for basic gate decision criteria that are the framework for identifying issues. The
criteria remain the same but the details evolve as a technology progresses from one gate to
another4.Thesedimensionsforbasicgatecriteriaareasfollows[29]:
• Strategicfit:businessstrategiesandneeds
• Marketandcustomer:potentialbreadthofthetechnologyinthemarket
• Businessincentivesandrisks:keyissuesanduncertainties
• Technicalfeasibilityandrisks:scienceandtechnologyuncertainties
• Competitiveadvantage:technologyorbusinessbenefitsoverthecompetition
• Killervariables:thatcompletelystopaproject
• Legalandregulatorycompliance:health,safety,environmentaloroperationalintegrity
• Criticalfactorsforsuccess
• Plantoproceed:planstoachievegoals,milestonesandtargetdatesforthenextgate.
Hoegletal.[30]discusstheimportanceofteamdynamics(andcoordination)duringallthephases
of development and emphasize that proper team dynamics in the early conceptual phases of
developmentcanultimatelyhavemajoreffectsonperformanceinthelaterstagesofdevelopment
[30].During thedifferent stagesof theproject,managers canexpect tohavedifferent views that
overall influence the project. Criteria for decision making should be integrated within all
developmentalphases[18].Managersordecision-makerscanbethoughtofasgate-keepersduring
aprojectwheretheycanstopprojectsthatarenotproducingaccordingtosetstandards,butmust
alsobeabletospotpotential innewideas,andmakechangesduringtheprojecttocapturethese
opportunities[29].
Metricsmustbesetbymanagementearly-on inorder forprogressevaluationtobecompletedat
each stage. As the project progresses from conceptual stages all the way to commercialization,
informationandevaluationmetricschange[17].
4 For example, for research, initially thequestion askedmaybewhether the research is feasible.
Stagesaftermaybecomeaboutwhethertheconceptsinvestigatedarefeasible.
10
It isalso importanttonotethatR&Dorganizationsmaynotbecorporateenvironmentsandtheir
goal may not be to maximize revenue5[10]. It is important to note that the nature of available
informationchangesduringtechnologydevelopment.Intheinitialstages,datacanbeexpectedto
be of the qualitative nature,while in the later stages of commercialization,managers can expect
morequantitativeinformation[17].
Terminationphases areoftennot included inR&Dprojects, however, reasons for terminationor
failure of a project should be considered6 as it could improve and drive the decision-making
processesinaproject[17].Figure4illustrateshowtheterminationphaserelatestootherstagesof
developmentandhowitfitswithinamodel.
2.1.2ExamplesofTechnologyDevelopmentModels
Another common tool used by organizations is the stage-gate development method where the
technology is assessed against a set of criteria and decisions aremade at each gate [15]. Exxon
Research andEngineering (ERE)7 Companyuses a stage-gate systemwhereR&D activities begin
with fundamental research, applied research development, validation, and concludes with a
commercializationstage[29].EREthenaddedthreenewresearchgatesthatprecedethestandard
stage-gateprocessdiscussed.Thisadditionincludedidentifyingopportunitiesandenablingscience
andideagrowth[29].Kelmetal.[24]emphasizethatregardlessofthedifferencesinthetheoretical
developmentmodelsinindustry,andthespecificsbehindeachphase,thereisanoverallconsensus
thattheearlystagesofatechnologydevelopedareheavilyinvolvedintechnicalinnovation,while
thelaterphasesarefocusedoncommercialization[24].
5Anorganizationmayhaveitsownideaofmetricsofsuccess.Forexample,anot-for-profitcould
aimtoonlywanttohavepositivesocietalimpactinthepublic.
6Thisiscanbedonethroughriskassessments.
7ThishasnowbecomeExxon-Mobil(EMRE)
11
Figure1.EREStagesofDevelopmentModel[Adaptedfrom[29]]
StageAshowninFigure1iswherebusinessmanagersandresearchersbegintotrytoestablisha
businesscaseandcompetitiveedgeforthepotentialtechnology,throughadetailedplanthatsets
technical and scientific variables. The plan includes resource requirements as well as plans of
action of how these deliverables can be met [29]. Stage B is where the plan from Stage A is
executed,andissuesrelatedtoscientificprocessandleadstobusinessopportunitiesareidentified
[29].
Usingthegate-processisastructuredmethodtoassessresearchprogressandallowsfordecisions
tomade in a timelymannerwhile tracking project progress fromboth a science and technology
aspect,aswellascommercializationaspect[29].Atthegateofeachstage,risksanduncertainties
andotherdriving factors shouldbediscussedandcommunicatedwithkeypersonnel [17].There
are many versions of technology development models in literature that have been adapted for
differentpurposesandindustries[17][29][31][32].
Figure2.DoD'sTechnologyStagesofDevelopment[Adaptedfrom[33]]
ConceptRefining
TechnologyDevelopment
SystemDevelopment&Demonstration
Production&Deployment
OperationsandSupport
DecisionMilestones
A B C
12
Figure2demonstratesthetechnologydevelopmentmodelusedbytheUSDepartmentofDefense
(DoD). Their model is comprised of five stages: concept refinement, technology development,
systemdevelopmentanddemonstration,productionanddeployment,andoperationsandsupport
[34][35]. The model identifies three major milestones as logical stops where technology
opportunitiescanbecaptured[34].This isshownattechnologyreadinesslevel(TRL)4,6,and7,
andOlechowskietal.suggestthatmappingTRLstostagesofdevelopmentisausefulpractice[36].
They argue that it allows expectations to be clear and consistent for all projects [36]. Concepts
related to technology readiness levels will be discussed in Section 2.2. The Milestone Decision
Authority(MDA)workswithstakeholdersinordertoassesswhetherthereisenoughinformation
at eachphase, beforemoving on to the next [34]. A projectmay start at any stage of themodel,
however, it is still a requirement that itmeet the entrance requirement of anyupcomingphases
[34].
LeeandGartner[25]discussedstagesoftechnologydevelopmentinasimplifiedmodelillustrated
inFigure3.Thereisonlyonephaseofresearchasopposedtotheclassicbasicandappliedresearch
phases and three major gates. This model does not view development as a linear sequential
process, instead, it is an iterativeprocess that responds to themarket andcompetition [25].The
potentialofwhetheratechnologicalbreakthroughhasanycommercialviabilityisdoneatthefirst
stagewiththehelpofamarketspecialist[25].
Figure3.LeeandGartner’s.DevelopmentModel[Adaptedfrom[25]]
TechnologyDevelopment
BasicResearch
TechnologyCommercialization Marketand
IndustryGo/NoGo Go/NoGo Go/NoGo
13
Thestagesof technologydevelopmentmodelbyTritleetal [17]articulates thatadevelopmental
processmustbealignedwiththevision,values,andgoalsofafirm.Theirmodelhassixstagesthat
aretheidea,concept,prototype,development,commercializationandtermination.Eachphaseisa
checkpointandhasadeliverablethatmustbeproduced[17].
Figure4.Tritleetal.StagesofDevelopment[Adaptedfrom[17]]
2.1.3TechnologyPush–MarketPull
Technology-push projects originate from researchers recognition of a new technological
phenomenon, this often causes scientists to become biased as the recognized benefits of a
technology override issues of how a scientific or technological phenomenon canmeet a market
need[37][38].
Figure5.Technology-Push&Market-Pull
Vision,Values&Goals
Idea Concept Prototype Development
CommercializationTermination
• Terminationdecisionreasons• Basisforimprovingphasereviews• Opportunitytoaddressmoraleof
theteam• Futuresourcesforideas
Receptormarket?/need?
R&D Scaleup&Production
Marketing&Commercialization
Marketing&Commercialization R&D
Scaleup&Production
Receptormarket/users
Expressedneed
TechnologyPush
MarketPull
14
Thepossibilitiesofthesetechnologiesareover-hypedinordertosecureinitialcapitalinvestment,
butoftenthereisnoidentifiedcustomeroruserneed[39].Wheatcraftarguesthatthetechnology-
pushapproachishighrisk,andprefersmanagersandresearchersadoptthemarket-pullapproach
[39].Technologiesdevelopedwiththemarket-pullordemand-pullapproachdefinedtheirproducts
features with a market of end users in mind [40]. Market conditions create opportunities for
technologies to satisfy unmet market needs [41]. Management’s attitude, in general, has an
influenceoninnovationwithinanorganizationandtheapproachtakenastheyhaveacriticalrole
indecision-making[38].
2.1.4CapabilitiesofaFirm
Capabilitiesofafirmhavealargeinfluenceonthefinancialcapital,technicalexpertiseandresource
requirements(andavailability)[38].Dynamiccapabilitiesisatopicthathasgainedpopularityover
theyears[42].Tosummarize,itistheabilityofanorganizationtointegrate,buildandreconfigure
theircompetenciesinreactiontofast-pacedanddynamicsenvironments[43].Anin-depthreview
ofthistopicisoutsidethescopeoftheresearch.
2.2TechnologicalMaturity
2.2.1TechnologyReadinessLevelsOverview
Thetechnologyreadiness level (TRL)scalewas firstdevelopedbyNASA in theearly1970’s [36].
The purpose of this scale was to set a standard, and provide a consistentmeasurement system
managers could use to assess technological maturity [16]. This can be validated through
demonstrationsthatincreaseinfidelity,andinrealisticoperatingenvironments[44][16].TheTRL
scale allows researchers and managers the opportunity to improve risk management,
communicationoftechnologydevelopmentprogress,andtheirdeliverables[45].Theoriginalscale
developed consisted of seven levels andwas later upgraded to nine levels in the 1980’s, where
NASA then published definitions of each level and their activities [44][46]. By 1999, the U.S
DepartmentofDefense(DoD)hadadoptedthisscalefortheirprogramsandsystems[47][46].The
scalewas then expandedby theDoD to allow the applicability of TRLs to softwaredevelopment
projects [48]. The terms “readiness” and “maturity” describe the developmental progress of
technologyandhavebeenusedinterchangeablyintheliterature[34].
15
WithlargeglobalcompaniessuchasGoogle,Bombardier,JohnDeere,BPandBoeingusingTRLs,it
is the most commonly used scale, utilized across many industries [31][36][49]. Evaluating
technologymaturityisimportantasitgivesmanagersinsightintosomeoftherisksassociatedwith
the technology development stages [46]. Brief descriptions of TRLs are outlined in Table 1. Full
descriptionsofsoftwareandhardwarerequirementsarefoundinAppendix1.
Table1.NASATechnologyReadinessLevels[50][51]
TRL Description
1 Basicprinciplesobservedandreported.
2 Technologyconceptand/orapplicationformulated.
3 Analyticalandexperimentalcriticalfunctionand/orcharacteristicproofofconcept.
4 Componentand/orbreadboardvalidationinalaboratoryenvironment.
5 Componentand/orbreadboardvalidationinarelevantenvironment.
6 System/subsystemmodelorprototypedemonstrationinanoperationalenvironment.
7 Systemprototypedemonstrationinanoperationalenvironment.
8 Actualsystemcompletedand“flightqualified”throughtestsanddemonstration.
9 Actualsystem“flightproven”.
The purpose of TRLs is to support decision-making processes during development and
implementation[12].Thescaleensuresacommongroundformanagersandengineerstoassessthe
statusof the technologyandensures riskmanagement is consideredduringdevelopment [12]. It
alsosupportsfundingprogramsfordevelopment,transfer,anddeployment[12].TheTRLscaleis
alsopromotedasagapassessmentbetweencurrenttechnologicalmaturity,andtherequiredtarget
TRL [52]. Technologies may be considered “low risk” in the engineering and manufacturing
developmentstageswhenprototypesdevelopedareofconsistentquality,andhavebeenprovento
workinsimilarenvironmentsasthetargetoperationalenvironment[52].Technologyreadinessis
a logical approach to systems and works to reduce risk through proof-of–concept and system
success[13].
16
2.2.2TRLs:CharacteristicsatEachLevel
As shown in Table 1, TRL 1 is the lowest level of maturity where activities might include
fundamental research and studying basic properties [30]. The costs during this level could vary
dependingontherigoroftheresearch[44].Theserelatedcostsarecompletelydependentonthe
scientific area of the research conducted and resources required (i.e. white board vs. super
computers)[53].TRL1isacommonlevelforuniversitiesandresearchorganizations[54][53].TRL
2 is where the practical application of a technology is identified, but without any experimental
prooforproperanalysistosupporttheclaim[39].Thecostswillstillberelativelyonthesamescale
asTRL1.AnyorganizationmaybeatTRL2,however,itiscommonforuniversities,entrepreneurs,
andsmallbusinessestobeinthisTRL[53].
AtTRL3,appliedresearchanddevelopmentbeginswherethetechnologyisputintocontext.This
levelcombinesanalyticalandexperimental(couldbelaboratory)methodologiestoproveconcepts.
Thespecificsbehindapproachesusedarespecifictothetechnologyandresearchers’discretion.For
software,proofofalgorithmisnecessary,whilehardwarewillrequirephysicalvalidation.Similar
to the previous TRLs, costs in TRL 3 can be expected to be unique to the technology being
developed [44][53], and because of the increase in costs, it can be expected that some kind of
funding would be attained at this point. This could be private sponsorship or government-type
funding.AttheselowTRLs(1to3),thetechnologicalriskishighwhichmeansthatleadtimesare
increasedandfundingopportunitiesmaybescarce.OftentechnologiesatTRL1-3mayfallintothe
technology-pushcategoryclassification,especiallyduringtheprocesseswhereknowledgeisgained
withoutanyspecificapplication[39].Duringtheseearlystages,managersshouldbegintoconsider
how the technology or processmay be interrelated and the potential risk and other parameters
needed for future development [26]. In TRL 4, low-fidelity validation is required and should be
developedinawaythatisconsistentwiththerequirementsofthepotentialsystemapplication,and
beabletosupporttheconceptsinpreviousTRLs.Theelementsofatechnologymustbeintegrated
to determine that theywill all operate with one another and achieve target performance at the
levelsofacomponent.Mankins[53]ranksTRL4costsasmoderate[44],andhedescribesthecost
requirementstobe“severaltimesgreater”[53]thanthepreviouslevels.AtTRL4,theuncertainty
isexpectedtohavedecreasedslightlywiththeproofofconceptandlaboratoryvalidation,whichis
argued to provide greater chances of securing funding sources [53]. TRLs 3-4 should identify
activitiesthatproveconceptsatalaboratoryscale,thatarealsorisk-reducing[26].
17
TRL 5 requires that elements of a technology be integrated into a component, sub-system or
system-level.Thismaymeanthatmoretechnologiescouldbe involved inthedemonstration.The
fidelityofthecomponenttestedinthislevelincreasesgreatly.CostsforR&DincurredinTRL5are
describedasmoderatetohigh[44],wheretheyaretwo(ormore)timesgreaterthanthatinTRL4
[44].TheactivitiescompletedinthisTRLaremostlikelydonebyR&Dorganizationswithaccessto
corporate laboratories. Therefore, it is expected that funding required increases due to these
increasedcosts[55].AtTRL6,theprototypesystemistestedinarelevantoperationalenvironment
andprovensuccessful.At thispoint,maturation isdrivenby instillingconfidence inmanagement
(inthetechnology’sfuturedeployment),ratherthantheR&D.Demonstrationmaybeofthesystem
applicationandanyothertechnologiesthatcouldbeintegrated.CostsinTRL6areexpectedtobe
highdue to intensive demonstrations of the technology [44]. Almost always, there is a source of
fundingwhetherfromthegovernmentorindustry.Pilot-scaletestingactivitiesinTRLs5-6address
risksandexposefurtherinformationabouttheconcept,andfurtherreducethem[26].Thecostsare
describedtobe“twoormoretimes”lessthantheinvestmentrequiredinTRL7[55].
LevelsbeyondTRL6aremajormaturationsteps.InTRL7,theprototypeshouldbecloseto,orat
the operational scale necessary, with the demonstrations occurring in the relevant operational
environments[44].Thepurposeofthislevelistoensuresystemengineering,aswellastodevelop
management confidence in concepts related to the market. Costs associated with TRL 7 are
describedas“veryhigh”[44].Dependingonthescaleandfidelityofthesystem,thiscouldbealarge
amount of the ultimate system cost, thus would always require formal sponsorship. TRL 8
representstheendofrealsystemdevelopmentformostelements[55].Thislevelmayincludenew
technologiesbeingintegratedintothesystem.CostsinTRL8arespecifictotherequirementsofthe
systemandareclassifiedas“veryhigh”withthemagnitudeofcostsbeing5-10timesgreater(this
isbecauseoffull-scalesystemdevelopment)[44]thanallthepreviousTRLscombined,andagain,
wouldexpectformalfunding.TRL9isthefinallevelwherethesystemisdeployed,andthefixingof
systembugsandglitchesbegins.AtTRLs7-9,researchersandmanagersshouldbeabletoassess
customer acceptance and real-world impact as the new product is introduced (or about to be
introduced) intothemarket[26].Thisalsoassumesthatcustomers’acceptanceofthetechnology
willbepositive,therefore,itisimportantthatthebusinesscaseisreviewedduringthistime[26].
18
It is important to note that reducing risk across TRLs is not done linearly in terms of cost [56].
Mankins[53]notedthatthecosttoincreasefromTRL5toTRL6ismorethan4timesthecostsof
the previous levels, and progressing to TRL 7 comes with even greater costs. He refers to
progression past TRL 6 as “the valley of death” [53][56] and discusses the struggle between
scientistsandmanagers.Withanynewtechnology,reducingriskisapriorityformanagerssothat
projectbudget andproject schedules arenot affected,while the scientists justwant tomaximize
theiradvancesanddiscoveries[56].MoreonthiswillbediscussedinChapter3.
2.2.2TRLAssessmentMethodologies
A technology readiness assessment (TRA) is the process of assessing the maturity level of a
technology [31]. This process relies on information during the technology development stages.
However, theU.SGovernmentAccountabilityOffice(GOA)suggests thatperformingaTRAbefore
developmentbeginsprovidesvaluableinformationformanagement[31].Mankins[16]statesthat
animportantpointduringdevelopmentiswhenmanagementmustdecideonwhethertechnologies
neededaspartofasystemhaveallcollectivelyreachedthetargetmaturity,risk,andperformance
levelforprogress[16].TheassessmentofTRLscanbeconductedattimesthatmanagementdeems
necessary,asaTRAneedstobespecificinthecontextofthetechnologyandtheaudiencethatwill
useit[31].TheabilitytoconductathoroughTRAwillultimatelydependontheavailabilityofdata,
reports,andaccuracyofitall[31].Inthecaseofnewtechnologydevelopments,scopeisnotalways
availableorunderstood[31].
Because theTRLscalemay lackobjectivityandrely toomuchon tacitknowledge,somematurity
assessmentmodelsandmethodshavebeendevelopedtotacklethisissue[34].Thesemodelshave
notonlybeenusedtoassessmaturity,buttoalsoassessriskssomanagementcanbetteranticipate
theminlaterstages[44][16].
19
Mankins [44] recommends that a general model to assess a technology should include five
categories:
1. Abasicresearchphasewheregoalsandtargetsareidentified.
2. An applied or focused research phase where a specific technology is considered for specific
applications.
3. Technologydevelopmentandprototypingforeveryidentifiedapplication(priortofullsystem
development).
4. Fullsystemscale-upandtesting.
5. Technologylaunchandoperations.
Mankinsalsosuggeststhatanassessmentshouldpossessthefollowingcharacteristics[16]:
• Clarity: cleardecision-making criteria todetermine risksand readiness.Criteria shouldallow
forindependentevaluationandverification.
• Transparency: technology risk assessment should be formal and consensus based, where all
participantseasilyunderstandtheassessmentprocessesandresults.
• Crispness:decisionsmadeduringtheassessmentshouldbetimelyanduptoprogrambudget
planningrequirements.
• Usefulinprogramadvocacy:processesusedduringtheassessmentshouldhavethebasicsfor
advocacyofaresult.
The model Mankins [16] introduces is an integrated technology readiness and risk assessment
framework (TRRA). This is a quantitative approach [34].He argues thatTRLs fail to address the
difficulty inR&Dprogress,andtheeffortrequiredtomovefromaTRLtothenextwithinasetof
criteriaorrequirements. ThismodelbuildsonanotherpaperbyMankins[57]anddescribes the
“research and development degree of difficulty” (R&D3) as a measure of the difficulty that is
expected during the process of maturation for a technology [55]. The purpose of this is to
supplementTRLmetrics[57].Itdeterminestheprobabilityofsuccess(orfailure)foragivensetof
technologyrequirementsatdifferentstagesofdevelopment [16].TheR&D3consistsof five levels
thataredescribedinTable2.The integratedassessmentmethoddevelopedincorporatesanother
dimension, “the technologyneedvalue” (TNV) [16].TheTNV isaweighting factor that isapplied
relative to the assessment of the importance of technological development (shown in Table 2)
[16][55].
20
Theapproachassessestheprobabilityofsuccess, identifiesthegapbetweenthecurrentTRLand
targetlevelanditsR&Deffort,andthenutilizestheTNVtoassesstheimportanceoftheprogram.
The factors are then applied into a technology riskmatrix that assess the technology on amore
coherentbasis[16].
Table2.R&D3LevelsandDescriptions[57]andTNVandDescriptions[16].
R&D3 Description TNV WeightingFactor
Description
R&D3–I
Averylowdegreeofdifficultyisanticipatedinachievingresearchanddevelopmentobjectivesforthis
technology.ProbabilityofSuccessin“Normal”R&DEffort99%
TNV1
40% Technologyeffortisnotcriticalatthistimetothesuccessoftheprogram.Advancesto
beachievedareusefulforsomecostimprovementshowever,theinformationprovidedisnotneededfordecisionsuntil
thefarterm
R&D3–II Amoderatedegreeofdifficultyshould
beanticipatedinachievingR&Dobjectivesforthistechnology.
ProbabilityofSuccessin“Normal”R&DEffort90%
TNV2
60% Technologyeffortisusefultothesuccessoftheprogram.Advancestobeachieved
wouldmeaningfullyimprovecostand/orperformancehowever,theinformationprovidedisnotneededfordecisionsuntil
themidtofarterm
R&D3–III Ahighdegreeofdifficultyanticipatedin
achievingR&Dobjectivesforthistechnology.ProbabilityofSuccessin
“Normal”R&DEffort80%
TNV3
80% Technologyeffortisimportanttothesuccessoftheprogram.Advancestobeachievedareimportantforperformanceand/orcostobjectivesandtheinformationprovidedisneededfordecisionsinthenear
tomidterm
R&D3–IV
AveryhighdegreeofdifficultyanticipatedinachievingR&Dobjectives
forthistechnology.ProbabilityofSuccessin“Normal”R&DEffort50%
TNV4
100% Technologyeffortisveryimportanttothesuccessoftheprogram.Advancestobe
achievedareenablingforcostgoalsand/orimportantforperformanceforperformanceobjectivesandinformationprovidedis
highlyvaluableforneartermmanagementdecisions
R&D3–V Thedegreeofdifficultyanticipatedin
achievingR&Dobjectivesforthistechnologyissohighthatafundamentalbreakthroughisrequired.ProbabilityofSuccessin“Normal”R&DEffort20%
TNV5
120% Technologyeffortiscriticallyimportanttothesuccessoftheprogram.Performanceadvancestobeachievedareenablingandtheinformationtobeprovidedisessentialfornear-termmanagementdecisions
21
Azizianetal.[34]discusstheTRAprocessusedbytheDoDfordefenseacquisitionprograms.Itisa
six-stepprocessthat isshowninthe figurebelow.Theprocess isstartedbysettingaschedule in
order for importantmilestones tobemet [34].Theassessment thencontinuesby identifying the
criticalelements8(CTE)acrossaWorkBreakdownStructure,dataisthencollectedandpresented
toanaudience(expertsintechnology)thatisindependentoftheteam[35].Reviewersthenassess
thematurityofCTEsagainstthemetricsthathavebeendecidedonandthenpassedupforapproval
bythechainofcommand[34][35].Ifnotapproved,then,theymayconductanotherTRA[35].
InthecasewhenacomponentisnotatthesameTRLastherestofthetechnologies,theDoDmay
doanyof the following [35]:restructure theprogramso thatonlymature technologiesareused;
delay the program in order to mature the technology; change the program requirements; not
initiatetheprogramandconsideranothersolution.
Figure.6DoDTRAProcess[Adaptedfrom[34]]
8 (CTE) is defined as an element that the system being acquired depends on in order to meet
operationalrequirements[64].
SetSchedule
IdentifyCriticalElements
CoordinateCriticalElements
AssessCriticalElements:PrepareTRA
Coordinate&SubmitTRA
OversightReview
DataCollection
22
TheU.SGovernmentAccountabilityOffice(GOA)drewuponthetechniquesandpracticesatNASA
and the DoD and produced their own TRA methodology that is often called the “best practice”
[58][31].Thepublished150pageguideoutlinesthesixstepstoimplementingthismethodindetail
[31].Thesestepsaresummarized inAppendix1.Anotherapproach toconductingTRAconsiders
moreholisticmethodstoredrawtheboundariesoftheproblemsandexaminestherateatwhicha
technology can mature, and larger issues related to technology-life cycles9, specifically its
obsolescence.Formoreinformationsee[59].
2.2.3OtherTRAToolsandTechniques
SimilartotheR&D3,therearemanyothermaturityassessmenttoolsthathavebeendevelopedto
work or leverage with the TRL scale and provide insight on different aspects of technology
development[34].Theadvancementdegreeofdifficulty(AD2)isaqualitativemethodthatisargued
tobuildontheR&D3approachandpossesses9 levels that integratetheaspectsofcost,schedule,
and risk [60]. Another qualitative technique developed, is the Manufacturing Readiness Level
(MRL)createdbytheDoD[61].Itisusedtoassessthemanufacturingmaturityofatechnologyona
scale of 1 to 10 [61][62]. This can be applied during system development of a technology and
continuesafterthetechnologyhasbeeninoperationforafewyears[62].
TheSystemReadinessLevel(SRL)isaquantitativeapproachthatmeasurestheindexofmaturity
onasystem-level[63].SRLsareafunctionofTRLs(andtheirmaturities)andareexpressedbased
on the Integration Readiness Levels (IRL) [63]. The IRL scale is a 9-point scale that measures
maturityandtherelationshipbetweentheinterfacesofotherreadiness levelsandcanbeusedto
determinetheriskofintegration(whenusedwithTRLs)[63][34].
9 Technology-life cycle has four stages. It starts at theR&Dphase and ends in the decline phase
wherethetechnologyeventuallybecomesobsolete.
23
Automated tools to measure maturity are also available, where they quantitatively assess the
maturitybasedontheinformationfedbytheuser[34].Themostwell-knownbeingtheTechnology
ReadinessLevelCalculator[58](andMRLcalculator).ThisisaMicrosoftExceltoolthatcalculates
the TRL level as an output at a specific time [58]. The calculator provides no information about
risksor theprobabilityofsuccessbutcangivemanagementageneral ideaabout therisk(as the
assumptionisthehighertheTRL,thelowertheoverallrisk[39]).
2.2.4LimitationsofTRLs
There can be a biaswhen conductingTRLswhere different priorities andmetrics for success or
levelsofacceptableriskcanplayafactorinwhichchoicesaremade.Thisincludesoptimism,which
can affect TRA results. Managers may also be tempted to accept higher risks and immature
technologies in hopes of future performance and stakeholder buy-ins [31]. In the case where a
technology is not developed at every level10, the risks related to skipping these levels should be
assessedagainstthecost[39].TRLsalonearenotsufficientasanentireframeworkfortechnology
and risk management. As discussed, many other complementary methodologies have been
introduced in order to better identify uncertainties during research and development, to take
actionupontheseuncertaintiesandtodeveloplong-termtechnologyopportunitiesbasedonneeds
[45][55]. Some other issues identified in literature include improper assessment of methods to
integrate two technologies, or an individual component of a system and the measurement of
uncertainty during the maturation process [47][46][36]. In addition, the lack of ability to
comparatively assess the alternate TRLs on the entire system [47], and failure to consider
technologyaging(obsolesce)[64].
10Managerscanchoosetoskipalevel;thisiscalledleap-frogging.
24
Finally, a paper published in 2015 by Olechowski et al. [36] identified amajor challenge as the
failuretoalignTRLswithtechnologydevelopmentstage-gates.Theyacknowledgethefactthatthe
aligningispracticedinindustry;however,arguethattherehasbeenlimiteddiscussionaboutthisin
theacademicworld.Theyarguethatthelackofpropermappingdoneinindustryandprocessesof
determining the minimum acceptable TRLs is related to the lack of understanding of the
consequencesofmissedmilestonesandreachingtargetTRLs11.
2.3TechnologyRiskManagement
2.3.1RisksinR&D&TechnologyDevelopment
There is a significant amountof risk that canbeexpectedduringanew technologydevelopment
project[27].ThereisalargeamountofliteraturethatidentifiescommonrisksassociatedwithR&D
andhigh-technologyprojects.However,thereisaweaknessinaligningtheserisksalongthestages
of development for a technology project and identifying trends of how one risk might affect
another.Thissectionhighlightsthemajortypesandcategoriesofrisksonagenerallevel.Chapter3
willapplythetopicssummarizedinthissectionanddiscussitinarelevantcontextthatbuildson
the framework inamanner thatcanbemappedoutalong technologystagesofdevelopmentand
generalcorrelationsofhowrisksmayberelated.Appendix1listsexamplesofpotentialrisksthat
mayariseduringdevelopmentasoutlinedby[65].
Thereisaninherentdifferencebetweenriskanduncertainty.Risksaredescribedasthedegreeof
whichanuncertaintyandlossmayoccurinanevent[32].Thereforeriskisaquantifieduncertainty
andoutcome[66].R&Dprojectsarelabeledas“risky”iftheprobabilityofabadoutcomeishigh,the
ability to control the risks within time and resource constraints is difficult, and if the potential
impact of the consequences is substantial [65]. The high uncertainty in R&D leads to high risks,
which may lead to project failure. Improving R&D probability of success requires managers to
controlrisksduringallstagesoftheproject[5].
11GoogleandJohnDeereweretheexamplesdiscussed.
25
Ingeneral,innovativeR&Dprojectscanexpecthighrisksofmarketandtechnologicaluncertainties
whichultimatelycauseproject failure [67].The literaturereviewedrevealed that individualshad
groupedR&Drisksaccordingtowhatwasviewedasmoredominantorrelevanttotheprojectsthat
werebeingconsidered.Asummaryofthegeneralriskcategoriesfromtheliteratureisoutlinedin
Table3.Critical issuesanddriving factorscanbe identifiedbymanagers interactingwithseveral
functional groups across an organization and other parties (such as customers, suppliers, and
experts)[17].
Table3.MajorR&DRisksinLiterature
RiskCategories Reference
Technology,market,finance,operations [65]
Economical,managerial,projectmanagement,organizational,quality,
market,social,legalpolitical,technical,supplier[32]
Incompetentmanagement,externalriskfactors,informationtechnology,
lackofmarketing,technologydevelopment,staffturnover,safetyfailures,
poorstrategy
[68]
Strategic,discovery/research,development,commercial,regulatory [67]
Firmspecificrisk,competitionrisk,marketrisk [69]
Financialrisk,projectrisk,owner’srisk [70]
Market,technology,environment,organization [71][72]
Market,competitive,development,commercialization [20]
Technology,business,organizational [73]
Economic,time/schedule,operational,customer,markets [74]
Technology,market,supplier/process,financial [75]
Technological,market,financial,institutional/regulatory [76]
26
Technologyriskscanbedescribedasthoserelatedtodesign,platformdevelopment,manufacturing
technology,IP[65],andtechnologylife-cycles[77],whiletechnicalrisksaredefinedasthetechnical
issues that comeupwithnew technologies (suchasglitches) [78].Otherexamplesof technology
riskelements includenotreducingthetechnologytopracticeorfailuretodemonstratefeasibility
[78],ortheabilitytoremainfeasibleleadingtobecomingobsolete[69].Technologyandtechnical
risksareimportantduringtheearlydevelopmentstagesofatechnologyastheyhaveaninfluence
onmilestones and feasibility. ThoseR&D firms that are able topossess technologieswithhigher
capabilitiesgenerallytendtohaveinvestorsthatlookforqualitiessuchasafirm’sabilitytomeet
technologychallenges[24].
Technicaluncertaintiesmaycreatepressureonmanagerstoinvestinordertolowertherisk[79].
Thelogicbehindthisisthatdelayinginvestment(i.e.actionsanddecisionstomitigatetheserisks)
may cause exposure to an increase in competition [79] and delayed market entry
(commercialization) could result in significant failure, as competition that possesses the right
resources may act quicker and enter the market (competitive risk) [69]. However, aggressively
enteringamarket tooearlymayposemanychallenges suchas large increases in costdue to the
infrastructure needed [79]. Another risk could be competition’s response to the new technology
launch, as their actions (releasing similar or better products) could destroy a firm’s competitive
advantage [69]. These competitive risks that could arise are directly related to actions by
competitorswhichcancausevaryinglevelsoflossesinprojectopportunities[69].
Manufacturingtechnologyrisksarealsoimportant,forexample,properscale-up(andthepotential
forscale-up)[65]isnecessarytoachievesuccessinlaterstages[65].Figure7illustratesthegeneral
trend for technology risk12 as technologiesmature acrossTRLs, as suggestedbyBatkovsky et al.
[12]. The figure depicts an increase in investment in funding requirements as higher TRLs are
reached.TheactualgrowthtrendasTRLsincrease(i.e.linear,exponential,etc.,)isnotreflectedin
Figure7.
12 The paper [12] only showed a table that outlined the relative qualitative magnitudes of risk
acrosstheTRLs.
27
Figure7.TechnologyRiskAcrossTRLsforHighTechnologies
Market risks can be characterized to be of any or a combination of the following: customer
demands, regulatorychanges thatmay influencedemand,customeracceptanceandadoption, the
effectofcompetitorsstrategiesandtheriseofnewerandcheapertechnologies[65][69].Wangetal.
[20]discussthatwhenhighmarketuncertaintiesarepresent,managersarestillatrisktoproperly
match market requirements, even with managerial flexibility. They advise that organizations
exercise their efforts to obtain reliable information about the market so that the range of their
optionstoreduceuncertaintycanproperlybeassessed[20].Theyalsoemphasizetheimportance
ofmaximizingmarket research capabilities as they argue evenwithwell thought-out technology
planning methodologies, a high market uncertainty will reduce any chances of capturing
opportunities [20] Often, managers are unsure of the market opportunities for a particular
technology [32]. Ifa firmhasaccess todataon thepotentialcustomers, competition,distribution
channels and other market information, then this can contribute to market uncertainty being
loweredoverall[32].
Market risks can be either strategic or operational and although different, there can be overlap
betweenthetwo[76].Operationalriskscanbemanagedbyprojectmanagerswhilestrategicrisks
moreoften thannot, require the involvementofhigherexecutives (suchasaboardofdirectors).
[76].Successfulcommercializationofatechnologyisdependentonthepresenceofamarketneed
andtheabilitytostrategizeaccordingly[80].Propermarketingstrategieswillexplorepositioning
opportunitieswithin themarket and devise action plans to gain a competitive advantage and to
maximizethevalueofaparticulartechnology[80].
TRL1 TRL3TRL2
TRL4 TRL5
TRL6 TRL7 TRL8 TRL9
Increasingcost
Absolutelyhigh
Extreme
lyhig
h
Veryhigh
High
Mediu
m
Low
VeryLow
Technologyrisklevels
28
Thiscalls forsomeonewithmarketingexpertise thatwouldbeable toconduct thisallwithin the
organizational (internal) and external constraints [80]. Formulation of marketing strategies for
early stage technology projects is an important milestone [80]. The concepts behind marketing
which strategies to select and when are outside the scope of this thesis. However, they are
recognizedasanimportantaspectofdevelopmentthatshouldbestudiedandconsidered.
Financeorfinancialrisks13canrefertotherisksassociatedwiththelimitedfinancingavailablefor
development in a project [69][76] and the challenges associated with obtaining funding for a
project[70].Thisisanimportantcategoryofrisk,astheavailabilityoffundingiscriticalincapital
intensive projects [65]. Scale-up costs of technology, time to develop, and human resource
additionstoanR&Dteamcanallbeexamplesofdevelopmentalrisksthatcancausefinancialrisks
to increase [17]. Technology viability, pricing sensitivities, inadequate investments, low-profit
marginsareexamplesoffinancialrisks[65].
Organizational risks can be firm-specific risks, related to internal organization factors or those
relatedtoR&Dteams’relationshipswiththirdparties[65].Regardlessofthetechnological,market
orfinancialopportunities,ifafirmisunabletoactuallyputoutaproductintothemarket(froma
resource point of view), then, they possess high organizational risks. Unavailability of resources
andmissedmilestonesareimportantrisks[68].Thesecangenerallybeattributedtoweaknessesin
management, structure of the organization, stakeholders [81], and failure of internal parties to
cooperate[69].R&Dmanagersmustalsofindabalancebetweenshort–termandlong-termproject
goals so that allocationof resources canbemanagedaccordingly, as theyareoftenan important
sourceofuncertainty[17].
13FinancialrisksarenotwidelydiscussedforR&Dandtechnologyprojects. Ibelievethatbecause
mostoftheliteratureisabouttheimportanceofobtainingcapital investments(andvaluingthese
projects), thus, its importance is inferred from literature that exposes the role of financing in
researchanddevelopment.
29
Organizationsshouldestablishmetricsforprojectsthatallowforclearguidelinesthatoutlinewhen
abandoningorterminatingaprojectbecomesnecessary14[17]assometimesthepersonalfeelings
or pride towards a belief about the potential applicability of a technology increase the overall
project risk andmay become amajor factorwhy delaying or ignoring termination of a project15
(thisthenbecomesanissueoftechnology-push)[17].
Environmental risks can be related to several facets such as political or social factors, public
interestandacceptance,andpublicacceptabilityoftheproduct[71].Regulatoryriskssuchaslegal,
industrialpolicies,andsourcingrequirementscanallbeclassifiedasenvironmental[76].
Theneed for competentmanagers is crucial inorder toavoidcosts,delaysoroverall failureofa
project [17], as it challenging for organizations to identify underlying latent causes of the
uncertainties [20]. R&D managers must define and address critical risks as soon as they are
identified [17]. Cooper and Kleinschmidt [21] argued that only having a “formal development
process” had no correlation with performance results. They state that many companies found
importanttaskssuchasmarketanalysisandcustomerresearchwerenotdoneordonetoolatein
thedevelopmentprocess.Theyadviseonfocusingonthequalityandnatureoftheseprocessesin
ordertobuildbestpractices,asthisiswhatwillreallydriveperformanceandpreventcompanies
fromfalselythinkingtheyareprogressing16[21].JanneyandDess[82]acknowledgethecomplexity
of risks and uncertainties in R&D projects. They believe the best approach for managers is to
identifyaprimaryuncertainty,trytocontrolitandexaminewhetheritaffectsotheruncertainties.
They argue that if different aspects of uncertainties are considered, the greater the chance is of
observingpotentialbenefits[82].
14Uponterminationreallocationofresourcesisnotonlyrequired,butalsoefficientfortheoverall
performanceoftheorganization.Noneedtotieupresourceswheretheycanbeusedelsewhere.
15Other factorscouldalsobereactions tocompetitorsorcustomersor technologyadvancements
[17].
16Asopposed to employees thinking they improvingprojectperformance solelybasedon “going
throughthemotions”oftheseassessmentsandprocesses.
30
2.3.2RiskManagementinR&D
The literature review conducted on risk assessment methodologies is a summary of the most
commonly used tools. There are qualitative andquantitativemethods that are used in industry.
Thepurposeofthisresearchisnottofocusonriskmanagementmethodsandtools.However,itis
important to discuss the possible optionsmanagersmay choose to incorporate in their projects.
Thesediscussedmethodsdonotrepresentanexhaustivelist.
“Risk”isatermthatcanreflectopportunities(i.e.positiverisk)orthreats(i.e.negativerisk).The
mostcommonusageofthisterm,however,usuallyreferstothedownside[83].Anytimetheterm
“risk”willbeusedthroughoutthisthesis,itwillreflectnegativerisk.Riskmanagementreferstothe
processesthatfirmsutilizetounderstand,evaluate,andtakeappropriateactionstodealwithrisks,
where project failure is reduced and the probability of success increased [74]. Therefore, risk
managementisacriticalfactorforbusinesssuccessandisavitalpartoftheoverallmanagementof
aproject[68].Thesubsectionswilloutlinetheimportantconstructsthatmakeupriskmanagement
thatcanbeapplied.
Thegeneralriskmanagementapplicationcanbesummarizedinafewsteps.Theseareidentifying
therisks,assessingthem,andfinally,applyingriskmanagementstrategiesthatcouldmitigateand
monitortherisks[74].
Riskidentificationisparticularlyimportantasitallowsmanagerstorecognizethecriticalrisksthat
mayprevent reachingproject goals [67].These critical risks canbe identified inmanyways that
may include, but not limited to, using the Delphi technique, scenario analysis, cause-and-effect
diagrams,fault-treeanalysis,interviews,surveys,questionnaires,aswellashistoricalandempirical
data[72][67][84].Identifyingcriticalriskscanbedividedintothreesteps:1)riskidentification,2)
riskanalysisand3)riskprioritization[67].
31
2.3.3RiskManagementTools&Techniques
Risks assessments are important as they rank and prioritize identified risks by estimating the
likelihood of occurrence, and severity of the consequence, either qualitatively, quantitatively or
semi-quantitatively[72].Therearemanyriskassessmentmethodsthatcanbeused.Failuremode
andeffectsanalysis(FMEA)isananalysistoolthatassessesforpossiblewaysfailurescouldoccur
andtheireffects[84].
Thecommonweakness in thesemethodologies (andmanyothers) is that they fail to identify the
correlation or relationships of different risk factors and compute their conditional probabilities
[72]. Sharma and Chanda [72] use the Bayesian Belief Network model (BBN) to establish
relationshipsbetweenriskfactorsinanR&Dproject.TheBBNapproachbeginsbyidentifyingthe
risksusinganyofthepreviouslymentionedmethods,thenidentifiestherisktriggers(causes)17and
theconsequencesof therisk factors.Finally, theBBNcanbeconstructed[72].Theapproaches to
constructingtheBBNareoutsidethescopeofthisthesis,however,see[85]forafullexplanationon
riskassessmentsandtheuseofBBNs.TheBBN’sinterfaceallowsdecision-makerstocalculatethe
conditionalprobabilities and their correspondingeffectsondependent risk factors in theproject
[72].
Aprobabilityriskmatrixcanbeused toqualitativelyassessrisks [86]. It isapopularandwidely
usedtoolasitsimpletounderstandandcanbecustomizedforriskcategoriesandlevelsthatreflect
management’sthresholdandtoleranceforrisk[87].Thematrixissimpletounderstandastherisks
areassessedwherethelikelihoodofoccurrenceandtheimpactoforconsequencearedetermined
based on predeterminedmetrics or scores [86]. Figure 8 is an example of a general probability
matrix.Amajordrawbackwiththeuseofprobabilityriskmatricesisthepotentialofinconsistent
assessmentsastheyarehighlysubjectivetoamanager(ororganization’s)aversiontorisk[87].
17 The authors identify risk triggers such as new technology, insufficient quality personnel, and
inexperiencedprojectleaders.Thesearecausesoftheactualriskfactors.
32
Figure8.3x3ProbabilityImpactMatrix
An arguably more comprehensive framework for evaluating business, organizational, and
technologyrisksforinnovativetechnologyprojectsistheRiskDiagnosingMethodology(RDM)that
wasstartedbyPhillipsElectronics[5].AstudybyKeizeret.aldiscussesUnilever’sadoptionofthis
methodology,whereemployeesunanimouslyagreedthattheRDMprocessesallowedthemtograsp
whatthecriticalriskswereandhowtohandlethembetterthantheirpreviousad-hoctechniques
[65].RDMisan8-stepprocessthatiscompletedwiththeexpertiseofriskfacilitatorsthathaveno
stakesintheprojectandcanprovideguidanceinanobjectivemanner[65].
In 1992, Kaplan and Norton developed the balanced scorecard method [88] that drives
management to connect their objectives to strategies, by prioritizing processes that are most
important [89]. A framework developed by Wang et al. [65] discusses that balanced scorecard
(BSC)isastrategictoolthatcanbeappliedtoR&Dprojectsforriskmanagementpurposes. They
proposethatBSCandqualityfunctiondeployment(QFD)beusedtocreateastreamlinedmethod
where managers can manage risks from a top-down approach. Risks are identified, assessed,
planned(mitigationplans)andcontrolled.FormoreinformationonHauser’sQFDandhowitwas
usedtoreducecycletimeofnewproductdevelopment,see[90].
Impact
Likelihood
Low Medium High
Low
Medium
High
33
2.4FinancialValuationMethods
2.4.1TraditionalValuationMethods
There isa largeamountof literatureavailable thatdiscussesvaluationmodelsandmethodsused
throughout the years. The two main approaches to value that have been used widely used to
forecastarethediscountedcashflow(DCF)andthenetpresentvalue(NPV)[91].
The biggest assumption associated with these methods is that the initial decisions made at the
beginning of the project are static and will not change [92][93]. These are now-or-never
approachestoinvestments[10]andtheydonottakeintoconsiderationmanagement’scapabilityto
strategizeduringprojectexecution,andability toactivelymanageaproject throughout itsentire
duration[92].Thecashflowsanddiscountratescarryahighamountofpotentialuncertainty,and
thedecision-making risk is high [94].Despite the limitationsof these approaches, the traditional
methodsshouldnotbescrappedastheyarestillnecessaryinputstoanoptions-basedapproachto
avaluation[6].
The benefits of these traditional valuation approaches are that they are fairly simple, widely
accepted,andareabletoreflectthemagnitudeoftheeconomicbenefits fromaninvestmentplan
[3][94]. DCF methods are derived from financial theory, however, Meyers argues that finance
theory and strategic planning have a large gap, which contributes to reasons behind their
shortcomings [95].Table4outlines theDCFassumptionsvs. therealitiesassummarizedbyMun
[7].
34
Table4.DCFAssumptionsvsDCFrealities[89]
DiscountedCashFlowAssumptions DiscountedCashFlowRealities
Decisionsdecidedup-frontandallcashflowsarestaticforthefuture
Uncertaintyandvariabilityinfutureoutcomes.Notalldecisionsaremadetodayassomemaybedeferredtothe
future,whenuncertaintybecomesresolved.
Projectsare“minifirms”,andinterchangeablewithwholefirms
Withtheinclusionofnetworkeffects,diversification,interdependencies,andsynergy,firmsareportfoliosof
projectsandtheirresultingcashflows.Sometimesprojectscannotbeevaluatedasstand-alonecashflows.
Allprojectsarepassivelymanagedoncetheylaunch
Projectsareusuallyactivelymanagedthroughprojectlifecycle,includingcheckpoints,decisionoptions,budgetconstraints,
etc.Futurefreecashflowstreamsare
deterministicandhighlypredictable
Itmaybedifficulttoestimatefuturecashflowsastheyareusuallystochasticandriskyinnature.
Projectdiscountrateusedistheopportunitycostofcapitalwhich
istheproportionaltonon-diversifiablerisk18
Therearemultiplesourcesofbusinessriskswithdifferentcharacteristics,andsomearediversifiableacrossprojectsor
time.
Allrisksarecompletelyaccountedforbythediscountrate.
Firmandprojectriskcanchangeduringthecourseofaproject.
AllfactorsthatcouldaffecttheoutcomeoftheprojectandvaluetotheinvestorsarereflectedintheDCFmodelthroughtheNPV
orIRR.
Becauseofprojectcomplexityandso-calledexternalities,itmaybedifficultorimpossibletoquantifyallfactorsintermsofincrementalcashflows.Distributed,unplannedoutcomes(e.g.,strategicvisionandentrepreneurialactivity)canbesignificant
andstrategicallyimportant.Unknown,intangible,or
immeasurablefactorsarevaluedatzero.
Manyoftheimportantbenefitsareintangibleassetsorqualitativestrategicpositions.
DCFmodelcanbepresentedbythefollowing[95]:
𝑃𝑉 = %&(()*)&
,-.( Where[95]:
PVisthepresentvalue
Ctistheincrementalcashflow
Tistheprojectlife
ristheexpectedrateofreturn
And,
NPV=PV–cashoutlayattime=0
18Non-diversifiableriskcanrefertoriskssuchasmarketorsystemicrisks.
35
Traditional NPV was initially established for stocks and bonds valuation, and assumes an
organizationholds realassetspassively [96].Thismethoddiscountsexpectedcash flowsand the
terminalvalueoftheprojectataspecificdiscountrate[97][8][10].Thechosendiscountrateusedis
a reflectionof theproject risks [97].Thisputsmanagersanddecision-makers inpassive rolesas
thisassumesthattheprojectwillplayoutasplannedandignoresthepossibilityoffuturechanges
intheproject[10][6][97].Becauseofthelackofmanagerialflexibility(whichisseentoaddvalueto
projects), it is seen that the traditional approach may actually undervalue a project [98][7],
especially forprojectsthatare longer,highlyuncertain,andpossesshigher interestrates,suchas
technologyandR&Dprojects[97][93].Theweightedaveragecostofcapital(WACC)isoftenused
as the discount rate as it is thought of as the opportunity cost of capital [99]. This, however, is
challenging for early R&D projects [99]. Managers involved in these types of projects require
methodologiesthatallowthemtojustifystrategiesfordevelopment[93]sinceR&Dprojectshave
differentphasesandshouldbeviewedasaseriesofdecisions,withvaryingrisksanduncertainties
[14]. The static NPV approach makes it seem like these qualities damage the investment
opportunity’s value19 [97]. Traditional DCF and NPV also assume that projects are independent,
whichisnotthecase.Thereareinherentinterdependenciesbetweenprojectsandtheseshouldbe
seenas links ina chainofprojects [97].Thesemethodsare suitable for short-termprojectswith
lowuncertainties,makingthesemethodslimitedforR&Denvironmentsthatarenaturallydynamic
andfluctuate[14][94].Thetraditionalvaluationmethodsmaybeadouble-edgedsword.Managers
maygiveupprojectswithnegativeNPVs[8]thathaveopportunitiestogrow,whiletheotherside
ofthisisthatmanagersmayendupmissingopportunitiesthatmaycomeupbecauseoftheinitial
positiveNPVthatwascalculated[9].
Another technique that is sometimesused is thepaybackperiod, and similar toDCF andNPV, it
lacks the ability to account for risks whichmay causemanagers to reject many long-term R&D
projects,as thismethodfavorsquickpayback[100].DCF,NPV,payback,andIRRarequantitative
valuationmethods.Aqualitativemethodthathasbeenusedisscorecardsthatguidemanagersto
shapetheirjudgmentsandstructurereasoning[1].Thismethodscoresprojectsqualitatively,such
ashowapplicableatechnologyisandranksthembasedontheirrespectivescores[101][3].
19Thisisnotalwaysthecaseastheoptiontodeferpartsoftheinvestmentcanbeconsidered[97].
36
Aweaknessofthisapproachisthatit isdifficulttojustifywhyascorewasgiven,andchallenging
formanagerstounderstandhowtoimprovethevalueofaproject[3].
2.4.2FinancialOptionsTheoryOverview
This section discusses general concepts that apply to the option theory models. Block [102]
identifiedthatthemostpopularvaluationapproachesincludedthebinomialmodel,Black-Scholes
model, and Monte-Carlo simulations [102]. The most popular models to solve options are the
binomialapproachandtheBlack-Scholes[103].Thebinomialmethodwasthemostfrequentlyused
asusersfoundittobesimplerwhencomparedtoothermethods,suchasMonte-Carlo[102].
Thevariablesinfinancialoptionsaretheunderlyingasset,volatility,exerciseprice,expirationdate,
interest rate and the dividends [104]. These variables are typically deterministic, except in real
options where these values are based on real assets and require more effort to define [104].
Volatility isan importantvariable inoptionpricingmodelsas it is theparameter that represents
uncertainty[104].Manyofthesevariablesarenotavailableforrealoptions,asparameterssuchas
timetomaturityortheexpiration,aredifficulttoestablishwhenmanagementhastheflexibilityto
makedecisionsthatalterthesevaluesduringtheproject[102].Theseoptiontheorieshaveabasis
on how discount rates are assigned depending on the levels of risk and uncertainty. This could
cause problems where the reliability of the valuation is questioned if there is no proper
documentation and justification [17]. Instead of varying the discount rate, factors that drive the
risksoftheNPVshouldbeidentifiedanddiscussed,andNPV(andrisk)rangesassessed[17].
Thesemethodsrelyheavilyonvolatilityvalues.Therelativevolatilityoftheoptionisnotconstant
anddependsonvariablessuchasthestockpriceandmaturity[105].Commonlyusedmethodsto
computevolatilityarenotapplicableforrealassets,andthereisanargumentthatthisvaluemaybe
manipulatedinordertoattainspecificdecisions[104].Managementmustensureanyassumptions
madeareonconservativegroundsfortheanalysistobepersuasive[104].Therearemanymethods
todeterminevolatility,however,thesemethodswillnotbediscussedastheyareoutsidethescope
ofthisthesis.Forfurtherreferenceondifferentmethodstocalculatevolatilitysee[7]and[106].
37
2.4.3Black-ScholesModel
TheBlack-Scholesequationisasfollows[103][7][107]:
𝐶 = 𝑁 𝑑( 𝑆3 − 𝑁 𝑑5 𝑋𝑒(8*,)
Where[103]
𝐶isthecalloptionvalue
𝑆3𝑖sthecurrentvalueofunderlyingasset
𝑟istherisk-freerateofreturn
𝑇isthetimetoexpiry
𝑑( =ln 𝑆3
𝑋 + 𝑟 + 0.5𝜎5 𝑇
𝜎 𝑇
𝑑5 = 𝑑( − 𝜎 𝑇
𝜎istheannualvolatility
𝑁 𝑑( and𝑁 𝑑5 representthevaluesofthestandardnormaldistributionat𝑑(and𝑑5
TheBlack-Scholesmodeloperateswiththefollowingassumptions[7][105]:
• Thevalueofacalloptionincreasesasthestockpriceincreases(andvice-versa).
• Ifthecalloptionexercisepriceincreases,thevalueoftheoptiondecreases.
• Thelongerthetimetomature,themorevaluableistheoption.
• If 𝑟 increases, the value of the option also increases. Short-term interest rate is assumed
constantthroughtime.
• Thehigherthe𝜎ofastockprice,thehigherthepossibilityofthestockpriceincreasingbeyond
thecalloption’sexercisepriceand,therefore,thehigherthevalueoftheoption.
• Therearenotransactioncoststobuyorselloptionsorstock.
38
Otherlimitationsinclude[103]:
• Itassumestheunderlyingassetsallowsfollowsalognormaldistribution20,thisisnotalwaysthe
casewithrealassets
• Itdoesnottakeintoaccountanyupsanddownsintheassetvalueandassumesthatincreases
invaluearecontinuousasdirectedbythevolatility
• The derivation of the equation is very complex mathematically and this causes a loss in
intuitionformanagerstryingtoapplyit,whichmakesitdifficultforthemtogetonboard
• Themodelwasdeveloped forEuropeanoptions that canonlybeexercisedona certaindate.
Thisdoesnotreflectthetruenatureofrealoptionsthatcanbeexercisedatanytime.
• Adjustments can bemade to themodel to help improve it, however, they tend tomake the
modelevenmorecomplex.
The Black-Scholes option theory has been used in industry to value real assets and investments
such as R&D and technology [105]. However, this model has limitations when applied to such
projects that are often exposed to varying types and levels of uncertainty. These projects that
usuallyincorporateaseriesofoptionsthatmayinteractwithoneanother(i.e.compoundoptions)
cannot be properly estimated by the Black-Scholes model, thus producing a value that may be
higherorlowerthantheactual[69].
2.4.4BinomialModel
Cox et al. [108] introduced the binomial model. This model is a discrete-time model that uses
binomial trees to show the changes in stock price with time. It assumes there are market
opportunities that createpayoff patterns for options [14]. There are twoapproaches to applying
thesemodels,risk-neutralprobabilitiesandmarket-replicatingportfolios[109].However,market-
replicating is amore complicated approach [7]. The risk-neutral approach assumes the option is
independentoftheriskpreferenceofaninvestor[108][110].Moreontheseapproachesin2.5.2.
20Variancerateisproportionaltothesquareofthestockpriceandthevariancerateofthereturn
onthestockisconstant[105].
39
The idea behind the binomial method is that it follows a multiplicative process over discrete
periods. The interest rate is assumed constant, and r, the riskless interest rate, follows the
following:u > r > dwhereu, is the upswing value andd, is the downswing21 [7].More detail on
binomiallatticeapplicationandoptionvaluationisshowninChapter3.
Figure9.GenericBinomialLattice[Adaptedfrom[7]]
There are two types of binomial lattices, recombining, and non-recombiningwhere recombining
aremorecommonlyusedthanthelatter[7].Bothlatticesyieldthesamevaluesattheendandthe
maindifferencebetweenthetwoisthatisthecenternodesofthelatticesaredifferent,wherenon-
recombiningassumesthatthevolatilityofanunderlyingassetchangeswithtime[7][111].Touse
thismodel,thebinomiallatticemustbedevelopedandthevaluesforu,dandpmustbecalculated.
pand1−paretheprobabilitiesthatarerisk-neutral[112].
21Otherwisetherewillberisklessarbitrageopportunitiesandrisklessborrowingandlending.No
arbitrageopportunityreferstothecurrentvalueofaninvestmentopportunitycannotbelessthan
theportfolionorgreaterthantheportfolio[14].
u
So
Sou
Sod
Sodu
Sou2
Sod2
Soun
Soundn
Soundn
Sodn
d
Year0 Year1 Year2 Yearn………
40
TheinitialvalueoftheprojectisrequiredtobuildthelatticeandcanbeestimatedbyaninitialNPV
withouttheuseofoptions[112][112][113].
Tocalculatetheupswing(whichisafunctionofthevolatility)[14][94][7]:
𝑢 = exp(𝜎 𝛿𝑡)
anddiscalculatedby[14][94][7]: 𝑑 = (W
Thevolatilityisrepresentedbythestandarddeviationofthelogarithmicfunctionoftheunderlying
freecashflowreturn,andtisthetimeassociatedwitheachstepofthetree[7].
Theriskneutralprobabilitypiscalculatedusing[14][94][7]:
𝑝 =exp 𝑟𝛿𝑡 − 𝑑
𝑢 − 𝑑
With all these variables, the values for all the nodes can then be calculated using simple
multiplication across the lattice [110]. To value an option, the backward induction method is
applied[112].Thedecisiontocontinueaprojectcanbedeterminedbyvaluingtheoption,whichis
completedusing thebackward inductionprocess that calculates the valueof a real option [113].
EachterminalnodehasavaluethatisthemaximumofzeroandthedifferencebetweenthevalueS
andtheexercisecostX(orinvestmentcost),whereMAX(S–X,0).IfthisdifferencebetweenSand
Xisnegative,thenthisvalueiszeroandconsideredasanabandonmentoption[114].Thisisdone
toeachpairofverticallyadjacentnodesrevealstheoptionvalueattheveryleftendofthelattice
[115].
Onelimitationofthisofapproachistheriskadjusteddiscountratethatisusedtovaluetheoption
acrosstheentiretree.Asthisvaluemaynotaccuratelyreflectthevaryinglevelsofriskacrossthe
tree[112].
41
2.5RealOptionsValuation
This section introduces fundamental concepts of real options (RO) and their value to R&D and
technologyprojectsthathavebeenwidelydiscussedintheliterature.
2.5.1IntroductiontoRealOptions:ValueasaDecision-MakingTool
RO research has been an important source contributing to financial economics and strategic
managementthroughanapproachthatencompasses investmentmethodologiesandflexibilityfor
decision-makersunderuncertainenvironments[9].StrategyresearchhasbeenfocusedonhowRO
adoption can create, sustain and improve an organization’s competitive advantage [9]. In 1977,
StewartMyerscoinedtheterm“realoptions”[116].TheROframeworkgivesdecision-makersthe
ability to invest, grow or abandon a project as more information is realized [117][112]. This
frameworkisanextensionoffinancialtheory,however,itreferstorealassets[112].Realoptions
valuation(ROV)wasdevelopedtoevaluatecapitalinvestmentsthatrequiredmanagerialflexibility
[112]. Option pricing methodology has been used to value real assets in the natural resources
industry but has now been applied in R&D, new technology developments and other areas [97].
Thereareseveralrealoptionsanalysisapproachesthatcanbeusedtotechnologyinvestmentsand
R&Ddecision-making[118][20].
ROtheoryisbuiltonthebeliefthatifdecision-makersareallowedtheflexibilitytoalterthecourse
of a highly uncertain project by incorporating strategic options, then, an organizations’ overall
valuecanincrease[9].Theserealoptionsarethoughtofasaguidethatcanleadtostrategicpaths
[9]andallowmanagers to favorablyutilizeuncertaintiesas theyevolve through theproject [10].
ROisavaluabletoolforriskyR&Dprojects[10]becauseitrecognizesthattheseprojectsoftenhave
longerhorizonsandarenotexpectedtogeneraterevenuesimmediately,butstillaccountsforthe
future potential [14]. This is accomplished by allowingmanagers tomake decisions at different
stages of theproject, such as,whether a project shouldbe continued [9][113]. This is unlike the
norm,wheremany financialdecisionsandresourcecommitmentsaremadeup-front,despite the
uncertainoutcomes[9].Abetterapproachwouldbetostagetheinvestments,asROallowsforthis
[9]. The sequence of decision-making allows managers to make justified decisions of when to
undertakeopportunities[14].
42
Manyofthetraditionalvaluationapproachesalso incorporatestandardinvestmentdecisionsthat
depend on creating profit, though thismay not be useful for R&D firms thatmay have different
goals,thatmaynotbetocommercializeandcreateprofit22[10].
TheframeworkforROemphasizesthatflexibilityofmanagementinhighlyuncertainenvironments
holdsagreatvalue[97].Thismethodgivesmanagerstheabilitytomakestrategicdecisionssuchas
stagingcapital investmentswhennecessary,wait tillmarketuncertaintiesaremanagedor future
prospectstobecomepromising,expand,orliquidatetheirassets[9].EachphaseofanR&Dproject
can be thought of as an option that depends on the success of previous stages (and options). If
successful,optionscanbeexercisedtomake larger investmentstocontinuetheprojectandgrow
(and launch a new technology), however, if there is a failure, then managers can decide to not
commitanylonger(andabandontheresearch)[3][20].Thislimitstheoverallrisktoonlytheinitial
investments of the project [3][20]. The RO managerial tool views strategic management as a
processthatallowsdecision-makerstoactivelyworktoreducethedownsidesofriskandcapture
the opportunities [119]. The application of RO affectsmanagers’ views on riskwhere it changes
from an attitude of avoiding it, to one that chooses to minimize or resolve it, leading to the
developmentofnewopportunitiesandotherpositiveoutcomes[82].
Figure10.ThestructureofRealOptions[Adaptedfrom[120]]
22Forexample,theycouldbedevelopingatechnologyforthe“greatergood”suchasaproductthat
isbeneficialtoimprovingtheenvironment.
Stage1
Stage2
LossofInitial
Investment
NetProfits
Donotinvest
InvestinOption
Unfavourablenews:AbandonOption
Favourablenews:exerciseoptionthroughfollow-upinvestment
43
2.5.2RealOptionsValuation:FinancialOptionsvs.RealOptions
Itisimportanttoclarifytherelationshipbetweenrealoptionsandfinancialoptionsandexplainthe
differences.ROapplyfinancialoptionstheorytoassessrealorphysicalassets.Infinancialoptions,
the underlying asset is the stock price [7]. In RO, this can be prices of real assets [7]. Financial
optionsprovideinvestorswithalternativechoices,whereriskaverseinvestorscanmakeaprofit,
andconservativeinvestorscanprotectthemselvesfromvolatilemarkets[116].
Table5.FinancialOptionsvs.RealOptions[Adaptedfrom[74]]
OptionTerminology FinancialOptions RealOptions
Writingtheoption Formalcontractincludinglegaltransactionterms
Initialdecisionthatcreatestheopportunity.No
requirementforformaloptioncontract
Exercisingtheoption Formallyactivatethetermsofthelegalcontract
Subsequentbeneficialdecisioninlightofthe
information
Strikeprice Transactionpricefortheoption
Thedecisionrulethatinformsthemanagertomakethedecision
ExercisepriceTheunderlyingasset’s
pricewhenthedecisiontoexerciseismade
Thecostofmakingthedecision
Liquidity&tradability Liquid,asmarketsexistforfinancialoptions
Rarelyliquid,difficulttotrade
Timing Pre-determined,precise,finite
Canbepre-determined,not-finiteandcanlast
indefinitelyPortfolio Acollectionofoptions Acollectionofdecisions
Underlyingasset Publiclyheldstock Tangibleand/orintangibleassets
Scholes[105]describedanoptionasthesecurityofhavingtherighttobuyorsellanassetunder
specifiedconditionsforacertainperiodoftime.Acalloptionistheright,butnottheobligationto
purchase, somethingofvalue (i.e. stock)ataprice thathasbeenagreedonbetweenabuyerand
seller [121][79][2][119][122]. The details behind call options vary from market to market.
American options canbe exercised any timebefore the date of expiration. Europeanoptions are
exercisedatadefinedfuturedate[105][97][116][7][106].Thereareotheroptiontypesthatexist,
suchasBermudanandexotic[7].
44
Aputoption is the right to sell somethingofvalueataprice thathasbeenagreedonbetweena
buyerandaseller,withtheexpirationdateandoptionrulesspecified[116].Anexercisepriceor
strikepricereferstothepricethatwaspaidforanasset(or fixedshareprice)whenanoption is
exercised.The final daywhich anoptionmaybe exercised is referred to as thematuritydate or
expirationdate[105][7][94].AROisdefinedastheright,withouttheobligationtomakedecisions
(suchasdefer,abandon,alter,etc.)regardingrealassets[123][124].
Thenatureofuncertainties alsodiffersbetweenROand financial options,where theuncertainty
liesinstockpricesandthevariabilityofthepriceofunderlyingfinancialassets[119].Factorsthat
affectthevolatility,thusalsoaffectthevalueofafinancialoption[79].
Infinancialoptiontheory,thehigherthestockprice,thehighertheoptionvalue.Therefore,when
thepriceofastockishigherthantheexerciseprice,investorsareverylikelytoexercisetheoption
[105]. If the stockprice is less than theexerciseprice, investorsarenot likely toexercise, as the
valuewillbeclosetozero[105].Thevalueofanoptiontypicallydecreasesastheexpirationdate
nearsandifthestockpricedoesnotchange[105].
InROtheory23,thevalueliesintheunderlyingasset,andiftheassetvalueincreases,thensodoes
thevalueoftheRO.Likewise,iftheexercisepriceincreases,thentheROvaluedecreases.Ifthetime
tomaturity is increased, then the RO values are consequently increased [121]. A longer time to
expire allows more time to learn about the uncertainties, which increases option value [94]. If
uncertaintyincreases,thensodoestheROvalue,asmanagementcantakeactionandcapturethis
opportunity (andavoid thedownside). If therisk-freeratevalue increases, sodoes theROvalue.
Finally,ifdividendspaidout(byanunderlyingasset)increase,sodoestheROvalue[121].
23Definingthesevariablesallowsarealoptiontobevalued.
45
Therearetwomainapproachestorealoptionsanalysis(ROA),thefirstoneisamarket-replicating
portfoliomethodthatimitatesthepayoffsofaproject,wherethevalueofaprojectisvaluedusing
theno-arbitrageapproach[96][7].Thesecondapproachistherisk-freediscountratemethodthat
calculatesadjustedprobabilitiesusingarisk-freediscountrateinordertovaluetheproject[96][7].
Both these approaches model the project’s uncertainty using geometric Brownian processes or
binomialtrees[7][96][107].
2.5.3TypesofRealOptions
There is a large amount of literature that describes types of options and their details24. R&D
projectsaregenerallyassociatedwithmanyrealoptions [98], and there isnospecificnumberof
real option types that researchers and academics fully agree on [10]. The basic real options are
defer or stage, grow, alter the scale (expand, contract), switch, and abandon [125][126]. The
specificsbehindmanagement’smotivationwhyeachoptionisconsideredandultimatelyexercised
isuptothemandtheopportunityvalueoftheoption.ThemostcommonROsaresummarizedin
thissection.
Optiontodelay/defer:ifthecurrenttechnologyisnotuptoparintermsofperformance,maturity,
etc., thenmanagerscanchoosetostrategicallydelayaprojectbyaspecificamountof timeto try
andachievetheirtargets[127].Managementcanwaitaspecifiedamountoftimebeforetheyneed
toexercise[94].Thisoptionisalsoexercisedwhenuncertaintyneedstoberesolved,therefore,the
valueofthisoptionisaffectedbytheuncertainty[67].
Optiontoabandon(termination):thedecisiontostopaprojectisalmostalwaysanoption[7].Ifa
technology or project may not meet specific organizational requirements25 such as market
requirements, changes in regulation, under-estimation of required resources or over-estimating
potential,thenaprojectcanbeterminatedtoavoidadditionalloss[20][104][127].
24 “Detail” refers to actual specifics of the option. For example, growth option may refer to
commercialization,however,commercializationpathsaremanyandthisisuptoanorganizationto
decide.
25Orforexample,notmeetingTRLtargettoprogresstothenextlevel.
46
Optiontoexpandorcontract:thisallowsmanagementtomakechangessuchasexpandproduction
scale or speed up resource allocation whenmarket conditions are favorable. In the case where
conditionsarenotfavorable,thenmanagerscanchoosetoreducethescales(i.e.contract)[121].
Optiontoswitch: ifconditionssuchasdemandorpricechange(orothers), thenmanagementcan
choosetoswitchresourcesorshiftindustries[69].Itgivesmanagerstheabilitytoswitchbetween
technologies,marketsorproducts[97][20].Ifthereisachangeindemand,thenthetypeofproduct
producedmay be changed (i.e. product flexibility) or the inputs to produce the productmay be
changed(i.e.processflexibility)[94][121].
Optiontogrow:Growingtypicallyreferstoanincreaseofcommitmentorinvestmenttoimprovea
technology’s performance [20][69]. This option aims to further develop a technology or build
experience [69] and can sometimes be treated as a compound option where future options are
chosenandexercisedbaseddependonthisoption[92].
Otheroptionsthatcouldaddvaluetotechnologyprojectsinclude:
Stage or compound options: this option refers to staging the investments so that the option to
abandon is created midstream if poor information is realized[92]. This is important in R&D
intensivefirms,capitalintensiveandstart-upventures[69][6].Thisoptionisadevelopedformofa
growthoption, as it has successive stageswith increased volumeof activity that depends on the
previousones[92].ThisoptioncommonlyusedinR&Dasanoptiontodevelopatechnologymay
beexercised,thenasubsequentoptiontocommercialize26[128].
Outsourcingoptions:reasonsforconsideringthisoptionincludecost-cutting,lackofemployeesor
expertise,andenhancingcompetitiveadvantagethroughtheexpertiseofavendor[20].
License-inoption:thisoptioncangiveearly-stagetechnologyprojectsaccesstonecessaryanalytical
validation methods and saves R&D organizations time and cost [20]. This option allows
organizations to identifyearly stageopportunities [67].Many factors candelay—orconceal—the
needtoterminateprojects, includingpersonalpride,competitoractions,customerprocesses,and
technology advances. It is thus important to establish the criteria for termination,may increase
uncertaintyasthetechnologyisathird-partyproductwhichmaycauseproblemswithtechnology
transfer[20].
26Thenatureof thestagedoption inR&Dmakes it clear thatusing theBlack-Scholes formula,or
anyofitsvariancethatallowdividendpay-outswillnotbeanappropriatemethod[128].
47
Collaborate/strategicalliance:thisisastrategicoptiontoincreaseefficiencyandreduceriskwhere
organizationsmayworktogethertocapturecompetitiveopportunitiesanddecreasethedownside
ofrisks[67].
Itisimportanttobuildabridgebetweenthetypesofrisksandtheireffectsonoptionconsideration
(andultimately selection). The higher the uncertainty, the greater the number of possible future
outcomesthatmustbeconsideredandassessed[3].Ifatechnologydoesnotlookpromisingfora
certain application, then other applications may be possible. Decision-makers are bound by
rationality where it is not possible for them to take into account all possible occurrences as a
technologydevelops[3].Thefollowingtableisasummaryoftheliteratureonsomeofthecommon
risksinR&Dandtechnology,andtheireffectsonthetypeofoptionsthatmaybechosen.Thisisnot
an exhaustive list anddoesnot replace theneed for experts or technology strategists todevelop
appropriateoptions.
Table6.RisksandOptionsTypes[adaptedfrom[69][20]]
RiskCategory Uncertainty(+/-) Options
Technology Technicaldesign(-) Switch,delay,abandon
Market Demand(+) Expand,switch,delay,license,abandon
Financial Capital&financingrequirements(-) Contract,delay,abandon,
CommercializationSupplychain&sourcing
(-)Regulatorychanges(-)
Switching,abandon,strategicalliance
Switching,delay,abandon
Organizational Resourcerequirements(-)
Outsource,delay,switch,abandonStrategicalliance,defer,abandon
48
2.6RealOptionsAnalysis
2.6.1ApplyingRealOptions:TheSteps
Although the specifics behind frameworks developed for each application may vary, the basic
methodisstandard.Thissectionwilloutlinethestepstoapplyingrealoptionsanalysis.Notethat
these steps listed are only the computational steps and do not include the steps that would be
required foracomplete framework for technologyandR&Dprojects(thedecision-makingaspect
too).Thestepsareasfollows[7][129][104][130][121]:
1. Computethebase-casetraditionalNPVanalysis
2. Modeltheuncertainty(usingMonte-Carlosimulationsorother)onthebase-case
3. Usingtheinformationfromsteps1and2,frametheROproblem:
a. Identifytheoptions
b. Select thevaluation toolormodel (Forexample:Black-Scholes,usinga replicating
portfolio[107])
c. Identifyallinputsforvaluation
4. ConductROA
a. Applythemodel
b. Obtaintheoptionvalue
c. Conductasensitivityanalysis
5. Reportandupdateanalysis
For example, if the binomialmodelwas applied, a general application canbe summarized in the
following steps [112][131]. A full application will be discussed in Chapter 3 and conducted in
Chapter4.ThestepstoapplyROAusingthebinomialmodelare:
1. Identifyalldecisionsequences,managerialflexibilities
2. Modeluncertaintiesbyspecifyingvariables’distributionsandrandomvariables
3. RunMonte-Carlosimulationsforthemodels(withouttheoptions)
4. Obtaincashflowforeachperiod
5. Estimatevolatilityoftheproject
6. Calculatebinomialmodelparameters(u,d,p)
7. Buildthebinomiallattice
8. Addoptionsandsolvethetreeusingtheriskfreerate(applybackwardinductiontovalue
option)
49
TheROvaluationmodelspreviouslyintroducedincorporateprobabilitieswhicharerequiredtobe
sufficiently reliable [66].Analystsusuallyestimate theprobabilityof successfuldevelopmentofa
projectoratechnologyfromstatisticsonsimilarprojectswhenthisisavailable[93].Thisisdifficult
toachievewithearly-stageR&Dprojects,however,itisbettertomodeluncertaintyandobtainbest
estimatesofprobabilities,as this typeofmodelingenablesmanagement tobetterchooseoptions
for different outcomes [66]. The process of valuing real options has several aspects, it estimates
financialgain,butalsostrategicpositioningandknowledgeisgained[107].
AnyROAshouldincludeasensitivityanalysisastheseR&Dprojectshavemanyassumptions[93].
Examplesofcommonsensitivityanalysisvariablesinclude[93]:
• Probabilityofsuccess(ofR&Dortechnologydevelopmenteffortorcommercialization)
• Costtoimplement
• Marketuncertainty
• Volatility
• Expirationoftheoption
Commonlyusedapproachestounderstandhowuncertaintybehavesinclude[117]:
• Conductingasensitivityanalysis
• Conducting a scenario analysis (best andworst-case scenarios to understand overall project
uncertainty)
• AMonte-Carlo simulation (probabilisticmethod that assesses the likelihoodof each outcome
andobtainsaprobabilitydistribution)
• ABayesiananalysis(toassesstherandomvariables)
Asdiscussedpreviously,volatilitystronglyinfluencestheoptionvalueoftheasset[117],andmany
of the R&D and technology projects do not have any historical or empirical data to accurately
estimate the volatility (and therefore, the probability of success) [117]. However, the literature
discussesestimatingthevolatilitywiththeuseofaMonte-Carlosimulationonthebase-casecash
flow,whereadistributionfortheprojectvalueisproducedandthestandarddeviationrepresents
thevolatility[130].
50
In general, technology projects carry a high amount of technology and market risk, which is
challengingtoestimatevaluesfortheparametersinthemodelduringtheinitialstages[20].During
theseearlyphases,ROcanbenaivelyappliedbymanagersastheuncertaintyinassumptionsmade
forvariablessuchasprobabilitiesofsuccessareexpectedtobehigh[1].
The use of RO as a decision-making tool has faced many organizational and implementation
challengesinpractice,asscholarshavenotbeenabletoencompasstheknowledgeinanorganized
and cohesivemanner [9]. Another point is that there is noway to actually enforcemanagers to
follow thedecision-makingoptions identified [102].Even though there is theoption to abandon,
managersmayalreadybedeeplyinvestedinthetechnologyandmaynotwanttoberecognizedas
havingpoorjudgementandriskbacklashfromhigherexecutives[102].
2.6.2Monte-CarloSimulations
ItisimportanttodiscussthemainpointsandideasbehindMonte-Carlosimulations.Moredetails
andstepstoapplyingarediscussedinChapter3.Tocountertheuncertaintyintheestimationsof
values in theROmodels,asensitivityanalysis isconducted toobservehowvalueschange,which
can be modeled using Monte-Carlo methods [3]. The name “Monte-Carlo” simulation originated
during World War II as a method applied to the development of the atomic bomb [132]. This
method isused to solvemanyproblemswithstochasticvariables in statistics thatarenot solved
analytically [132]. Monte-Carlo simulations are valuable tools in the financialworld and in real
options[117],andtheprocessisprogramming-intensive,withoutcomesthatarehardtoverify,as
theyrequirerigorousassumptionsaboutprobabilitydistributionsoftheinputparameters[104].
Usually,asensitivityanalysisisfirstperformedonthediscountedcash-flowmodel;thatis,setting
thenetpresentvalueorROIastheresultingvariable;wecanchangeeachofitsprecedentvariables
andnotethechangeintheresultingvariable[133].Precedentvariablesincluderevenues,costs,tax
rates,discountrates,capitalexpenditures,depreciation,andsoforth,whichultimatelyflowthrough
themodeltoaffectthenetpresentvalueorROI.Bytracingbackalltheseprecedentvariables,we
canchangeeachonebyapre-setamountandseetheeffectontheresultingnetpresentvalue[133].
Input variables to themodel are assigned probability distributions. Formore information about
howtoselectprobabilitydistributions,see[132][7].
51
Thesimulationallowscontinuousdistributionstobemodeled,resultinginafullrangeofvariability
ofparameters[132].Theoutputsofthesimulationprovideimportantinsightintothebehaviourof
variablesandmaybeusedasinputsfortheROvaluation.Atornadochartisthenproducedwhere
the most sensitive input variable is shown first. Using this information, managers can then
determinewhichvariablesaredeterministicandwhichareuncertain.Someofthesefactorsmaybe
correlated,which could requiremultidimensional simulations. Correlations are often determined
fromhistoricaldata[133].
2.6.3RealOptionsinIndustry
Therearemanycompaniesthathaveadoptedrealoptionsframeworksaspartoftheirinvestment
methods such as Airbus, GE, Hewlett Packard, Intel, and Toshiba [119]. Hewlett-Packard has
employedRO tomatch supplyanddemandsand the trade-offsbetween the costsof components
andtheflexibility[104].IntelandToshibahavebeenusingROtoevaluatelicensingopportunities,
whilemany companies inSiliconValleyhaveadoptedROas ithas led to increased collaboration
between companies [104]. Pharmaceutical company,27 Merck, has famously analyzed their R&D
investments using RO frameworks, allowing them to evaluate their decisions under high risks
[104].TheyhavealsousedBlack-Scholestovaluethebenefitsofjointventures[109].Eventhough
theBlack-Scholesmodel isnotappropriateforR&Dcompoundoptions,Merckbelievedtheyhave
enoughhistoricaldatafromtheirR&Dprojectstobeabletomakeappropriateassumptions[128].
ShellOilusesROAtoassesscapitalprojectsand their investmentstrategiesand tomodel for the
perfectextractiontimesfortargetoil-fields[104].
27ForanotherapplicationofRObinomialapproachinthepharmaceuticalindustrysee[98].
52
OtherindustriesthathaveheavilyusedROincludeITandtheenergysector(renewableandnon-
renewable)[121][134][2][104][94].AlthoughtherearemanydifferentapplicationsofRO28usedin
different industries,most have been utilized to value capital investments and project selections.
Although useful, this is not themain point of this thesis. For this reason, only themost relevant
frameworksrelatedtoR&D29andtechnologydevelopment(andtheirrisks)willbediscussed.
Perhapsthe frameworkthatmostcloselyresemblestheworkproposed inthis thesis isShishko’s
andEbbler’s frameworkapplication [135][136] toNASA’s technologies.Theirworkexplores real
options and decision-making in a technology context that uses TRLs [135][136]. The model
developed targets long, high-risk technology investments up to and including TRL 6 [135][136].
Themodel isacommon frameworkapplicable to the threeclassesof technologiesatNASA:cost-
reducing,mission-enhancingandmission-enabling [136]. Theydiscuss themostobviousrisks in
theinitialstagesoftheirdevelopmentandtheexpectedshifttomarketrisksbeyondTRL6[136].
They identifiedthat theunderlyingassumptionsofBlack-Scholeswasnotapplicable for theearly
stageprojectsandcomplementedtheirmethodologywiththeuseofMonte-Carlosimulationsand
decision trees to model the variability of their stochastic variables [135] [136]. Their main
challengewas estimating the risk-neutral probabilities for these long-termprojects, the need for
assumptionstobemade,andwhetherNASAasawholeshouldberisk-neutral[135][136].
Wangetal.[20]developedaROframeworkforR&Dplanningthatspecificallytargetstechnology-
based organizations, and allows managers to identify risks and capture opportunities in three
stages. The first stage is opportunity identification, using market-product-technology linkages
wheremarketdriversareidentifiedandperformancetargetsareset.Thesecondstagedevelopsthe
opportunities,where the effects of uncertainties on the technology or performance are assessed
andidentifiedintermsofpossibleoptions[20].
28 There have been many ROmodels that have been adapted depending on the application, for
examplesee[93] forahybridmodeldevelopedforriskyproducts.Thismodelsplitsanalysis into
financialandtechnologicalaspectsoftherisk.
29LookingatR&Dasastep-by-developmentalprocessandconsideringallaspectsofrisk.
53
Forexample,specificuncertaintieswereanalyzedonwhethertheywouldpositivelyornegatively
influencetheproject,andasetofoptionsforeachpotentialriskwasconsidered.Thethirdstageis
theopportunityevaluation,wherethedecisionsareevaluatedandproductdiffusionspeedandits
effectonpayoffsarestudied[20].Theframeworkaccountsfortheevaluationofoptimaldecisions
tomaximizemarketopportunitiesundervariousdemandstructuresusingthediffusionmodel[20].
This work was applied to an R&D biochip project and results showed that by assessing
uncertainties and exploring options tomanage them, the expectedmarket payoff increased [20].
Wanget al. emphasize the importanceof growinga firm’smarket researchcapability inorder to
fully capture market requirements, and avoid the technology-push approach. They suggest
considering theuseofMonte-Carlomethods in futureresearch, to fullyunderstand the impactof
uncertaintiesondecisions[20].
Thenextchapterbuildsonconceptsdiscussedintheliteraturereviewtodevelopaframeworkthat
canbeappliedbymanagers,engineers,anddecision-makersintechnologyandR&Dprojects.
54
Chapter3DevelopmentoftheROADecisionFramework
ThepurposeofthischapteristodeveloparealoptionsframeworkthatcanbeappliedacrossR&D
and technology projects. The framework is expected to guide decision-makers into exercising
flexibilityduringprojectssothattheupsideofopportunitiesismaximizedduringriskytechnology
development processes. The proposedmethods establish the groundwork formanagers to start
thinkingahead,andtoconsiderthechallengesandpotentialopportunitiesoffuturestagessuchas
commercialization, in order to create innovative productswith amarket need. It creates a basic
understandingoftheprominentrisksduringdevelopmentandaimstodrawonsomeofthebasic
trendsandbehavioursoftheserisksduringdifferentstagesofaproject.Thetrendsintheserisks
arelinkedtocommonlyoverlookedfactorssuchasresourcerequirements.
3.1FrameworkMethodology
The proposed framework for risk evaluation of a technology development project combines a
stage-gateprocessfordecision-makingbasedontechnologyreadinesslevelwiththeriskprojection
methodofrealoptionsvaluation.
Theframeworkdescribesaprocessforrecommendingasmallnumberofoptionstobeplacedalong
theTRL scale and identifies important risks associatedwith technologyprojects. The framework
alsodiscussesanapproachtoapplyingROvaluationmethodstoguidemanagerstoevidence-based
decisions in an individual technology-developmentproject andacross aportfolioof projects that
mayeachentailadifferentlevelofrisk.
The rationale behind using TRLs as opposed to any other maturity assessment approach was
primarily due to the popularity of this toolwith large organizations (as discussed in Chapter 2),
suchasNASAandNationalResearchCouncilofCanada[54]thatfrequentlyuseit.Theapplication
of RO captures the expected value of decisions at different stages of development, using the
binomial lattice approach. The remainder of this chapter explains how to select points for
determininginvestmentoptions,howtodevelopthevaluation,andimportantconsiderationswhen
usingthisframework.
55
This frameworkcanbedivided into threephases. The first twophasesbeing largelyqualitative,
with management screening for appropriate strategies and projects to consider. Defining
organizational goals, project milestone planning using TRLs, identifying risks, and conducting
analysisareallcompletedduringthesephases.ThethirdphaseistheapplicationofROvaluationto
decisionoptions.
Phaseonefocusesondefiningtheproblem,collectingdataandrelatedinformationforpreliminary
planning and analysis. Phase two consists of understanding requirements needed to advance to
higherTRLsinthecontextofthetechnologyandorganization.Thiscallsforformal&completeTRL
assessments,wheretechnologydevelopmentphasesaredescribedinsixgeneraltypesofactivities:
• Scientificresearch
• Engineeringdevelopment
• Marketing
• Businessdevelopment
• Financing
• Commercialization
Critical risk factors are identified and assessed using relevant risk assessment tools. Decision
milestone schedules and a plan to collect information to improve estimates as project proceeds
(marketing, technology readiness, etc.) are developed. Managers must identify these key risk
factors that drive the decisions behind optionalities and affect their overall value. Phase three
quantifies the value of different options through the use of the binomial lattice approach. The
decisionsarevaluedandthevariabilitiesofparametersareanalyzedanddiscussed.
56
Figure11.FrameworkSetup
3.2Phase1:CollectionofData&InitialPlanning
Phaseonegivesanoveralldescriptionoftheentireproject.Datarelatedtotheprojectdescription
canbeacombinationofqualitativeandquantitative.Managersbeginbydefiningtheproblemand
motivationbehindwhy theparticularprojectwill bepursued, andpossibleproject options.Data
and informationmaybe collected fromvarious sources,whichmay includemanagers, engineers,
stakeholders,andinvestors.Inthecaseswherearelevantorsimilarprojectisavailable,thismaybe
usedasasourceofdataforcomparativeassessments.
Interviewsandsurveyscanalsobeusedasamethodofinformationgathering.Questionnairesand
interviewswithmanagersandstakeholdersmaybe themainapproach for informationgathering
duringtheearlystages.Preliminaryprojectschedulesaredevelopedandmajormilestonescanbe
identified during this phase. Technological area(s) of application and target industries should be
considered.PotentialR&Dteam(s)andotherkeyprojectpersonnelcanbeselected,orplanscanbe
madetoacquiretheappropriatetalent.Capitalcostrequirementsandpotential financingoptions
maybediscussed(ifnotalreadysecured)andpreliminaryplansmade.
Defineproblem&goals
Collectdata&plan
TRLassessment&riskassessment ROValuation Decision
Phase1 Phase2 Phase3
57
Themoreinformationgatheredduringthisphase,themoredetailedtheTRLandriskassessment
can be. R&D data are often limited for high technology projects, and somanagers can expect to
exercisetheirjudgementandmakeeducatedassumptionsduringthesestages.
3.3Phase2:TechDevelopmentStages-gates&TRLAssessment
It is important to use a consistent definition of project development, especiallywhen applying a
methodofassessment fora rangeofprojects inaportfolio.Phase twobeginsbyestablishing the
stages of technology development against the TRL scale as part of the framework. Mapping the
developmenthelpswiththeunderstandingofthetypeandnatureofworkinwhichanorganization
(and potentially its key partners) will engage in during different TRLs. Figure 12 is a simple
illustrationthatsummarizesthemajorphasescapturedintheTRLframework.Theproposedmodel
mapsthestagesofdevelopmentasinspiredbytheDoD’smodel[35]andLeeandGartner’s[25].-
Proposing a very specific framework with too many milestones this early in the research may
actuallyhavenegativeeffectsondevelopmentand takeaway from themanagerial flexibility that
hasbeenstressedasanimportantcomponenttobetterdecision-making.
Figure12.StagesofTechnologyDevelopmentAgainstTRLs[adaptedfrom[36][25]]
SystemDevelopment&Demonstration
TRL7TRL1 TRL3TRL2
TRL4 TRL5 TRL6 TRL8 TRL9TechnologyDevelopment
Production&DeploymentS
yst
Pre-conceptRefinement ConceptRefining
ScaleUpandCommercializationBasicResearch
TechnologyDevelopment
MarketandIndustry
Go/NoGo Go/NoGoGo/NoGo
58
The pre-concept refinement refers to the fundamental research conducted to support
understandingscientificphenomenon.Thecostassociatedwith resourcesduring these levelsare
typically low (compared to commercial implementation activities at later TRLs). More resource
requirements,costsandtherelativeriskswillbediscussedin3.4.DuringTRLs1to3,researchers
should be in the business of linking scientific and technological advancements with a potential
market,toavoidthepitfallsassociatedwithtechnology-pushproducts.Therefore,earlymarketand
feasibility studies are important during this stage, as well as direct contact with first potential
users.Conceptrefinement inTRL4refers toappliedresearchwhereresearchershave linked the
technologytoapotentialapplicationandoperatingenvironment.TRL4isalsowherethecreation
of IP assets begins, and patent strategy decisions are made. Industry and competition studies
shouldalsobecompletedduringtheearlyTRLs.
Thetechnologydevelopmentstage(TRLs5to6),isprototypingandpilot-testingintensive.Atthe
endofthisstage,thetechnologyshouldprovetoaddeconomicvalueforitsintendedapplicationor
industry.Duringtheselevels,organizationsmaybeginstrategicallianceswithotherorganizations
of interest30. TRL 4 to 6 is a critical period during development as there is a dramatic shift in
financialandresourcerequirementsandobservedrisks.Thefunctionalcapabilityofthetechnology
moves from a laboratory environment, to an operational environment, and activities that rely
heavilyonscale-up.Thisperiod isoftenreferredtoas the technology“valleyofdeath”andmany
organizations face great difficulties progressing beyond TRL 6 [56] due to the predominant risk
factorsinthisstage,whichisdiscussedfurtherinthenextsection.
NearingtheendofTRL6,atechnologyprototypeshouldhavebeensuccessfullydemonstratedina
relevant operating environment. Early-stage IP should shift to validated IP, as this has a strong
influenceoncommercializationoptionsandfinancingandinvestmentactivities.Afiledorpending
patentmaybea favorable featuretosomeinvestorsas itprotectsaspectsof thetechnologyfrom
competitorsandmaybea“securityblanket”assellingtherightstothepatentmayyieldareturn
[137]ifthingsdonotgoaccordingtoplans.
30Forexample,ifthereisacompetitorinthesamemarket,thenastrategicoptionmaybetoThis
joinforcesandcollaborateinsteadofabandoningaprojectcompletelyorlosingtocompetition.
59
TRLs past TRL 6 are heavily involved in scale-up work, and aggressively seeking marketing
opportunities and other commercialization options. Business development managers and other
marketingexpertsrequiredtofillknowledgegapsshouldhavebeenhiredbythisstage.However,
buildingamulti-disciplinaryteamshouldbeginatearlyTRLs.ByTRL7,commercializationoptions
shouldhavebeenconsidered(andpossiblyselected).Strategicmarketinganddistributionplansfor
technology positioningwithin an industry should also be completed. This includesmarket entry
strategiesaswellmarketpenetrationplans,andcompetitivedifferentiation(ifany).Organizations
should begin building relationships with key market players and early customers and the
community.AttheendofTRL7,theproductshouldhavebeenproventohavecommercialization
ability.Thefinancingexpectationstoreachtargetmaturitylevelsshouldbeexplicitandconsistent.
This is to ensure that the technology can pass through the relevant stage-gate assessments and
reviews.
ThefinalstageofproductionanddeploymenttakesplaceinTRLs7through9.Thetechnologyhas
been proven suitable for full commercial use (as the technology is now fully implemented and
tested).Humanandorganizationalrequirementsarestillhigh,asnewchallengestomeetdemands
arise.MaintainingandcontrollingIPisimportantduringthisstage.Relationshipswithkeymarket
playersandusersaremaintainedandstrengthened.Thesocialimpactoftechnologyshouldalsobe
considered.Fixingofsystembugsandtechnicalglitchesisalsoexpectedwiththelaunchofanew
technologyandinnovationiscontinuouslydrivenbyfeedbackfromendusers.
Theframeworksuggestsaminimumofthreemainstage-gatesoneatTRL4,TRL6,andTRL7.The
frameworkisalsodevelopedwiththeassumptionthatorganizationsutilizingthisframeworkhave
TRLtargetsbeyondTRL6.However, this frameworkcanbeappliedforthosethatdonotwishto
commercializeandlaunchandwishtoremainatlowerTRLs.
60
Thestage-gatesdiscussedarenaturalpositionswhereactivities, resourcerequirements(whether
financialororganizational) changeandmanagement is required to strategize toavoid fatal risks.
Stage-gates are logical, reasonable points during to place real options during the project. These
gatesorlogicalstoppingpointsarewheremanagementshouldexpecttoapplyrealoptionswhere
their decisions can be evaluated using ROA. Management may choose to add more options as
projectsprogressorrisksarerecognized,buttherecommendationofthreeoptionsisconsideredas
thebareminimum.
Stage-gates drive decision-making in R&D and empower researchers tomake an evidence-based
businesscaseforprojectcontinuation.Inthecasewherecomponentswithinasystemareassessed
tobeatdifferentTRLs,thenanoptionmaybeplacedtore-allocateresourcesandfocusonacritical
pathofacomponenttoadvancetohigherTRLs,ordecidetoreachanearlydecisiontoabandonthe
project.
Figure13.FrameworkSetupofStage-gates&RealOptions
CompletinganddocumentingaformalTRLassessmentisimportant,however,willnotbeshownin
this framework as any of the assessment methodologies discussed in Chapter 2. Ultimately, the
assessmentmethodologyusedwillbeup tomanagement’sdiscretionandmustbeapplied in the
contextofthetechnologyandtheorganization.
Go/NoGo
TRL1 TRL2 TRL3 TRL4
TRL5 TRL6 TRL7 TRL8 TRL9
Go/NoGo Go/NoGo
MinimumRecommendationofRealOptionsPlacementforTechnology
Development
LogicalStage-GatesDuringTechnologyDevelopment
61
AsampleTRAusingtheNASAworkbook[138]requiresorganizationstoidentifyalternativeroutes
in the case the technology does not mature according to plans. NASA recommends developing
schedules that includemilestones, how thesemilestones will be assessed, and requirements for
funding during the first stages to help achieve target TRLs. Potential (known) costs and risks
associatedwithtransitioningfromearlyresearch,toproductionandbeyond,shouldbeidentified.
This ties in well with the proposed framework as the project plan guides decision-making
processes. It also allows managers to consider a variety of applicable options that support
technologydevelopment.
Forcompleteness,theuseofthestructuredandanalysisdesigntechnique(SADT)isrecommended
tobeusedtohelpfurtherthebreakdownanddevelopmentoftheapplicationofaTRLs.Usingthese
block diagrams can help visualize each TRL as a process with a specific set of inputs, outputs,
resource requirements and constraints. These parameters may be specific to a particular
technologyorfirm.FormoredetailsonSADTprocesses,see[139].
Figure14.GenericSADTModel
UnderstandingtheactivitiescompletedateachTRLandwhatproperlydocumentingandtracking
projectprogressisimportantasitcanenablebetterriskidentificationandassessmentduringthis
phase.Thenextsectionlinksthediscussionofactivitieswithassociatedrisksandtheirbehaviours
andeffectsontheproject.
Process Output(s)Input(s)
Resources
Constraints
62
3.4RisksinTechnologyDevelopment
Theprevalent risks associatedwith technology development are discussed in this section. These
risksareinnowayanexhaustivelistbutcanprovideageneralframeofreferenceformanagersto
beabletounderstandthebehavioursofthemajorrisksateachstage.Decision-makerscanusethis
information when they develop risk mitigation plans and decide on where options need to be
placedalongtheTRLscale.Itisfromanalyzingtheliteratureandunderstandingtherequirements
expected at different stages of technology development that these risks were outlined in this
section. Figure 15 illustrates the critical risk factors identified for this framework. These risk
categories and can be further broken down into specific risks that apply to certain stages of
development.Exampleswillbediscussedlaterinthesection.Aclearunderstandingofthestagesof
development and having clear expectations can lead to a better understanding of risk and the
importanceofmanagingthemaccordingly.DuringearlyTRLs,onlymajorrisksandchallengesare
knownoridentifiable.AsthetechnologyprogressestohigherTRLs,theassessmentbecomesmore
refinedandtheserisksbecomebetterunderstoodinthecontextofthetechnologyofinterest.
Figure15.CriticalRiskFactorsforR&D
Science&TechnologyRisks:Theserisksareduetoengineeringprocesses, technicaldesignand
development. There aremany risks associated with this category. They can vary from technical
risks such as integration and connectivity of components within a system, contradictory
specificationsorgoals,flaweddesigns,andtechnologylifecycles.
FinancialTechnology
Market
CriticalRiskFactorsinR&D
Organizational
63
Financial Risks: This is a major risk category as it can be thought of as the “fuel” for an
organization’sactivities.Capitalandfinancingareimportantastheyarethedriversthatwillallow
technology development to continue, demand to be met, and resources to be obtained and
maintained. The need for investors and funding increases as higher TRLs are reached since
investment costs go up, as well as the cost of materials and need for human resources begin
increasing.Therefore,itcanbeexpectedthatfinancialriskwillalsoincrease.Financialrisksdonot
disappearasthetechnologymatures, infact, thegoaloranorganizationshouldbetocontroland
mitigatethisrisktoaminimumacceptablelevel.Financialrisksmayalsoincreaseinthecasewhere
patentsareinfringed,asnewcostsmayarise.
MarketRisks31:Theserisksplayalargerolewhattechnologiesactuallygetpushedthroughtolater
stagesofdevelopment,andwhichoneshavepotentialmarketacceptance.Theidentificationofan
active market from early research stages prevents technology-push products and reduces the
chanceofprojectfailure.Examplesofmarketrisksincluderisksassociatedwithcommercialization,
competitiverisk,marketdemandandstructure,productionanddistribution.Marketrisksareoften
overlooked by engineers andmanagers as prioritization is given to technology risks in the early
stages.Thisbehaviourmayleadtomissingonmarketopportunitiesandincorrectassessmentsdue
tocompetitorproductsandparalleltechnologies.Marketconditionswillalsodeterminewhetheran
option can be exercised in time to meet a market window [23]. Customer acceptance and the
presence of a receptor market is important. Supply chain management weaknesses, demand
fluctuations (which may be caused by other underlying factors such as regulatory or policy
changes)32canalsobedescribedasmarketrisks.
31Specificriskssuchasregulatorypoliciesincreasing(duetosocio-politicalreasonsforexample),
may add risk in different categories such asmarket, financial and technological. In this example,
marketandfinancialrisksmayincreaseandtacticsmustbeemployedtoreducerisksindifferent
areas.
32 RegulatoryRisks:Couldalsobecalledsocio-politicalrisksandincludesafetyrisks,healthrisks,
environmentalrisks,industryregulationandpolicychanges.
64
Organizational Risks: These risks are characterized by the organizational structure, leadership
available,stakeholders,andtheorganization’sculture[72].Theserisksresult fromfailed internal
processes, includinghumanresourcesandorganizationaldynamics.PoorselectionofR&Dteams,
talentacquisitionandretentionandthedomainknowledgeofteamsduringkeystagesofconcept
development.Weakprotectionandmanagementofintellectualpropertyrightscanalsobeasource
oforganizationaluncertaintyandweakness.
Figure16.CriticalRiskFactorsDuringStagesofDevelopment[Adaptedfrom[140]]
It is important to understand that these risk categoriesmay be correlated. For example, during
TRLs4 to6, technologydevelopmentrisksshouldbegin todecreaseasactive learningcontinues,
knowledge is gained, and technical risks are defined and controlled.However, intensive physical
modeling and prototyping require large financing requirements, this is turn increases financial
risks, since the lack of financing threatens the quality of work and ability to complete tasks
required.This,inturn,mayincreasetechnologyrisksinceprogressmaybestoppeduntilfinancing
is obtained. It is critical to note that the consequences of any one of these risks could be
catastrophic to the entire likelihoodof successof theproject, as risks are interrelated. Figure16
illustrateshowclassesofriskmaychangeacrossthestages.Therelativemagnitudeoftheclassesof
risksisnotaccuratelydepictedinFigure16.Thisisdiscussedlaterinthissection.
65
Understanding relative risk creates a frame of reference for determining the effort that may be
required tomitigatemajor risks that changeas theprojectevolves.Technologyandmarket risks
aremorecloselyrelatedthanonewouldthink,astechnicalrisksaresometimesdependentonthe
knowledgeavailableaboutmarketssincetechnologyspecificationsareoftenconstrainedbymarket
opportunities.Therefore, themoremanagementunderstands themarket, the lower the technical
risk may be [78]. Financial risks could also be correlated with market risks, as well as
organizational risk. For example, market demand could increase, but because of financial
constraints (reduced margins) or organizational inabilities to meet demands, overall profit
decreaseswhich in turn increases financial risks.Theserelationshipsshouldbeconsideredwhen
organizationsbegin tobuild theirprofitmodelsduring their financialanalysis. It isnecessary for
management to consider suitable strategies during the analysis. The motivation behind the
selectionofthesestrategiesisoutofthescopeofthisthesis.
Figure17illustratestheriskprofilesofmajorrisksandtheirrelativemagnitudesduringstagesof
development.BytheendoftheTRL9,thegoalofmanagementshouldbetoreducetheriskswhere
theytaperofftoanacceptablecontrolledlevel.It isvaluabletounderstandhowrisksbehaveand
the relativedegreeofprominenceas it canalloworganizations tobettermanage these risksand
establishresourcerequirements.PlanningiscriticalinR&Dandstart-upsbecausethesefirmsface
challengesdue to sporadic cash flows,which can cause limitationson their ability toprogress to
higherTRLs.
66
Figure17.Riskprofileofmajorrisksduringtechnologydevelopment
Technology risk is expected to be high in the early stages of fundamental research and applied
research.Theriskofmissingamarketneed,incorrectscale-up,andconflictingrequirementsexist
during this period. Technology risks begin to decrease as technological capabilities grow around
TRL 4 and 5. This risk decreases with time and maturity, however at the time of launch
(implementation), it can be expected to increase slightly. This is due new system bugs, glitches
commonly seen with new technologies [26]. The scale up/commercialization and market
entry/production stages reached cause a shift in the nature of risks. Financial risks and
organizationalrisksareexpectedtoincreaseduetothefollowing:asthestart-uptriestogrow,this
becomesariskypointduringthelifeoftheprojectashavingenoughcapitalandaccesstohuman
resources (organizational) becomes vital. This implicates start-ups to streamline these business
processesquickly(leanoperationsaswell)astheyhavelimitedavailableresources.Thefocusisno
longerontechnologyinnovationandknowledgecreation,instead,itshiftstoproductcreationand
theknowledgedomainrequiredandnatureofmanagerialcompetencieschanges.Therebecomesa
needformarketexpertisewherecustomersareacquiredandretained.
CommercializationBasicResearch TechnologyDevelopment
High
Low
Risk
Time
Implementation
Science&Technology
Financial
Organizational
Market
67
The more comprehensive management’s understanding of the market and the position of the
product, the better the chances to a receptor market, and the higher the value of the product.
Therefore, the higher chance to raise capital as investors may have a higher confidence in the
successofthetechnology[141].ThegapshowninFigure18isdescribedasthe“valleyofdeath”and
isprimarilyusedtodiscuss thegapbetweenascientificbreakthroughstageandthecommercial-
readyprototypestage [78].Thegap is comprisedofavarietyof factors.The financialgapoccurs
fundingrequirementsincrease,andthesearenotmetoravailable.Capitalislimitedandcashflow
isirregularandfinancialriskbeginstoincrease,asdiscussed.Withtheshiftfromsciencetomarket
driven activities, human resource requirements begin to shift in a similar manner, increasing
organizationalandmarket/commercializationriskaccordingly.
TherisksfactorsandtrendsacrossTRLsdiscussedinthissectionareexaminedonabasiclevelfor
general cases. It can be expected that any R&D technology project could face these risks. The
specificsaroundtheuncertaintieswillbedeterminedbasedonthetechnology,R&Dandmarketing
team,aswellasotherfactorssuchastheindustryandgeographiclocation(tonameafew).Itisup
tomanagementtoconstructadetailedriskmanagementplanearlyon.
Figure18.Resourcerequirementsduringtechnologydevelopment
68
Similar to the TRL assessment, it is up to management to construct a plan in the relevant
technologicalcontext.RiskassessmentmethodologiessuchasonessummarizedinChapter2could
be used in the R&D context. Identifying risks alone is not enough, as they need to be assessed
(whetherqualitativelyorquantitatively)asthe impactof theseriskswilldrivemanagerstoplace
optionsaccordingly.
Figure19.PossibleRealOptionsConsideredatTRL4
Ina standardscenario, anorganizationwould reachTRL4andanoptionwouldbeplacedanda
decision tree would be mapped out. Figure 19 is only an example of the optionalities an
organization may choose to exercise. A certain amount of risk is always accepted in order to
progress to later stages of development, and depending on an organization’s appetite for risk,
optionscanbeexercisedaccordingly.
Scaleup
GrowthOptions Switchup
TRL4:Fataltechnologicaloranalyticalrisks
Yes
Abandon:StopProject/Abandon
Identifiedpotentialapplicationand/ormarket?Competitive
advantages?
Yes
No
Doesthefirmhaveresources&capability
tostrategize?
No
Yes
Delay/AbandonOptions
WaitandSee(Strategicdelay)
No
69
3.5Phase3:RealOptionsAnalysis
The previous sections outlined the qualitative portion of the framework. At this point of the
analysis, there should be a basic understanding of the important stages of development the
technologywillhave togo throughand the important logical stops that signify the change in the
natureofthedevelopmentandrisksassociated.Withanylongandriskyhigh-technologyandR&D
project, management should establish well-defined connections between the investment
opportunityandarealoptionbyproperlyframingtheproblem.Then,theoptioncanthenbevalued
[142].
GeneralconsiderationsforROvaluationcanbesummarizedasfollows:
• Themorevolatileaproject,theriskierit isandtheriskofsuccessandriskoffailureareboth
increased.
• Whenprobabilitydistributionfunctionsarenotavailablefromanalogouspriorprojects,expert
opinionisusedtodevelopestimatesofprobabilities.
• Plans to improve estimates as project proceeds should be made and the analysis should be
revisitedasdecisionsareincorporated.Thisapproachisiterative.
• Allassumptionsneedtobestatedsothatdiscussionswithstakeholdersareonacommonbasis
ofunderstanding.
3.5.1Base-CaseDiscountedCashFlow&SensitivityAnalysis
As discussed in Chapter 2, the forecast revenues produced through the use of DCF are highly
uncertainandunreliablefornewtechnologydevelopmentprojects.DCFcalculationsarestillused
asabase-casewhereassumptionscanbedevelopedandappliedfortheROanalysis.Withlimited
data and financial history for new start-ups and technology projects, accurate estimations and
rationalassumptionsarecriticalindrivingthedecisionsinthelaterstagesoftechnologyprojects.
The valuation begins by conducting DCF and NPV calculations for projects that have passed the
qualitativescreening[7]andinitialplanningstagesofthetechnology.Equationsusedtocalculate
theNPVarein2.4.1.Thenumberofyearsusedtoforecast,aswellasthediscountratesused,areup
to the discretion of themanagers. A particular challengewhen calculating DCF is estimating the
weightedaveragecostofcapital(WACC)whichisoftenusedasthediscountrate.
70
General practice assumes that the entire firm and the project share the same risks, and this
approach is not appropriate for innovative technology programs. This means that management
mustuse their judgement in selecting theproject’s discount rate [110]. Estimating theminimum
acceptable rate of return (MARR) for a new start-up is also a challenge in the early stages of a
technology when an organization does not yet know parameters such as administrative and
overheadcostsanddonothaveanypositivecashfloworrevenue.
AsensitivityanalysisshouldbecompletedfortheDCFtogiveasnapshotofthevariabilityofprofit
and other parameters. The analysis should be completed for key variables, with special
considerationtothedrivingriskfactorsidentifiedalongtheTRLs.Thesensitivityanalysisscenarios
chosenbymanagementshouldbebasedonthe“what-ifs”ofperceivedrisksandtheirinfluencethe
variables.TheeffectsonthesevariablesarethennotedandthechangesinNPVvaluesaretracked.
Sensitivityanalysisresultsmaydrivethestrategicoptionsplacedduringthecourseoftheproject.
3.5.2Monte-CarloSimulation
The Monte-Carlo simulation is an extension of the sensitivity analysis where continuous
distributionscanbemodeledgivingmanagersafullrangeofvariabilityofparameters.Theoutputs
of the simulation provide important insight into the behaviour of variables andmay be used as
inputs for the RO valuation. The application of a Monte-Carlo simulation can be challenging for
early-stagetechnologiesasthereislimitedinformationthatcanbeusedtoproducereliableresults.
ThisisdiscussedinmoredetailinChapter4ofthecasestudy.Thisisanoverviewoftheapplication
asitassumedthereaderiswell-versedwithbasicprobabilisticmethods.Theprocessisthesystem
model,where themodel could be a profitmodel in the case ofmany organizations thatwant to
reachhigherTRLs.Theuncertaintyintheparametersisestimated.Theassumptionsforthe“worst-
case” and “best-case” scenarios from the sensitivity analysis can be carried and used for the
simulation.Thiscanbereflectedintheassignedupperandlowerboundsofthevariables,andother
parametersassociatedwithselecteddistributions.Probabilitydistributionsforinputvariablesare
oftenusingempiricaldata,actuarialdataorothersystemmodels[143].Thiscanbechallengingfor
R&Dwhenthereisnodataavailable.
71
Figure20.Monte-CarloStochasticUncertaintyModeling[Adaptedfrom[144]]
Asimplifiedmethod todeterminedistributionscanbedonebyexaminingavariable’s conditions
based on the analysis conducted in the previous stages of the analysis. Distributions are either
continuous or discrete. For parameters that can possess negative values such as NPV, a normal
distributioncanbeassigned.Forparametersthatcannotpossessnegativevaluesbutincreasewith
nolimitorbecomepositivelyskewed,alognormaldistributionmaybeassigned[129].Forvalues
whereminimum andmaximummay occurwith equal likelihood, a uniform distributionmay be
assigned.Forvariablesorscenarioswhereaminimum,maximumandmost-likelyvaluesmayoccur
(themost-likelyfallssomewherebetweenthetwofixedbounds),atriangulardistributionmaybe
assigned [7]. These distributions do not represent a complete list and aremere examples of the
commonlyuseddistributions.However,otherdistributionsmaybeused.Ahome-grownsimulation
tool canbeused inMicrosoftExcel,MATLABoranyother commercial simulation toolwhere the
numberoftrialscanbeselected.Thesimulationoutputyieldsanestimateoftheprobabilityofthe
NPV, a stochastic DCF distribution [143]. As discussed in previous sections, variables may be
correlated.Thesecorrelationsarenotaccountedforinthisframework.
Model
f(x)
y1 y2yn
x1x2 xn
72
3.5.3RealOptionsProblemFraming&OptionValuation
Atthisstageoftheanalysis,managementshouldhaveagoodunderstandingoftheelementsofrisk,
theirvariabilityandtheeffectonprofit.Actionneedstobetakentonotonlymitigatethedownside
oftherisksbutalsotakeadvantageoftheupswings.
Byframingtheproblemduringthequalitativestagesofassessment,strategicoptionalitiesatTRLs
becomemoreapparent,asmoreknowledgeisgained[133].Atechnologydevelopmentprojectcan
be considered as a series of optionswhere each stage is an option on the value of future stages
[145]andmostdevelopmentswillhavemorethanoneoptionapplicabletothematonestageand
all need to be consideredduring valuation [6][96]. Thenature of the options considered at each
TRLdepends on these risk elements’ effect on themodel.Managers should be proactive in their
approach to R&D planning. The options considered, and ultimately selected, are ones that are
deemedtomaximizethevalueofatechnology.Thiscouldbeobservedduringashorttimeframeor
over a longer period (for example, for the delay option). Ultimately, their goal should be to
competitivelycapturemarketopportunities,whichrequirescarefulconsiderationof thepotential
endusersandmarketlandscapeandwarrantsmarketexpertise.
Thisframeworkutilizesthebinomiallatticeapproach(discussedinChapter2),astheyareeasyto
implement,flexible,andsimpletoexplaintomanagement[7].Inaddition,mostmanagerswithout
technicaltraininginrealoptionswouldfindthebinomialtreerepresentationstraightforwardand
the modeling of problems to have a more intuitive appeal, as well as the ability to include
underlyinguncertaintiesandaligningtheoptionsmoreeasilythanotherapproaches[110].Figure
21isarepresentationofasimplebinomiallatticethatisusedtovalueanoption.
73
Figure21.BinomialLatticeExample
Therearetwoimportantcalculationsthatwillbecompletedduringtheanalysis.Thefirststepisto
determinethevalueoftheunderlyingasset.Thisisdonefromthelefttotherighttoillustratehow
valuescanchangewith time.The firstnode,So, represents theNPVcalculated fromtheDCF.The
DCFusedatthispoint is theupdatedonethatreflectsthechangesmadeasnewinformationwas
obtained from the previous steps of the analysis. The nodes to the right represent the possible
distribution of future values, and the last nodes represent the range of values at the end of the
optionlife[103].Thesevaluesarecalculatedbysimplemathusinguanddasmultiplicationfactors.
The upswing value,u, is always a value above 1,while the downswing,d, is always less than 1.
Section2.4.4presentstheequationsforu,d.
u
So
Sou
Sod
Sodu
Sou2
Sod2
Sou3
Sou2d
Soud2
Sod3
d
Year0 Year1 Year2 Year3
A
B
C
Max(S-X,0)XisExerciseCost/Initial
Investment
74
Usingthebinomialmodeltocalculatevaluessuchasu,dandp(s)ischallengingforearlystageR&D
projects.Thesevaluesarea functionofvolatility,whichcannotbecalculated formost innovative
technologyprojects.Fromvolatility, theprobabilityof success,p(s), canbeestimated.Becauseof
thelimiteddataavailablefortheseprojects,managementshouldexpecttoestimatetheprobability
of success during the early stages. At low TRLs, estimating p(s) = 0.5may be accepted by some
managersandinvestors.TheentirepremiseofstrategicROapplicationistodrivethevalueofthe
projectwith time. This should overall improve the conditional probability of success in the later
stages.Thisisveryimportantasmarketrisksbecomemoredominantduringthelaterstagesandif
improvementsinp(s)arenotobserved,abandonmentbecomesanoption.
Thenextstepisthelatticevaluation–alsoreferredtoasbackwardinduction[112](arrowsgoing
righttoleftfromthetoprightnodeshownonFigure21).Thisprocessdetermineswhattheoptimal
decision(oroptionvalue)isateachperiod[112].Eachterminalnodehasvaluesofthemaximumof
zeroandthedifferencebetweenvalueSandexercisecostX(orinvestmentcost),whereMAX(S–X,
0). If thisdifferencebetweenSandX isnegative, then this iszeroandconsideredasanabandon
option.Thisisdonetoeachpairofverticallyadjacentnodesrevealstheoptionvalueattheveryleft
endofthelattice.
Whenestimatingvaluessuchastheprobabilityofsuccess,itcanbeexpectedthatthiswillbelowat
lowerTRLs.Thevalueofexercisinganoptionandacceptingtherisk,isastrategytolowerriskas
thetechnologyprogressesalongtheTRLscale.Thus,intuitivelyincreasingvaluefortheprobability
ofsuccess.Closertoimplementation,theprobabilityofsuccessbecomesmoreunderstoodthanthat
intheexploratoryphase.Estimatesofthelikelihoodofsuccessfuldevelopmentofatechnologycan
beestimatedfromstatistics,anddata fromcomparableprojectswithinaportfolio,which ismore
likely ina largeR&D-intensiveorganization.Inthecasewherethis isnotpossible,seekingexpert
adviceisacommonpracticetoobtaintheseestimates[93].Thisisaniterativeprocessandasmore
informationbecomesknown,calculationsandassumptionsshouldberevisited.
Other considerations are to develop metrics for non-economic factors that influence an
organization’sreputation.This isofparticular importance forgovernmentresearchorganizations
and smaller start-ups that need to build relationships with clients and their respective
communities.
75
Unlikefinancialoptions,ROsdonothaveexpirationdates.Generallyspeaking,managersshouldbe
able toestimatea time-frame forwhenthedecisionneeds tobemadeandbecauseofchanges in
businessconditions,competitionandtechnology,checkpointsthroughouttheprojectareneeded,
andshouldbeplacedaccordingly[104][103].Thisstrengthenstheframework’sneedforoptionsto
beplaced(atleast)atstage-gatesidentifiedintheprevioussection.
SimilartothesensitivityanalysisconductedontheDCF,managersshouldanalyzethevariabilityof
theoption.Thevalueinthisanalysisitwillhelpestimatetherangeofthebenefits,theprobabilityof
successoftheresearcheffort,thecosttoimplement,andthemarketuncertainty.
TheoutcomeoftheentireROanalysisshouldgivejustificationtomanagerialdecisionstoinvestina
technology, despite a negative NPV. This can also allow for better resource allocation across a
projectorportfolio.
76
Chapter4ApplicationtoCaseStudy:CopperstoneTechnologies
Thischapter isa casestudyonCopperstoneTechnologies (CST),where theproposed framework
willbeapplied.Thechapterwillbeginbyintroducingbackgroundinformationrelatedtothecase
study.Theanalysiswillbedividedintothreephases.Phase1willdiscusshowrelevantinformation
wascollectedandsummarizethedata.Phase2willdiscusstheresultsfromphase1,outlinemajor
risks,anddiscusscurrenttechnologicalmaturity.Phase3willapplyROthinkinginordertoassess
decisionsatkeystagesoftheproject.
Not all aspects of the proposed framework were formally completed. This includes a formal
documentedriskassessmentandaTRA.Thiswasduetothe limitedaccesstodata fromCSTand
what information theywerewilling to share for this thesis. Despite the limitations, critical risks
were identified and discussed in 4.2.2. The analysis will conclude by discussing some of the
limitations of the calculations and assessments and how they could be improved for future
applicationsthatarespecifictoCST.
Ashortsectionofthischapterwillbrieflydiscusshowtheframeworkcouldbeappliedtoaproject
withinalargeportfolioatNationalResearchCouncilofCanada.Unfortunately,afullanalysiscould
notbecompletedastherewasnoinformationavailableaboutthetechnology,projectsorportfolio.
Importantpointsabouthowtheframeworkcanbeusedasacomparativeassessmenttoolwithina
portfoliowillbediscussedforfutureapplications.
4.1Phase1:DataCollection&Background
The main challenge of the analysis for this case study was collecting enough data. Available
information was very limited and CST was going through many changes within their company,
where new talent was being acquired and investors were aggressively being sought after. This
madeitdifficulttoobtaininformationefficiently.
77
The main method used for information gathering was through informal discussions with CST’s
ChiefTechnologyOfficer(CTO)33.Thesediscussionshelpedshapetheoptionalitiesconsideredand
the estimated values. The remaining portion of data collectedwas through ameetingwith CST’s
business development manager, and a thesis published by one of CST’s founders. The thesis
discussed details around the technology developed and some of the team challenges. Any other
miscellaneousinformationwasobtainedthroughonlinesourcesfromCST’scompanywebsite.
4.1.1CompanyBackground:CopperstoneTechnologies
CopperstoneTechnologies(CST)isanengineeringstart-upfoundedin2014bythreeUniversityof
Alberta students. CST currently develops amphibious robots (AR), they also offer electrical,
mechanical, and consulting services related to the field of robotics. CST currently has office
locations in Edmonton and Calgary, Alberta, Canada. The current team is comprised of four
founders, and a business development manager – who is responsible for all marketing, sales &
businessstrategyrelatedwork.AfulldescriptionofCST’scurrentteambiosandthepositionsthey
are looking to fill in thenear futureare inAppendix2.CST is currentlydevelopingstrategies for
futuregrowthopportunitiesinmarketingandsales,customerservice,andproductdevelopment.A
summaryoftheirtimelineofactivitiesisinAppendix2.
CST’smain product offering is the AR technology. This technology allows access to soft andwet
fresh tailings ponds that are inaccessible using current sampling andmeasuring equipment. The
rover is unmanned and therefore poses a lower risk to personnel safety when compared to
traditionallyusedequipment[146].TheARtechnologyalsorequireslesspersonnelsupportoverall
andprovidesautonomousoperationforsamplingandmeasurements.TheARcanbeusedforseed
broadcasting and planting for reclamation, and tailings mapping and monitoring. CST’s main
potential customers are oil sands producers, and they are looking at applications that target
customersinthemining,goldandpotashindustry[147].
33CST’scurrentCTOisDr.Lipsett.
78
CST has been awarded one patent for their “Soft Soil Sampling Device and System” on June 16,
2016.Thescopeofthispatentprotectsadevicecollectingsurfacesamplesneededforcalibrations
forother instrumentsandtomeasuretailings’properties.Thepatentalsoprotects themethodof
deployment. They also have another pending patent filed in 2016, “All-terrain Vehicle”, which
protects the frame, instrumentdeploymentanddriveconfiguration,andanymodifications to the
drivesystem.CSTplanstoacquiremorepatentsasthetechnologydevelopsfurther.Theaimisto
protect all new drive and control mechanisms for operations in soft terrains, autonomous
operations of robots and their instrumentation for monitoring and data processing algorithms
[146].
4.1.2InterviewwithBusinessDevelopmentManager
Ameetingwith CST’s current business developmentmanagerwas conducted. Thismeetingwas
valuableas itrevealed informationaboutCST’s futuregrowthplans,marketing,andsalestargets.
Someofthekeytakeawaysfromthemeetingwere:
• Thelargestriskispoorrevenuegeneration.
• Latent causes of potential failures were discussed. Competition, market scale-up, and legal
regulatoryrequirementswereconcerns.
• The initial market analysis conducted was almost non-existent. Now requiring a rigorous
approachtosizethemarketandacquire(andmaintain)customers.
• CSTiscurrentlylookingforinvestors.
• Conflicting team goals is a large risk. The risk of founders leaving may be unfavorable to
investorsandcanbeseenasunstable.
• MarketscalingofARhasbeenaconcernforCST.
• CST iscurrentlyconsidering the followingcommercializationandmarketpositioningoptions:
salesofARunits,licensingofIP,orenvironmentalmonitoring.
• An informal market risk assessment for CST’s current positioning was done. There was no
formaldocumentation.
• CSTiscurrentlyfieldtestingwithalmostallcomponentsfullyintegratedwithinthesystem.
• CSThasonecompetitorintheenvironmentalmonitoringmarketintheoilsands.
79
TheultimategoalistothepenetratetheAlbertatailingpondsmarketandgrowthecompanywhile
continuingtoinnovate.TherewasnoinformationobtainedaboutCST’ssalesandrevenuetargets.
Maximizationofprofitpotentialwasan important factor forCST.Therefore,aconsistentrevenue
stream is preferable. Selling AR units may not be the most viable opportunity for CST due to
potential inconsistent revenueassociatedwith sales.Theorganizationdoesnothaveanycurrent
plans, or the resource capability to expand to othermarkets at this time. This, however,will be
considered in the future.CurrentARunitsarebuilt in-house,butCSTmaysourcemanufacturers
dependingonthedemandandchosencommercializationpath.Thebusinessdevelopmentmanager
was only recently hired, this indicated that therewere no realmarket considerations or studies
done.CSTplansonexpandingtheirhumantalentbaseandplansongrowingtheteam.Thereisalso
aneedforfundinginordertogrowandCSThasbeenworkingtotryandsecurefuturefunding.CST
alsohasplansfornewversionsoftheAR,withanexpandedrangeofapplications.Thiscouldbring
CST a new competitive advantage and can allow them to remain innovative. These future
applicationswillnotbeconsideredintheanalysis.Managerialvisionsandgoalswerediscussed,as
therewasadifferenceinopinionbetweenthefoundersonthebestcommercializationpathforAR.
Majorriskfactorsthataffectsuccessfulcommercializationwerediscussed.Moreontheserisks in
4.2.2.A sampleof the interviewquestions is inAppendix2.Thesequestionsarebynomeansan
exhaustivelist,buttheydosummarizesomeofthequestionsthatwereasked.
4.1.3InformationfromRelevantSources
OthersourcesofinformationusedincludedathesispublishedbyoneofthefoundersofCST,which
involved a case study on the AR project [148]. Detailed notes from the thesis are presented in
Appendix2.
Someofthekeypointsfromthecase-studyincluded:
• CST’s business model was not developed at the beginning of the project. Although
commercializationmodelsconsideredincluded:salesofARunits,rentalofARunits,leasingto
operators,ormonitoringservices.
• TherewashighuncertaintyinrequirementsandtechnicalrisksbecauseCSTfailedtoproperly
plan,collectdata,andassesstheoperatingenvironment.Thisledtoadecreaseinthetechnical
successoffieldtrialsconductedandresultedinfield-testingbeingdelayed.Thiscauseddelays
inthescheduleandbudgetoverruns.
80
• No formal market analysis or verification of market conditions was conducted. CST’s initial
marketanalysiswascompletedusingqualitativedatafromasmalldataset.Thereweremany
uncertaintiesrelatedtothemarketastherewaslimitedunderstandingoftherequiredtechnical
specifications wanted by customers. CST collaborated with potential customers through
informaldiscussionstotryandestimatethesespecifications.Thepotentialprimarymarketfor
ARsis inenvironmentalmonitoring,withonlyasmallnumberofpotentialclients,buta large
amountofpotentialapplication(land).
• Therewasnoformalriskmanagementcompletedduringdesign.However,safeworkpractices
andriskmitigationwereconductedduringfabricationandtesting.
• The project funding came internally fromwithin the organization. Therefore, available funds
werelimited.
ConversationswithCST’scurrentCTOalsorevealedthatCST’sARprojectbeganataroundTRL4
and is currently about to enterTRL734.Thiswasachieved throughan informalTRLassessment.
BasicfinancialprojectionswereobtainedfromCSTfortheROA.Thevaluesshownintheworkdone
in this sectionmay not reflect the true values for the AR project as CST did not share all their
financialsforthisthesis.CST’sfinancialsarediscussedin4.3.
4.2Phase2:CurrentTRLandCriticalRisks
Thissectionassessestheinformationanddatacollectedfromphase1.Aspreviouslymentioned,no
formalriskassessmentorTRAwasconducted.However, itwasrecommendedtoCSTthat formal
assessments be completed and documented. The assessment process should identify alternative
routes in the case the technology does not mature according to initial plans. TRA and risk
assessmentsshouldbeconductedthroughouttheprojectandnotjustatstage-gates,althoughthose
provideageneralrecommendationofwhenprojectprogressneedstobeassessed.
34AsofJuly2017.
81
4.2.1ARCurrentTechnologicalMaturity
Figure22 illustratesCST’s initialmaturity in2014and theirmostcurrentTRLpositionasof July
2017.
Figure22.CSTCurrentTRLPositioning
CST’s AR project began at around TRL 4. Thiswas determined and confirmed throughmeetings
withtheirCTO.ROAwillbeappliedatthatstage-gate.Thiswillbedonefordemonstrativepurposes
andtoprovideadiscussionofthecommonfactorsthatmaybeconsideredduringthosestagesof
development, and how they could influence the overall success of the project. This can provide
insighttoimportantinformationthatmayhavebeenoverlookedinthepast.
CST’s AR is designed around amultiple screw drive propulsion system, and the initial proof-of-
conceptsystem isbatterypoweredwithanelectricdrive thatallowscontrolof themotor. Italso
allows for design of assumptions during the process of proving the system [146]. Some of the
activitiesconductedduringthetechnologydevelopmentstagebetweenTRL5andTRL6included
developingseedbroadcasterandseedlingplantingmechanismsanddevelopmentoftheindustrial
controllerforteleoperation.Screw-drivecleaningmechanismsandanewdrivesystemtoimprove
mobility on frozen sand and other conditions were also activities completed during this phase.
Someofthemostrecentactivitiesrelatedtomudlinemonitoringincluded:systemintegrationand
mapping, testing of long-term deployment for remote monitoring, as well as autonomous data
analyticsandreporting.
SystemDevelopment&Demonstration
TRL7TRL1 TRL3TRL2
TRL4 TRL5 TRL6 TRL8 TRL9TechnologyDevelopment
Production&DeploymentS
yst
Pre-conceptRefinement ConceptRefining
ScaleUpandCommercializationBasicResearch
TechnologyDevelopment
MarketandIndustry
Go/NoGo Go/NoGoGo/NoGo
CSTstarted2014
201
CST2017
82
Asof July2017, theARproject is in the systemdevelopment anddemonstrationphase, about to
enter TRL 7, where some of the field-testing has been completed. According to the proposed
framework,CSTisatthesecondstage-gatewheretheywouldconsider(commercialization)options
thatwouldsupportprogresstohigherTRLs.
Future R&D activities for commercial prototyping will include optimization of the screw-drive
propulsion system, innovations for long-term reliability for longer operational durations and
samplinganddeploymentcapabilities.Inaddition,CSTplanstousehigherdensitypowersources.
TherewasnoformaltimelineprovidedbyCST,however,theaimistohaveaninitiallaunchforAR
withmonitoringsystemanalyticscapabilitiesbetween2018and2019andanotherlaunchofARfor
deepdepositsandlongerdurationsaround2019.Asfordevelopmentrequirementsforthenext2-4
years,CSTpredictsfabricationresourcerequirementswillneedtobeexpanded,andalargershop
may be needed. A laser cutter,mill, andwelderwill be required for any future prototyping and
assemblywork.CSTpredictsthatonlyoneshopemployeewillbeneeded.Anyotherfabricationor
manpowerrequirement that isnot supportedbyCST’s capabilitieswillbeoutsourced.Forshort-
term requirements, CST requires a part-time electrician, as well as a technologist that will be
neededformanufacturingandproductionneeds.
In addition to the increase in resource requirements, additional fundingwill be required. CST is
currentlyseekingnewinvestors.Theneedformarketingexpertisehasalsobeenrecognizedatthis
stage. CST’s business manager is formulating growth strategies and strategies to keep CST’s
competitiveadvantagewithinthenichemarketofoilsands.Commercializationoptionshavebeen
identifiedbutnotyetselected.Moreoneachoptionwillbediscussedin4.3.5.
83
4.2.2CriticalRiskFactors
The general risk categories are financial, science & technology, marketing and organizational as
discussedinChapter3. Inordertobetterframetheproblemtoidentifythecriticalriskfactors, it
was important to assess the information from the interview, conversations and othermaterials.
CST’sultimategoalwastolaunchtheARtechnologyintothemarketandgenerateaprofit.Although
there had been discussions about possible commercialization paths (i.e.whether theywanted to
selltheirunits,license,etc.),thechoicewasstillunclear.ThisiswherethevalueofROAiscaptured.
These commercializationoptions canbe valued so that an evidence-baseddecision canbemade.
Themainrisksthatareconsideredtohaveadirect impactonprofitarediscussedinthissection.
Theserisksarenotanexhaustive list.Theyare,however, themost importantasviewedbyCST’s
teamatthispointintime(i.e.attheTRL6stage).Table735givesabriefrepresentationoftwoofthe
risksidentifiedandtheireffectsonprofit.
A factor that has a substantial effect on themarket risk for the AR project are regulatory laws.
Provincialandfederallawsshouldstronglybeconsidered.Ifregulatorypoliciesregardingtailings
are increased, CST could potentially see an increase in the customer demand, as the technology
currentlyfallsunderthecategoryofcomplianceindustry.Conversely, ifthepoliciesarenolonger
strictandarereduced,themarketforARscanexpecttodramaticallydecrease.It isimportantfor
managementtobeproactiveforfuturedevelopmentandapplicationsofthistechnology.CSTmust
strategically plan to position their AR technology for operational use, where the technology is
criticalforimprovingefficiencyandperformance,asopposedtobeingintheregulatorymonitoring
andcompliancespace.
35Thistabledoesnotaccuratelyreflectfulltheextentofhowmulti-dimensionalriskelementsare
andhowtheymayaffectoneanother.Thisaddressedinthissection.Asensitivityanalysiscanhelp
betterunderstandthebehavioursoftheserisks.
84
An area of concern for CST at this point is related to scaling-up, as there are three aspects to
consider.Thereisaneedforthedesignprocesstoencompassaphysicalscalingoftechnologyi.e.
from lab to operating environment. There is the market aspect of scaling up, where suppliers,
manufacturers and distribution channels need to be considered. Finally, there is a performance
relatedscale-upwheredesigncomplexityisaccountedfor.
Competition in themarket is alsoan important factor thatwas identified.Because financial risks
arealsodominantthroughouttheproject(sinceR&Discapitalintensive),thereistheconcernthat
if CST does not commercialize within an appropriate time-frame and capture current market
opportunities,ARtechnologyreplicasmayariseanddamageCST’sfutureplanstolaunch.Financial
constraints are an important risk identified that may delay technology growth and delay
commercialization.FinancialrisksarecurrentlybeingmitigatedasCSTtriestoacquirefundingand
plan accordingly. It is also important to be aware that each commercialization optionwill come
withitsownrisksandchallenges.Distributionandsupplychannelconstraintsarealsoanotherrisk
that may delay or prevent successful commercialization of technology to end users. In order to
mitigatethisrisk,CSTiscollaboratingwithservicecompaniesfortheirprototypingandfield-trials,
inhopesofhavingbetteraccesstocustomers.
Demand is also another important factor. It can also be linked to regulatory policies and
competition. For example, demandmay become lower if competition is high. Demandmay also
decreaseorincreasedependingonhowgovernmentregulatorypolicieschange.Iftheyincrease,the
demandcanexpecttogoup, if theyloosenthepoliciesthendemandmaydecrease.This isavery
one-dimensionalwaytolookathowvariablesmaychangeasitignorestheeffectsofotherpossible
variables. To elaborate, demand may be high, however, profit may still decrease due to CST’s
reducedmarginorinabilitytomeetdemand.Asensitivityanalysisoramulti-dimensionalMonte-
Carlosimulationcanbeusedtobetterunderstandhowthesevariablesmightaffectoneanother.
85
Table7.ExampleofPossibleEffectsRiskonProfit
CriticalFactors Change EffectonVariable(s) Potential
Strategy
RegulatoryPolicies
Increase Demandincreases,expensesincreased,profitincreases
Sellingpriceincreased
Decrease Demanddecreases,profitdecreases
Pivottooperationalpositioninginstead
ofregulatory
Competition
Increase Demandmaydecrease,profitdecreases
Marketstrategyfornewstart-upistoofferdiscounts
Decrease DemandmayincreaseHigherprices,monopolizethearea/industry
Technologydevelopmentrisksarestill importantduringthisstageofdevelopment,however, less
prominentasCSTbeginstoshiftfromknowledgecreationtoproductcreation.CSThaspassedthe
“valleyofdeath”atthispoint.However,thereisalwaysariskwhennewimprovementsareadded
to the technology. CST hopes to add updates to their design in order keep their competitive
advantage in the industry. In order for them to accomplish thiswithout having large costs, they
developedadesignthatallowsforeasymodificationwithouttheneedtomakesignificantchanges
tosystemdesign.
Other factorssuchascustomerrelationshipsandreputationwithinthecommunitymayalsobea
factor.However, forCST, this isnot a critical issueas theyhavebuilt strong relationships (some
furtheralongthanothers)withoperatorsintheAlbertaoilsandsanddonotconsiderthisariskat
thistime.
86
4.3Phase3:ROAApplication
TheapplicationofROA to theARproject assessesdecisions at two stages-gates.AlthoughTRL4
wasreachedinthepast,ROAwillstillbeappliedatthatstage-gate.Thiswillprovideadiscussion
onkeyfactorsassociatedwiththisstageofdevelopment.AttheTRL6stage-gate,optionsshouldbe
valuedbyCSTwherethedecisiononwhethercommercializationcostswillbepaidoutrightaway
or whether timing of the investment (waiting) may be an option. The trade-offs between
proceedingrightawayorachoosingtowaitcanbeaconsideredandevaluated.
4.3.1ROAProblemSet-up
Thefollowingwereassumptionsandconsiderationsappliedduringtheentireanalysis.Thesewill
remain consistent and be carried to the next steps of the analysis. Any changes to these
assumptionswill be noted in the relevant sections. The general assumptions and considerations
are:
• AllthefinancialsarebasedoninformationprovidedbyCST.Thebasefinancialevaluationswill
notbechanged(i.e.CST’sestimationforexpenses,grossrevenue,etc.).Thisisbecausethere
wasnotenoughinformationprovidedforthisthesis.Therefore,itisassumedthatCST’scurrent
plans,whichmayincludefield-testingwithcustomers,newR&Dinvestmentswillcontinueas
originallyplanned.TheROAwilladdontothebasefinancialsforCST(i.e.costsassociatedwith
optionswillbeaddedtotheoriginalestimatedexpenses).
• Allrevenueandexpensesfromconsultingactivitieswereomittedfromtheanalysis.Thisisnot
a core element of the AR project, and the revenue is considered minuscule compared to
potentialrevenuefromAR.
• Equityandfinancingwerealsoignoredduringtheanalysis.Thisismainlyduetounavailability
ofinformationaboutCST’scurrentandpreviousfinancingactivities.
• Year0at2014isthefoundingyearofCSTwheretheybeganatTRL4.AWACCof15%wasused
and a MARR of 20%. The WACC and MARR values were selected based on CST’s business
manager’s recommendation. A nine-year cash flow was completed. There is no underlying
reasonwhynineyearswereselected,thiswasCST’spreference.
87
• TheexpensesatearlyTRLs(startingat2014)includemostlyR&Dexpenses,incurredexpenses
for materials and equipment. At medium TRLs, at years 2017 and beyond, the nature of
expenses would begin to shift from R&D to activities that involve engagement of other
companies and a projected revenue and estimated expenses. These activities included field
testing. Expenses related to patentingwere also incurred, aswell as other professional fees.
TheseexpensesareallreflectedinCST’sfinancials.
• The technology life cycleofARswas ignoredduring theanalysis.Thiswasmainlydue to the
limitedinformationprovidedaboutthetechnology.
• Taxes and inflation were ignored in the calculation. This was done in order to simplify the
calculationspresentedinthisthesis.
4.3.2Base-caseDCF
DCFisusedasabase-casewhereestimatedvaluesandassumptionscanbedeterminedandapplied
to the RO application. With limited information and no historical data for CST, estimations and
assumptions can have serious effects on the decisions in the later stages of technology projects.
Theseassumptionscan influencethevalueofanoption,whichmayultimatelydrive thedecision.
Basedonthegeneralassumptionsandconsiderationsin4.3.1,aDCFwascompleted.Asummaryof
thecashflowisoutlinedinTable8.
Table8.Base-caseDCFResults
Year FV PV0 2014 ($121K) ($121K)1 2015 ($37K) ($33K)2 2016 ($39K) ($30K)3 2017 ($30K) ($18K)4 2018 $336K $162K5 2019 $1.40M $564K6 2020 $2.31M $773K7 2021 $2.96M $827K8 2022 $4.03M $938KWACC 15%MARR 20%Income $3.26MNPV $3.06M
88
At first glance, the positive NPV looks promising. However, this is not a reliable source of
informationforayoungstart-upsuchasCST,andaninnovativetechnologyinitsearlystages,such
astheAR.Thiscalculationassumestherewillbenochangesthroughoutthedevelopmentalprocess
as thecompanygrowsand the technologymaturesalong theTRLscale,which is reflectedby the
constantdiscountrate.Thisanalysisdoesnottakeintoconsiderationpossiblechallengesthatmay
ariseduringthecourseoftheproject,andtheeffectsofdecisionsmaderelatedtodevelopmentand
commercialization.Forexample,CSTdoesnotcurrentlyknowwhichcommercializationpaththey
willpursue.Eachoptionconsideredwillrequiredifferentconditions.SellingofARunitswillrequire
differentresourcerequirementsthantheIP licensingoption.Therefore, it isnaïvetoassumethat
decisions made during the project will yield the static results in Table 8 and have no effect on
operatingcostsandincome.
AnotherissuewiththeDCFandNPVestimatesisthevalueusedfortheMARR.Thechoiceinvalue
fortheMARRwassubjective.WhendiscussingthiswithCST’sbusinessdevelopmentmanager,the
argumentwasCSTisataveryearlystageandthereisnoMARRatthistime.Thisisattributedto
CSTnotknowinganyoftheirparameters(suchasoverheadcosts),andareestimatedtothebestof
their abilities. This strengthens the argument that these values cannot be considered accurate
estimates.
Despite the unreliability of the DCF and NPV values, the results from the DCF should not be
overlooked,astheyareusedasabase-casecalculationfortheROanalysis.FromtheDCFresults,a
well thought-outsensitivityanalysiscanbeproduced,whichwillprovideCSTwith insightonthe
variabilityofparameters.
89
4.3.3SensitivityAnalysis&Monte-CarloSimulation
A sensitivity analysis is valuable as it provides managers with a tool to better understand how
factorsmay influenceprofitanddecisionsmadeduring theproject.ForCST’sanalysis,ascenario
analysis was chosen over the Monte-Carlo simulation. This is mainly due to unavailability of
informationonparametersandtheirvalue. Instead,a threescenarioanalysiswasanappropriate
andacceptableanalysisforCST.Theyweremoreinterestedintheboundsorlimitsofthecashflow
valuesbasedonthemainvariables.AMonte-Carlosimulationcouldhavebeeneasilyapplied,which
iswhyitwasfullyset-upinthissectionandwillbeexplained.AlthoughaMonte-Carlosimulation
wasnot completed, therewas insight from setting it up. It gaveperspectiveonhow someof the
parameters couldbehave andhow theymight be related to one another. The thought-process of
howthesensitivityanalysiswascompletedasoutlinedinthissection.
Thesensitivityanalysiscalculationsbeganbyfirstconsideringtheriskfactorsthatwereidentified
in 4.2.2. CST’s goal is to maximize profit. This meant that all major parameters needed to be
carefully examined. Demand was an important factor as it ultimately could be affected by
competition, regulatory changes, and other socio-political factors such as community trust and
reputation.Therefore,thesefactorsalsohaveadirecteffectonanyofthegrossrevenueobtained
from AR operated activities. If any of these factors change, they may cause CST to increase or
decreasetheircostsofservicesaccordingly.Operatingexpenseswerealsoasignificantfactorthat
mayaffectprofit,asanyincreaseinexpensesmayreduceprofitmargins.Asimpleprofitmodelthat
describedtheseparameterswasassumedtohavefixedindirectcostsasfollows:
𝑃𝑟𝑜𝑓𝑖𝑡 = 𝑆𝑒𝑙𝑙𝑖𝑛𝑔𝑃𝑟𝑖𝑐𝑒 − 𝐸𝑥𝑝𝑒𝑛𝑠𝑒𝑠 ×𝐷𝑒𝑚𝑎𝑛𝑑 − 𝐹𝑖𝑥𝑒𝑑𝐼𝑛𝑑𝑖𝑟𝑒𝑐𝑡𝐶𝑜𝑠𝑡𝑠
Wheretheunitsareasfollows:
𝑃𝑟𝑜𝑓𝑖𝑡𝑖𝑠𝑖𝑛$
𝑅𝑒𝑣𝑒𝑛𝑢𝑒𝑖𝑠$/𝐴𝑅𝑈𝑛𝑖𝑡
𝐸𝑥𝑝𝑒𝑛𝑠𝑒 𝑠 𝑖𝑠$/𝐴𝑅𝑈𝑛𝑖𝑡
𝐷𝑒𝑚𝑎𝑛𝑑𝑖𝑠𝐴𝑅𝑈𝑛𝑖𝑡𝑠
𝐹𝑖𝑥𝑒𝑑𝐴𝑑𝑚𝑖𝑛𝐶𝑜𝑠𝑡𝑠𝑎𝑟𝑒𝑖𝑛$
90
The revenue specifically refers to that from operations involving AR. The source of AR revenue
couldbeduetoseveraloptions.DependingontheoptionCSTchoosestoexercisewillinfluencethe
profitmodelaccordingly.Examplesofpotentialsourcesofrevenuecouldincludesellingorrenting
ARunits tooperators in theoil sands.All of theseoptions couldalso includea sourceof income
fromtechnicalsupportprovidedtocustomers.
The next step was to examine the variables and assign each a probability distribution.
Administrativeorfixedindirectcostswereassumedtobeconstant.Demandisadiscretevariable
and was assumed to follow a triangular distribution. There would be a maximum number for
demandonARunitsorservices,aminimumvalueandanestimatedmost-likelyvaluethatwouldlie
inbetweenthosetwonumbers.Theminimumvaluewouldlogicallybeademandofzero.Expenses
perARunitandgrossrevenuewerebothsaidtofollownormaldistributions.Theseinsightshelped
with the understanding of some of the parameters and shape the sensitivity analysis. Three
scenarioswereanalyzed.Eachscenariocalculateda “best-case”and “worst-case”.Theresultsare
summarizedinTable9andatornadodiagramisshowninFigure23TheNPVwascalculatedforthe
followingscenarios:
1. CapitalcostsandcostsrelatedtoARwereincreasedanddecreasedby20%.
2. Grossrevenuewasincreasedanddecreasedby20%(i.e.demandorsalesincrease).
3. AMARRof17%andat23%.
Table9.DCFSensitivityAnalysisResults
ExpectedNPV Input
Variable Downside Upside Range Downside Upside BaseCaseARExpenses $2.80M $3.22M $340K $2.63M $1.75M $2.19MARR $2.57M $3.65M $1.08M 23% 17% 20%GrossRevenue $2.25M $3.87M $1.61M $10.60M $15.89M $13.24MCFYear0 ($135K) ($106K) $29K ($135K) ($106K) ($121K)CFYear1 ($40K) ($25K) $15K ($46K) ($29) ($37K)CFYear2 ($32K) ($29K) $4K ($42K) ($37) ($39K)CFYear3 ($41K) $1K $42K ($62K) $2K ($30K)CFYear4 $104K $220K $116K $216K $456K $336KCFYear5 $423K $704K $281K $1.05M $1.75M $1.40MCFYear6 $589K $957K $368K $1.76M $2.85M $2.30MCFYear7 $631K $1.02M $391K $2.26M $3.66M $2.96MCFYear8 $729K $1.15M $419K $3.13M $4.93M $4.03M
91
A major limitation of these results is that they are based on estimated values that may not be
accurate.However,thereisstillvalueinthistoolasitallowsCSTtobetterunderstandhowsomeof
thevariablesmightbehaveundercertainconditions.Thissensitivityanalysisalsofailstoproperly
define relationships between the variables. As mentioned in Chapter 3, multidimensional
relationshipsbetweenvariableswillbeignoredforthisthesis,however,isrecommendedforfuture
applications.
Figure23.TornadoDiagramRangeofNPVResults
AMonte-CarlosimulationMicrosoftExceltemplatecanbeusedtoconductthisanalysis.Usingthe
randomnumbergeneratorinExcel,thefunctionRAND(),wouldeasilysimulatetherangeofprofit
values.Values thencanbeeasilycategorizedaccording to thosebelowzeroorgreater thanzero.
The resultingprofit’sprobabilitydistributionwill yieldvaluable informationabout themeanand
standarddeviation.
$2,000,000 $2,500,000 $3,000,000 $3,500,000 $4,000,000
AR1Expenses
MARR
GrossRevenue
NPV
Upside Downside
10%Error
92
4.3.4ROValuationatTRL4
Aspreviouslydiscussed,CSTisabouttoenterTRL7andhascontinuedwiththeARproject.There
is value in applyingROA at TRL4. Thiswill help develop a complete analysis and allow gaps or
importantaspectsthatmayhavebeenoverlookedtobeidentified.Thismayalsohelpguidesomeof
the decision making that is done at TRL 6 and 7. At TRL 4, CST had completed the necessary
analytical studies and has assessed the context in which the technology will operate. Analytical
studies and assessments were completed through preliminarymodeling and simulation [23]. At
TRL4,CSTbegantestingtheanalyticalassumptionswithinthecontextoftherelevanttailingponds
environment.Between2014and2015,CSThadjuststartedandhadverylimitedfundsandlimited
humanresources.Thetechnologywasalsolimited,whereinitialfeasibilitystudiesanddetermining
applications and potential markets for the technology were in the process of being conducted.
Abandonmentorterminationofaproject isanoption inthe flexibleROapproachandbecauseof
theselimitationsmentioned,abandonmentwassetasthefirstpossibleoption.Itshouldbenoted
thatdelayingaproject isanimportantoptionthatcouldbeusedbyR&Dandtechnologyfirmsto
reassesstheirworkandtrytoaddresssomeofthefactorsorrisksdiscussed.AsmallfirmlikeCST
did not have the capital or resources to be able to delay the project and re-invest. This is why
delayingwasnotconsidered.
Theothertwooptionsconsideredweregrowthtypeoptions.IfCST’stechnologywasfoundtonot
have any fatal design risks, but managers had failed to identify a market application for their
technology, a market pivot could have been considered. There are several factors that may
contribute to the choosing of this option. For example, high competition or low demand, or no
identifiedmarketreceptors.ThisoptionwouldhaveaddedvaluetoCSTas itwouldhaveallowed
them tomodify thebusinessmodel andmakenecessary changes,without losingon theprevious
investmentanddomainknowledgetheteamhadacquiredsofar.ThiswouldhaveenabledCSTan
opportunity to exploit a prior investmentwith a sequential one [1]. Exercising themarket pivot
optionmayactuallyhavewarrantedtheneedforanotheroption(afewmonthstoayear)after it
had been exercised, depending on the CST’s requirements and their threshold for risk. This is
becausetherewouldbea“sanitycheck”thatmaybeneededafterchoosingtopivot,whereCSTmay
need togobackaftersometimeandreassessagain tomakesure theirshort-termand long-term
goalsarebeingconsidered.
93
Thefinaloptionconsideredistheoptionthat isthepaththatCSTpursuedwhichhasledthemto
theirpositiontoday.Thisoptionwastocontinuetogrowandbuildcapabilitiesaccordingtotheir
initial project plans. There were no fatal risks identified and CST was able to continue to the
technologydevelopment stage. Figure24 illustrates a simpledecision tree forCSTatTRL4. It is
importanttounderstandthatCSTisinthegrowthstageandcannotexpectincometobesignificant
(ifany). In fact,becauseofhowtheproposed framework is structured, it isa logicalapproach to
estimatecashflowsshorttermsinceCSTcanexpecttoexerciseanotheroptionatTRL6,therefore
needinganotheroptionandresultinginnewestimatedvalues.
Figure24.DecisionTreeatTRL4
Thenextstepistobuildthebinomiallatticeandvaluetheoptions.Onlythemarketpivotandbuild
capabilitiesoptionswillbeshown.Theabandonmentoptionwouldresult inthe lossof the initial
investmentsmade into theproject.Theprobabilityof success,p(s), isassumed tobe0.5 forboth
options.Thiswassatisfactory forCSTas theyare in theearlystagesof theprojectandwilling to
accepthigherrisk.Theexpectationisthattechnologydevelopmentwilllowertheriskandimprove
p(s)as theprojectmatures.Otherwise, thetechnologyprojectshouldbereconsidered,unless the
investorshaveahigherappetiteforriskandarewillingtogamble.ItisimportanttonotethatCST’s
initialestimationsforexpensesandgrossrevenueswerenotchanged.Thisisbecauseinformation
wasnotsharedintermsofwheretherevenuewascomingfromandwhyexpensesincreasedone
yeartothenext.
BuildCapabilities
GrowthOptions MarketPivot
TRL4:NoFataltechnologicaloranalyticalrisks
Yes
NoStopProject/Abandon
Identifiedpotentialapplicationand/ormarket?
Yes
No
94
These base-case values were left as is and results of the ROA were added onto the base-case
estimations.Allotherassumptionsandresultsaresummarizedbelow.Financialprojectionsareall
providedbyCSTareshowninAppendix2.AllspreadsheetcalculationsareinAppendix3.
Marketpivot
Thepivotoptionwasvaluedwiththefollowingassumptions:
• Assumingtheyearis2014.
• No identifiedmarketapplicability for the technologyaccording toCST’s initialbusinessplans
forthetechnology.Nofataltechnologyoranalyticalriskidentified.
• Theoptionistobeexercisedin2015.
• Taxesandinflationareignoredinthecalculations.
• CST will outsource market experts to identify market opportunities. This is estimated to
increaseexpensesby20%foroneyear.
• The probability of success, p(s)=0.5. The u=1.5 and d=0.67. The reasoning behind p(s) was
previouslydiscussed.
• Allequityorfinancingimplicationsassociatedwiththisoptionareignoredforsimplicity.
• Acceptabledomainexpertiseandtechnicalknowledgeispresentwithintheexistingteam.
• Notechnologicalordesignflaws.
• AR will remain in the same TRL for a maximum of one year. Assuming that the operating
environment,marketorindustrywillbeidentifiedbythen.Nomajortechnologicalmaturityor
projectchangesareassumedduringthistime.
• Thetimetoexerciseisallowedflexibility,asthespecificCSTactivitiesthatwereconductedin
2014 -2015and thedetailsabout thepaceatwhich theprojectwasdevelopedareunknown.
Thisisallestimated.
• Finally,arevisedcashflowwillbeconducteduptilltheyear2019,asaccordingtoCST’sproject
plans(andinformationknown),anotheroptionwillbeexercisedatthenextstage-gateatthat
time.AllothercashflowvaluesandassumptionsareconsistentwiththeDCFcalculations.
95
OptionValuation:
Figure25.MarketPivotLatticeValuation
Buildcapabilities
Thebuildcapabilitiesoptionwasvaluedwiththefollowingassumptions:
• Financialsofactualexpensesincurredin2014byCSTwereusedforthisoption.
• Theoptionistobeexercisedin2015.
• Taxesandinflationareignoredinthecalculations.
• CSThasenoughresourcestobeabletoexercisethisoption.
• Noknowncompetitionwiththesamepotentialtechnologicalcapabilities.
• The probability of success p(s)=0.5. The u=1.5 and d=0.67. The reasoning behind p(s) was
previouslydiscussed.
• Thetimetoexerciseisallowedflexibility,asthespecificCSTactivitiesthatwereconductedin
2014 -2015and thedetailsabout thepaceatwhich theprojectwasdevelopedareunknown.
Thisisallestimated.
• Finally,arevisedcashflowwillbeconducteduptilltheyear2019,asaccordingtoCST’sproject
plans(andinformationknown),anotheroptionwillbeexercisedatthenextstage-gateatthat
time.AllothercashflowvaluesandassumptionsareconsistentwiththeDCFcalculations.
2015
$757K$840K
2014 2015
$1.26M
$560K
2014
$1.22M
$521K
ProbabilisticPV
$621K PVofOption
($135K)PVCosttogettoOption
96
OptionValuation:
Figure26.BuildCapabilitiesLatticeValuation
The cost to implement themarket pivot option is $135Kwith anoverall present option value of
$621K.Thebuildingcapabilitiesoptionisvaluedat$706Kwithanimplementationcostof$121K.
Thedifferenceincostsbetweenthetwooptionscanbeattributedtotheincreaseinexpensesdueto
theneedtohiremarketexpertsinthemarketpivotoption.Fromthosevalues,buildingcapabilities
yield a higher PV and costs less overall, making it the most desirable option by the numbers.
However, thedifferencebetween thevalues is$85K. Inaproject thatmaypotentiallyyield large
revenues,thisisnotasignificantdifference.Thiscalculationcanbeusedtoreinforceandsupport
thedecisionmadein2015tocontinueandgrow.
2015
$827K$840K
2014 2015
$1.26M
$560K
2014
($121K)PVCosttogettoOption
$1.22M
$680K
ProbabilisticPV
$706K PVofOption
97
If CST had exercised themarket pivot option, then the need for another optionmay have been
warranted (staged-option). This can be argued to be necessary in order for managers to make
educatedandjustifieddecisionsbeforemovingforwardwiththeirnewredefinedgoals.Thevalue
oftheoptionsintheearlystagesisnotsomuchabouttheincomegeneration.It isimportantthat
managers do not become obsessedwith the numbers during early development. Awell-rounded
decisionconsidersallaspectsofanoption,thebenefits,andchallenges,andreferstothenumbers
to reinforce general estimates. The process of conducting ROA can open up important
conversationsaboutwhattheshort-termgoals, long-termgoalsandexpectationsmovingforward
are.Thecosttoimplementanoptioncanprovidesomeinsightonthemagnitudeofeffortrequired
andresourcesandfinancing.
4.3.5ROValuationatTRL6
CST is currently at the commercialization stage-gate. At this point,major AR system integration
issueshavebeenaddressedandahugeportionoftestinghasbeencompleted.CST’sactivitiesand
concerns have begun to shift from technology development, to the preparation for scale-up and
commercialization.Theresourcerequirementschangeatthisstagewheretheneedformarketand
scale-upexpertisebecomesanimportantkeyforsuccess.CSThasbeenlookingfornewinvestors
andclients,andaccordingtotheirbusinessmanager,theyhavepotentialclientsandinvestorsthat
are interested in the technology. There have been plans made for field trials with potential
customers.Atthispointintime,itisappropriateforCSTtoconsideroptionsthatsupporttheirgoal
to launch the technology into themarket. Growth for a small start-up such as CST is very risky
because of the need for human resources and financial capital, as the business shifts from
knowledgecreationtoproductcreation.It is importantthatbusinessprocessesmaturequicklyin
ordertobringthetechnologytothemarket.
The three options that were of particular interest to CST were to operate as a datamonitoring
servicefortailingponds,tolicensetheirIP,ortosellARunits.Figure27illustratesadecisiontree
atTRL6.CST’scurrentequityandfinancingpositionwasignoredduringtheanalysis,mainlydueto
limited information and ongoing investor related activities. Once the conditions are better
understood,theanalysisshouldbeincludedtoreflectthesechanges.
98
Theprobabilityofsuccessatthisstageisexpectedtobegreaterthantheestimatedvalueof0.5at
TRL4,astechnologicalcapabilityhasbeenprovenandnofatalrisks(technologyormarket)have
been identified.However, for thepurposeof this thesis, aprobabilityof0.5wasnot changed for
TRL6’sROA.Itisimportanttounderstandthatatthisstageofdevelopment,realistically,investors
wouldexpectahigherp(s).Alowp(s)atlaterstagessignifiesalotofunresolveduncertaintyand
couldbetooriskyandunfavorableforinvestors.
Table10.SummaryofCommercializationOptionsProsandCons
MonitoringServices Licensing&Royalties Sales
ProsCompetitive
advantage,highmargins
Lowoverhead,consistentrevenue&predictablecosts,product/marketexpansion,
innovationfocus
Capitalinflux,highmargins
ConsComplex&highoverhead,saleschannelaccess
Lowermargin,maintenanceresponsibilities
Highcapitalrequirements,distributionchallenges,
inconsistentrevenue
The current technological capability of the AR allows operations in soft tailing ponds, aswell as
highstrengthtailings.ThisgivesCSTacompetitiveadvantage,high-profitmargins,andallowsthem
tostrategicallydominateinsofttailingpondsmonitoringastherearecurrentlynocompetitors.Itis
alsoalegalregulatoryrequirementforproducerstomonitorandmeetspecificdatacollectionrates.
The cons associated with the implementation of the monitoring option include high and
complicated overhead costs, as well as challenges accessing sales channels. CST will be fully
responsible for allmanufacturing,maintenance, and operation of the rover, which increases the
load on their resources.Other risks CST faces include the potential of new strategic competitors
enteringthemarketaswellaspolicyandregulatorychanges,whichinturnmaynegativelyaffect
thedemandforservicesandthetechnology.
99
CSTmaychoosetolicenseoutoveratermorofferoutone-timeperpetuallicenses,andtheymayor
may not choose to implement royalties. The choice behind the licensing structures and fees are
ultimatelyuptoCST’spreference.However,CSTiskeenonaone-timelicensefeeasitmeantless
overall resource commitments to this project and a greater ability to continue developing new
technologiesandstayinginnovative.Areasonableoutcomewiththeimplementationofthisoption
wouldbethatCSTcontinuestobuildstrongrelationshipswiththeclientstheyarelicensingtoand
will later develop more products to sell or license to these customers, depending on the
circumstances.
Selling of AR units is the final option considered by CST. The major benefit is the high-profit
marginsandcapitalinfluxassociatedwithpursuingthisroute.However,forasmallstart-upsuchas
CST,thisoptionmayfacesomechallenges.Forexample,inconsistentrevenuethatmayarisedueto
factorssuchasdistributionchannelchallengesandhighcapital requirements tomanufacture the
rovers.ThereisalsothequestionofresourceconstraintsandwhetherCSTcouldhandlecustomer
demandswiththecurrentteamsizeandcapability.Amajor factorthatmustbeconsideredwhen
choosingthesalesrouteisthelifeofthetechnologyasthiscanimpacttherevenueanddemandfor
AR. Special consideration should be takenwith the sales option as it possesses capital intensive
requirements thatmay have negative effects on CST’s focus on innovation and their competitive
advantage. Some the requirements for this option include shop facilities, procurement channels,
staffforfabricationandprovisionstoofferwarrantyandafter-salesservices.Therefore,thisoption
maynotbeasustainablepathtochooseasnewtechnologydevelopmentprojectsmayneedtobe
put on hold if the sales option is exercised. In addition, assuming there is a fixed number of
potential clients for the next few years in Alberta, CST may sell one unit to a customer, and
depending on the choice of maintenance services, that customer may no longer be a source of
revenueforCST(i.e. theyareaone-timeclient).Therefore, important factorssuchasmarketsize
andwellastechnologylifeareimportant.
100
Figure27.DecisionTreeatTRL6
Similar to options at TRL 4, CST’s initial estimations for expenses and gross revenueswere not
changed.Thesebase-casevalueswereleftasisandnewROAestimationswereaddedontothebase.
Forall threecommercializationoptions, itwillbeassumedthat ifchosen,CSTwill implementthe
optionforamaximumoftwoyears.Theywill thenreassesswhethertheyareontrackwiththeir
project goals. Two years is only a recommendation, however, CST can choose to place an option
whenever they see fit.Basedon this assumption, a cash flowwill onlybe conducted till theyear
2021.Allquotesandserviceschargesstatedforall threeoptionsareasreportedbyCopperstone
Technologies.
MonitoringServices
Commericialization
Licensing/Royalties
SalesTRL6:NoFatalrisksintechnologyscale-uporinpotentialmarketacceptance
Yes
No
StopProject/Abandon
101
MonitoringServices
Themonitoringservicesoptionwasvaluedwiththefollowingassumptions:
• Thisoptionistobeexercisedin2019.
• Operations and rental of AR is quoted at $5,000/day for 8 hours of operation (for the
calculations,thepricewillremainconstantforthenexttwoyearsbutitmaybeastrategicmove
toraisepricesinayeariftherearestillnocompetitors).
• Minimummonitoring requirements of 100 days per year at 8 hours per day resulting in an
estimatedgrossrevenueof$500,000peryearpercustomer.
• Maintenanceandtechnicalsupportfeesarecoveredwithinthequotedmonitoringfees.
• Regular maintenance requirements are estimated at three weeks per year. This is based on
technicalexpertiseandknowledgeofengineersatCSTandexpectedminorglitchesorbugfixes
thatmaybeneededafterthetechnologyislaunched.
• ThemaintenanceexpensesforCSTarequotedat$25Kperclient.
• CSTwillrequireatleastamonthcommitmentsignedcontractbycustomers.
• Thebreakingof contractor cancellation isnot refundable, andany refundsareup to the full
discretionofCST’smanagement.
• There are no competitors in the monitoring of soft tailing ponds and assumed that no new
competitorswillarisewithinthenexttwoyears.
• Overhead costs and expenses associated with manufacturing and production are assumed
constantforthenexttwoyears.
• Noregulatoryorpolicychangesfortwoyearsaftertheoptionisexercised.
• CSTcurrent’sresourcesareabletosupportthepotentialcustomerdemandswithintheareaof
operation.InthecasewhereanewARunitneedstobebuilttomeetdemands,CSTwillbuildin-
house,financedfromprivateinvestors.
• Potentialof3customersforthefirsttwoyearsafterimplementation.ForROAbasecalculations,
thenexttwoyearswillhaveaconstantnumberofcustomers.
• CSTwillonlyoperateinAlbertafor2yearsafterimplementation.
• Taxesandinflationareignoredinthecalculations.
• Additional service charges imposed by CSTmay be added at their discretion in caseswhere
damagetoequipmentoccursonsiteandisdeemedthecustomer’sfault.
102
• Theprobabilityofsuccessp(s)=0.5inordertoremainconservative.However,theprobabilityis
expectedtobehigherthanitwasatTRL436.Theu=1.5andd=0.67aspreviouslydiscussed.
OptionValuation:
Figure28.MonitoringLatticeValuation
36Thishasadirectrelationshipwiththeperceivedrisksatthispointintime.Marketriskcontrolis
animportantfactorinalldecision-makingfromthispointonandCSTisstrategicallyplanning
$2.45M
Today2017 2018 2019
$3.68M
$1.64M
$5.52M
$2.45M
$1.09M
$1.03M
Today201720182019
$2.02M
$458K
$4.12M
$1.05M
$0
($11K)PVCosttogettoOption
ProbabilisticPV
$1.02M PVofOption
103
LicensingandRoyalties
Thelicensingoptionwasvaluedwiththefollowingassumptions:
• CSTwilllicensetheIPasaone-timeperpetuallicenseatafeeof$400K.
• TargetingonlythesmallmarketintheAlbertaoilsandsandIPusagerestrictedtoAlberta(i.e.
notworldwideuse).Limitationsofusewillbepresentedduringthelicensecontractoffering.
• Aconservativeassumptionoftwocustomersintwoyears.
• Implementationof licensingoptionwill increaseexpensesby10%as lawyersmaybeadvised
and other process or legal fees may be associated. This increase in expenses is expected to
occurin2018.
• Anyrequests for improvementsfromthe licenseewillresult inanadditionalcharge.Thiswill
benegotiateduponrequest.
• Maintenanceorconsultingservicesarechargedtocustomersat$150/hour.
• Thecostsassociatedwithmaintaining thepatentareconstantand factored into theexpenses
(sincetheIPhasalreadybeenapprovedandfeeshavebeenpaidout).
• AnyincomedirectlyfromtheuseofIPwillresultina20%royaltystructureforthefirst5years.
ROAwillnotincludethisinthecalculations,however,onceinformationisknown,theyshould
beupdatedtoreflectabetterestimate.
• Taxesandinflationareignoredinthecalculations.
• Theprobabilityofsuccessp(s)=0.5inordertoremainconservative.However,theprobabilityis
expectedtobehighthanitwasatTRL437.Thep(u)=1.5andp(d)=0.67aspreviouslydiscussed.
37Thishasadirectrelationshipwiththeperceivedrisksatthispointintime.Marketriskcontrolis
animportantfactorinalldecision-makingfromthispointonandCSTisstrategicallyplanning
104
OptionValuation:
Figure29.LicensingLatticeValuation
$2.43M
Today2017 2018 2019
$3.65M
$1.62K
$5.48M
$2.43M
$1.08M
$1.16K
Today201720182019
$2.20M
$449K
$4.07M
$1.03K
$0
($25K)PVCosttogettoOption
ProbabilisticPV
$1.14K PVofOption
105
Sales
Thesalesoptionwasvaluedwiththefollowingassumptions:
• CSTestimatestosell2unitsin2years.
• Theoptionistobeexercisedin2019.
• There will be no major technology improvements on the hardware or software and system
integrationwillbeassumedtobereadyformarketlaunch.
• Thesellingpriceperunitis$200K.
• The cost for CST to build anAR unit is approximately $120K including the cost of labor and
parts.
• Ifthisoptionischosen,CSTplanstobuildoneARunitin2017andoneunitin2018inorderto
meetexpectedfuturedemand.
• Any unit sold will also require customers to purchase maintenance services quoted at
$150/hour.Regularmaintenance isestimatedtobescheduled foramaximumof threeweeks
peryearandisbasedontechnicalexpertiseandknowledgeofengineersatCSTandexpected
minorglitchesandbugsthatmayariseafterlaunch.
• CST is theonly companyqualified toprovided technical supportormaintenance services.No
competitorscanprovidethisservice.
• Manufacturingcostsareassumedconstantforthenext2years.
• RefundsonARpurchasesareonlyconductedunderfulldiscretionofCST.
• Thecosttoimplementisbasedonthesamevaluesprovidedinthebase-caseanalysis(i.e.fixed
admincostsandotherexpenses, thegrowthofCST isnotanticipated for thenext twoyears-
however,ispossible).
• Thereisnodirectcompetitionforthenexttwoyearsaftertheoptionisexercised.
• Noregulatoryorpolicychangesforthenexttwoyearsaftertheoptionisexercised.
• Taxes and inflation are ignored in the calculations. Service charges imposed by CSTmay be
addedattheirdiscretion.
• CSThasthecapabilitiesandresourcestobuildin-housethefiveunitsestimatedtosellwithin
thefirstyear.
• Theroverswillbebuiltinhouseforthefirsttwoyears,andifdemandishigh(i.e.morethan5
unitsperyear)outsourcingoptionscanbeconsideredlateron.
106
• Theprobabilityofsuccessp(s)=0.5inordertoremainconservative.However,theprobabilityis
expectedtobehighthanitwasatTRL438.Thep(u)=1.5andp(d)=0.67,aspreviouslydiscussed.
OptionValuation:
Figure30.SalesLatticeValuation
38Thishasadirectrelationshipwiththeperceivedrisksatthispointintime.Marketriskcontrolis
animportantfactorinalldecision-makingfromthispointonandCSTisstrategicallyplanning
$1.72M
Today2017 2018 2019
$2.58M
$1.15M
$3.87M
$1.72M
$765K
$558K
Today201720182019
$1.14M
$139K
$2.48M
$320K
$0
($80K)PVCosttogettoOption
ProbabilisticPV
$479K PVofOption
107
Theestimatedpresentcosttoimplementthemonitoringoptionwas$11Kwithavalueof$1.02M.
Thepresent cost to implement the licensing optionwas estimated to cost around $25K andwas
valuedat$1.14Mandtheestimatedpresentcosttoimplementthesalesoptionwas$80Kwhilethe
optionwasvaluedat$479K.
Theassumptionsmadeduringthecalculationshaveasignificantimpactonthemagnitudeofvalues
forcostimplementationandvalue.Forexample,thecosttoimplementforsalescanbeconsidered
“low”because the assumption thatCSThad twoavailableunits for salewasmade, and that they
would build one unit in 2017 and one in 2018. This of coursewas based on an assumption for
potentialdemand.AdditionaloverheadcostswereassumedtoalreadybeincludedinCST’s initial
projections forgrowth.Therealisticcost to implement theseoptionsareexpected tobedifferent
once CST updates the values. A more accurate estimation was not possible during the study as
detailsbehindsomeofthevaluesontheirfinancialprojectionswerenotsharedanditwasdifficult
togaugeestimatesonexpensesandoverheadswhenCSTalreadyhadincorporatedtheirownplans
forgrowth in theseprojections.Themonitoringoptionalsodeemstohavea low implementation
cost; this is because it was assumed that CST had basic capabilities to meet minimum demand.
Similartothesalesoption,abetterestimationcanbeexpectedoncethesevaluesareupdated.This
isespeciallythecaseoncetruemaintenancecostsarereflectedinthecalculations.
Fromtheseresults, licensingandmonitoringcostsarecomparatively lowerthanthesalesoption.
Thisalignswellwith the initialassumptionsmadeabouteachoption.The licensingoptionyields
the highest value, again, reinforcing some of the expectations related to this option. As it is low
maintenanceandhighmargincosts.Ofcourse,tobetterhaveanideaoftherangeofthesevalues,a
sensitivityanalysisshouldbeconducted.
108
The value in ROA as explained throughout this thesis comes in the ability tomake stream-lined
decisionsbasedonimportantassessmentsmadethroughouttheapplicationofthisframework.The
choicebehindwhytheseoptionswereimplementedafterforaperiodofmaximumtwoyearshas
alreadybeenaddressedatthestartofthissection.Inthecasewhereanimplementedoptiondoes
notgoaccordingtoplan,managerscanreassess,orinextremeconditions,abandon.Theprocessof
estimatinghowmuchanoptionwillcostwillgiveinsightontheeffortthatwillberequiredandthe
resources.ThisisbeneficialforCSTbecauseitaidsintheprocessofprojectplanning.Estimatesof
theoptionvaluescanalsobeused tosupport theirprojectprogressdiscussionsand futureplans
whenspeakingtoinvestorsandtryingtosecurefinancingDuringthecourseofthisthesis,CSTdid
notselectanoption.
4.3.6SensitivityAnalysisforExercisedOption
Asensitivityanalysisshouldbeconductedontheoptionsconsidered,andmorespecificallyonthe
optionchosen.Whenconsideringmorethanoneoption,valuesofdifferentoptionsmaybeclosein
numbers.Itisimportanttoconductawellthought-outsensitivityanalysisinordertoobtainafull
range of values (or at least understand the boundaries), that show the variability in parameters.
Thisisalsosimilarfortheoptionchosen.
A full sensitivityanalysiswasnotcompleted in thiscasestudyas thecalculationsare trivial.The
valueisinthethoughtprocess,andthevariablesshouldbevariedwithriskelementsinmind.CST
did not select an option during the course of this study. However, it is recommended that CST
conduct a sensitivity analysis when they decide whether they will proceed with any of the
recommended options in this study, or any other options in the future. The list of the possible
scenarioslistedbelowfortheanalysisisnotanexhaustivelist.Someapplicablesensitivityanalysis
scenariosrelevanttoCSTinclude:
1. Varyingtheprobabilityofsuccess(thiscouldbefrom0.4to0.6orwhatCSTchooses).
2. Yeartheoptionisexercised.ThiswillallowCSTtostudyhowearlyorlateimplementation
ofanoptioncanaffectitsvalueandtheoverallPVofincome.
3. Varyingtheupswinganddownswingvaluesusedinthelatticevaluation.
4. Varyingtheexpensesrelatedtoimplementation(increase/decreaseby20%forexample).
5. Varyingcustomerdemandandcompetition.
6. Modelingtheeffectsofregulatorychanges(whichwillalsoaffectdemand).
109
4.3.7OtherCaseStudyConsiderations
Thecommercializationoptionsconsideredwererelativelywellunderstoodintermsofadvantages
anddisadvantages.However, inthecasewhereCSTmaybeconsideringbetweenoptionsthatare
not as clear, a SWOT analysis39 could be used to qualitatively assess these options. Ultimately,
marketconditionswilldeterminewhethertheoptionexercisedcandosointimetomeetamarket
window.
DataanalyticsandcollectionareanintegralpartofsuccessfulandefficientARoperations.Thereis
a need for the implementationof IT systems that allowaccess to real-timedata collectedbyAR.
Data exchange programs have been developed for oil and gas industry allowing for seamless
automation of data transfer and remote access. An example of such IT platforms include is the
Partner Data Exchange [2] that was developed by CGI. Integrating a platform as such into AR
technologycanallowforoilsandsproducers,governmentstointerpretrealtimedatafromtailings.
39Strengths,weaknesses,opportunities,threatanalysisthatorganizationsemploywhenassessing
theircapabilitiesandfactors(internalorexternal)thatcanimpactaprojectordecision.
110
4.4.NationalResearchCouncilofCanadaCaseStudy
TheinitialprojectplanforthisthesiswastoconductROAonaNationalResearchCouncilofCanada
(NRC) project. The project would be within the Security and Disruptive Technologies (SDT)
portfolio.Thegoalwasthatacomparativeassessmentwouldbecompletedusingasimilarproject
withinthatportfolio.
4.4.1NRCCaseBackground
OnepotentialapplicationoftheproposedframeworkwouldbetheNRCBoronNitrideNanotubes
(BNNT)technologywithintheSDTportfolio.Theprojectwaschallengedbymanyunforeseenrisks,
mainly due to misinterpreting the potential market. BNNT’s were discovered in 1995. They
remainedatalowerlevelofmaturitythanexpectedfortwodecadesandhavenotreachedastage
ofdevelopmentwhereanyreceptormarkethasbeen identified.Thishas ledBNNTs toremain in
theR&Dsphere.Althoughtherewasno identifiedbusinessorconsumermarket forBNNT’s,NRC
optedtousethestructurallysimilarCarbonNanotubes(CNT)asaproxymarket forBNNT[149].
BNNTssharesimilarmechanicalpropertiesandconductivityofCNTs.However,aremoresuperior
as they possess greater thermal and chemical stability, electrical insulation and the ability to
producecurrentwhensubjectedtomechanicalstress[150].
BNNTsarelight-weight,extremelystrongandtiny.Theyaredescribedtobe100timesthestrength
ofsteelandcanwithstandupto2,000degreesCelsius.Despitethesepositivequalities,NRCargues
thatthesequalitiesarealsothereasonthathasledtodifficultiestocommerciallyproduceBNNTs
[151]. The capabilities and potential future applications of BNNTs have been considered for
applicationsas transparentmilitaryarmor,strongenoughtoholdagainstexplosions,orasBNNT
coatings for buildings to shield against ultraviolet light and fire. For more information about
BNNTs,see[152].
111
4.4.2RecommendationsforFutureNRCPortfolios
TherewasnotenoughinformationprovidedbyNRCtoconductacomparativestudyontheBNNT
case.However,withenoughdata,theproposedmethodologyinthisprojectcouldbeconsideredas
a valuation tool across their entire SDT portfolio. It is important to identify the potential
approachestocommercializetheBNNTtechnology.ThereisanimplicationthatR&Dorganizations
can no longer fully rely on their internal R&D capacity and do not have the ability to cover all
disciplines that contribute to the firm [153]. Therefore, it is crucial that NRC employ external
marketexpertise(ifpossible)inordertoobjectivelystudythepotentialmarkets.Basedonseveral
informalconversationswithmanagersatNRC,thereisatendencyforthesamegroupofemployees
to staywithin the sameportfolio for several years (15+) [54]. Further researchon the skills and
competenciesacquiredanddevelopedovertime,andtheirinfluenceonthedecisionsandstrategies
during the cross-functional processes of developing and commercializing technologies should be
consideredbyNRC’sportfoliomanagers.Onepossibleway is toanalyzehistoricaldata frompast
projectsandidentifytrendsorpatternsindevelopmentalandmanagerialactivitiesthatmayhave
contributedtothesuccess,orincreasedtheriskoffailureforaparticularproject.
NRCisheavyonearlytheoreticalresearchandcanexpecttobeginanyprojectatTRL1-2.Theycan
expecttoexerciseatleastfouroptionsatTRL2,4,6and7.ThiswouldbeexpectedfortheBNNT
project.Projects forwhich there ishighNRCcapabilitybut lowpotential impact forCanada,may
not be pursued. Projects for which there is low NRC capability, but very high potential and
sustainableimpactforCanada,maybepursuedaspartofanevolvingNRCstrategyforprograms,
hiring, and facilities [23]. Contrary to RO application for CST, options exercised for NRCmay be
years frompresent time.Thiscanbeexpectedwithvery long,projectswithextensive theoretical
research. It is important thatPortfoliomanagers atNRCare able toplanandbudget accordingly
with importantmilestones clearly defined so that project progress can occur. This can avoid the
pitfallofendlessyearsoftheoreticalresearchwithnofutureapplicationandnoshort-termorlong
termgoals,tyingupresourcesthatcanbeallocatedelsewhere.
112
AgeneralrecommendationfortheBNNTcaseistobeginbyreassessinginformationavailableand
assessingtechnologicalmaturityasoftoday,usingaformalTRA.Anycomponentsthatarenotyet
at the target TRLneed to be addressed accordingly. The use of the CNTs’ success as a proxy for
BNNTs should be approached with caution to avoid oversimplifying assumptions and avoid
personalbiasesthatmaycausemanagerstoviewoutcomesmoreoptimistically.Theeffectsofthese
biases can encourage investments in technology-push projects that could have a higher risk of
failure. Distinctions between the CNT application and BNNT application must be made and any
marketoverlapshouldbeidentified.TheimpactofmarketoverlaponthesuccessoftheCNTsand
potentialBNNTsneedtobeconsidered.
Short-termand long-termprojectandportfoliogoalsneed toberedefined,andclearandspecific
milestones must be set. A formal market analysis needs to be conducted to reflect the market
landscape today. An option towait-and-see for one yearmay be possible forNRC depending on
theircurrentfinancialandresourceconstraints.NRCmustidentifyallcriticalrisksmovingforward
anddevisealternativeoptionsandback-upplans toavoidprojectoverrunsandsloworstagnant
projectprogress.UsingstagedoptionsisvaluablefortheBNNTprojectasitwillbreakdownavery
long project into smaller andmanageable tasks. Thiswaymanagers can keep updating progress
and value their decisions for short-term and long-term progress as they continue to grow. This
approach can overall help the SDT portfolio managers to better budget and allocate resources
accordinglywhilekeepingtheportfoliobalancedandrunningefficiently.
113
Chapter5Conclusion
5.1Conclusion
The aim of the research conducted was to develop a decision-making framework that can be
appliedtosingleR&Dandtechnologyprojects,oracrossentireportfolios.Theproposedframework
isastrategictoolformanagers,researchers,anddecision-makersusedtovaluedecisionsmadein
highlyriskyprojects.Itcombinesastage-gateapproachforrisk-baseddecision-makingbyusingthe
technologyreadinesslevelscalealongsidearealoptionsapproach.Thisapproachisarguedtodrive
organizations to make evidence-based decisions for technology development and project
continuation.
The framework recommends a minimum number of three real options to be placed along the
technology readiness levels at different points during the project. These are key decision points
during technology development as there is a significant change in activities, risks, resource
requirements and financial requirements. The first option is at the TRL 4 stage-gate where the
technologyisabouttoenterthetechnologydevelopmentphase.Thesecondoptionistobeassessed
atTRL6wheretheprojectshifts toproductcreation.Beyondthisstage,scale-up,marketing,and
commercialization activities are dominant. The final option is recommended at TRL 7, before a
technology is about to enter the production and deployment phase. The major risk categories
associatedwithtechnologyprojectsareidentifiedasscienceandtechnology,financial,marketand
organizational.Theresearchdiscussesthegeneralbehaviouraltrendsoftheseriskcategoriesand
how they change during a project, and their effects on different aspects of a technology
development. This is a flexible framework that allowsmanagement to addoptions as needed, as
more information is obtained during a project. This approach was developed as a streamlined
methodology that supports risky technology andR&Dprojects inways that traditional valuation
methodssuchasdiscountedcash-flow(DCF)andnetpresentvalue(NPV)couldnot.
114
Theframeworkwasthenappliedtoanautonomousrover(AR)projectcurrentlybeingdeveloped
byCopperstoneTechnologies(CST).CST’smaingoals fortheARprojectandtheoverallcompany
was tomake a profitwhile allowing the founders towork on new technologies and bringmore
innovations to the market. Factors such as competition, demand, and regulatory policies were
determined as critical risks through interviews with CST’s business development manager and
theirchieftechnologyofficer.
Theeffectsoftheserisksontheoverallprojectsuccessandpotentialprofitwerediscussed.Scale-
upactivitiesatthesecondstage-gatewerealsodeterminedtobeimportant.Technologyanddesign
riskswerealsodiscussed.The importanceofCSTensuringtheyhaveasystemdesignthatallows
futuremodifications thatreflect improvementswithouthaving tomake largedesignchangeswas
criticaltotheirsuccessandtheirfuturecompetitiveadvantage.
Options were retroactively valued at TRL 4. CST’s decision to continue development and build
capabilitiesbackin2014wasdeterminedtohavebeentheappropriateoptionforthematthetime.
Itwasin-linewithCST’sgoaltoremaininnovativeandthevaluationsupportedtheirinitialplanto
commercialize the technologyasa tool thatwillbeutilized in thenichemarketof theAlbertaoil
sands. Decisions were also valued at the second stage-gate, which is where CST’s currently
positioned today.Basedon the results from the interview, three commercializationoptionswere
consideredandassessed.LaunchingtheARtechnologyinthemarketandoperatingasamonitoring
service was the first option considered. The second option was to license the IP to oil sands
operators through a one-time perpetual license fee. The final option was to sell AR units.
Advantagesanddisadvantagesofeachoptionwerediscussed.Duringthecourseofthisthesis,CST
didnotselectacommercializationoption.Therefore,noadditionalinformationwascollectedafter
thevaluationatTRL6.
115
Monitoring and sales optionswere found to take away fromCST’s ability to continue to develop
newtechnologiesandremaininnovative.Thiswasduetothehighresourcerequirementsneeded
fromthesetwooptions.ThesalesoptionalsoposedissuesasitdidnotmeetCST’spreferencefor
anoption thatwill produce consistent revenue.Licensingof IPwasdetermined to allowCST the
most flexibility to continue to innovate and create new technologies. These observations were
proven through the cost to implement the option, aswell as the potential income that could be
generateduponimplementation.
Thereliabilityofthevaluationwasalsoaddressed.Duetothelimitedavailableinformationforthis
case study,many estimations and assumptionsweremade and stated accordingly. An important
assumption that had an effect on the valuation of option was the probability of success (p(s)).
DuringearlystagesofatechnologyitwasarguedthatCSTwaswillingtoaccepthigherriskasthey
enteredthetechnologydevelopmentphase.Realisticallyspeaking, thep(s)shouldhave increased
atthesecondstage-gatewherethetechnology’scapabilitieshadgrown,andrisksfromtheprevious
stageshadbeenmitigatedaccordingly.However,becauseofthelimitedavailabilityofdataforthis
study,thep(s)of0.5wasusedforbothstage-gatevaluations.Itwasexplainedthatinvestorswould
expectahigherp(s)athigherTRLsandloworunchangedp(s)atlaterstagessignifiedunresolved
uncertaintyandahighervolatilitywhichwouldandcouldbeunfavourableformanyinvestorsand
asignthattheprojectmayneedtobereassessedorabandoned.
Finally,apotentialapplicationtoaBoronNitrideNanotubes(BNNT)projectbyNationalResearch
CouncilCanada(NRC)wasintroduced,andthevalueoftheproposedframeworktotheirportfolio
wasbrieflydiscussed.Unfortunately,therewasnodataorinformationobtainedbyNRCinorderto
conductacomparativeassessment.
In conclusion, the framework developed is a useful tool that can allow managers to better
understand risks and how they change as a technology progresses relative to the TRL scale. It
enablesamoreconsistentdiscussionaboutthecomparativevalueofdifferentoptionsandcanbe
used to justify spending on long and risky technology development projects. Framework
limitations,challengesanddirectionsforfutureresearcharediscussedinthefollowingsections.
116
5.2FrameworkLimitations
The framework has limitations in its current applicability. It does not outline an alternative
approach to estimate upswing, downswing, and the probability of success for early-stage
technology projects, which may weaken the quantitative aspect of the assessment and raise
questions about reliability from managers and others using it. Difficulties in obtaining exact or
relativelyaccurateinputsforROAmayresultinsomemanagersunderminingthevalidityofresults
duringdecision-making.Although thiswasstressed throughoutChapter4, somedecision-makers
maybecomeobsessedwiththenumbersproducedfromtheROAanddisregardtherealvalueofthe
framework. The proposed methodology does not replace the need for technology development
strategists.Theoutlinedframeworkcouldbefurtherrefinedtoprovidecleartiesbetweenstrategy
andoptionselection.Valuedriverssuchasbuildingupofscaletogaincompetitiveadvantageina
market,orinnovativeproductdifferentiationshouldbeclearlylinkedtostrategiesandaportfolio
ofrealoptionsthatareapplicable.Thiscanhelpinexperienceduserscaptureopportunitiesifthey
are able to understand RO application better. This can also reduce some of the vagueness that
inexperienced managers may feel towards the concept of real options valuation. An improved
methodtopresenttheseconceptsinanorganizedmannerthatissimpleforeveryoneinvolvedina
project to understand is necessary. This can be done using step-by-step approaches to option
valuationorperhapsanin-housetoolbuiltspecificallytoaportfoliothatcanbeusedbydecision-
makers.
The framework discusses the possibility of correlated variables and risk elements but does not
outlineadefinedapproachtoassesstheserelationshipsandproperlyquantifythembeyondusinga
sensitivity analysis.More detailedmapping of risks along theTRL scale should be considered as
well as their potential effects on other categories. The research also stresses the importance of
havingavailableandappropriateresources,butdoesnotreallydefineamethodthatorganizations
can use to assess their capabilities and need for additional capabilities as a whole. A possible
approach toovercome this limitation is toperformanorganizational capabilitiesaudit.Formore
informationsee[154].
117
Thedevelopedframeworkdoesnotdirectlytakeintoconsiderationtechnology-lifecyclesandtheir
effect on the option value and cost to implement. During the development of the framework
(Chapter 3) and the analysis (Chapter 4), therewasmentionof the technology-life cycle and the
possibleeffectsonoptionvalue.Therewasnothingbeyondstatingthatthisfactorcaninfluencethe
value of option. The framework can be strengthenedby incorporating technology-life cycles into
theanalysis.Thiscanaidmanagerstobetterestimatewhenandwhether(andthedegreeofwhich)
thepotentialprofitswilloffsetR&Dcosts.This cansupportdecisionsas the technology-life cycle
can help organizations to predict adoption and decline of the technology and can providemore
insightonhowrisksmightaffectthelifespanofatechnology.Theeffectofintellectualproperty(IP)
may also have an influence on the technology-life cycle. This should be considered for future
applications.
5.3RecommendationsforFutureResearch
Inadditionto improvementsthatcanbemadetothe limitationsdiscussed in5.2, thereareother
areasforfutureresearchthatcanbeconsidered.
The methodology should be refined further as more data becomes available and we develop a
better understanding of internal capabilities and resource restraints. As start-ups begin to grow
theirportfolios,thiswarrantstheneedtoconsidermarketcannibalization.Thisreferstotheeffect
that new products being developed within an organization have on the performance of other
existing products. The performance is often measured through sales of these products [1]. One
aspectthatcouldbefurtherexaminedarethedecisionsbehindwhichproductstoreleasefirstand
the strategy behind how technology risks could be blended across several products40. Projects
should not always be assumed to be independent, as theymay be linked in severalways. These
linkages can have effects on other projects thatmay be kept at lower TRLs in order to support
higherTRLprojects.
40BasedonconversationswithDr.MichaelLipsettandDr.SahilRainaattheUniversityofAlberta
118
Futureresearchshouldtakeintoconsiderationthedifferencesbetweenpublicandprivatesectors
when implementing such a framework. Metrics for success need to be defined as some R&D
organizations may be driven by scientific competence that achieve a non-economic impact in
Canada41. Aspects such as types of financing whether government funded or private investors,
shouldbestudiedandtheireffectonstrategicdecisionmakingduringdevelopment.Potentialrisks
and critical factors of success should be studied in more depth and organized by industry and
magnitudeof impact inorder to further streamline themethodology.Thismay requireextensive
datafromvariousindustriesinordertoproduceasomewhatcompletelist.
Futureresearchcouldalsoincorporateaformalprocesswithintheframeworkwhendealingwith
residualriskfromlowerTRLsthatmaybecarriedforwardintothelaterstagesofdevelopmentand
how these can be accounted for quantitatively. Keeping in mind that the lowest TRL for a
component,meansthattheentiresystemisatthatlowestTRL,regardlessofwhatTRLstheother
componentsareat.
Futureimprovementsshouldincorporateportfoliomanagementandresourceallocationguidelines
fororganizationsthataremoreseasonedandhavebeenoperatingforseveralyearsandaremore
“set in their ways”. Ensuring resources are properly managed and utilized across projects is a
success factor in new product development [21]. The research should explore a process for
resource allocation andmanagement so that resources can be optimized across a portfoliowith
projects that have varying levels of risk. Research should also consider concepts related to
technology transfer42 between functional groupswithin an organization and processes to ensure
smoothtransitionsbetweenthetwogroups.
41Frome-mailcommunicationbetweenNRCandDr.Lipsett
42Thisreferstothetransferthatoccursfromtechnologytonewproductdevelopment.Sometimes
itmaybeanentirelynewteamthattakesoverthenewproductdevelopment.
119
References
[1] M.C.Dissel,R.Phaal,C.J.Farrukh,andD.R.Probert,“ValueRoadmapping,”Res.-Technology
Manag.,vol.52,no.6,pp.45–53,2009.
[2] G. Martin-Barrera, C. Zamora-Ramirez, and J. M. Gonzalez-Gonzalez, “Application of real
options valuation for analysing the impact of public R&D financing on renewable energy
projects:Acompany’sperspective,”Renew.Sustain.EnergyRev.,vol.63,pp.292–301,2016.
[3] F. Hunt, R.Mitchell, R. Phaal, and D. Probert, “Early valuation of technology: real options,
hybridmodelsandbeyond.,”J.Soc.Instrum.ControlEng.,vol.43,no.10,pp.730–735,2004.
[4] G.AjamianandP.Koen,“TechnologyStage-Gate:astructuredprocessformanaginghigh-risk
newtechnologyprojects,”PDMAToolb.newProd.Dev.,vol.1,pp.267–295,2002.
[5] J.Wang,W.Lin,andY.H.Huang,“Aperformance-orientedriskmanagementframeworkfor
innovativeR&Dprojects,”Technovation,vol.30,no.11–12,pp.601–611,2010.
[6] L.Trigeorgis, “Topics inRealOptionsandApplicationsRealOptionsand InteractionsWith
FinancialFlexibility,”Financ.Manag.,vol.22,no.3,pp.202–225,1993.
[7] J. Mun, Real Options Analysis Tools and Techniques for Valuing Strategic Investments and
Decisions,SecondEdi.WileyFinance,2006.
[8] L. Trigeorgis, “The Nature of Option Interactions and the Valuation of Investments with
MultipleRealOptions,”J.Financ.Quant.Anal.,vol.28,no.1,pp.1–20,1993.
[9] R. Ragozzino, J. J. Reuer, and L. Trigeorgis, “RealOptions in Strategy and Finance: Current
GapsandFutureLinkages,”Acad.Manag.Perspect.,vol.30,no.4,pp.428–440,2016.
[10] A.MenichiniandM.Celiktas,Case:RealOptionsinDefenseR&D.2016.
[11] “Real Option,” Investopedia. [Online]. Available:
http://www.investopedia.com/terms/r/realoption.asp.[Accessed:01-Jul-2017].
[12] A.M. Batkovskiy, A. V. Leonov, A. Y. Pronin, E. G. Semenova, andA. V. Fomina, “Models of
EconomicEvaluationofHigh-TechProducts,”IndianJ.Sci.Technol.,vol.9,no.28,2016.
[13] D. F. Rico, “ROI of Technology Readiness Assessments Using Real Options: An Analysis of
GAODatafrom62U.S.DoDPrograms,”Unpubl.WhitePap.,2007.
[14] H.S..HerathandC.S.Park,“EconomicAnalysisofR&DProjects:AnOptonsApproach,”Eng.
Econ.,vol.44,no.1,pp.1–35,1999.
[15] R.G.Cooper,“ManagingTechnologyDevelopmentProjects,”IEEEEng.Manag.Rev.,vol.35,
no.1,pp.67–76,2007.
[16] J.C.Mankins,“Technologyreadinessandriskassessments:Anewapproach,”ActaAstronaut.,
vol.65,no.9–10,pp.1208–1215,2009.
120
[17] G.L.Tritle,E.F.VScriven,andA.R.Fusfeld,“ResolvingUncertaintyinR&DPortfolios,”Res.-
TechnologyManag.,vol.43,no.6,pp.47–55,2000.
[18] L.M.Meadeanda.Presley,“R&Dprojectselectionusingtheanalyticnetworkprocess,”Eng.
Manag.IEEETrans.,vol.49,no.1,pp.59–66,2002.
[19] W.D.Bodensteiner,E.A.Gerloff,andJ.C.Quick,“Uncertaintyandstress inanR&Dproject
environment,”R&DManag.,vol.19,no.4,pp.309–322,1989.
[20] J.Wang,C.Y.Wang,andC.Y.Wu,“ArealoptionsframeworkforR&Dplanningintechnology-
basedfirms,”J.Eng.Technol.Manag.,vol.35,pp.93–114,2015.
[21] R. G. Cooper and E. J. Kleinschmidt, “Winning businesses in product development: The
criticalsuccessfactors,”Res.Manag.,vol.50,no.3,pp.52–66,2007.
[22] C. Carlsson, R. Fullér, M. Heikkilä, and P. Majlender, “A fuzzy approach to R&D project
portfolioselection,”Int.J.Approx.Reason.,vol.44,no.2,pp.93–105,2007.
[23] M.G.Lipsett,“PersonalCommunication.”2017.
[24] K. M. Kelm, V. K. Narayanan, and G. E. Pinches, “Shareholder Value Creation during R&D
InnovationandCommercializationStages,”Acad.Manag.,vol.38,no.3,pp.770–786,1995.
[25] Y. Lee and R. Gaertner, “Technology Transfer from University to Industry: A Larg–Scale
Experimentwith Technology Development and Commercialization,”Policy Stud. J., vol. 22,
no.2,pp.384–399,1994.
[26] D.Klar, J.Frishammar,V.Roman,andD.Hallberg, “ATechnologyReadinessLevelscale for
ironandsteel industriesATechnologyReadinessLevelscale for ironandsteel industries,”
Ironmak.Steelmak.,vol.43,no.7,pp.494–499,2016.
[27] E.W.EldredandM.E.Mcgrath, “CommercializingNewTechnology—II,”Res.Manag., vol.
40,no.2,pp.29–33,1997.
[28] J.BirkinshawandM.T.Hansen,“TheInnovationValueChain,”no.July,2007.
[29] L. Y. Cohen, P. W. Kamienski, and R. L. Espino, “Gate system focuses industrial basic
research,”Res.Manag.,vol.41,no.4,pp.34–37,1998.
[30] M.Hoegl,K.Weinkauf,andH.G.Gemuenden,“InterteamCoordination,ProjectCommitment,
andTeamworkinMultiteamR&DProjects:ALongitudinalStudy,”Organ.Sci.,vol.15,no.1,
pp.38–55,2004.
[31] U.S Government Accountability Office, “Technology Readiness Assessment Guide Best
Practices for Evaluating theReadiness of Technology forUse inAcquisitionPrograms and
Projects,”2016.
[32] Y. Z. Mehrjerdi and M. Dehghanbaghi, “A Dynamic Risk Analysis on New Product
121
DevelopmentProcess,”Int.J.Ind.Eng.Prod.Res.,vol.24,no.1,pp.17–35,2013.
[33] J. M. Eckhause, D. R. Hughes, and S. A. Gabriel, “Evaluating real options for mitigating
technicalriskinpublicsectorR&Dacquisitions,”Int. J.Proj.Manag.,vol.27,no.4,pp.365–
377,2009.
[34] N.Azizian, S. Sarkani, andT.Mazzuchi, “A comprehensive reviewand analysis ofmaturity
assessment approaches for improved decision support to achieve efficient defense
acquisition,”inProceedingsoftheWorldCongresson…,2009,vol.2.
[35] C.Dion-schwarz,“HowtheDepartmentofDefenseUsesTechnologyReadinessLevels,”2008.
[36] A.Olechowski,S.D.Eppinger,andN.Joglekar,“Technologyreadinesslevelsat40:Astudyof
state-of-the-artuse, challenges, andopportunities,” inPortland InternationalConferenceon
ManagementofEngineeringandTechnology(PICMET),2015,pp.2084–2094.
[37] I. Ameka and W. Dhewanto, “Technology Push vs. Market Pull in Technology University
InnovationCommercialization,”Inf.Manag.Bus.Rev.,vol.5,no.7,pp.337–341,2013.
[38] H. Munro and H. Noori, “Measuring Commitment to New Manufacturing Technology:
IntegratingTechnologicalPushandMarketingPullConcepts,”IEEETrans.Eng.Manag.,vol.
35,no.2,pp.63–70,1988.
[39] L. S. Wheatcraft, “Developing Requirements for Technology-Driven Products,” INCOSE Int.
Symp.,vol.15,no.1,pp.681–693,2005.
[40] G. Di Stefano, A. Gambardella, and G. Verona, “Technology push and demand pull
perspectives in innovation studies: Current findings and future research directions,” Res.
Policy,vol.41,no.8,pp.1283–1295,2012.
[41] G. F. Nemet, “Demand-pull, technology-push, and government-led incentives for non-
incrementaltechnicalchange,”Res.Policy,vol.38,no.5,pp.700–709,2009.
[42] M.Faizal,A.Zaidi,andS.N.Othman,“UnderstandingtheConceptofDynamicCapabilitiesby
DismantlingTeece,Pisano,andShuen(1997)’sDefinition,”Int.J.Acad.Res.Bus.Soc.Sci.,vol.
2,no.8,pp.367–378,2012.
[43] D.J.Teece,G.Pisano,andA.M.Y.Shuen,“DynamicCapabilitiesandStrategicManagement,”
Strateg.Manag.J.,vol.18,no.7,pp.509–533,1997.
[44] J.C.Mankins,“TechnologyReadinessLevels.AWhitePaper,”1995.
[45] K.Tomaschek,A.L.Olechowski,S.D.Eppinger,andN.R.Joglekar,“ASurveyofTechnology
ReadinessLevelUsers.,” inINCOSEInternationalSymposium,2016,vol.26,no.1,pp.2101–
2117.
[46] C.Piaszczyk, “ModelBasedSystemsEngineeringwithDepartmentofDefenseArchitectural
122
Framework,”Syst.Eng.,vol.14,no.3,pp.305–326,2011.
[47] B. Sauser, J. E. Ramirez-marquez, and W. Tan, “A Systems Approach to Expanding the
TechnologyReadinessLevelwithinDefenseAcquisition,”2009.
[48] DepartmentofDefense,“TechnologyReadinessAssessmentDeskbook,”2005.
[49] Dan Newman, “Technology Readiness Level and they Valley of Death,” 2017. [Online].
Available: http://www.boeing.com/features/innovation-quarterly/may2017/feature-
thought-leadership-newman.page.[Accessed:04-Jul-2017].
[50] NASA,“NASAEngineeringHandbook,”2007.
[51] NASA, “NASA TRL Definitions.” [Online]. Available:
https://www.nasa.gov/sites/default/files/trl.png.[Accessed:01-Jan-2017].
[52] R. Valerdi and R. J. Kohl, “An Approach to Technology Risk Management,” in Engineering
SystemsDivisionSymposium,2004,vol.3,pp.1–8.
[53] J.C.Mankins,“Technologyreadinessassessments:Aretrospective,”ActaAstronaut.,vol.65,
no.9–10,pp.1216–1223,2009.
[54] NationalResearchCouncilofCanada,“PersonalCommunication.”2017.
[55] J.C.Mankins, “Approaches toStrategicResearchandTechnology(R&T)AnalysisandRoad
Mapping,”ActaAstronaut.,vol.51,no.1–9,pp.3–21,2002.
[56] R. A. Bauer, P. S. Millar, and C. D. Norton, “Bridging the Technology Readiness ‘Valley of
Death’UtilizingNanosats,”2015.
[57] J. C. Mankins, “RESEARCH & DEVELOPMENT DEGREE OF DIFFICULTY. A White Paper.,”
1998.
[58] W.L.Nolte,B.M.Kennedy,andJ.DziegielRogerJ.,“TechnologyReadinessLevelCalculator,”
WhitePap.AirForceRes.Lab.2004.,pp.1–16,2003.
[59] J. E. Kasser, “Applying Holistic Thinking to the Problem of Determining the Future
AvailabilityofTechnology,” IEEETrans.Syst.Man,Cybern.Syst.,vol.46,no.3,pp.440–444,
2016.
[60] J. W. Bilbro, “Using the Advancement Degree of Difficulty (AD2) as an input to Risk
Management,”Multi-dimensionalAssess.Technol.Matur.Technol.Matur.Conf.,2008.
[61] S.Gavankar,S.Suh,andA.A.Keller,“TheRoleofScaleandTechnologyMaturityinLifeCycle
AssessmentofEmergingTechnologiesACaseStudyonCarbonNanotubes,”J.Ind.Ecol.,vol.
19,no.1,pp.51–60,2014.
[62] B.BasuandS.Ghosh,“Chapter11AssessmentofTechnologyandManufacturingReadiness
Levels,” inBiomaterials forMusculoskeletal Regeneration: Applications, Springer, 2016, pp.
123
235–246.
[63] B. Sauser, D. Verma, J. Ramirez-Marquez, and R. Gove, “From TRL to SRL: The concept of
systems readiness levels,” inConference on Systems Engineering Research, Los Angeles, CA,
2006.
[64] J.Fernandez,“ContextualRoleofTRLsandMRLsinTechnologyManagement,”SandiaNatl.
Lab.SAND2010-7595,no.November,2010.
[65] J. A. Keizer, J. I. M. Halman, andM. Song, “From experience: Applying the risk diagnosing
methodology,”J.Prod.Innov.Manag.,vol.19,no.3,pp.213–232,2002.
[66] V.Thorn,F.Hunt,R.Mitchell,D.Probert,andR.Phaal, “Internal technologyvaluation :real
worldissues,”Int.J.Technol.Manag.,vol.53,no.2–4,pp.149–160,2011.
[67] J.Wang andC. Y. Yang, “Flexibility planning formanagingR&Dprojects under risk,” Int. J.
Prod.Econ.,vol.135,no.2,pp.823–831,2012.
[68] H.Mikkelsen,“Riskmanagementinproductdevelopmentprojects,”Int.J.Proj.Manag.,vol.8,
no.4,pp.217–221,1990.
[69] M.Benaroch,“Option-BasedManagementofTechnologyInvestmentRisk,”vol.48,no.4,pp.
428–444,2001.
[70] K. Rudnik and A. M. Deptuła, “System with probabilistic fuzzy knowledge base and
parametricinferenceoperatorsinriskassessmentofinnovativeprojects,”ExpertSyst.Appl.,
vol.42,no.17–18,pp.6365–6379,2015.
[71] R. Balachandra and J. H. Friar, “Factors for Success in R&D Projects and New Product
Innovation :AContextualFramework,”IEEETrans.Eng.Manag.,vol.44,no.3,pp.276–287,
1997.
[72] S.K.SharmaandU.Chanda,“DevelopingaBayesianbeliefnetworkmodelforpredictionof
R&Dprojectsuccess,”J.Manag.Anal.,pp.1–24,2017.
[73] J. a Keizer, J. Vos, and J. I. M. Halman, “Risks in New Product Development Risks in New
ProductDevelopment,”no.February2003,pp.1–26,2005.
[74] T.Daim,ManagingTechnologicalInnovation:ToolsandMethods,Volume1.WorldScientific
Publishing,2017.
[75] Y.H.Park,“AStudyofR&DInvestmentFrameworkandSuccessFactors,”AsianJ.Qual.,vol.9,
no.1,pp.103–112,2008.
[76] J. Wu and Z. Wu, “Integrated risk management and product innovation in China: The
moderatingroleofboardofdirectors,”Technovation,vol.34,no.8,pp.466–476,2014.
[77] R. G. McGrath and I. C. MacMillan, “Assessing technology projects using real options
124
reasoning,”Res.…,vol.43,no.4,pp.35–49,2000.
[78] L.M. Brandscomb andP. E. Auerswald,TakingTechnical Risks:How Innovators,Managers,
andInvestorsManageRisk.MITPress,2003.
[79] R. G. McGrath, “A Real Options Logic for Initiating Technology Positioning Investments,”
Acad.Manag.Rev.,vol.22,no.4,pp.974–996,1997.
[80] L. Vincent, “Marketing Strategies for Commercialization of New Technologies,” Technol.
Innov.Gener.Econ.Results,pp.257–287,2016.
[81] B.Kayisetal.,“IRMAS-developmentofariskmanagementtoolforcollaborativemulti-site,
multi-partnernewproductdevelopmentprojects,” J.Manuf.Technol.Manag.,vol.18,no.4,
pp.387–414,2007.
[82] Jay J. Janney and Gregory G. Dess, “Can Real-Options Analysis Improve Decision-Making?
PromisesandPitfalls,”Acad.Manag.,vol.18,no.4,pp.60–75,2004.
[83] D.Hillson,“Extendingtheriskprocesstomanageopportunities,”Int.J.Proj.Manag.,vol.20,
no.3,pp.235–240,2002.
[84] S.L.Murray,K.Grantham,andS.B.Damle,“DevelopmentofaGenericRiskMatrixtoManage
ProjectRisks,”J.Ind.Syst.Eng.,vol.Vol.5,no.1,pp.35–51,2011.
[85] N. Fenton andM.Neil,Risk Assessment andDecision Analysiswith BayesianNetworks. CRC
Press,2012.
[86] V.Dumbrava,“UsingProbability–ImpactMatrixinAnalysisandRiskAssessmentProjects,”
J.Knowl.Manag.Econ.Inf.Technol.,no.SpecialIssue,pp.76–96,2013.
[87] L.AnthonyCox, “What’swrongwithriskmatrices?,”RiskAnal.,vol.28,no.2,pp.497–512,
2008.
[88] R.S.KaplanandD.P.Norton, “Thebalancedscorecard.Measures thatdriveperformance,”
Harvardbusinessreview,pp.71–79,2005.
[89] S.A.Mastroianni,“RiskManagementamongResearchandDevelopmentProjects,”2011.
[90] J.R.Hauser,“HowPuritan-Bennettusedthehouseofquality,”SloanManage.Rev.,vol.34,no.
3,pp.61–70,1993.
[91] E.Schwartz,“Therealoptionsapproachtovaluation:Challengesandopportunities,”Lat.Am.
J.Econ.,vol.50,no.2,pp.63–177,2013.
[92] U.Dzyuma,“Realoptionscomparedtotraditionalcompanyvaluationmethods:Possibilities
andconstraintsintheiruse,”e-Finanse,vol.8,no.2,p.51,2012.
[93] J.E.N. IiiandR.DeNeufville, “Hybridrealoptionsvaluationofriskyproductdevelopment
projects,”Int.J.Technol.PolicyManag.,vol.1,no.1,p.29,2001.
125
[94] S.-C.Lee,“Usingrealoptionanalysisforhighlyuncertaintechnologyinvestments:Thecaseof
windenergytechnology,”Renew.Sustain.EnergyRev.,vol.15,no.9,pp.4443–4450,2011.
[95] S.C.Myers,“Financetheoryandfinancialstrategy,”Midl.Corp.Financ.J.,vol.5,no.1,pp.6–
13,1987.
[96] B.DeReyck, Z.Degraeve, andR.Vandenborre, “Projectoptionsvaluationwithnetpresent
valueanddecisiontreeanalysis,”Eur.J.Oper.Res.,vol.184,no.1,pp.341–355,2008.
[97] E. S. Schwartz and L. Trigeorgis, Real Options and Investment under uncertainty: Classical
ReadingsandRecentContributions.MITPress,2004.
[98] N.T.Nang,N.Takezawa,andN.Takezawa,“Realoptionsandtheevaluationofresearchand
developementprojectsinthepharmaceuticalindustry:Acasestudy,”J.Oper.Res.Soc.Japan,
vol.45,no.4,pp.385–403,2002.
[99] J. Yliopisto, “Real Options Analysis as a tool for Start-Up Company Investment Valuation,”
UniversityofJyväskylä,2015.
[100] K.MilisandR.Mercken,“TheuseofthebalancedscorecardfortheevaluationofInformation
andCommunicationTechnologyprojects,”Int.J.Proj.Manag.,vol.22,no.2,pp.87–97,2004.
[101] R.G.Cooper,S.J.Edgett,andE.J.Kleinschmidt,“BestpracticesformanagingR&Dportfolios,”
Res.Manag.,vol.41,no.4,pp.20–33,1998.
[102] S.Block,“Are‘RealOptions’ActuallyUsedintheRealWorld?,”Eng.Econ.,vol.52,no.3,pp.
255–267,2007.
[103] P.KodukulaandC.Papudesu,ProjectValuationUsingRealOptions:APractitioner’sGuide. J.
RossPublishing,2006.
[104] C.W.-N..Chan,C.-H. .Cheng,A. .Gunasekaran,andK.-F. .Wong,“Aframeworkforapplying
realoptionsanalysistoinformationtechnologyinvestments,”Int.J.Ind.Syst.Eng.,vol.10,no.
2,pp.217–237,2012.
[105] F.BlackandM.Scholes,“ThePricingofOptionsandCorporateLiabilities,”J.Polit.Econ.,vol.
81,no.3,pp.637–654,2016.
[106] T. A. Luehrman, “Investment Opportunities as Real Options: Getting Started on the
Numbers,”Harv.Bus.Rev.,vol.76,no.4,pp.97–105,1998.
[107] C. M. Sipp and E. G. Carayannis, “Real Options and Strategic Technology Venturing,” Real
OptionsStrateg.Technol.Ventur.ANewParadig.Decis.Mak.,vol.31,pp.15–39,2013.
[108] J. C. Cox, S. A. Ross, andM. Rubinstein, “Option Pricing: A Simplified Approach,” J. financ.
econ.,vol.7,pp.229–263,1979.
[109] G. Sick, “Chapter 21 Real options,” inHandbooks in Operations Research andManagement
126
Science,vol.9,no.C,1995,pp.631–691.
[110] L.E.Brandão,J.S.Dyer,andW.J.Hahn,“UsingBinomialDecisionTreestoSolveReal-Option
ValuationProblems,”Decis.Anal.,vol.2,no.2,pp.69–88,2005.
[111] P.KodukulaandC.Papudesu,Projectvaluationusingrealoptions:apractioner’sguide.J.Ross
Publishing,2006.
[112] B.Tan,E.Anderson,J.Dyer,andG.Parker,“UsingBinomialDecisionTreesandRealOptions
Theory to Evaluate System Dynamics Models of Risky Projects,” Syst. Dyn. Rev., pp. 1–15,
2009.
[113] S.BenningaandE.Tolkowsky, “RealOptions -An IntroductionandanApplication toR&D
Valuation,”Eng.Econ.,vol.47,no.2,pp.151–168,2002.
[114] W.Bailey,B.Couët,andA.Bhandari,“Unlockingthevalueofrealoptions,”Oilf.Rev.,pp.4–19,
2003.
[115] C.Wernz,I.Gehrke,andD.R.Ball,“Managerialdecision-makinginhospitalswithrealoptions
analysis,”Inf.Syst.E-bus.Manag.,vol.13,no.4,pp.673–691,2015.
[116] O.Yu, “Applicationof realoptionsanalysis to technologyportfolioplanning: a case study,”
Int.J.Qual.Reliab.Manag.,vol.25,pp.52–59,2008.
[117] L. T. Miller and C. S. Park, “Decision Making Under Uncertainty—Real Options to the
Rescue?,”Eng.Econ.,vol.47,no.2,pp.105–150,2002.
[118] A.K.DixitandR.S.Pindyck,Investmentunderuncertainty.PrincetonUniversityPress,1994.
[119] M.Boyer,P.Christoffersen,P.Lasserre,andA.Pavlov,“Valuecreation,RiskManagementand
RealOptions,”2003.
[120] R. Adner andD. A. Levinthal, “What Is Not a Real Option: Considering Boundaries for the
ApplicationofRealOptionstoBusinessStrategy,”Acad.Manag.Rev.,vol.29,no.1,pp.74–85,
2004.
[121] B.Fernandes, J.Cunha,andP.Ferreira, “Theuseof realoptionsapproach inenergysector
investments,”Renew.Sustain.EnergyRev.,vol.15,no.9,pp.4491–4497,2011.
[122] S.M.Ross,“AnElementaryIntroductiontoMathematicalFinance:OptionsandOtherTopics,”
J.Am.Stat.Assoc.,vol.99,no.466,pp.563–563,2004.
[123] T. Driouchi and D. Bennett, “Real options in multinational decision-making: Managerial
awarenessandriskimplications,”J.WorldBus.,vol.46,no.2,pp.205–219,2011.
[124] E.A.Martínez-Ceseñaand J.Mutale, “Applicationofanadvancedrealoptionsapproach for
renewableenergygenerationprojectsplanning,”Renew.Sustain.EnergyRev.,vol.15,no.4,
pp.2087–2094,2011.
127
[125] L.TrigeorgisandJ.J.Reuer,“RealOptionsTheoryinStrategicManagement,”Strateg.Manag.
J.,vol.38,no.1,pp.42–63,2017.
[126] D.M.LanderandG.E.Pinches,“Challengestothepracticalimplementationofmodelingand
valuingrealoptions,”Q.Rev.Econ.Financ.,vol.38,no.3,pp.537–567,1998.
[127] T. Eschenbach, N. Lewis, M. Henrie, E. Baker, and J. C. Hartman, “Real Options and Real
EngineeringProjects,”Eng.Manag.J.,vol.19,no.4,pp.11–19,2007.
[128] A. Kumaraswamy, “A real options perspective of firms’ R&D investments,” New York
University,1996.
[129] T.E.CopelandandV.Antikarov,RealOptions:APractitioner’sGuide.NewYork,2001.
[130] M. Amram and N. Kulatilaka, Real options: Managing strategic investment in an uncertain
world.OxfordUniversityPress,1998.
[131] B.Tan,E.G.Anderson, J. S.Dyer,andG.G.Parker, “Evaluatingsystemdynamicsmodelsof
risky projects using decision trees: alternative energy projects as an illustrative example,”
Syst.Dyn.Rev.,vol.26,no.1,pp.1–17,2010.
[132] A.M.Law,SimulationModeling&Analysis,4thEditio.2007.
[133] J.MunandT.Housel,“APrimeronApplyingMonteCarloSimulation,RealOptionsAnalysis,
KnowledgeValueAdded,Forecasting,andPortfolioOptimization,”2010.
[134] M. Benaroch, “Real Options Models for Proactive Uncertainty-Reducing Mitigations and
ApplicationsinCybersecurityInvestmentDecision-Making,”2017.
[135] R. Shishko, D. H. Ebbeler, and G. Fox, “NASA technology assessment using real options
valuation,”Syst.Eng.,vol.7,no.1,pp.1–12,2004.
[136] R. Shishko and D. H. Ebbeler, “A Real-Options Approach for NASA Strategic Technology
Selection,”1999.
[137] H.ELKUS, “Why InvestorsLike toSeeaPatentBeforeBackinganEntrepreneur,”TheWall
Street Journal, 2013. [Online]. Available: https://blogs.wsj.com/experts/2013/12/03/why-
investors-like-to-see-a-patent-before-backing-an-entrepreneur/.[Accessed:01-Jun-2017].
[138] NASA,“NASAESMPTRLAssessmentWorkbook.”2015.
[139] D. A. Marca, “Augmenting SADTTM To Develop Computer Support for Cooperative Work
DavidA.Marca1,”Architecture,1991.
[140] SDTC, “Risk Profile vs. Stages of TechnologyDevelopment& Commercialization.” [Online].
Available: https://roelfwoldring.files.wordpress.com/2009/07/enterpriserisk.jpg.
[Accessed:01-May-2017].
[141] O.FuerstandU.Geiger,FromConcepttoWallStreet:ACompleteGuidetoEntrepreneurship
128
andVentureCapital,1stEditio.FinancialTimesPrenticeHall,2003.
[142] A.Triantis,“ApplyingaRealOptionsAnalysis,”AccentureAcademy,2013.[Online].Available:
https://www.accentureacademy.com/d/blog/257/?GoBackTo=74F5C9A9#/u/blog/257.
[143] M.Lipsett,“ENGM620:RealOptionsValuations.”2017.
[144] P. Balogh, P. Golea, and V. Inceu, “Profit Forecast Model UsingMonte Carlo Simulation in
Excel,”Rom.Stat.Rev.,vol.61,no.12,pp.33–40,2013.
[145] J. Flatto, “The application of real options to IT valuation process a benchmark study,”
UniversityofNewHaven,1996.
[146] CopperstoneTechnologies, “CopperstoneTechnologiesEngineeringConsultingandRobotic
SystemsforRemoteEnvironmentalMonitoring,”2017.
[147] CopperstoneTechnologies, “CopperstoneTechnologiesRobotic Systems forEnvironmental
Monitoring,”2017.
[148] S. Dwyer, “Agile Design Project Methodology for Small Teams Developing Mechatronic
Systems,”UniversityofAlberta,2017.
[149] W.Baird,“BoronNitrideNanotubes,”2014.
[150] NationalResearchCouncilofCanada,“NRCMakesWorld-FirstDemonstrationofPilot-Scale
Boron Nitride Nanotube Production,” 2014. [Online]. Available:
http://www.marketwired.com/press-release/nrc-makes-world-first-demonstration-of-
pilot-scale-boron-nitride-nanotube-production-1941973.htm.
[151] National Research Council of Canada, “Pure Strength Super Tubes: the strong-and-petite
type.” [Online]. Available: http://www.nrc-
cnrc.gc.ca/eng/about/centennial/100_years/2010_beyond.html/#s86. [Accessed: 11-Aug-
2017].
[152] National Research Council of Canada, “A big opportunity for microscopic fibres,” 2016.
[Online]. Available: http://www.nrc-
cnrc.gc.ca/eng/achievements/highlights/2016/microscopic_fibers.html. [Accessed: 11-Aug-
2017].
[153] J.BusbyandC.Pitts, “Realoptionsandcapital investmentdecisions,”Manag.Account., vol.
75,no.10,pp.38–39,1997.
[154] N.SmallwoodandD.Ulrich,“CapitalizingCapabilities,”HarvardBusinessReview,2004.
Appendix1SupportingInformationforLiteratureReview
1.1DefinitionsofTRLs
Table11.TRLSoftwareDefinitions[Adaptedfrom[31]]
TRLDefinition Description1Basicprinciplesobservedandreported.
Lowestlevelofsoftwaretechnologyreadiness.Anewdomainisbeinginvestigatedbythebasicresearchcommunity.Thislevelextendstothedevelopmentofbasicuse,basic
propertiesofsoftwarearchitecture,mathematicalformulations,andgeneralalgorithms.2Technologyconceptand/orapplicationformulated
Oncebasicprinciplesareobserved,practicalapplicationscanbeinvented.Applicationsarespeculative,andtheremaybenoproofordetailedanalysistosupporttheassumptions.
Examplesarelimitedtoanalyticstudiesusingsyntheticdata.
3Analyticalandexperimentalcriticalfunctionand/or
characteristicproofofconcept.
ActiveR&Disinitiated.Thelevelatwhichscientificfeasibilityisdemonstratedthroughanalyticalandlaboratorystudies.Thislevelextendstothedevelopmentoflimited
functionalityenvironmentstovalidatecriticalpropertiesandanalyticalpredictionsusingnon-integratedsoftwarecomponentsandpartiallyrepresentativedata.
4Moduleand/orsubsystem
validationinalaboratory
environment(i.e.,softwareprototype
developmentenvironment).
Basicsoftwarecomponentsareintegratedtoestablishthattheywillworktogether.Theyarerelativelyprimitivewithregardtoefficiencyandrobustnesscomparedwiththe
eventualsystem.Architecturedevelopmentinitiatedtoincludeinteroperability,reliability,maintainability,extensibility,scalability,andsecurityissues.Emulationwith
current/legacyelementasappropriate.Prototypesdevelopedtodemonstratedifferentaspectsofeventualsystem.
5Moduleand/orsubsystem
validationinarelevant
environment.
Levelatwhichsoftwaretechnologyisreadytostartintegrationwithexistingsystems.Theprototypeimplementationsconformtotargetenvironment/interfaces.Experimentswithrealisticproblems.Simulatedinterfacestoexistingsystems.Systemsoftwarearchitecture
established.Algorithmsrunonaprocessor(s)withcharacteristicsexpectedintheoperationalenvironment
6Moduleand/orsubsystem
validationinarelevantend-to-end
environment
Levelatwhichtheengineeringfeasibilityofasoftwaretechnologyisdemonstrated.Thislevelextendstolaboratoryprototypeimplementationsonfull-scalerealisticproblemsinwhichthesoftwaretechnologyispartiallyintegratedwithexistinghardware/software
systems.
7Systemprototypedemonstrationinanoperational,high-
fidelityenvironment.
Levelatwhichtheprogramfeasibilityofasoftwaretechnologyisdemonstrated.Thislevelextendstooperationalenvironmentprototypeimplementations,wherecriticaltechnical
riskfunctionalityisavailablefordemonstrationandatestinwhichthesoftwaretechnologyiswellintegratedwithoperationalhardware/softwaresystems.
8Actualsystemcompletedandmissionqualifiedthroughtestand
demonstrationinanoperationalenvironment.
Levelatwhichasoftwaretechnologyisfullyintegratedwithoperationalhardwareandsoftwaresystems.Softwaredevelopmentdocumentationiscomplete.Allfunctionality
testedinsimulatedandoperationalscenario
9Actualsystemproventhrough
successfulmission-provenoperational
capabilities.
Levelatwhichasoftwaretechnologyisreadilyrepeatableandreusable.Thesoftwarebasedonthetechnologyisfullyintegratedwithoperationalhardware/softwaresystems.
Allsoftwaredocumentationverified.Successfuloperationalexperience.Sustainingsoftwareengineeringsupportinplace.Actualsystem.
130
Table12.TRLHardwareDefinitions[Adaptedfrom[31]]
TRLDefinition Description1Basicprinciplesobservedandreported.
Scientificknowledgegeneratedunderpinninghardwaretechnologyconcepts/applications.
2Technologyconceptand/orapplicationformulated
Inventionbegins,practicalapplicationisidentifiedbutisspeculative,noexperimentalproofordetailedanalysisisavailabletosupporttheconjecture.
3Analyticalandexperimentalcriticalfunctionand/or
characteristicproofofconcept.
Analyticalstudiesplacethetechnologyinanappropriatecontextandlaboratorydemonstrations,modelingandsimulationvalidateanalyticalprediction.
4Componentand/orbreadboard
validationinlaboratoryenvironment.
Alowfidelitysystem/componentbreadboardisbuiltandoperatedtodemonstratebasicfunctionalityandcriticaltestenvironments,andassociatedperformancepredictionsare
definedrelativetothefinaloperatingenvironment.
5Componentand/orbreadboard
validationinrelevant
environment.
Amediumfidelitysystem/componentbrassboardisbuiltandoperatedtodemonstrateoverallperformanceinasimulatedoperationalenvironmentwithrealisticsupport
elementsthatdemonstratesoverallperformanceincriticalareas.Performancepredictionsaremadeforsubsequentdevelopmentphases.
6System/sub-systemmodelor
prototypedemonstrationinan
operationalenvironment.
Ahighfidelitysystem/componentprototypethatadequatelyaddressesallcriticalscalingissuesisbuiltandoperatedinarelevantenvironmenttodemonstrateoperationsunder
criticalenvironmentalconditions.
7Systemprototypedemonstrationinan
operationalenvironment.
Ahighfidelityengineeringunitthatadequatelyaddressesallcriticalscalingissuesisbuiltandoperatedinarelevantenvironmenttodemonstrateperformanceintheactual
operationalenvironmentandplatform(ground,airborne,orspace).
8Actualsystemcompletedand"flightqualified"throughtestanddemonstration.
Thefinalproductinitsfinalconfigurationissuccessfullydemonstratedthroughtestandanalysisforitsintendedoperationalenvironmentandplatform(ground,airborne,or
space).
9Actualsystemflightproven
throughsuccessfulmissionoperations.
Thefinalproductissuccessfullyoperatedinanactualmission
131
1.2GOARiskAssessment
Table13.GOATRASteps[Adaptedfrom[31]]
Steps BestPractices AssociatedTasks1 Designtheoveralltechnologymaturityassessmentstrategy
fortheprogramorproject.Identifiesallthetechnologymaturityassessmentsfortheoverallprogramstrategythroughouttheacquisition,includingguidanceonreachingagreementwith
stakeholdersonthescopeandschedule
• Thetechnologyneedsofaprogramarewell-understoodandtheassessmentstrategyreflectsthoseneeds.
• Thescheduleandeventsneededtoconductassessmentswasdiscussed,developed,anddocumentedinoneormorestrategydocuments
• Thetechnologymaturityassessmentstrategyisalignedwiththesystemsengineeringplan,acquisitionstrategy,orsimilarplans.
2
DefinetheindividualTRA’spurpose,develop,aTRAplan,andassembletheassessmentteam.
Includesdevelopingaplanforaspecificassessmentofcriticaltechnologiesandcriteriaforselectingtheteamthat
willconducttheTRA,includingagreementssuchasstatementsofindependence
• Acharter,chargememorandumorsimilarinstrumentwasdevelopedtoidentifytheTRA’spurpose,requiredlevelofdetail,overallscope,TRLdefinition,andwhowillreceivetheTRAreportwasdetermined.
• TheexpertiseneededtoconducttheTRAandspecificteammemberswhoareindependentoftheprogramweredetermined
• Theassessmentapproachwasoutlined,includingappropriateTRLcalculators(ifused)
• Anapproachforhowthedataistobedocumentedandinformationreportedwasdefined
• Aplanforhandlinghowdissentingviewswasidentified• PertinentinformationneededtoconducttheTRAwasobtained
3
SelectcriticaltechnologiesIncludesthecriteriaandstepstoidentifyandselectcriticaltechnologiesforevaluation;responsiblepartiesfacilitatingtheselectionofcriticaltechnologiesmayincludethespecificorganizations,people,andsubjectmatterexpertswithkey
knowledge,skills,andexperience
• Theprogram’spurpose,system,andperformancecharacteristicsandsystemconfigurationswereidentifiedinatechnologybaselinedescriptiondocument
• Aworkbreakdownstructure,processflowsheet,orotherdocumentsthatcharacterizetheoverallsystem,subsystems,andelementswereusedtoselectcriticaltechnologies
• Programmaticandtechnicalquestionsandthetechnology’soperationalenvironmentwereusedtodetermineifatechnologywascritical
• Relevantenvironmentforeachcriticaltechnologywasderivedfromtheoperationalenvironment
4 EvaluatecriticaltechnologiesIncludesthecriteria,analyticalmethods,steps,people,and
guidanceusedtofacilitatetheevaluationofcriticaltechnologies;thesourcesanddata,analyses,test
demonstrations,testenvironmentscomparedtoderivedrelevantenvironments,pilots,simulations,andotherevidenceusedtoevaluatethematurityandreadinessofcriticaltechnologies;theagreementoftheprogram
manager,technologydeveloper,andTRAleadonwhatconstitutesaspecifiedTRLlevel,goal,orobjective
• TRLs,oranothermeasurewereusedasacommonmeasureofmaturity• ConsistentTRLdefinitionsandevidenceneededtoachievethedesignatedcategory
orTRLweredeterminedbeforetheassessment• Theassessmentclearlydefinedinclusionsandexclusions;theassessmentteam
determinedwhetherthetestarticlesandenvironmentswereacceptable• Theassessmentteaminterviewedtestingofficialstodeterminewhetherthetest
resultsweresufficientandacceptable• Theassessmentteamdocumentedallpertinentinformationrelatedtotheir
analysis
5 Prepare,coordinateandsubmitTRAreportIncludestheelementstobeincludedintheTRAreportandhowthereportisdeveloped,submittedforinitialandfinalreview,andcommunicated;alsoincludeshowdissentingviewsareaddressed,documented,andreportedandwhois
involved
• AnofficialTRAreportwaspreparedthatdocumentedactionstakeninsteps1-4above
• OfficialcommentsontheTRAreportwereobtainedanddissentingviewswereexplained
• IftheTRAwasconductedbythetechnologydeveloperorprogrammanagerfortheirowninternalusewhereanofficialreportisnotrequired,itshouldbedocumentedforfuturereferenceanduse.ThismayincludeaTRAself-assessmentconductedduringearlydevelopmentandlaterusedasareferencesourcetoascertaininitialrisks.
6 UsingTRAresultsanddevelopingaTechnologyMaturationPlan
Describeshowtechnologydevelopers,programmanagers,andgovernancebodiesusetheTRAresultstomake
informeddecisionsandhowpotentialrisksandconcernsareidentifiedandtheuseofsuchinformationinother
• TRAresultswereusedtomakedecisionsabouttheprogram’sdevelopmentpriorities
• ProgrammanagementidentifiedrisksandconcernsrelatedtotheTRAwereprovidedasinputstorisk,cost,andplanningefforts
• Atechnologymaturationplanwasdevelopedtotrackprogresstowardhighertechnologymaturitylevelsfortroubledorselectedtechnologies
134
Appendix2CaseStudySupportingInformation
TheinformationinA2.1toA2.5iscourtesyofCopperstoneTechnologies[146][147].
2.1CopperstoneTeamBios
Co-founderandPresident
JamieYuen,M.Sc.,EIT,MechanicalEngineering
Jamie’s focus is onmechanical, electronics hardware, software design and testing. His role is to
managetheday-to-dayproductionoperations,contractsandprocurement.
Co-founderandDirector
NicolasOlmedo,B.Sc.,EIT,MechanicalEngineering
Nicolasworks on the design anddevelopment of robotic system includingmechanical, electrical,
andsoftwarecomponents.Heisalsoinvolvedinbusinessdevelopmentactivities.
Co-founderandDirector
StephenDwyer,B.Sc.,EIT,MechanicalEngineering
Stephen is the primary firmware developer, implements hardware and firmware embedded
systemsdesignanddevelopment,aswellasmechanicalandroboticdesignanddevelopment.
CTOandAdvisor
MichaelLipsett,Ph.D.,P.Eng,MechanicalEngineering
Michael is a Professor in theEngineeringManagementGroup at theUniversity ofAlbertawith a
Ph.D. from Queen’s University on Robot Looseness Fault Detection. He has been a Research
Engineer forAtomicEnergyofCanadaLimited,developedremoteandrobotic tooling,performed
labandfieldprototypeevaluation,andhasexperienceinprojectmanagement.Michaelhasbeena
Research Associate at Syncrude and supervised the development of BMI remote monitoring
technologyandallCopperstoneinitiatives.
135
BusinessDevelopment,Marketing,Sales&Strategy
SarahPrendergast,MBA(Finance)
Sarah is the most recent addition to CST. She joined in April 2017 after completing an MBA in
FinancefromtheUniversityofAlberta.ShehasaBScinBiologicalSciencesfromtheUniversityof
Alberta and brings a wide-range of expertise to the team. She is currently working on business
strategy and positioning for CST, aswell as operating organization and themarketing, sales and
businessstrategy.
Pre-VMSProgram
Currently have twobusinessmentorswith a combinationof industrybackground and successful
entrepreneurialexperience.Theirnameshavenotbeenshared.
Positionsthathavenotbeenfilled:
________–AdvisoryBoard
PhDinFinance–Corporatefinance,venturecapitalandinnovation
_______–AdvisoryBoard
MBAFinance,JD
LegalCouncil(______,_______&________)
________–AdvisoryBoard
ComputerEngineeringTechnologyandProfessionalManagement
SoftwareDeveloper(_____,_____&_______)
SuccessfulEntrepreneur(_________)
136
2.2ActivitiesTimeline
The strategy that Copperstone is deploying focuses on commercialization of the AR1 and
developmentof theAR2. Inorder toprovide the services requiredby thepilotprojects through
BGC and ArcelorMittal for the summer of 2017, Copperstone is focused on production of seed
broadcasting and seedling planting technology to be included in the AR, as well as improved
mobility through centrifuged tailings and frozen sand. The field project for ArcelorMittal for the
spring of 2018 will include development of an autonomous system for the rover and mudline
mapping technologies. These product development improvements allow Copperstone to deliver
theservicesthatoilsandsandminingcompanieshaverequestedandhaveindicatedarehighlyin
demand.ThedevelopmentofthesetechnologiesarethekeymilestonesforCopperstone’sin2017
and will provide an excellent entry into the primary market of end users by catering to their
productneeds.
In addition to production technology milestones, Copperstone has expected purchase order
expectationsafterthecompletionofthepilotandfieldprojectsforBGCandArcelorMittal.Afterthe
successful utilization of the AR technology in the field, Copperstone expects to sign at least two
larger tailings monitoring or reclamation contracts by the end of 2017. Coinciding with this
increaseofmarketshare,Copperstoneexpectstobeginhiringoutsideofthecorefoundingteamin
order to bring in expertise in sales and marketing, business development, and overall business
strategy.
2015-2016Initial technology development and field demonstrations
2017Launch of AR1 first contracts beyond technology demonstrations
2018Commercial launch of ARwith mud-line monitoring system & data analytics capabilities
137
2.3CopperstoneTechnologiesFinancials
2.4SampleofInterviewQuestionswithBusinessDevelopmentManager
SampleofInterviewQuestions
• ChallengesCSTisfacingrightnow?
• Updatesonthemarketstudy
• Wherewouldyoubeginensuringyouhaveacompletecompetentteam?
• WhatiscurrentdetailorplanaroundIP?Needmoreinformationtoregardingtheupdates
• WhendidCopperstoneapply/getthepatentandwhatdoesitcover?
• CurrentdistributionchannelplanforCopperstone?Whatweresomeofthechallengesfaced?
• DoyouuseTRLs?Ifnot,wouldyouusethem?
• AtwhatTRLdidCSTstartatin2014?
• HowwasthepricefortheAR1reached?i.e.then150/hour,5000/day
• HowmanyAR1’sdoesCSTcurrentlyhave?(forrentalpurposes)
• CurrentR&Dactivities:Copperstone’scurrentplan/targetforthenexttwoyears?
• Stakeholderanalysis:whoarethecurrentstakeholders?Wasthereaformalanalysisdone?
• Arethereanyestablishedrelationshipswithsuppliers?
• WhatisCST’scurrentpositioninthemarket?Aretheyplanningonre-positioningwithinthe
nexttwoyears?
• Anyformalriskassessmentconducted?Ifso,whatstageswasthisdone?
• Areyouhiringnewtalent?
• Whatdoesthefinancingcurrentlylooklike?Areyoulookingfornewinvestors?
• Whatisthebiggestperceivedatthispointduringtheproject?
• Whataretheplansfortheconsultingsideofthebusiness?
• ArethereplanstooperateoutsideNorthernAlberta?
138
• Whatcommercializationoptionswillyoubeconsideringandwhichoneareyoulearning
towards?
• WhatisyourrelationshipwithyourcompetitorConeTec?
• WherearetheARscurrentlybeingbuilt?Howlongdoesittaketobuildone?Howmuchdoesit
cost?
• Whatwerethebiggestdifficultiesduringtechnologydevelopment(besidesneedforcapital)?
• Doyouhaveanycontractssignedwithcustomers?
• Howdidyouconductamarketstudyanalysis?
2.5SummaryfromRelevantThesisDocument
Thefollowingisasummaryfrom[148].
Ingeneral:
• CSTdidnot communicatewith the clientas frequentlyas they shouldhave,whichmeant the
feedback was limited- this resulted in wasted efforts developing and configuring design.
parametersontheuserend,thatwerelaterdeemedunnecessarybytheclient.
• Therewasalackofscheduledprogressreportsandupdates.
• Work-breakdown was a challenge for the team. This affected team collaboration and
understandingofrequirementswhichleadtoimproperprioritization.
• Lackoftechnicalandmanagerialplanningleadtooverrunsonbudgetandschedules.
• Lackofstructuredscopeorvision.
• Lack of proper documentation throughout the project. Initial scope documents were not
updatedtoreflectchanges.Thiswassimilartodesigndocumentationaswell,whichleadtoan
increaseindifficultyinreviewingchangeswhichledtoamoretimedemandingprocesses.
• Technicalsuccessfeedbackwaslimited.
• Disciplineofprogresstrackingwasanissue.
• Duetolimitedtimeandimpropertimemanagementprototypeswerenotproperlytested.
• Issueswithinsufficientinfrastructureandcapabilitiestotestandaddresstechnicalproblems.
139
SpecifictotheARTechnologyProject:
• High uncertainty in requirements and technical risks because team did not properly collect
informationregardingoperatingenvironmentandconditions,whichaffectedtheteam’soverall
understanding and affect field testing. There was a lack in taking appropriate action when
dealingwithareasofhighuncertainty(thisoverallreducedthetechnicalsuccessoffieldtrials
conducted)–resultinginfieldtestingbeingpushedtoa latertimei.e.scheduleoverrunsand
additionalcosts.
• Limited to non-existent effort into conducting a formal market analysis. There was no
verificationofmarketconditionsthatwereprojectedatthestartof theproject.entiremarket
analysisconsistedofqualitativedatathatrepresentaverysmalldatasetfromlimitedsources.
• Theteamwassuccessfulincompletingtheprototypewithinashorttimeframe.CST’steamdid
betterthanpreviousworktheyhaddoneinthepast,soprototypingthisroundwasimprovedin
design,assemblyandmaintenanceareas.
• CSTwasabletoactquicklytomakechangessuggestedbasedonclientfeedback.Thiswasalso
duetothefactthatthedesignwasmorerobustandplannedbetterforchanges.
• ThisprototypewasdevelopedwithMOREstakeholderfeedbackandwastestedintheoperating
environment.
• Logicsandtransportationofrovertooperationalenvironmentwaschallenging.
• Highambiguityabouttheoperatingconditions,andtheteamshouldhaveidentifiedtheneedto
addresstheseareasofhighuncertaintyandshouldhavetakenappropriateaction.Failuretodo
soresultedinreductionintechnicalsuccessduringfieldtesting.
• Therewasnoformalriskmanagementcompletedduringdesign.However,safeworkpractices
andriskmitigationwereusedduringfabricationandtesting.
• High uncertainties due to limited understanding of the market and required technical
specifications (most of the specifications were obtained through discussion with a potential
customer).
• Theprojectfundingcameinternallyfromwithintheorganization.Fundswerelimited.
• Thebusinessmodelof thetechnologywasnotdecidedonat thestartof theprojecthowever,
the models considered for commercialization included sales of AR units, rental, leasing to
operators,orprovidingmeasurementdataservices.
• ThepotentialprimarymarketforARsisinenvironmentalmonitoring,withonlyasmallnumber
ofpotentialclients,butlargeamountofpotentialapplication(land).
• CSTcollaboratedwithapotentialclientduringtestinganddemonstrationactivities.
140
• CST spent time reviewing and completing design and development o of the mechanical
subsystemcomponents.Thesewereexpensiveactivitiesanddiligencewas importantascosts
tomakechangeslaterwouldbehigh.
Appendix3RealOptionsAnalysisSupportingCalculations
3.1Base-CaseDCF
144
3.3BuildingCapabilitiesOptionValuation
SummaryofAnalysis
Variable Downside Upside Range Downside Upside BaseCaseAR1Expenses $2,890,319 $3,229,878 $339,559 $2,632,370 $1,754,914 $2,193,642
MARR $2,573,586 $3,651,133 $1,077,547 23% 17% 20%GrossRevenue $2,253,772 $3,866,425 $1,612,653 $10,595,200 $15,892,800 $13,244,000
CFYear0 ($135,318) ($106,488) $28,829 ($135,318) ($106,488) ($120,903)CFYear1 ($40,136) ($24,888) $15,248 ($46,157) ($28,621) ($37,389)CFYear2 ($31,806) ($27,850) $3,956 ($42,063) ($36,831) ($39,447)CFYear3 ($40,990) $1,100 $42,090 ($62,341) $1,673 ($30,334)CFYear4 $104,258 $219,999 $115,741 $216,189 $456,189 $336,189CFYear5 $422,854 $704,168 $281,314 $1,052,195 $1,752,195 $1,402,195CFYear6 $588,537 $956,924 $368,388 $1,757,361 $2,857,361 $2,307,361CFYear7 $631,206 $1,021,920 $390,714 $2,261,724 $3,661,724 $2,961,724CFYear8 $729,043 $1,147,665 $418,622 $3,134,750 $4,934,750 $4,034,750
ExpectedNPV Input