Seediscussions,stats,andauthorprofilesforthispublicationat:https://www.researchgate.net/publication/311825391
Towardarationalandmechanisticaccountofmentaleffort
ArticleinAnnualReviewofNeuroscience·January2017
DOI:10.1146/annurev-neuro-072116-031526
CITATION
1
READS
829
7authors,including:
Someoftheauthorsofthispublicationarealsoworkingontheserelatedprojects:
DevelopingtoolsandtheoriesforhelpingpeoplemakebetterdecisionsViewproject
Resource-rationalityViewproject
AmitaiShenhav
BrownUniversity
26PUBLICATIONS742CITATIONS
SEEPROFILE
SebastianMusslick
PrincetonUniversity
6PUBLICATIONS2CITATIONS
SEEPROFILE
FalkLieder
UniversityofCalifornia,Berkeley
26PUBLICATIONS148CITATIONS
SEEPROFILE
MatthewMBotvinick
PrincetonUniversity
64PUBLICATIONS17,812CITATIONS
SEEPROFILE
AllcontentfollowingthispagewasuploadedbyFalkLiederon22December2016.
Theuserhasrequestedenhancementofthedownloadedfile.Allin-textreferencesunderlinedinblueareaddedtotheoriginaldocument
andarelinkedtopublicationsonResearchGate,lettingyouaccessandreadthemimmediately.
1
Annu.Rev.Neurosci.2017.40:X--X
doi:10.1146/annurev-neuro-072116-031526
Copyright©2017byAnnualReviews.Allrightsreserved
SHENHAVETAL.
MECHANISMSANDRATIONALITYOFMENTALEFFORT
TOWARDARATIONALANDMECHANISTICACCOUNTOFMENTALEFFORT
AmitaiShenhav,1,2SebastianMusslick,3FalkLieder,4WouterKool,5ThomasL.Griffiths,6JonathanD.Cohen,3,7andMatthewM.Botvinick8,91DepartmentofCognitive,Linguistic&PsychologicalSciences,BrownUniversity,Providence,RhodeIsland02912;email:[email protected],BrownUniversity,Providence,RhodeIsland029123PrincetonNeuroscienceInstitute,PrincetonUniversity,Princeton,NewJersey085444HelenWillsNeuroscienceInstitute,UniversityofCalifornia,Berkeley,California947205DepartmentofPsychology,HarvardUniversity,Cambridge,Massachusetts021386DepartmentofPsychology,UniversityofCalifornia,Berkeley,California947207DepartmentofPsychology,PrincetonUniversity,Princeton,NewJersey085408GoogleDeepMind,LondonM1c4AG,UnitedKingdom9GatsbyComputationalNeuroscienceUnit,UniversityCollegeLondon,LondonW1T 4JG,UnitedKingdom■ Abstract In spite of its familiar phenomenology, the mechanistic basis for mental effort remains poorlyunderstood. Althoughmost researchers agree thatmental effort is aversive and stems from limitations in ourcapacitytoexercisecognitivecontrol, it isunclearwhatgivesrisetothose limitationsandwhytheyresult inanexperienceof controlas costly.Thepresenceof thesecontrol costsalso raises furtherquestions regardinghowbesttoallocatementalefforttominimizethosecostsandmaximizetheattendantbenefits.Thisreviewexploresrecent advances in computationalmodeling and empirical research aimed at addressing these questions at thelevelofpsychologicalprocessandneuralmechanism,examiningboththelimitationstomentaleffortexertionandhowwemanagethoselimitedcognitiveresources.Weconcludebyidentifyingremainingchallengesfortheoreticalaccountsofmentaleffortaswellaspossibleapplicationsoftheavailablefindingstounderstandingthecausesofandpotentialsolutionsforapparentfailurestoexertthementaleffortrequiredofus.
Keywordsmotivation,cognitivecontrol,decisionmaking,reward,prefrontalcortex,executivefunction
2
1.INTRODUCTION
Allhighlyabstractconceptions,unaccustomedreasons,andmotivesforeigntotheinstinctivehistoryoftherace…prevail,whentheyeverdoprevail,witheffort;andthenormal…sphereofeffortisthusfoundwherevernon-instinctivemotivestobehavioraretoruletheday.(James1891,p.536)
Cognitiveeffortisamongthemostfamiliarandintuitivefixturesofmentallife.Differenttaskstransparentlydemanddifferentlevelsofcognitiveexertion,withsuccessorfailuredependingonhowhardwetry.Insomecases,difficultiespromptustoapplyourselvesmoreintently.Inothers,wedisengage,judgingthedemandedeffortnottobeworthit,orperhapsexperiencingourselvestobedepletedorfatigued(Botvinick&Braver2015,Hockey2011,Kurzbanetal.2013,Westbrook&Braver2015).
Giventhisseeminglyimmediateavailabilitytointrospection,mentaleffortissurprisinglydifficulttopindownasanobjectofscientificstudy.Whatexactlyismentaleffort,fromanobjectiveratherthanintrospectivepointofview?Whatexactlyisgoingonwhenwetryharderonacognitivetaskordecidethatthistryingisnotworthit?Whatisbeingconservedwhenweconserveourcognitiveresources,andhowdowedecidethemannerinwhichthoseresourcesgetallocated?Andhowcanweidentifytheneuralmechanismsunderlyingsuchasubjectiveconstruct?Ouraiminthepresentarticleistoreviewsomeareasofrecentprogressinaddressingthesequestions.
Toconvertmentaleffortintoanapproachableobjectofscientificstudy,ausefulfirststepistooperationalizeitnotinpurelysubjectiveconativetermsbutinsteadintermsofinformationprocessing.Drawingonpreviouswork(Bonner&Sprinkle2002,Camerer&Hogarth1999,Hockey1997,Kahneman1973),weadoptthefollowingworkingdefinition:Effortiswhatmediatesbetween(a)thecharacteristicsofatargettaskandthesubject’savailableinformation-processingcapacityand(b)thefidelityoftheinformation-processingoperationsactuallyperformed,asreflectedintaskperformance.Thefirsttwofactors,taskcharacteristicsandcapacity,determinewhatlevelofperformanceisattainableinprinciple.Effortreferstothesetofinterveningprocessesthatdeterminewhatlevelofperformancewillinfactberealized;thequalityofthisperformanceisquantifiedthroughsuchmeasuresasresponselatencyandaccuracy.Drawingonthefamiliaranalogybetweenmentalandphysicaleffort,wecansaythattaskcharacteristicsandinformation-processingcapacityareanalogoustotheweightofanobjectandthephysicalstrengthofapersontryingtoliftit,andthattaskperformanceisanalogoustotheswiftnessofthelift.Effort,then,isthethingthatmediatesbetweenweightandstrength,ontheonehand,andtheactualliftoutcomeontheother.
3
Inthecaseofmentaleffort,theheavyliftaccomplishedthrougheffortcantakeavarietyofforms,manyofwhichwediscussfurtherbelow:theoverridingofdefaultactionsorhabits(Miller&Cohen2001),engagementincomplicatedmentalfeatssuchasreasoning(Kahneman2003),workingmemorymaintenance(Braver2012),andswitchingbetweentaskswithdifferentkindsofdemands(Monsell2003).Andthecumulativeeffectofsuchliftingcandetermineimportantlifeoutcomes,includingacademicsuccess,socialcompetence,andabilitytocopewithenvironmentalstressors(Caseyetal.2011,Duckworthetal.2007,Mischeletal.1989,Tangneyetal.2004).
However,thesedownstreameffectsdonotimmediatelytellushoweffortdoesitswork.Whatexactlyisthenatureofthismediator?Ifphysicaleffortregulatestheengagementofmuscles,whatisitthatcognitiveeffortisregulating?
Aplausibleanswertothisquestionhasemergedthroughseveraldecadesofresearchonperformanceincognitivetasks.Thisworkhasshownthatinformationprocessingfallsalongacontinuumofautomaticity(Shiffrin&Schneider1977),withsomeprocesses(typicallyheavilypracticedones)abletobedeployedmorereflexivelyandwithlessthreatofinterferencefromotherongoingthoughts.Processesontheotherendofthiscontinuumaresaidtorequireincreasingcommitmentsofcognitivecontroltoreconfigureinformationprocessingawayfromdefault(i.e.,moreautomatic)settings(Botvinick&Cohen2015,Cohenetal.1990).Thenotionofeffortwasinfactcentraltotheearliestcharacterizationsofautomaticprocessingandcontrol-dependentprocessing,withtheformerdescribedaseasyandeffortless,andthelatteraseffortful.Thus,cognitivecontrolmaybeviewedastheforcethroughwhichcognitiveeffortisexerted.
Althoughthispointhelpstofirmupadefinitionforcognitiveeffort,itleavesopenonemorecriticalissue.Ifcognitiveeffortregulatesthedegreetowhichcognitivecontrolisengaged,howisthetargetlevelofcontrolchosen?Thenotionofeffortimpliesadecisionproblem:Howmuch(andwhatform)ofcontrolshouldbeallocated,givencurrentcircumstances?
Thissetofquestionsprovidesthecentralfocusofourpresentreview.Inparticular,weexploreresearchaimedatunderstandingmentaleffortasadomainofdecisionmaking,focusinginparticularonrecentapproachesthatidentifycognitiveeffortastheoutputofreward-basedchoice.Accordingtosuchapproaches,individualsweighthebenefitsofcognitivecontrolagainstsomeinherentcost,thenatureofwhichwediscussnext.Thecoreofthisreviewfocusesoncomputationalcognitiveapproachestounderstandingthiscost-benefitanalysis,withtheaimofprovidingaframeworkforinvestigatingassociatedphenomenologyandunderlyingneuralsubstrates.Weconcludebysummarizingthosesubstratesandconsideringhowthesehelpexplainthesubjectiveexperienceofeffortandhowthiscanbequantified.
4
2.WHYISCONTROLCOSTLY?
Ourworkingdefinitionofmentaleffortexposesanintriguingriddle:Whyshouldtherebeanymediatingfactorbetweencognitivecapacity,ontheonehand,andperformanceontheother?Why,inotherwords,don’tpeoplealwayssimplyperformatthehighestlevelofwhichtheyarecapable?Theintuitiveanswersuggestedbyintrospectionisthatweareconstitutivelyreluctanttomobilizeallavailablecognitiveresources.Thatis,mentaleffortisinherentlyaversiveorcostly.Inadditiontoaccountingforitsphenomenology,theideathatcontroliscostlyhelpstoexplainwhyincentivesarefoundregularlytoimprovecognitiveperformance,suggestingthatindividualscanincreasetheircontrolallocationwhenhigherincentivesareonoffer(i.e.,theyarenotconstrainedbyability)butholdbackfromdoingso,owingtotheaversivenessoftheeffortrequired(Botvinick&Braver2015).Forexample,participantsrespondfasterandmoreaccuratelywhenexpectinggreaterrewardfornamingthecolorofaStroopstimulus(e.g.,whenthewordGREENissetinredtype)(Krebsetal.2010).Similareffectshavebeenfoundwithtaskdemandsincludingselectiveattention(Engelmannetal.2009,Padmala&Pessoa2011)andtaskswitching(Aartsetal.2010,Umemoto&Holroyd2014).Evenperformanceonintelligencetests,traditionallyassumedtobeoneofthepurestmeasuresofcognitiveability,isaffectedbyincentivelevels(Duckworthetal.2011).
Additionalevidenceforthepresenceofcognitiveeffortcostscomesfromworkonthedemand-selectiontask(DST).IntheDST,participantsfacearecurringchoicebetweentwooptions,eachassociatedwithdifferentlevelsofdemandforcognitiveeffort(e.g.,higherversuslowerfrequenciesoftaskswitching).Thekeyfindingfromthissetoftasksisthatparticipantsgenerallypreferthecourseofactionassociatedwiththefewestcognitiveeffortdemands(Dunnetal.2016,Kooletal.2010,McGuire&Botvinick2010).Thisisconsistentwithfindingsthatparticipantsdemandgreaterrewardstoengageintasksthatdemandincreasinginhibitorycontrol(Dixon&Christoff2012)orworkingmemorymaintenance(Westbrooketal.2013).Inotherwords,cognitiveeffortisexperiencedascarryingdisutility(i.e.,assomethingtobediscountedfromtheexpectedreward),anobservationthathasbeenfurthersubstantiatedbyfindingsthatcognitiveeffortevokesnegativeemotions(Dreisbach&Fischer2015,Inzlichtetal.2015,Spuntetal.2012),negativelybiaseslearningofstimulus-rewardassociations(Cavanaghetal.2014),anddiscountsneuralresponsestotherewardpresentedafterapersoncompletesaneffortfultask(Botvinicketal.2009a).
Theseandotherfindingssubstantiatetheintuitionthatcontrolisregisteredascostlyandlaythegroundworkfortreatingcontrolcostsasacentralexplanatoryvariableintheoriesofcontrolallocation.However,beforeexaminingthesetheoriesingreaterdepth,itisworthfirstconsideringwhyafunctionsoseeminglyimportantascognitivecontrolmightbeencodedascostlyatall.
Therearetwobroadcategoriesofexplanationthathavebeenofferedinresponsetothisquestion,thatareinfactcloselyrelatedtooneanother:intrinsiccosts,andopportunitycosts.Thefirstsuggeststhattheallocationofcontrolitselfcarriesacost,and
5
thusthereisalimitonhowmuchcontrolcanbeallocatedatonetime.Onthisaccount,theinvestmentofcontrolmayregisterasmentaleffortinordertoindexthatcost,andensureaproperevaluationoftheworthofthatinvestment.Thesecondtypeofexplanationfollowsnaturallyfromthefirst:giventhatthecapacityforcontrol-dependentprocessingislimitedbyitscost,thenallocatingcontroltoonesetofprocessesmeansforgoingpursuitofothersthatmayalsohavevalue.Onthisaccount,thedurationofacontrol-demandingprocessposesanopportunitycost(Kurzbanetal.2013),thatmayalsoregisterasmentaleffort.Inthesectionsthatfollow,weconsiderthesetwotypesofexplanation.
2.1.IntrinsicCostsofControl.Thenotionoflimitedcapacityofcontrolwasadefiningfeatureintheearliestconceptualizationsofcontrolledprocessing(e.g.,Posner&Snyder1975,Shiffrin&Schneider1977).However,itbegsafundamentalquestionthatcontinuestovexresearchinthisarea:Whyisthecapacityforcontrolsolimited?Prominenttheoreticalaccountsofferthreepossibleexplanations:limitedmetabolicresourcesinthebrain,constraintsonthecapacitytomaintaintask-relevantinformation,and/orinterferencethatarisesfromtheuseofsharedrepresentationsformultiplepurposes(Figure1).Ineachofthesecases,thecostofcontrolplaysaroleinprotectingalimitedresource.Whatdistinguishesthetheoriesisthenatureoftheprotectedresourceitself.
Figure1.Schematicsummaryofpossiblecontrollimitations.Differentaccountsofthepotentialsourcesofcontrolcostsareshownforanexamplecaseofadrivertryingtoattendmultiplestreamsofinformation.Resource-basedaccounts(green)proposethatcontrolcostsreflectthelimitationsofacentralmetabolicresourcethatdepleteswithextendeduseofcognitivecontrol.Controlcapacity--basedaccounts(blue)proposethatcontrolcostsreflectanupperboundonthe
Constraintsduetointerferenceintask
processingpathways
Constraintsduetolimitedcapacityofcontrolsystem
Constraintsduetometabolicresourcesdepletedbycontrolledprocessing
Glucose,glycogen
Taskperformance
Localprocessing
Cogni/vecontrol
Environmentals8muli
6
controlsignalsthatcanbedeployedorcontrol-relevantinformationthatcanbestoredatanygiventime.Interference-basedaccounts(red)proposethatcontrolcostsreflecttheriskstoperformanceassociatedwiththeover-allocationofcontrol.Theserisksresultfromcrosstalkthatcanarisefromoverlapamongthepathwaysrequiredtoprocesstask-relevantstimuli,crosstalkthatitistheverypurposeofcontroltoavoid.Assuggestedbythevisual,theseaccountsarenotmutuallyexclusiveofoneanother.
2.1.1.MetabolicConstraints
Byanalogytotheexertionofphysicaleffort,cognitiveefforthasbeenlinkedhypotheticallytoalimitedphysiologicalresourcethatdepleteswithuse,muchlikeamuscledepletesenergy(oracquirestoxicbyproducts)asittranslatescontractionintoforce(Baumeister&Heatherton1996,Baumeisteretal.1998,Muravenetal.1998).Accordingtothisidea,exertionofcognitiveeffortislimitedbyaresourcethatdepletesinproportiontotheamountanddurationofexertionandthereforeencouragesindividualstojudiciouslyallocatetheiravailablereserves(Figure1,green).Forinstance,anindividualthatengagesinacontrol-demandingtaskoveranextendedperiodoftimewouldfindherselfimpairedatasubsequenttaskthatrequirescontrolorself-regulation(e.g.,choosingahealthymeal).Aspredictedbythisresource-basedaccount,initialevidencesuggestedthatexperimentalparticipantsindeedexertlesscognitiveeffortontasksthatfollowsomeamountofeffortexertion(i.e.,post-depletion)relativetotasksthatfollowminimaleffortexertion(Haggeretal.2010).
However,thisaccountofcontrolcostsraisestwoquestionsthatremainunderdebate.First,whatistheresourcebeingdepleted?Researchershypothesizedinitiallythatitmaybebloodglucose,showingforinstancethatparticipantswerelessdepletedwhenadministeredasweeteneddrink(Gailliotetal.2007).However,subsequentexperimentsprovidedstrongevidenceagainstthishypothesis(reviewedinKurzbanetal.2013;see,e.g.,Moldenetal.2012,Vadilloetal.2016)andsuggestedthat,totheextentglucoseimprovescontrol,itdoessothroughanincreaseinmotivationtoperformthesubsequenttaskratherthanareplenishmentofaphysiologicalresource(Hockey2011,Inzlicht&Schmeichel2012).Moreover,thebrain’sglucoseutilizationisheavilyweightedtowardprocessesthatdonotdepleteinthismanner(e.g.,vision)andthemarginalincreaseinconsumptionforcontrol-demandingtasks–atleastsomeofwhichseemfarlesscomputationallydemanding(suchastwo-digitarithmeticvs.recognizingaface)—isestimatedtoberelativelysmall(Kurzbanetal.2013).
Somehavesuggestedthattheseconcernsaboutglucosecanbeavoidedbyalternateresourcemobilizationaccountsthataremoresensitivetocontrolledprocessing,focusingforinstanceonanindividual’seffortstomaximizeastrocyticglycogen(astoredformofglucose)(Christie&Schrater2015)ortominimizebuildupoftheneurotoxinamyloid-βintheinterstitialfluid(Holroyd2015),buttheseproposalshaveyettobetestedempirically.Alltheseaccountsmustalsoaddressasecondopenquestionpertainingtothetimescaleoverwhichputative
7
resourcesdeplete.Classicfindingshavesuggesteddepletionofcognitivecontroloccursoverrelativelyshortperiodsoftime(i.e.,followinglessthananhourofmentaleffortexertion).However,recentmeta-analysesandreplicationattemptshavecalledsuchfindingsintoquestion(Carteretal.2015,Haggeretal.2016).Collectively,thesesuggestthat,totheextentcontroldepletesorfatigues,itmaydosoonlyoverlongertimescales(Blainetal.2016).Wereturntopotentialsourcesoftheselongertimescalecontrolcostslater(e.g.,boredom),butfornowsimplyreiteratethattheresourcemobilizationaccountssuggestthatthesecostsareoperativeatshortertimescalesaswell(e.g.,individualtrialsofatask).
2.1.2.StructuralCapacity:LimitationsonStorageandMaintenance
Asecondlineofresource-basedcontrolcostaccountssuggeststhatcontrolcostsmayarisefromcomputationallimitationsinthecapacityforcontrolledprocessing,ratherthanthedepletionofanykindofmetabolicresource.
Thetraditional,andstilldominant,accountexplainsthisbymakingtwokeyassumptions:(a)Controlreliesonacentralizedmechanism,and(b)assuggestedabove,thecapacityofthissystemislimited.Theseassumptionsaretypicallyjustifiedbyarguingthatcontrolisdependentonworkingmemorytorepresentthecontextinformation(e.g.,instructions,intentions,taskconditions,goals)usedbythecontrolsystemtoguidebehavior(e.g.,Anderson1983,Cohenetal.1990).This,inturn,linkstheconstraintsoncontroltothewell-knownlimitationsofworkingmemorycapacity(e.g.,Cowan2012,Luck&Vogel1997,Miller1956)(Figure1,blue).Thus,thecapacitylimitationsofcognitivecontrolcanbetracedtothefactorsthatlimitworkingmemorycapacity,ofwhichseveralhavebeenproposed:aresourcelimitationinactivelymaintainedworkingmemoryrepresentations,intermsofdiscreteslots(Cowanetal.2012,Luck&Vogel1997)orcontinuousresources(Maetal.2014);interferencebetweentherepresentationsheldinworkingmemory(Nairne1990,Oberauer&Kliegl2006);and/orpassivedecay(Jensen1988,Page&Norris1998)(foracomparativereviewoftheseaccounts,seeOberaueretal.2016).
However,theseexplanationshaveyettobejustifiedadequatelyintermsoftheunderlyingmechanisms(e.g.,theneuralmechanismsinvolved)and,wherethishasbeenattempted(e.g.,Elmoreetal.2011,Maetal.2014),thefocushasbeenonsimpleformsofshort-termmemory(e.g.,visualmemory)andnotonsystemsmoredirectlyinvolvedincognitivecontrol.Moregenerally,itseemsoddtoimaginethatthecontrolsystem---onesocriticaltoadaptivebehaviorandwithaccesstosuchvastresources(therearebillionsofneuronsinthehumanprefrontalcortexalone)---wouldbesubjecttosuchastultifyinglimitation:theinability,inmanyinstances,tocarryoutmorethanasinglecontrol-dependenttaskatatime.Evolutionanddevelopmentwouldhavetoberatherpoorengineersindeedtoarriveatthissolution,ifstructuralresourcesavailabletothecontrolmechanism(s)weretheonlyconsiderationinvolved.(Foranormativeconsiderationoftheconstraintsonworkingmemoryitself,seeElman1993,Toddetal.2009.)
8
2.1.3.RepresentationalCapacity:LimitationsArisingfromInformation-ProcessingPathways
Analternativetoanaccountattributingcapacitylimitstothecontrolsystemitselfwassuggestedbyearlyattentiontheoristsandreferredtoasthemultipleresourceshypothesis(Navon&Gopher1979;seealsoAllport1980,Allportetal.1972,Logan1985,Wickens1984).Thisproposedthatrestrictionsincontrol-dependentbehaviorreflectcrosstalkarisingfromlocalbottlenecksinprocessing,whendifferenttaskscompetetousethesamesetofrepresentationsorapparatusfordifferentpurposes.Asatrivialexample,evenwithlimitlesscapabilityforcontrol,itwouldbeimpossibletosaythewordsgreenandredatthesametimebecausewehaveonlyonesetofvocalchords,onemouth.However,theconstraintsneedn’tariseonlyineffectorsystems(e.g.,theremaybeonlyasinglephonologicalsystemthatdrivesmultitaskingconstraintsonspeech),andtheconstraintsontheseeffectorsclearlycan’texplainmorecommonexamplesofmulti-taskingfailures,suchasourinabilitytocarryouttwomentalarithmeticproblemsatthesametime(seeShaffer1975foramoreinterestingexampleandclassicexperimentaldemonstrationoftheproblemposedbycross-talkinvolvinginternalrepresentations).Fromthisperspective,restrictionsoncontrol-dependentprocessingreflecttheverypurposeofcontrol---tolimitthedeleteriouseffectsofcrosstalkintheprocessingsystemoverwhichcontrolpresides(Figure1,red)---ratherthananintrinsiclimitationofthecontrolsystemitself.
Thisaccountofcapacityconstraintsincontrol-dependentprocessing---intermsofrepresentationandcomputationalpropertiesoftheprocessingsystem,ratherthanstructuralpropertiesofthecontrolsystem---givesrisetoanotherpairofquestions,bothofwhichhavebeenaddressedbyrecentcomputationalwork.Thefirstquestioniswhethercollisionsinprocessingthatgiverisetocrosstalkarereallyaseriousprobleminasystemaslargeasthebrain.Simulationstudies(Fengetal.2014),followedbyrecentanalyticwork(Musslicketal.2016b),indicatethatevenmodestamountsofoverlapamongprocessingpathwayscanimposedramaticandnearlyscale-invariantconstraintsonhowmanyprocessescanbeexecutedatonetime.Suchconstraintsonparallelprocessinghavebeenshowntoholdevenincasesinwhichmultitaskingisexecutedasrapidsequentialswitchesratherthanperformancethatisstrictlyparallel(Musslicketal.2016a),undertheassumptionthatsequentialtasksbleedintooneanother(referredtoastasksetinertia)(Allportetal.1994,Allport&Wylie1999).Thus,itisatleastplausiblethateveninaverylargenetwork,pathwayoverlap(i.e.,theshareduseofrepresentationsbydifferentprocesses)quicklyproducesbottlenecksthatdemandmanagementbytheinterventionofacontrolsystem,andthattheselocalbottlenecks,ratherthantheconstraintsonacentralizedcontrolmechanism,mayexplainlimitsinthecapacityforcontrolledprocessing.Inotherwords,ourlimitedcapacityforcontrolledprocessingmayreflectthepurposeofcontrolratherthanaconstraintonitsabilitytooperate.
Thedeleteriousimpactofpathwayoverlaponprocessingraisesasecondquestion:Ifthebottleneckstheycreatearesoproblematic,whynotavertthisproblembydiminishingtheshareduseofrepresentations?Insightgainedfromthe
9
studyoflearningandrepresentationinneuralnetworksprovidesadirectanswertothisquestion:Sharedrepresentationsupportsinferenceandgeneralizationandiscriticaltothediscoveryanduseofabstractstructure.Thisinsightdrovetheconnectionistrevolutioninpsychologyinthe1980s(Rumelhart&McClelland1986)andisdrivingthecurrentexplosionofinterestindeeplearningnetworkswithinthemachinelearningcommunity(Bengioetal.2013,Caruana1998,LeCunetal.2015).
Thus,theuseofsharedrepresentationimposesatrade-offbetweenitsvalueforlearningandabstractionononehandandtheconstraintsitimposesonthesimultaneousexecutionofmultipleprocessesontheother.Musslicketal.(2016a)haveexploredthistrade-offdirectly.Theyhaveshownthatwhennetworksaretrainedtoperformavarietyoftasks,thereisastrongbiastowardtheemergenceofrepresentationsthataresharedacrosstaskswithsimilarrequirements,andforcontrolrepresentationstodevelopthatdisambiguatethesharedrepresentationsappropriately,accordingtotaskcontext.Furthermore,thisbiasconsiderablyfacilitateslearninginlargetaskspaces.However,thiscomesatthecostofseverelydegradedperformanceifanyofthetasksinvolvedmustbeperformedconcurrently.Additionaltrainingcanovercomethislimitationbyseparatingtherepresentationsforthedifferenttasks,whichalsodiminishestheirrelianceoncontrol(Garner2015).Theseobservationsconcurwithavast,longstandingcognitivepsychologicalliteratureonthetrajectoryfromcontrolledtoautomaticprocessingduringskillacquisition(Cohenetal.1990,Graybiel2008,Shiffrin&Schneider1977).
Musslicketal.(2016a)describetheirobservationsintermsofafundamentalcontinuumofcomputationalarchitectures,withthoseatoneendthatmakeuseofindependent(sometimesreferredtoasembarrassing)parallelismtosupportconcurrentmultitasking,andarchitecturesattheotherendthatexploitsharedrepresentationstosupportinteractiveparallelismintheserviceofabstractinferenceandefficientlearning.Fromtheperspectiveofsuchacontinuum,thecapacityconstraintsincontrolledprocessingreflectthebrain’schoice---inthosesituationsthatdemandrapidlearning,theflexibilityaffordedbyabstractinferenceandgeneralization,orboth---toexploitthevalueofsharedrepresentation,atthecostoflimitsonconcurrenttaskexecution.Assuggestedabove,mentaleffortcanthenbeviewedasanexplicitindicator(computationalandsubjective)ofthiscost---thatis,thecostassociatedwithsituationsinvolvingprocessingconfigurationsthatdemandtheengagementofcontroltoavertcrosstalk.
2.2OpportunityCostsofControlAlthoughunderstandingthesourceoftheconstraintsoncontrolled-processingremainsanimportantpriorityforresearch,theconstraintitselfsufficestoimposeacloselyrelatedcost:thatoftime.Engagingthecontrolsystemintheserviceofonecontrol-demandingtaskmeansforgoingothersthatcouldhavebeenperformedoverthatsameperiod.Thus,inadditiontoindexingthedegreeofinvestmentincontrolitself,mentaleffortmayalso
10
reflecttheopportunitycostofthatinvestment,asawayofensuringthattheyarerespectedwhenmakingdecisionsabouthowtoallocatecontrol(Kurzbanetal.2013).
Theideathatmentaleffortreflectstheopportunitycostsassociatedwithallocatingavaluablebutlimitedresource---thecapacityforcontrol---isintuitivelyappealing.Italsosuggeststhattheperceivedeffortmayscalewiththedurationoftheinvestment.Thisideaaccordswellwithworkonhowpeopleassessthevalue(andcost)ofcomputation,andhowthisinfluencestheirdecisionsaboutstrategiestopursueinproblemsolvingandotherbehaviors—alineofworkthathasanintimaterelationshiptothecostofcontrol.Wediscussthisindetailinthesectionthatfollows.
3.HOWSHOULDWEALLOCATECOGNITIVEEFFORT?
Asdiscussedabove,thebrain’sinherentcapacitylimitations---irrespectiveoftheircause---provideabasisforunderstandingmentaleffortcosts:Ifaresourceisvaluablebutlimited,itshouldbeconservedwheneverpossiblesothatitcanbeusedjudiciously.Thatis,cognitiveeffortshouldbeexpendedtotheextentitisworthit.Thisideaiscentraltoadecades-oldliteratureontheoriesofboundedrationality,whichemphasizethathumancognitionhastomakedowithlimitedinformation,littletime,andboundedcognitiveresources,andhasbeenfurtherdevelopedintotheoriesofboundedoptimality,whichspecifytheoptimalwaytousetheselimitedresources(seethesidebartitledBoundedRationalityandBoundedOptimality).
Recentaccountshavebuiltuponthistheoreticalgroundingtoproposeseveralwaysinwhichcomplexcognitiveprocessescanbeselectedsoastomaximizerewardswhileminimizingthecostsassociatedwithmentaleffort.Inthissection,wefocusontwocomplementaryoptimizationapproachesthathavebeendevelopedinparallelwithinresearchonstrategyandalgorithmselectionandwithinresearchoncognitivecontrol.
BOUNDEDRATIONALITYANDBOUNDEDOPTIMALITY
People’sjudgmentsanddecisionssystematicallyviolatethenormativeprinciplesoflogic,probabilitytheory,andexpectedutilitytheory(Tversky&Kahneman1974).Theoriesofboundedrationality(Simon1956,1982;Gigerenzer2008,Simon1982,Todd&Gigerenzer2012)attributethesesuboptimalitiestopeople’srelianceonsimplifyingheuristicsinlieuofmoreexhaustivedeliberation.Thesetheoriesarguedthatsuchcognitivefrugalityreflectspeople’smentallimitationsaswellastheirlimitedtimeandinformation.
Whereasearlypsychologicaltheoriesofboundedrationalitywerelargelyqualitative,researchinartificialintelligencesubsequentlydevelopedamathematicallyprecise,normativetheoryofhowboundedagentsshouldallocatetheircomputationalresources.Accordingtothetheoryofboundedoptimality(Russell&Subramanian1995),theobjectiveofrationalinformationprocessingistomaximizetheagent’sexpectedrewardperunittimeoverthelongterm,subjecttotheconstraintsoftheagent’sperformance-limitedhardware.
11
Thenotionofboundedoptimalityhasinspiredcomputationalmodelsofhumancognition,accordingtowhichthebrainmakesoptimaluseofitsfiniteresources(Gershmanetal.2015,Griffithsetal.2015,Lewisetal.2014).Fromthisperspective,limitationsinthebrain’scapacityforparallelprocessing(Fengetal.2014,Musslicketal.2016b)anditsfiniteinformation-processingspeedgiverisetoopportunitycostsforeachcognitiveoperation:Committingtooneorasetofcomputationsprecludesthesimultaneousexecutionofothervaluableones(cf.Kurzbanetal.2013).Asaresult,peoplehavetotradeofftheexpectedqualityoftheirchosencomputationsagainstthenumberofdecisionsthatcanbemadeperunittime,andinmanycasestheoptimalamountofdeliberationforeachindividualdecisionissurprisinglylow(Vuletal.2014).Thelimitedeffortthatpeopleinvestincertaindecisionsandjudgmentsmayreflectthisoptimalityprinciple(Liederetal.2012,F.Lieder,T.L.Griffiths,Q.J.M.Huys&N.D.Goodman,submittedmanuscript,Vuletal.2014;butseealsoOudetal.2016).
3.1.TheValueofComputation
Toachieveboundedoptimality,peoplemayperformacost-benefitanalysistoselectthecognitivestrategywiththebesttradeoffbetweeneffortandaccuracy(Beach&Mitchell1978,Payneetal.1988).Researchinartificialintelligencehasprovidedamathematicallyprecisedefinitionofwhatconstitutesthisoptimaltradeoffforagivensetofcomputeralgorithms(i.e.,sequencesofcomputations)(Russell&Wefald,1991).Inbrief,thebestalgorithmshouldmaximizethevalueofcomputation(VOC),whichisdefinedastheexpectedutilitygainedbyengagingthosecomputationsminustheexpectedcostofthecomputationalresourcesitwillconsume(e.g.,CPUcyclesandmemory).Suchmethodsfordecidinghowtoallocatecomputationalresources(termed‘rationalmetareasoning’)havebeendevelopedtoenableintelligentsystemstointeractwiththeirenvironmentinrealtimetomakeoptimaluseoftheirlimitedtimeandfinitecomputationalresourcesbyselectingtheircomputationsadaptively(Hay,etal.,2012).
Liederandcolleagueshaverecentlyappliedthisoptimalityprincipletocognition,proposingthatindividualsmightsimilarlyselectcognitivemechanisms(e.g.,decisionstrategies)basedontheirrelativeVOC(Griffithsetal.2015,Liederetal.2012,Lieder,Plunkett,etal.,2014;Lieder&Griffiths,2015).Underthisformulation,peopleshouldrationallytradeoffthequalityofaselectedcognitivestrategyagainstthecostofthecomputationsitentails(Liederetal.2014).Specifically,mentaleffortshouldbeallocatedtoachievetheoptimaltrade-offbetweentheexpectedutilityofitsoutcomeandtheopportunitycostoftherequiredtime.Thissimpleprinciplecanaccountfortheadaptiveflexibilitywithwhichpeopleswitchbetweendifferentcognitivemechanismsdependingontheproblemtobesolved(F.Lieder&T.L.Griffiths,submittedmanuscript).
Theseresearchershaveproposedthatpeoplelearntoselectcognitivemechanismsinawaythatapproximatesrationalmetareasoningefficiently(Lieder&
12
Griffiths2015,Liederetal.2014);thatis,theylearntopredicttheVOCofcandidatecognitiveoperationsfromfeaturesoftheproblemtobesolvedandselectthesequenceofoperationswiththehighestpredictedVOC(seeFigure2a).Suchafeature-basedlearningmechanismcanaccountforadaptivechangesinstrategyselectionacrossabroadarrayoftaskdomains,includingdecisionmaking,problemsolving,andmentalarithmetic(Lieder&Griffiths2015,F.Lieder&T.L.Griffiths,submittedmanuscript).Forexample,thisaccountwasabletopredictpeople’sadaptivechoicesofsortingstrategywhenfacedwithanout-of-orderlistmuchmoreaccuratelythanpreviousmodelsofstrategyselection(Figure2b)(Liederetal.2014).Overall,thesefindingssuggestthatwhenselectingbetweensequencesofcognitiveoperations,peopleinvestmentaleffortrationallywithrespecttotheirmentalmodelofthe(time)costsandrewardsforpotentialstrategies.
Figure2.(a)Thevalueofcomputation(VOC)modellearnshowfeaturesoftheenvironment(e.g.,thelengthandsortednessofalistofnumbers)predicttheeffectivenessandefficiencyofcandidatecomputations(e.g.,strategiesforsortingthatlist),basedontheirexpectedrewardandtimecost(thedifferencebetweenwhichdeterminestheirVOC).(b)TheVOCmodelperformed
13
substantiallybetteratpredictingactualchoicesofsortingstrategythanpreviousmodelsofstrategyselection(namedStrategyChoiceAndDiscoverySimulation[SCADS],StrategySelectionLearning[SSL],andReinforcementLearningfromCognitiveStrategies[RELACS]).AdaptedfromLiederetal.(2014).(c)Accordingtotheexpectedvalueofcontrol(EVC)theory,agivencognitivecontrolsignalsetting(consistingofbothitsidentityandintensity)determinestheexpectedpayoffsandcostsforcontrol.Theoptimalcontrolsignalsettingsmaximizethedifferencebetweenthesetwoquantities,ortheEVC.AdaptedwithpermissionfromShenhavetal.(2013).(d)EVCmodelsimulationscapturetheimprovementsinbehavioralperformance(higheraccuracyandfasterresponses)observedwhenvaryingtheincentivesforapicture-wordStrooptask(Musslicketal.2015,Padmala&Pessoa2011).Participantsperformingthistaskwereinstructedtoindicatewhethertheimagedisplayedabuildingorhousewhileignoringthecontentoftheoverlaidtext.AdaptedwithpermissionfromPadmala&Pessoa(2011)andMusslicketal.(2015).
3.2.TheExpectedValueofControl
TheVOCmodeldescribeshowindividualsselectbetweensetsofsequentialoperationsbasedontheexpectedrewardaswellasthecostsassociatedwiththetimerequired.However,asnotedabove,cognitivecontrolisgenerallyconceivedasfallingalongacontinuum;onecanapplyvaryingdegreesofcontrol(e.g.,attention)tothetaskathand,withconcomitantchangesinperformance.Moreover,peopleexperiencehigherlevelsorintensitiesofcontrolexertionasmorecostlyoraversive,independentlyoftheassociatedtimecosts(Dixon&Christoff2012,Kooletal.2010,Westbrooketal.2013).Thus,acost-benefitanalysisisnecessarytodeterminenotonlywhattypesofcontrolledprocessesareworthinvestinginbutalsohowmuchcontrolisworthinvestingineach,basedonthereturnsexpectedforagivenlevelofcontrol.Inspiredbyreinforcementlearning(RL)modelsofactionselectionandmotorcontrol(Sutton&Barto1998,Wolpert&Landy2012),Shenhavandcolleagues(2013)recentlydevelopedatheorythatformalizesthisideaofacost-benefitanalysisformaximizingtheexpectedvalueofcontrol(EVC).
TheEVCtheoryproposesthatcontrolsignalsarespecifiedalongtwodimensions:anidentity(e.g.,whattoattend,suchasthecolorofaStroopstimulus)andanintensity(e.g.,howstronglytoattend,relativetoadefaultorautomaticlevel).Adjustingtheintensitiesofcontrolsignalsshouldinfluencethelikelihoodofobtainingreward,avoidingpunishment,orboth(e.g.,basedonprovidingacorrectversuserroneousresponse),aswellastheefficiencyofdoingso(e.g.,howlongittakestorespond).Collectively,thesefactorsdefinetheoverallrateofrewardreceipt(rewardperunittime),whichisakeyvariableanimalsseektomaximize(Bogaczetal.2006,Nivetal.2007).Inadditiontodiscountingthevalueofcontrolbythetimespentonthetask,thetheoryimportantlyalsoassumesanintrinsiccostordisutilityassociatedwithincreasingcontrolintensity.TheEVCisdefinedasthedifferencebetweentheexpectedgains(e.g.,rewardrate)andtheexpectedcontrolcostassociatedwithagivenconfigurationofcontrolsignals(Figure2c).OptimalcontrolallocationcanbeachievedbyselectingthecontrolsignalconfigurationthatmaximizesEVCinthecurrentsituation.Recently,acomputationalimplementationoftheEVCtheoryhasbeenshowntoaccountforavarietyofphenomena
14
associatedwiththeallocationofcontrol(Musslicketal.2015),includingsequentialadaptationeffectssuchaspost-errorslowingandincentive-drivenimprovementinperformanceoninhibitorycontroltasks(Figure2d).
EVCisoneofafamilyofrecenttheoriesthathaveappliedRLapproachestotheallocationofcontrol(Frank&Badre2012,Holroyd&McClure2015,O'Reilly&Frank2006,Toddetal.2009,Vergutsetal.2015).Forinstance,onerecentmodelusestemporaldifferencelearningtoestimatethevalueandeffortcostsofbothcognitiveandmotoractions(Vergutsetal.2015)andcombinesthetwoestimatestodeterminewhetherornottoincreasethegain(signal-to-noiseratio)ofeitherformofactionselection.BuildingonahierarchicalextensionofRL(HRL)---wherebyanagentcanlearnthevalueofindividualactionsaswellastemporallyextendedsequences(Botvinicketal.2009b)---Holroydandcolleagues(Holroyd&McClure2015,Holroyd&Yeung2012)haveproposedanothermodel,accordingtowhichcontrolmaybeusedtomitigatetheshort-termcostofphysicallyeffortfulobstaclesinfavoroflonger-termrewardsthatwillbeobtainedaftercompletingthecurrentsequenceofactions.Inotherwords,theHRLmodelselectshigh-levelactions(calledoptions)tomaximizelong-termvalue(e.g.,walkacrosscampustomeetafriend),andcontrolprotectsthoseoptionsfrombeingdelayedoroverturnedinthefaceofeffortfulobstaclesalongtheway.OthermodelshaveaddressedthecontrolofbehaviorbyusingRLasthebasisforselectingtheinformationthatisallowed(gated)intoworkingmemory(Alexander&Brown2015,Frank&Badre2012,O'Reilly&Frank2006,Toddetal.2009),enablingthemaintenanceofappropriatehigher-levelgoalsandtheassociatedmobilizationofcontrolmechanismsgivenfeedbackfromtheenvironment.
Theseaccountsdrawuponsimilarbasicprinciplesoflearningandactionselectionandarethereforepotentiallycomplementary,addressingdifferentcomponentmechanisms,levelsofmechanisticdetail,orboth.AdistinctiveelementoftheEVCtheoryisitsfocusonthecostofcontrolasafactorintheselectionprocess.ThisallowstheEVCtheorytonotonlymakecontactwithbroaderresearchintorationalmetareasoninganditshistoricalpredecessors,butalsotomakedetailedcontactwiththelarge,butheretoforeratherqualitative,literaturelinkingeffortwithincentivemotivation(reviewedinBotvinick&Braver2015).
3.3.ValueofComputationandExpectedValueofControl
Inadditiontodifferentlyemphasizingdifferentkindsofcontrolcosts(i.e.,thosethatscalewithtimealoneorinadditiontothosethatscalewithlevelofcontrolengagement),theVOCandEVCframeworksalsomaximizethevalueofdifferentkindsofpotentialoutputs.TheVOCframework,withitsoriginsintheartificialintelligenceliterature,framesthecentralcost-benefitanalysisunderlyingeffortintermsofcomputation.TheEVCframework,incontrast,framesitintermsofcognitivecontrol.Thismayappeartobeadiscrepancybetweenthetwotheories.However,analignmentcanbeestablishedbyconsideringtheamountofcomputationrequiredforcontrolledversusautomatic
15
processing.Incomparisonwithautomaticprocessing,controlledprocessingrequiresaricherrepresentationoftaskortemporalcontext.Automaticprocessing,bydefinition,yieldsdefaultbehaviors;tooverridethese,theprocessingsystemmustinstantiatearepresentationofcontext(suchas,forexample,arepresentationofrecentlyreceivedverbalinstructions).Settingupthisrepresentationandderivingappropriateactionsfromitrequirescomputation,andstillmorecomputationisrequirediftheappropriatecontextrepresentationmustbediscoveredthroughsearchordeliberation.Thishighlightsthattherepresentationalcostofcognitivecontroliscloselyrelatedtothecostofcomputation.RecentresearchbyOrtega&Braun(2011,2013)formalizesthispoint,relatingittothelargerthemeofboundedrationality.
4.WHATARETHENEURALMECHANISMSFORTRACKINGCONTROLCOSTSANDALLOCATINGCONTROL?
Thetheoreticalperspectiveswehavereviewedaboveprovideaformallyrigorousframeworkforexaminingtheneuralsubstratesunderlyingmentaleffortallocation.Thesetheoriessuggestthatneuralcircuitsmediatingcontrolallocationshouldbesensitivetothepossiblewayscontrolcouldbeallocatedatagiventimeandthepotentialrewardsgained(orpunishmentavoided)byengagingcontrol,andthecostsincurredbytherequisitecontrol.Furthermore,inactivatingthesecircuitsshouldresultinmotivationaldeficits(i.e.,impairedeffortallocation).Thesepredictionshavebeenlargelyborneoutbyresearchintothecircuitryforcognitivecontrolandinparticulartheroleofthedorsalanteriorcingulatecortex(dACC)withinthatcircuit.
4.1.EffortandtheExecutiveNetwork
Abroadnetworkofcorticalstructureshasbeenimplicatedintasksthatrequireanindividualtoexertcognitiveeffort(Dosenbachetal.2008,Duncan2010,Power&Petersen2013,Shenhavetal.2013).ThisincludesdACC,anteriorinsula(AI),lateralprefrontalcortex(lPFC),andlateralparietalcortex.Theregionsinthiscontrol-relatednetworkaremoreengagedwhenanindividualmustperformataskthatdemandssustainedattention,maintenanceofinformationinworkingmemory,and/oroverridingofprepotentresponses;theyarerelativelydisengagedwhenperformingmorehabitualand/orexternallyguidedbehaviors.However,althoughtheinvolvementofthisnetworkinresearchoncognitivecontrolisrobust,thefunctionalrolesoftheconstituentsofthiscircuitareheavilydebated,particularlywithrespecttopotentialrolesindecisionsabouttheallocationofcontrol,ascontrastedwiththeexecutionofcontrol.GiventheparticularrelevanceofdACC,AI,andlPFCtothisquestion,andthequestionofcontrolcostsignaling,wereviewrelevantfindingsconcerningthesecorticalstructuresandtheirinteractionswithsubcorticalstructuresandneuromodulatorysystems.
ResearchershavelongvieweddACC(and,inparticular,anteriormidcingulatecortex)(Shackmanetal.2011,Vogt2016)asplayingaroleindetermininghow
16
physicalactions,cognitiveactions,orbotharedeployedoralteredbasedonavailableevaluativesignals.EarlyviewsfromthecognitivecontrolliteraturesuggestedthatdACCmayeffectthesechangesbycapitalizingonexternalsignalsthatanerrorhadbeencommitted(Holroyd&Coles2002),internalsignalsofconflictbetweenpotentialresponses(Botvinicketal.2001),and/orinternalestimatesofthelikelihoodofcommittinganerrorinagivencontext(Brown&Braver2005).Consistentwiththesesuggestions,dACChasbeenshowntosignaleachofthesequantities(Ridderinkhofetal.2004;Shackmanetal.2011;Shenhavetal.2013,2016;butseeNieuwenhuisetal.2007)aswellasotherpotentialindicatorsofcontroldemandssuchassurprise(Cavanagh&Frank2014,Wesseletal.2012).However,patternsofactivityindACCandbetweenthisregionandotherssuggestabroaderandmorenuancedrolethanonethatsimplyindicateshowmuchcontrolmightbedemandedataparticularpointintime(Heilbronner&Hayden2016,Shenhavetal.2016).Thesepatternssuggestrolesineffortavoidance(Waltonetal.2007),reward-baseddecisionmaking(Rushworthetal.2004,2011),andmotivation(Holroyd&Yeung2012,Stuss2011).TheapparentinvolvementofdACCinthesedifferentfunctionscanbereadilyexplainedthroughthelensofthecontrolallocationmodelsdescribedintheprevioussections.
4.2.AnExpectedValueofControlPerspectiveonDorsalAnteriorCingulateCortex
IthasbeenproposedthatdACCintegratessignalsrelevanttotheEVCandspecifiestodownstreamregionsthetypesandintensitiesofcontrolthatwouldmaximizethisquantity(Shenhavetal.2013,2016).Assuggestedabove,thismonitoringanddecision-makingprocessisinformedbysignalsthatindicatethelikelihoodofacertainoutcome(e.g.,acorrectresponse)givenaparticularallocationofcontrol,thetimerequiredtoobtainthatoutcome,andtherewardorpunishmentassociatedwiththatoutcome.ThisaccountsfordACC’sassociationwithindicatorsofcognitivedemandmentionedabove(e.g.,errors,conflict,surprise),whichcanserveasproxiesforperformancecosts(timeanderrorlikelihood).ItalsoaccountsforfindingsofdACCsignalsindicatingthevaluesofpotentialoutcomes(Heilbronner&Hayden2016,Kapingetal.2011,Kouneiheretal.2009).Moreover,thepredictionthatEVCincorporatestheintrinsiccostofcontrolisconsistentwithfindingsthatdACCtrackshowaversivecontroldemandsaretoanindividual,includingtheirexperiencedfrustration(Spuntetal.2012),theirpreferencesagainstperformingthetask(McGuire&Botvinick2010),andhowmuchtheydevaluerewardsassociatedwiththecognitivelyeffortfultask(Botvinicketal.2009a,Cavanaghetal.2014).Interestingly,dACChasalsobeenfoundtotrackthecostofphysicaleffort(Croxsonetal.2009,Hillman&Bilkey2010,Prévostetal.2010),suggestingitmayplayasuperordinateroleincomputingeffort-sensitivecost-benefitanalysesacrossdomains.
WithinthecontextoftheEVCtheory,dACChasbeenfurtherproposedtooutputsignalsspecifyingboththetypesandamountsofcontroltoallocateinordertomaximizeEVC;thesesignalshavetheeffectoflicensingtheeffortrequiredbytheallocatedcontrol.ThispredictionissupportedbyevidencethatdACCisableto
17
differentiatebetweentypesofcontrolthatareneededinagivensituation(Kapingetal.2011,Shenetal.2014);thatchangesindACCactivityduringmonitoringpredictsubsequentcontroladjustments(e.g.,responseslowing,attentionalshifts)(Kernsetal.2004,Shenhavetal.2016,Ullspergeretal.2014);andthatcausalmanipulationsofdACCcaninfluencetheseadjustments(Reinhart&Woodman2014,Shethetal.2012;seealsoMansourietal.inpress)or,inextremecases,moredrasticallyinfluenceone’swillingnesstoengageincontrol-demandingtasks(i.e.,one’sdecisiontodeemthosetasksworththeeffort)(Holroyd&Yeung2012,Parvizietal.2013,Stuss2011,Waltonetal.2007).Forinstance,dACC-lesionedratsarelesslikelytopursuethegreateroftworewardsifdoingsorequiresovercominganeffortfulobstacle(Waltonetal.2007,Holroyd&McClure2015).
TheproposedroleofdACCinspecifyingEVC-maximizingcontrolsignalsisbroadlyconsistentwithmanyotheraccountsofthisregion’sroleinintegratingrelevantevaluativesignalstohelpguideadaptivebehavior(reviewedinCavanagh&Frank2014;Heilbronner&Hayden2016;Shenhavetal.2013,2016;Ullspergeretal.2014).Forinstance,theHRLmodelbyHolroydandcolleagues(Holroyd&McClure2015,Holroyd&Yeung2012)proposesthatdACClearnsthevalueofextendedsequencesandonthatbasisdetermineswhethertodiscountthecostofeffortfulobstaclesthatpreventlower-levelactionselectionregionsofstriatumfromperseveringoverfutureactionswithinthatsequence.Vergutsandcolleagues(2015)similarlyproposethatdACClearnsthecost-discountedvalueofadjustingthegainonactioninagivencontextandmodifiesprocessingaccordingly.AndAlexander&Brown(2015)proposethatregionsofdACCservetotunepredictionsabouttheerrorsthatwillresultfromfailingtoactappropriately,failingtomaintaintheappropriateinformationinworkingmemory,orboth,leadingtoadaptiveimprovementsindecisionsaboutwhichinformationtogate.AlthoughacomprehensivecomparisonofthesemodelsandtheirabilitytoaccountfordACCfunctionisbeyondthescopeofthecurrentreview,thesetheoriesshowcollectivelythatanappropriatelynuancedmodelofcontrolallocationmayprovideamoreparsimoniousaccountofthevarietyofsignalsthathavebeenobservedinthisregion,withouthavingtopositadditionaluniquefunctionsassociatedwitheachsignal(Shenhavetal.2016).
4.3.ExpectedValueofControlandtheBroaderExecutiveNetwork
Twobrainregionsthatarecommonlyincludedwithinthesameexecutive/controlnetworkasdACCaretheAIandlPFC.Thesehavebeenproposedasinputstoandoutputsofcontrolallocationdecisions,respectively(Bushetal.2000,Caietal.2016,Shackmanetal.2011,Shenhavetal.2013,Ullspergeretal.2014).LikedACC,AIhasbeenshowntorespondphasicallytoabroadarrayofsalienteventsthatmaysignaltheneedtoadaptcontrol(includingrewardsanderrors)(Menon&Uddin2010)andalsodisplayssustainedelevatedresponsesoverthecourseoftaskperformance(possiblyrelatedtothemaintenanceofcontrol)(Dosenbachetal.2006).However,thesetworegions(whichsharereciprocal
18
connections)differintheirrelativepatternsofconnectivitywithsensoryinputsversusmotoroutputs,withAIsharingmoreconnectionswiththeformerthanthelatter(Craig2009).Accordingly,whilesubstantialevidenceofdissociationisstilllacking,basedontheirrelativetimingandpatternsofactivationacrosstheliterature,ithasbeensuggestedthatdACCmayplayamoredirectroleineffectingchangesincontrolallocationwhileAImaysignalsalientstatesthatbearonthosecontrolallocationdecisions(e.g.,internalandexternalsignalsassociatedwitherrorcommission)(Craig2009,Magnoetal.2006,Medford&Critchley2010,Ullspergeretal.2010,2014,Caietal.2016).
ResearchershaveshownlPFCplaysakeyroleintheexecutionorregulationofcertaincontrolpolicies,andtheythereforeargueitistheexecutorofspecifickindsofcontrolsignalsthathavebeenspecifiedbydACC(e.g.,maintainingtasksets)(Botvinicketal.2001,Holroyd&Yeung2012,Ridderinkhofetal.2004,Shenhavetal.2013).Thisproposedfunctionalrelationshiphasbeensupportedbystudiesthatexaminethetimescaleofprocessingacrossthesetworegionsduringcontrolmonitoring,adjustment,andexecution(Kapingetal.2011,Oehrnetal.2014,Tangetal.2016,Womelsdorfetal.2010).LikedACC,lPFChasalsobeenfoundtobeassociatedwithpreferencestoavoidcognitiveeffort(McGuire&Botvinick2010)andwithsignaturesofcognitivefatigue(Blainetal.2016,Tanakaetal.2014).Thismaysuggestaroleinsignalingcontrolcosts,perhapsatamoreabstractorheuristiclevel(cf.McGuire&Botvinick2010)butcouldalsosuggestthatlPFCprovidesamoresensitivereadoutofthelevelofcognitiveeffort(i.e.,controloutput)beingexertedatagiventime(Wangetal.2016).
AlthoughregionsoflPFCappeartobethenodesoftheexecutivenetworkmostcloselyassociatedwiththeregulationofcontrol,itisimportanttonotethatcontrolsignalsareheterogeneousandtheirregulationisthereforeunlikelytobethesolepurviewoflPFC.Rather,theexecutionofcontrolsignalsislikelymediatedbyseveralofdACC’sdownstreamtargetsthatcanimplementdifferenttypesofcontrolsignals(Shenhavetal.2013,Ullspergeretal.2014),includingglobalmodulatorychangesinresponsethreshold(drivenbysubthalamicnucleus)(Cavanagh&Frank2014,Cavanaghetal.2011,Keukenetal.2015)andinthegainofneuralprocessing(drivenbylocuscoeruleus)(Aston-Jones&Cohen2005,Eldaretal.2013,Jepma&Nieuwenhuis2011).
Itisimpossibletodiscusstheroleofcorticalandsubcorticalcircuitsineffortallocationwithoutalsoconsideringthecentralroleofthemidbraindopaminergicsystem(reviewedinCools2016,Salamoneetal.2009,Westbrook&Braver2016).Atshortertimescales,dopaminergiccircuitshavebeenshowntodiscountphasicresponsestorewardsbythelevelsofphysicalorcognitiveeffortrequired(Botvinicketal.2009a,Pasquereau&Turner2013,Varazzanietal.2015;butseealsoGanetal.2010).Atlongertimescales,dopaminergicinputtocortex,andtodACCinparticular,hasbeenshowntobecausallynecessaryforengagementineffortfulbehavior(Holroyd&McClure2015,Salamoneetal.2009,Waltonetal.2007),particularlywhentheeffortrequirestheagenttoovercomeabiastowardasalientdefaultalternative(Nicola2010).Forinstance,pharmacologicalattenuationof
19
dopaminelevels,ordamagetothewhitemattertractsconnectingthenucleusaccumbensandcingulatecortex,rendersanimalslesswillingtoexpendefforttoachieveagreaterreward.
ThemotivationalimpairmentsresultingfromdopaminemanipulationaresimilartothosediscussedabovethatresultfromdamagetodACCand,often,adjacentregionsofsupplementarymotorarea(SMA).VariabilityinstructuralandfunctionalconnectivitybetweendACCandSMAhasalsobeenimplicatedinmoresubtlemanifestationsofapathy,andtheirrelationshiptodecisionsaboutphysicaleffortinvestment(Bonnelleetal.2015).Mostnotably,transcranialmagneticstimulationofSMAhasbeenfurthershowntoattenuatetheaversiveexperienceofaphysicallydemandingtask,renderingparticipantsmorewillingtoexerteffortforlessreward(Zenonetal.2015).Collectively,thesefindingssuggestthatinteractionswithinandamongmedialprefrontalcortexanddopaminergiccircuitsarecriticalfortransformingevaluativeinputsintoeffortinvestments,bothinthephysicalandcognitivedomain(seealsoHoskingetal.2014).However,inthefinalsectionbelow,wediscussimportantquestionsthatremainregardingthedegreeofneuralandcomputationaloverlapthatexistsacrossthesedomains.
5.CHALLENGESANDFUTUREDIRECTIONS
Asourreviewshows,thereisanexcitingconfluenceofresearchonthecomputationalandneuralbasisofcontrolcostsandourabilitytoallocatecontrolwhileaccountingforthesecosts.Yetseveralimportantquestionsremaintobeaddressed.
5.1.OperationalizingandMeasuringCognitiveEffort
First,althoughoverwhelmingevidencesuggeststhatcontroliscostly,itisfarfromclearhowthosecostsshouldbeoperationalized(i.e.,whataretheirconstituents)or,moreimportantly,howtheyshouldbemeasured.Controlcostshavebeeninferredfromavarietyofmeasuresincludingresponsetimes(RTs)(Anderson1996,Liederetal.2014,Ratcliff1978,Sternberg1969),avoidantpreferences(Kooletal.2010,McGuire&Botvinick2010,Westbrooketal.2013),affectivepriming(Dreisbach&Fischer2015),pupildiameter(Kahneman&Beatty1966),contractionofspecificfacialmuscles(corrugatorsupercilli)(Elkins-Brownetal.2015),sympatheticarousal(Critchleyetal.2003),andneuralactivitymeasuredfromdACC(Cavanagh&Frank2014,Cavanaghetal.2014,McGuire&Botvinick2010,Spuntetal.2012)andotherregions(Blainetal.2016,McGuire&Botvinick2010;reviewsinInzlichtetal.2015,Westbrook&Braver2015).However,noneofthesemeasureshasbeenshowntobeselectivetocontrolcosts;rather,manyhavebeenshowntoindexother,moregeneral,factorssuchassympatheticarousal;evenaseeminglydirectmeasuresuchasdemandavoidancecanbesusceptibletoancillaryfactorsrelatedtoexperimentaldemandandone’sabilityormotivationtodetectthepresenceofcontrolcostdifferencesacrosstasks(Goldetal.2014).Moreover,someofthesemeasuresmaybesensitivetoboththecostsofcontrolandthecontrolbeingallocated,thusconfoundingtheirinterpretationasreflectingoneortheother.Forinstance,longerRTscouldreflecta
20
moredifficulttask(i.e.,indicatinghighercost)orlesseffort(i.e.,indicatingafailuretoallocateadequatecontrol).
Complicatingmattersfurther,thereisampleevidencethatcognitiveeffortcanhavepositiveassociationsinadditiontotheaversiveonesdescribedaboveandthatthismaydifferconsiderablyacrossindividuals.Forexample,certaincontextsorpersonalitytraits[e.g.,needforcognition(Cacioppo&Petty1982),learnedindustriousness(Eisenberger1992)]willleadanindividualtoassociatementaleffortexertionpersewithreward,independentoftheoutcomeoftheeffort.Moreover,itiswellknownthatpeoplefindtasksrequiringlowlevelsofcognitiveengagementtobeboringandthereforeaversive,andtheyinsteadseektofindanoptimalmidpointbetweentoolittleandtoomuchcognitiveeffort(Nakamura&Csikszentmihalyi2002).Whereastheformerexampleslinkrewardmoredirectlytoeffortexertion---leadingtopredictionsthataredifficulttodissociatefromintensity-basedcontrolcostpredictions---boredomhasbeenproposedtoinsteadbeareactiontolowlevelsofinformationorarousalincertainenvironments(Eastwoodetal.2012;Geanaetal.2016a,b;Zakay2014)andisthereforepotentiallymorereadilydissociatedfromcontrolcosts.Dissociatingthesecostandrewardfunctionswillrequirecarefulexperimentaldesignandmeasurementofparticipant-specificestimatesofrewardandeffort.
Theseconcernsregardingmeasurementandoverlappingcostandrewardfunctionshighlighttheimportanceofgeneratingpreciseandquantitativepredictionsregardingthefactorsinfluencingcontrolallocationandconstrainingthesepredictionswithmultipleclearlyspecifiedpredictorvariables.Inadditiontotheaforementionedmeasures,thisnaturallyincludesmeasuringself-reportedsubjectiveexperiencesofeffort.Itremainsanopenquestionwhetherconsciousawarenessisaprerequisitefortreatingtheexertionofcontrolascostly(cf.Desenderetal.2014,Dunnetal.2016,Mulertetal.2005,Naccacheetal.2005)andwhethercognitivecontrolevenfunctionsinparttoregulatethesenegativeexperiences(Inzlichtetal.2015).Regardless,subjectivemeasurescanundoubtedlyhelpconstrainatheorybyidentifyingsimilaritiesanddifferencesbetweentask-andcontext-relatedvariablesthattriggerdifferentkindsofavoidantreactionsandtherebydefineatopologyofsubjectiveexperiencescharacterizedbytermssuchasdifficult,frustrating,tiring,stressful,challenging,andboring(cf.Saundersetal.2015,Spuntetal.2012).Theymayalsohelpteaseapartthecomponentsofcontrolcoststhatrelatetoeffortpersefromotheraversivereactionsassociatedwithindividualcontroldemands,suchaserrorsandconflictoruncertainty.
5.2.DisentanglingCompetingModelsandMechanisms
Anothercriticalstepisformodelsofcontrolcostsandcontrolallocationtobecomparedwithoneanotherdirectly,whichinturnwillrequiremodelstoreconciledifferencesinrelevantterminology.Thisprocessislikelytorevealsimilaritiesorevensubstitutabilityinindividualalgorithms,aswellassubstantivedifferencesatthelevelsoftheory,
21
implementation,orboth.Thegoalshouldbetogeneratecommonbenchmarksforthesevariedmodelsintermsofbothselectionandexecutionofcontrolandtoperformdedicatedexperimentsthatexaminewhichmodelbestpredictshowpeopleallocatementaleffort.Forinstance,futureresearchshouldevaluatemorecarefullytheextenttowhichcontrolcostincreases(a)monotonicallywithcontrolsignalintensityandduration(Musslicketal.2015,Shenhavetal.2013);(b)withtherichnessofcontextrequiredtosupportnondefaultresponses(asindexedbythedifferencebetweenthedistributionofpotentialresponsesforthecontrolledversusautomaticpolicy)(Ortega&Braun2011,2013);and/or(c)withthevalueofopportunitiesforegonewhileengagedincandidatecomputations(Kurzbanetal.2013,F.Lieder&T.L.Griffiths,submittedmanuscript,Liederetal.2014).
Suchmodelcomparisonwillfurtheraideffortstointerprettheroleofneuralmechanismsthathavebeenimplicatedincontrolallocationandgeneratemoreprecisepredictionsregardingthetime-courseofactivationacrossregionsinresponsetosignalsthatdemandachangeincontrolsettings.ThiswillinturnhelptosettlelongstandingdebatesregardingthefunctionalroleorrolesofdACCandotherregionsacrossresearchintodecisionmaking,cognitivecontrol,andaffectiveprocessing(Heilbronner&Hayden2016;Holroyd&Yeung2012;Shackmanetal.2011;Shenhavetal.2013,2016).ItwillfurtherhelptoconstrainpredictionsforagiventheoryregardinghowtointerpretdACC’softenunderdeterminedengagementinaparticularcontext---forinstance,whetheritreflectsthecosts,demands,and/oroutputofcontrol(Shenhavetal.2013).
5.3.RelationshipBetweenDifferentFormsofEffort
Attemptstounderstandthecomputationalandneuralunderpinningsofcognitiveeffortfrequentlydrawconnectionstothosesameunderpinningsforphysicalormotoreffort.Thismaybeunavoidablegiventheempiricallinksbetweenthephenomenologyandmechanismsofthesetwoformsofeffort(Marcoraetal.2009,Schmidtetal.2012),butitpromptsaveryimportantquestion:Ifphysicaleffortisinfactassociatedwithphysicalresourcedepletion(Cabanac2006),andcognitiveeffortturnsoutnottobe(Inzlicht&Schmeichel2012,Kurzbanetal.2013),thenwhyaretheresimilarunderpinningsforthesetwo?Aprovocativepossibilityisthatthecostlycomponentofphysicaleffortinfacthaslittletodowithphysicalresources(Marcora2009,Marcora&Staiano2010,cf.Huangetal.2012)butratherisasimilarcontrolcostasforcognitivecontrol---thecostofovercomingamoreautomaticoption(e.g.,morehabitualbehavior).Giventhefundamentalnatureoftherelationshipbetweenthesetwoformsofeffort,futureworkshouldbettercharacterizetheirrespectivecostfunctionsandthebasisforanysimilaritiesamongthese.Thisworkshouldalsoaimtocomparethedegreetowhichdifferentspeciesdiscountrewardsforasimilartypeofcognitiveorphysicaleffort.
5.4.ClinicalandPolicy-RelatedApplications
22
Appropriateallocationofcognitiveeffortiscentraltoourabilitytothriveashumans,particularlygiventhedemandsofmodernenvironments.Individualswhoarewillingand/orabletoexertcontrolmoreconsistentlyandinspiteofapparentobstaclesareabletoperformbetterinacademicandworkenvironments(Duckworthetal.2007,Eigstietal.2006,Mischeletal.1989).Conversely,varietiesofimpairmentsincontrolallocationresultingfromdisorderssuchasmajordepression,schizophrenia,addiction,obsessive-compulsivedisorder,andattention-deficitdisordercanbeparticularlydebilitating(Cools2016,Salamoneetal.2016,Holroyd&Umemoto2016,Westbrook&Braver2015).
Yetevenhigh-functioningindividualsarepronetofailingtoexertthecontrolrequiredtooverridehabitual,impulsive,andothershort-sightedresponsetendencies;suchself-controlfailureshaveseriousramificationsforthehealth,safety,relationships,educationalattainment,andfinancesoftheseindividuals(Duckworthetal.2007,Heatherton&Wagner2011,Mischeletal.1989)andeventheevolutionofourspecies(Cohen2005,Tomlinetal.2015).Fromtheperspectiveofboundedoptimality,someofthesefailuresmightbeinevitableincomplexenvironmentswithtoomanymisleadingtemptationsbecausepeople’scognitiveresourcesarefiniteandtheirtimeislimited.Indeed,toleratingoccasionalself-controlfailuresmaybemoreboundedlyoptimalthanensuringsuchfailuresneveroccur.
Accordingtothisview,peopleshouldinsteadrestructuretheirenvironmentsothatgooddecisionscanresultfromsimpleheuristicsratherthanrequiringextensivecognitiveoperations(Gigerenzer2008,Lieder&Griffiths2016).Oneexampleofthisapproachthathasprovedeffectiveistoimposedefaultoptionsforcertaindecisionsthatmaximizethetypicaldecisionmaker’slong-termrewards(e.g.,retirementsavings),apolicyreferredtoaspaternalisticlibertarianism(Thaler&Sunstein2008).Anotherapproachistoaligntheimmediaterewardsofeachchoicewithitslong-termvalue,potentiallyenablingpeopletorelyonautomatic,short-sighteddecisionmechanismsinsteadofhavingtooverridethemandinsteadengageeffortfullong-termplanning(Lieder&Griffiths2016).Theseapproachesofferusefuldirectionsforpoliciesaimedatimprovingpeople’sabilitytodeploycognitiveeffortadaptivelywithincomplexenvironments.
Formalmodelsofcontrolallocationofferacriticalpathforwardinunderstandingthemechanismsbywhichindividualssucceedorfailatachievingthedesiredreturnontheircognitiveeffortinvestment.Thesemodelscanprovideinsightintohowtomakethecontrolallocationmostappropriatetothetaskathand---forinstance,byimprovingstrategiesforlearningaboutthevalueofexertingcognitiveeffortandhowitdependsondifferentattributesofthetaskorsituation.Inaddition,suchmodelsmayalsoprovideinsightsintohowtomakecontrollesscostly,forinstancebyallowingpeopletorelymoreonprocessesthatarewelllearnedandoverlapminimallywithotherprocessesthatmayneedtobeengaged.Thesemodelsmightthereforefacilitatethedesignofinterventionstoimprovetheallocationofcognitivecontrolandrestructuretheenvironmenttomaximizethelikelihoodthatattemptsatcontrolsucceedintheirgoals.Advancesinmodelingandempiricallyvalidatingsuchinterventionswillredoundtothebenefitofour
23
understandingofthemechanismsunderlyingcognitiveeffortallocationand,moregenerally,toansweringtheage-oldquestionofwhatmakesithardtothinkandwhatwecandoaboutit.
DISCLOSURESTATEMENT
Theauthorsarenotawareofanyaffiliations,memberships,funding,orfinancialholdingsthatmightbeperceivedasaffectingtheobjectivityofthisreview.
ACKNOWLEDGMENTS
TheauthorswouldliketothankCeydaSayaliandAndrewWestbrookforhelpfulcommentsonanearlierdraftofthisreview.
LITERATURECITED
AartsE,RoelofsA,FrankeB,RijpkemaM,FernándezG,etal.2010.Striataldopaminemediatestheinterfacebetweenmotivationalandcognitivecontrolinhumans:evidencefromgeneticimaging.Neuropsychopharmacology35:1943--51
AlexanderWH,BrownJW.2015.Hierarchicalerror representation:acomputationalmodelofanteriorcingulateanddorsolateralprefrontalcortex.NeuralComput.27:2354--410
Allport A, Styles EA, Hsieh S. 1994. Shifting intentional set: exploring the dynamic control oftasks. In Attention and Performance XV: Conscious and Nonconscious InformationProcessing,ed.CUmiltà,MMoscovitch,pp.421--52.Cambridge,MA:MITPress
AllportA,WylieG.1999.Task-switching:positiveandnegativeprimingoftask-set.InAttention,Space, and Action: Studies in Cognitive Neuroscience, ed. GW Humphreys, J Duncan, ATreisman,pp.273--96.NewYork:OxfordUniv.Press
AllportDA.1980.Attentionandperformance. InCognitivePsychology:NewDirections ed.GIClaxton,pp.112--53.London:Routledge
AllportDA,AntonisB,ReynoldsP.1972.Onthedivisionofattention:adisproofof thesinglechannelhypothesis.Q.J.Exp.Psychol.24:225--35
AndersonJR.1983.TheArchitectureofCognition.Cambridge,MA:HarvardUniv.Press
AndersonJR.1996.ACT:asimpletheoryofcomplexcognition.Am.Psychol.51:355--65
Aston-Jones G, Cohen JD. 2005. An integrative theory of locus coeruleus-norepinephrinefunction:adaptivegainandoptimalperformance.Annu.Rev.Neurosci.28:403--50
Baumeister RF, Bratslavsky E,MuravenM, TiceDM. 1998. Ego depletion: Is the active self alimitedresource?J.Pers.Soc.Psychol.74:1252--65
BaumeisterRF,HeathertonTF.1996.Self-regulationfailure:anoverview.Psychol.Inq.7:1--15
24
BengioY,CourvilleA,VincentP.2013.Representationlearning:areviewandnewperspectives.IEEETrans.PatternAnal.Mach.Intell.35:1798--828
BlainB,HollardG,PessiglioneM.2016.Neuralmechanismsunderlyingthe impactofdaylongcognitiveworkoneconomicdecisions.PNAS113:6967--72
Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD. 2006. The physics of optimal decisionmaking:aformalanalysisofmodelsofperformanceintwo-alternativeforced-choicetasks.Psychol.Rev.113:700--65
BonnelleV,Manohar S, Behrens T,HusainM. 2015. Individual differences in premotor brainsystemsunderliebehavioralapathy.Cereb.Cortex26:807--19
Bonner SE, Sprinkle GB. 2002. The effects of monetary incentives on effort and taskperformance: theories, evidence, and a framework for research. Account. Organ. Soc.27:303--45
Botvinick MM, Braver T. 2015. Motivation and cognitive control: from behavior to neuralmechanism.Annu.Rev.Psychol.66:83--113
Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD. 2001. Conflict monitoring andcognitivecontrol.Psychol.Rev.108:624--52
Botvinick MM, Cohen JD. 2015. The computational and neural basis of cognitive control:chartedterritoryandnewfrontiers.Cogn.Sci.38:1249--85
BotvinickMM,HuffstetlerS,McGuireJ.2009a.Effortdiscountinginhumannucleusaccumbens.Cogn.Affect.Behav.Neurosci.9:16--27
Botvinick MM, Niv Y, Barto AC. 2009b. Hierarchically organized behavior and its neuralfoundations:areinforcementlearningperspective.Cognition113:262--80
Braver TS. 2012. The variable nature of cognitive control: a dual mechanisms framework.TrendsCogn.Sci.16:105--12
Brown JW, Braver TS. 2005. Learned predictions of error likelihood in the anterior cingulatecortex.Science307:1118--21
BushG,LuuP,PosnerM.2000.Cognitiveandemotionalinfluencesinanteriorcingulatecortex.TrendsCogn.Sci.4:215--22
CabanacM.2006.Exertionandpleasurefromanevolutionaryperspective.InPsychobiologyofPhysicalActivity,ed.EOAcevedo,PEkkekakis,pp.79--89.Champaign,IL:Hum.Kinet.
CacioppoJT,PettyRE.1982.Theneedforcognition.JPers.Soc.Psychol.42:116--31
CaiW,ChenT,RyaliS,KochalkaJ,LiC-SR,MenonV.2016.Causalinteractionswithinafrontal-cingulate-parietalnetworkduringcognitivecontrol:convergentevidencefromamultisite--multitaskinvestigation.Cereb.Cortex26:2140--53
CamererC,HogarthR. 1999. Theeffectsof financial incentives inexperiments: a reviewandcapital-labor-productionframework.J.RiskUncertain.19:7--42
25
CarterEC,KoflerLM,ForsterDE.2015.Aseriesofmeta-analytictestsofthedepletioneffect:Self-controldoesnotseemtorelyonalimitedresource.J.Exp.Psychol.Gen.144:796--815
CaruanaR.1998.Multitasklearning.InLearningtoLearn,ed.SThrun,LPratt,pp.95--133.NewYork:Springer
Casey BJ, Somerville LH, Gotlib IH, Ayduk O, Franklin NT, et al. 2011. Behavioral and neuralcorrelatesofdelayofgratification40yearslater.PNAS108:14998--5003
CavanaghJF,FrankMJ.2014.Frontalthetaasamechanismforcognitivecontrol.TrendsCogn.Sci.18:414--21
Cavanagh JF, Masters SE, Bath K, Frank MJ. 2014. Conflict acts as an implicit cost inreinforcementlearning.Nat.Commun.5:5394
CavanaghJF,WieckiTV,CohenMX,FigueroaCM,SamantaJ,etal.2011.Subthalamicnucleusstimulation reverses mediofrontal influence over decision threshold. Nat. Neurosci.14:1462--67
ChristieST,SchraterP.2015.Cognitivecostasdynamicallocationofenergeticresources.Front.Neurosci.9:289
Cohen JD. 2005. The vulcanization of the human brain: a neural perspective on interactionsbetweencognitionandemotion.J.Econ.Perspect.19:3--24
Cohen JD, Dunbar K,McClelland JL. 1990. On the control of automatic processes: a paralleldistributedprocessingaccountoftheStroopeffect.Psychol.Rev.97:332--61
CoolsR.2016.Thecostsandbenefitsofbraindopamineforcognitivecontrol.WileyInterdiscip.Rev.Cogn.Sci.7:317--29
CowanN.2012.WorkingMemoryCapacity.NewYork:PsychologyPress
CowanN,Rouder JN,BlumeCL, Saults JS. 2012.Models of verbalworkingmemory capacity:Whatdoesittaketomakethemwork?Psychol.Rev.119:480--99
CraigAD.2009.Howdoyoufeel—now?Theanteriorinsulaandhumanawareness.Nat.Rev.Neurosci.10:59--70
CritchleyHD,MathiasCJ,JosephsO,O’DohertyJP,ZaniniS,etal.2003.Humancingulatecortexandautonomiccontrol:convergingneuroimagingandclinicalevidence.Brain126:2139--52
Croxson PL, WaltonME, O'Reilly JX, Behrens TEJ, RushworthMFS. 2009. Effort-based cost--benefitvaluationandthehumanbrain.J.Neurosci.29:4531--41
Desender K, VanOpstal F, Van den Bussche E. 2014. Feeling the conflict: the crucial role ofconflictexperienceinadaptation.Psychol.Sci.25:675--83
DixonML, Christoff K. 2012. The decision to engage cognitive control is driven by expectedreward-value:neuralandbehavioralevidence.PLOSONE7:e51637
Dosenbach NUF, Fair DA, Cohen AL, Schlaggar BL, Petersen SE. 2008. A dual-networksarchitectureoftop-downcontrol.TrendsCogn.Sci.12:99--105
26
DosenbachNUF,VisscherKM,PalmerED,MiezinFM,WengerKK,etal.2006.Acoresystemfortheimplementationoftasksets.Neuron50:799--812
Dreisbach G, Fischer R. 2015. Conflicts as aversive signals for control adaptation. Curr. Dir.Psychol.Sci.24:255--60
DuckworthAL,PetersonC,MatthewsMD,KellyDR.2007.Grit:perseveranceandpassion forlong-termgoals.JPers.Soc.Psychol.92:1087--101
Duckworth AL, Quinn PD, Lynam DR, Loeber R, Stouthamer-Loeber M. 2011. Role of testmotivationinintelligencetesting.PNAS108:7716--20
DuncanJ.2010.Themultiple-demand(MD)systemoftheprimatebrain:mentalprogramsforintelligentbehaviour.TrendsCogn.Sci.14:172--79
DunnTL,LutesDJ,RiskoEF.2016.Metacognitiveevaluationintheavoidanceofdemand.J.Exp.Psychol.42:1372--87
EastwoodJD,FrischenA,FenskeMJ,SmilekD.2012.Theunengagedmind:definingboredomintermsofattention.Perspect.Psychol.Sci.7:482--95
Eigsti I, ZayasV,MischelW, ShodaY,AydukO,et al. 2006.Predicting cognitive control frompreschooltolateadolescenceandyoungadulthood.Psychol.Sci.17:478--84
EisenbergerR.1992.Learnedindustriousness.Psychol.Rev.99:248--67
Eldar E, Cohen JD, Niv Y. 2013. The effects of neural gain on attention and learning. Nat.Neurosci.16:1146--53
Elkins-BrownN,SaundersB,InzlichtM.2015.Error-relatedelectromyographicactivityoverthecorrugator supercilii is associatedwith neural performancemonitoring. Psychophysiology53:159--70
Elman, J. L. (1993). Learning and development in neural networks: the importance of starting small. Cognition, 48(1), 71–99.
ElmoreLC,MaWJ,MagnottiJF,LeisingKJ,PassaroAD,etal.2011.Visualshort-termmemorycomparedinrhesusmonkeysandhumans.Curr.Biol.21:975--79
Engelmann JB, Damaraju E, Padmala S, Pessoa L. 2009. Combined effects of attention andmotivationonvisualtaskperformance:transientandsustainedmotivationaleffects.Front.Hum.Neurosci.3:4
FellowsLK,FarahMJ.2005. Isanteriorcingulatecortexnecessaryforcognitivecontrol?Brain128:788--96
Feng SF, Schwemmer M, Gershman SJ, Cohen JD. 2014. Multitasking versus multiplexing:toward a normative account of limitations in the simultaneous execution of control-demandingbehaviors.Cogn.Affect.Behav.Neurosci.14:129--46
FrankMJ,BadreD.2012.Mechanismsofhierarchical reinforcement learning incorticostriatalcircuits1:computationalanalysis.Cereb.Cortex22:509--26
27
GailliotMT,BaumeisterRF,DeWallCN,Maner JK,PlantEA,etal.2007.Self-control reliesonglucose as a limited energy source: Willpower is more than a metaphor. J. Pers. Soc.Psychol.92:325--36
Gan, J.O.,Walton,M. E.,& Phillips, P. E.M. 2010.Dissociable cost and benefit encoding offuturerewardsbymesolimbicdopamine.NatureNeuroscience,13:25--27.
GarnerKG,DuxPE.2015.Trainingconquersmultitaskingcostsbydividingtaskrepresentationsinthefrontoparietal-subcorticalsystem.PNAS112:14372--77
GeanaA,WilsonRC,DawND,CohenJD.2016a.Boredom,information-seekingandexploration.Proc.38thAnnu.Conf.Cogn.Sci.Soc.,Philadelphia,pp.1751--56
GeanaA,WilsonRC,DawND,CohenJD.2016b.Information-seeking,learningandthemarginalvaluetheorem:anormativeapproachtoadaptiveexploration.Proc.38thAnnu.Conf.Cogn.Sci.Soc.,Philadelphia,pp.1793--98
Gershman SJ, Horvitz EJ, Tenenbaum JB. 2015. Computational rationality: a convergingparadigmforintelligenceinbrains,minds,andmachines.Science349:273--78
Gigerenzer G. 2008.Rationality forMortals: How People Copewith Uncertainty. Oxford, UK:OxfordUniv.Press
GoldJM,KoolW,BotvinickMM,HubzinL,AugustS,WaltzJA.2014.Cognitiveeffortavoidanceanddetectioninpeoplewithschizophrenia.Cogn.Affect.Behav.Neurosci.15:145--54
GraybielAM.2008.Habits,rituals,andtheevaluativebrain.Annu.Rev.Neurosci.31:359--87
GriffithsTL,LiederF,GoodmanND.2015.Rationaluseofcognitiveresources:levelsofanalysisbetweenthecomputationalandthealgorithmic.Top.Cogn.Sci.7:217--29
HaggerMS,ChatzisarantisNLD.2016.Amultilabpreregisteredreplicationoftheego-depletioneffect.Perspect.Psychol.Sci.11:546--73
HaggerMS,WoodC,StiffC,ChatzisarantisNLD.2010.Egodepletionandthestrengthmodelofself-control:ameta-analysis.Psychol.Bull.136:495--525
HayN,RussellS,TolpinD,ShimonySE.2012.Selectingcomputations:theoryandapplications.Uncertain. Artif. Intell. Proc. Twenty-Eighth Conf., Catalina Isl., Aug. 15--17, pp. 346--55.Corvallis,OR:AUAIPress
Heatherton TF, Wagner DD. 2011. Cognitive neuroscience of self-regulation failure. TrendsCogn.Sci.15:132--39
Heilbronner SR, Hayden BY. 2016. Dorsal anterior cingulate cortex: a bottom-up view.Annu.Rev.Neurosci.39:149--70
HillmanKL,BilkeyDK.2010.Neurons in the rat anterior cingulate cortexdynamically encodecost-benefitinaspatialdecision-makingtask.J.Neurosci.30:7705--13
HockeyGRJ.1997.Compensatorycontrolintheregulationofhumanperformanceunderstressandhighworkload:acognitive-energeticalframework.Biol.Psychol.45:73--93
28
Hockey GRJ. 2011. A motivational control theory of cognitive fatigue. In Cognitive Fatigue:Multidisciplinary Perspectives on Current Research and Future Applications, ed. PLAckerman,pp.167--87.Washington,DC:Am.Psychol.Assoc.
HolroydCB.2015.Thewastedisposalproblemofeffortfulcontrol.InMotivationandCognitiveControl,ed.TSBraver,pp.235--60.NewYork:Routledge
Holroyd CB, Coles MGH. 2002. The neural basis of human error processing: reinforcementlearning,dopamine,andtheerror-relatednegativity.Psychol.Rev.109:679--709
HolroydCB,McClureSM.2015.Hierarchical controlovereffortfulbehaviorby rodentmedialfrontalcortex:acomputationalmodel.Psychol.Rev.122:54--83
HolroydCB,UmemotoA.2016.Theresearchdomaincriteriaframework:thecaseforanteriorcingulatecortex.Neurosci.Behav.Rev.71:418--43
Holroyd CB, Yeung N. 2012. Motivation of extended behaviors by anterior cingulate cortex.TrendsCogn.Sci.16:121--27
Horvitz E, Rutledge G. 1991. Time-dependent utility and action under uncertainty. In Proc.SeventhConf.Uncertain.Artif.Intell.,LosAngeles,pp.151--58.Corvallis,OR:AUAIPress
HoskingJG,FlorescoSB,WinstanleyCA.2014.Dopamineantagonismdecreaseswillingnesstoexpend physical, but not cognitive, effort: a comparison of two rodent cost/benefitdecision-makingtasks.Neuropsychopharmacology40:1005—15
Huang,H.J.,Kram,R.,Ahmed,A.A.2012.ReductionofMetabolicCostduringMotorLearningofArmReachingDynamics.J.Neurosci.32:2182—2190.
InzlichtM, BartholowBD,Hirsh JB. 2015. Emotional foundations of cognitive control.TrendsCogn.Sci.19:126--32
InzlichtM,SchmeichelBJ.2012.What isegodepletion?Towardamechanisticrevisionoftheresourcemodelofself-control.Perspect.Psychol.Sci.7:450--63
JamesW.1891.ThePrinciplesofPsychology,Volume2.London:Macmillan
Jensen AR. 1988. Speed of information processing and population differences. In HumanAbilitiesinCulturalContext,ed.SHIrvine,JWBerry,pp.105--45.Cambridge,UK:CambridgeUniv.Press
Jepma M, Nieuwenhuis S. 2011. Pupil diameter predicts changes in the exploration--exploitationtrade-off:evidencefortheadaptivegaintheory.J.Cogn.Neurosci.23:1587--96
KahnemanD.1973.AttentionandEffort.EnglewoodCliffs,NJ:Prentice-Hall
KahnemanD.2003.Aperspectiveonjudgmentandchoice:mappingboundedrationality.Am.Psychol.58:697--720
KahnemanD,BeattyJ.1966.Pupildiameterandloadonmemory.Science154:1583--85
KahnemanD,TverskyA.1979.Prospecttheory:ananalysisofdecisionunderrisk.Econometrica47:263--91
29
Kaping D, VinckM, Hutchison RM, Everling S,Womelsdorf T. 2011. Specific contributions ofventromedial,anteriorcingulate,andlateralprefrontalcortexforattentionalselectionandstimulusvaluation.PLOSBiol.9:e1001224
Kerns JG,Cohen JD,MacDonaldAW,ChoRY, StengerVA,CarterCS.2004.Anterior cingulateconflictmonitoringandadjustmentsincontrol.Science303:1023--26
Keuken MC, van Maanen L, Bogacz R, Schäfer A, Neumann J, et al. 2015. The subthalamicnucleusduringdecision-makingwithmultiplealternatives.Hum.BrainMapp.36:4041--52
KoolW,McGuire JT, Rosen ZB, BotvinickMM. 2010. Decisionmaking and the avoidance ofcognitivedemand.J.Exp.Psychol.:Gen.139:665--82
Kouneiher F, Charron S, Koechlin E. 2009. Motivation and cognitive control in the humanprefrontalcortex.Nat.Neurosci.12:939--45
Krebs RM, Boehler CN,WoldorffMG. 2010. The influence of reward associations on conflictprocessingintheStrooptask.Cognition117:341--47
Kurzban R, Duckworth A, Kable JW,Myers J. 2013. An opportunity costmodel of subjectiveeffortandtaskperformance.Behav.BrainSci.36:661--79
LeCunY,BengioY,HintonG.2015.Deeplearning.Nature521:436--44
LewisRL,HowesA,SinghS.2014.Computationalrationality: linkingmechanismandbehaviorthroughboundedutilitymaximization.Top.Cogn.Sci.6:279--311
Lieder F, Griffiths TL. 2015.When to usewhich heuristic: a rational solution to the strategyselectionproblem.Proc.37thAnnu.Conf.Cogn.Sci.Soc.,Austin,TX,pp.1362--67
LiederF,GriffithsTL.2016.Helpingpeoplemakebetterdecisionsusingoptimalgamification.Proc.38thAnnu.Conf.Cogn.Sci.Soc.,Philadelphia,pp.2075--80
LiederF,GriffithsTL,GoodmanND.2012.Burn-in,bias,andtherationalityofanchoring.Adv.NeuralInf.Process.Syst.25(NIPS2012),LakeTahoe,NV,pp.2690–2798
LiederF,PlunkettD,Hamrick JB,Russell SJ,HayNJ,GriffithsTL.2014.Algorithmselectionbyrationalmetareasoning as amodel of human strategy selection.Adv.Neural Inf. Process.Syst.27(NIPS2014),Montreal,pp.2870--78
Logan GD. 1985. Skill and automaticity: relations, implications, and future directions.Can. J.Psychol.39:367--86
LuckSJ,VogelEK.1997.Thecapacityofvisualworkingmemoryforfeaturesandconjunctions.Nature390:279--81
Ma WJ, Husain M, Bays PM. 2014. Changing concepts of working memory. Nat. Neurosci.17:347--56
MagnoE,FoxeJJ,MolholmS,RobertsonIH,GaravanH.2006.Theanteriorcingulateanderroravoidance.J.Neurosci.26:4769—73
30
Mansouri, F. A., Egner, T., & Buckley, M. J. (in press). Monitoring Demands for ExecutiveControl: Shared Functions between Human and Nonhuman Primates. Trends inNeurosciences.
Marcora SM. 2009. Perception of effort during exercise is independent of afferent feedbackfromskeletalmuscles,heart,andlungs.J.Appl.Physiol.106:2060--62
MarcoraSM,StaianoW.2010.The limit toexercise tolerance inhumans:mindovermuscle?Eur.J.Appl.Physiol.109:763--70
Marcora SM, Staiano W, Manning V. 2009. Mental fatigue impairs physical performance inhumans.J.Appl.Physiol.106:857--64
McCarthyJ.1959.Programswithcommonsense.Proc.Symp.Mech.ThoughtProcess.,Vol.1,pp.75--91.London:HerMajesty’sStation.Off.
McCarthy J, Hayes PJ. 1969. Some philosophical problems from the standpoint of artificialintelligence.InReadingsinArtificialInteligence.,ed.BLWebberNJNilsson,pp.431–50,LosAltos,CA:MorganKaufmannPublishers,Inc.
McGuire JT, BotvinickMM.2010. Prefrontal cortex, cognitive control, and the registrationofdecisioncosts.PNAS107:7922--26
Medford N, Critchley HD. 2010. Conjoint activity of anterior insular and anterior cingulatecortex:awarenessandresponse.BrainStruct.Funct.214:535--49
MenonV,UddinLQ.2010.Saliency,switching,attentionandcontrol:anetworkmodelofinsulafunction.BrainStruct.Funct.214:655--67
Miller EK, Cohen JD. 2001. An integrative theory of prefrontal cortex function. Annu. Rev.Neurosci.24:167--202
MillerGA.1956.Themagicalnumberseven,plusorminustwo:somelimitsonourcapacityforprocessinginformation.Psychol.Rev.63:81--97
MischelW,ShodaY,RodriguezML.1989.Delayofgratificationinchildren.Science244:933--38
Molden DC, Hui CM, Scholer AA, Meier BP, Noreen EE, et al. 2012. Motivational versusmetaboliceffectsofcarbohydratesonself-control.Psychol.Sci.10:1137--44
MonsellS.2003.Taskswitching.TrendsCogn.Sci.7:134--40
MulertC,MenzingerE,LeichtG,PogarellO,HegerlU.2005.Evidenceforacloserelationshipbetweenconsciouseffortandanteriorcingulatecortexactivity.Int.J.Psychophysiol.56:65--80
Muraven M, Tice DM, Baumeister RF. 1998. Self-control as limited resource: regulatorydepletionpatterns.J.Pers.Soc.Psychol.74:774--89
MusslickS,DeyB,ÖczimderK,PatwaryMMA,WillkeTL,CohenJD.2016a.Parallelprocessingcapabilityversusefficiencyofrepresentation inneuralnetworks.Presentedat15thNeuralComput.Psychol.Workshop,Aug.8--9,Philadelphia
31
MusslickS,DeyB,ÖzcimderK,PatwaryMMA,WillkeTL,CohenJD.2016b.Controlledversus.automaticprocessing: a graph-theoretic approach to theanalysisof serial versus.parallelprocessing in neural network architectures. Proc. 38th Annu. Conf. Cogn. Sci. Soc.,Philadelphia,pp.1547--52
Musslick S, Shenhav A, Botvinick MM, Cohen JD. 2015. A computational model of controlallocationbasedontheexpectedvalueofcontrol.Presentedat2ndMultidiscip.Conf.Reinf.Learn.Decis.Mak.,June7--10,Edmonton,Can.
NaccacheL,DehaeneS,CohenL,HabertM-O,Guichart-GomezE,etal.2005.Effortlesscontrol:executive attention and conscious feeling of mental effort are dissociable.Neuropsychologia43:1318--28
NairneJS.1990.Afeaturemodelofimmediatememory.Mem.Cogn.18:251--69
Nakamura J, Csikszentmihalyi M. 2002. The concept of flow. In Handbook of PositivePsychology,ed.CRSnyder,SJLopez,pp.89--105.Oxford,UK:OxfordUniv.Press
Navon D, Gopher D. 1979. On the economy of the human-processing system. Psychol. Rev.86:214--55
Nicola SM. 2010. The flexible approach hypothesis: unification of effort and cue-respondinghypothesesfortheroleofnucleusaccumbensdopamineintheactivationofreward-seekingbehavior.J.Neurosci.30:16585--600
Nieuwenhuis S, Schweizer TS, Mars RB, Botvinick MM, Hajcak G. 2007. Error-likelihoodpredictioninthemedialfrontalcortex:acriticalevaluation.Cereb.Cortex17:1570--81
NivY,DawND, JoelD,DayanP.2007.Tonicdopamine:opportunity costsand thecontrolofresponsevigor.Psychopharmacology191:507--20
Oaksford M, Chater N. 2007. Bayesian Rationality: The Probabilistic Approach to HumanReasoning.Oxford,UK:OxfordUniv.Press
OberauerK,FarrellS,JarroldC,LewandowskyS.2016.Whatlimitsworkingmemorycapacity?Psychol.Bull.142:758--99
OberauerK,KlieglR.2006.Aformalmodelofcapacitylimitsinworkingmemory.J.Mem.Lang.55:601--26
Oehrn CR, Hanslmayr S, Fell J, Deuker L, Kremers NA, et al. 2014. Neural communicationpatternsunderlyingconflictdetection,resolution,andadaptation.J.Neurosci.34:10438--52
O'ReillyRC,FrankMJ.2006.Makingworkingmemorywork:acomputationalmodeloflearningintheprefrontalcortexandbasalganglia.NeuralComput.18:283--328
Ortega PA, Braun DA. 2011. Information, utility and bounded rationality.Proc. 4th Int. Conf.Artif.Gen.Intell.,MountainView,CA,Aug.3--6,pp.269--74.Berlin:Springer-Verlag
OrtegaPA,BraunDA.2013.Thermodynamicsasatheoryofdecision-makingwithinformation-processingcosts.Proc.R.Soc.A469:20120683
32
OudB,Krajbich I,MillerK,CheongJH,BotvinickM,FehrE.2016. Irrational timeallocation indecision-making.Proc.R.Soc.B283:20151439
Padmala S, Pessoa L. 2011. Reward reduces conflict by enhancing attentional control andbiasingvisualcorticalprocessing.J.Cogn.Neurosci.23:3419--32
PageM,NorrisD.1998.Theprimacymodel:anewmodelof immediateserialrecall.Psychol.Rev.105:761--81
ParviziJ,RangarajanV,ShirerWR,DesaiN,GreiciusMD.2013.Thewilltopersevereinducedbyelectricalstimulationofthehumancingulategyrus.Neuron80:1359--67
Pasquereau B, Turner RS. 2013. Limited encoding of effort by dopamine neurons in a cost-benefittrade-offtask.J.Neurosci.33:8288--300
Payne, J.W., Bettman, J. R., & Johnson, E. L. (1988). Adaptive strategy selection in decisionmaking. Journal of Experimental Psychology: Learning,Memory, and Cognition, 14, 534–552.
PosnerMI, SnyderCRR.1975.Attentionand cognitive control. In InformationProcessingandCognition:TheLoyolaSymposium,ed.RLSolso.Hillsdale,NJ:ErlbaumAssoc.
Power JD, Petersen SE. 2013. Control-related systems in the human brain. Curr. Opin.Neurobiol.23:223--28
Prévost C, Pessiglione M, Météreau E, Cléry-Melin M, Dreher J. 2010. Separate valuationsubsystemsfordelayandeffortdecisioncosts.J.Neurosci.30:14080--90
RatcliffR.1978.Atheoryofmemoryretrieval.Psychol.Rev.85:59--108
Reinhart RMG, Woodman GF. 2014. Causal control of medial-frontal cortex governselectrophysiological and behavioral indices of performance monitoring and learning. J.Neurosci.34:4214--27
RidderinkhofKR,UllspergerM,CroneEA,NieuwenhuisS.2004.Theroleofthemedialfrontalcortexincognitivecontrol.Science306:443--47
Rumelhart DE, McClelland JL. 1986. PDP models and general issues in cognitive science. InParallelDistributedProcessing:Explorations in theMicrostructureofCognition,Vol.1,ed.DERumelhart,JLMcClelland,pp.110--46.Cambridge,MA:MITPress
RushworthMFS,NoonanMP,BoormanED,WaltonME,BehrensTE.2011.Frontalcortexandreward-guidedlearninganddecision-making.Neuron70:1054--69
RushworthMFS,WaltonME,KennerleySW,BannermanD.2004.Actionsetsanddecisions inthemedialfrontalcortex.TrendsCogn.Sci.8:410--17
RussellSJ,SubramanianD.1995.Provablybounded-optimalagents.J.Artif.Intell.Res.2:575--609
RussellSJ,WefaldE.1991.Principlesofmetareasoning.Artif.Intell.49:361--95
SalamoneJD,CorreaM,FarrarAM,NunesEJ,PardoM.2009.Dopamine,behavioraleconomics,andeffort.Front.Behav.Neurosci.3:13
33
Salamone JD, Yohn SE, López-Cruz L, SanMiguel N, CorreaM. 2016. Activational and effort-related aspects of motivation: neural mechanisms and implications for psychopathology.Brain139:1325--47
SaundersB,MilyavskayaM, InzlichtM.2015.Whatdoescognitivecontrol feel like?Effectiveand ineffective cognitive control is associated with divergent phenomenology.Psychophysiology52:1205--17
SchmidtL,LebretonM,Cléry-MelinM-L,DaunizeauJ,PessiglioneM.2012.Neuralmechanismsunderlyingmotivationofmentalversusphysicaleffort.PLOSBiol.10:e1001266
ShackmanAJ,SalomonsTV,SlagterHA,FoxAS,WinterJJ,DavidsonRJ.2011.Theintegrationofnegative affect, pain and cognitive control in the cingulate cortex. Nat. Rev. Neurosci.12:154--67
ShafferLH.1975.Multipleattentionincontinuousverbaltasks.InAttentionandPerformanceVed.PMARabbitt,SDornic,pp.157--67.London:Academic
ShenC,ArdidS,KapingD,WestendorffS,EverlingS,WomelsdorfT.2014.Anterior cingulatecortex cells identify process-specific errors of attentional control prior to transientprefrontal-cingulateinhibition.Cereb.Cortex25:2213--28
ShenhavA,BotvinickMM,CohenJD.2013.Theexpectedvalueofcontrol:anintegrativetheoryofanteriorcingulatecortexfunction.Neuron79:217--40
ShenhavA,Cohen JD,BotvinickMM.2016.Dorsal anterior cingulate cortex and the valueofcontrol.Nat.Neurosci.19:1286--91
Sheth SA, Mian MK, Patel SR, Asaad WF, Williams ZM, et al. 2012. Human dorsal anteriorcingulatecortexneuronsmediateongoingbehaviouraladaptation.Nature488:218--21
Shiffrin RM, Schneider W. 1977. Controlled and automatic information processing: II.Perceptuallearning,automaticattending,andageneraltheory.Psychol.Rev.84:127--90
SimonHA.1956.Rationalchoiceandthestructureoftheenvironment.Psychol.Rev.63:129--38
SimonHA.1982.ModelsofBoundedRationality,Vol.3:EmpiricallyGroundedEconomicReason.Cambridge,MA:MITPress
Spunt RP, Lieberman MD, Cohen JR, Eisenberger NI. 2012. The phenomenology of errorprocessing:thedorsalACCresponsetostop-signalerrorstracksreportsofnegativeaffect.J.Cogn.Neurosci.24:1753--65
SternbergS.1969.Memory-scanning:mentalprocessesrevealedbyreaction-timeexperiments.Am.Sci.57:421--57
Stuss DT. 2011. Functions of the frontal lobes: relation to executive functions. J. Int.Neuropsychol.Soc.17:759--65
SuttonRS,BartoAG.1998.ReinforcementLearning:AnIntroduction.Cambridge,MA:MITPress
TanakaM, Ishii A,WatanabeY. 2014.Neural effects ofmental fatigue causedby continuousattentionload:amagnetoencephalographystudy.BrainRes.1561:60--66
34
Tang H, Yu H-Y, Chou C-C, Crone NE,Madsen JR, et al. 2016. Cascade of neural processingorchestratescognitivecontrolinhumanfrontalcortex.eLife5:4671
Tangney JP,BaumeisterRF,BooneAL. 2004.High self-control predicts goodadjustment, lesspathology,bettergrades,andinterpersonalsuccess.J.Personal.72:271--324
Thaler RH, Sunstein CR. 2008. Nudge: Improving Decisions about Health, Wealth, andHappiness.NewHaven:YaleUniv.Press
Todd MT, Niv Y, Cohen JD. 2009. Learning to use working memory in partially observableenvironmentsthroughdopaminergicreinforcement.Adv.NeuralInf.Process.Syst.21:1689--96
Todd PM, Gigerenzer G. 2012. Ecological Rationality: Intelligence in the World. Oxford, UK:OxfordUniv.Press
Tomlin D, Rand DG, Ludvig EA, Cohen JD. 2015. The evolution and devolution of cognitivecontrol:thecostsofdeliberationinacompetitiveworld.Sci.Rep.5:11002
Tversky A, Kahneman D. 1974. Judgment under uncertainty: heuristics and biases. Science185:1124--31
UllspergerM,DanielmeierC,JochamG.2014.Neurophysiologyofperformancemonitoringandadaptivebehavior.Physiol.Rev.94:35--79
UllspergerM,HarsayHA,WesselJR,RidderinkhofKR.2010.Consciousperceptionoferrorsanditsrelationtotheanteriorinsula.BrainStruct.Funct.214:629—43
Ullsperger, M., Danielmeier, C., & Jocham, G. 2014. Neurophysiology of Performance Monitoring and Adaptive Behavior. Physiological Reviews. 94: 35–79.
UmemotoA,HolroydCB.2014.Task-specificeffectsofrewardontaskswitching.Psychol.Res.698--707
VadilloMA,GoldN,OsmanM.2016.Thebitter truthabout sugarandwillpower: the limitedevidentialvalueoftheglucosemodelofegodepletion.Psychol.Sci.27:1207--14
VarazzaniC,San-GalliA,GilardeauS,BouretS.2015.Noradrenalineanddopamineneuronsinthereward/efforttrade-off:adirectelectrophysiologicalcomparisoninbehavingmonkeys.J.Neurosci.35:7866--77
Verguts T, Vassena E, Silvetti M. 2015. Adaptive effort investment in cognitive and physicaltasks:Aneurocomputationalmodel.Front.Behav.Neurosci.9:57
Vogt BA. 2016. Midcingulate cortex: Structure, connections, homologies, functions anddiseases.J.Chem.Neuroanat.74:28--46
VonNeumannJ,MorgensternO.1944.TheTheoryofGamesandEconomicBehavior.Princeton,NJ:PrincetonUniv.Press
VulE,GoodmanND,GriffithsTL,TenenbaumJB.2014.Oneanddone?Optimaldecisionsfromveryfewsamples.Cogn.Sci.38:599--637
35
WaltonM,RudebeckPH,BannermanD,RushworthMFS.2007.Calculatingthecostofactinginfrontalcortex.Ann.N.Y.Acad.Sci.1104:340--56
Wang C, Trongnetrpunya A, Samuel IBH, Ding M, Kluger BM. 2016. Compensatory neuralactivityinresponsetocognitivefatigue.J.Neurosci.36:3919--24
WesselJR,DanielmeierC,MortonJB,UllspergerM.2012.Surpriseanderror:commonneuronalarchitecturefortheprocessingoferrorsandnovelty.J.Neurosci.32:7528--37
Westbrook A, Braver TS. 2015. Cognitive effort: a neuroeconomic approach. Cogn. Affect.Behav.Neurosci.15:395--415
Westbrook A, Braver TS. 2016. Dopamine does double duty in motivating cognitive effort.Neuron89:695--710
WestbrookA,KesterD,BraverTS.2013.Whatisthesubjectivecostofcognitiveeffort?Load,trait,andagingeffectsrevealedbyeconomicpreference.PLOSONE8:e68210
Wickens DD. 1984. Processing resources in attention. In Varieties of Attention, ed. RParasuraman,DRDavies,JBeatty,pp.63--102.NewYork:Academic
WolpertDM,LandyMS.2012.Motorcontrolisdecision-making.Curr.Opin.Neurobiol.22:996--1003
WomelsdorfT,JohnstonK,VinckM,EverlingS.2010.Theta-activityinanteriorcingulatecortexpredictstaskrulesandtheiradjustmentsfollowingerrors.PNAS107:5248--53
ZakayD.2014.Psychologicaltimeasinformation:thecaseofboredom.Front.Psychol.5:917
ZenonA, SidibeM,Olivier E. 2015.Disrupting the supplementarymotorareamakesphysicaleffortappearlesseffortful.J.Neurosci.35:8737--44
View publication statsView publication stats