@JackS'lgoe
Machinelearning,sociallearningandself-drivingcars
Outline
• Responsibleinnova'onassociallearning• Socialisingmachinelearning• Self-drivingasa‘solvableproblem’• Self-drivingasmisnomer• Limitstoresponsiveness
Source:electrek.com
“Perfectsafetyisreallyanimpossiblegoal.”ElonMusk,Sept2016
Being“inBeta”“Becausetheprobabilityissosmall,thereisnoprac'calpossibilityofdeterminingthisfailureratedirectly-i.e.,bybuilding,letussay,1,000reactors,opera'ngthemfor10,000yearsandtabula'ngtheiropera'nghistories”.
(AlvinWeinberg,1972)“Eachnuclearpowerplantisitsowntestcase.Itmaybecategorizedasanimplicitexperiment,anditsmostrevealingcaseistheaccident”.
(KrohnandWeingart,1987)
“Emergingprac'cesdefine‘rules’,ratherthanrulescontrollingprac'ces.“
(Wynne1988)
Thewhat,thehowandthewhyofinnova'on
Products• Whatarethelikelyrisks
andbenefits?• Howwilltherisksand
benefitsbedistributed?• Whatotherimpactscan
wepredict?• Howmightthese
changeinthefuture?• Whatdon’tweknow
about?• Whatmightwenever
knowabout?
Processes• Howshouldresearch
andinnova'ontakeplace?
• Howshouldstandardsbedrawnupandapplied?
• Howshouldrisksandbenefitsbedefinedandmeasured?
• Whoisincontrol?• Whowilltake
responsibilityifthingsgowrong?
• Whatifwearewrong?
Purposes• Whyshouldthis
researchbeundertaken?
• Whowillbenefit?• Whatarethe
alterna'ves?
Responsibleinnova'onis‘collec'vecareforthefuturethroughthestewardshipofinnova'oninthepresent’
(S'lgoe,OwenandMacnaghten2013)(alsoS'lgoeandGuston2016)
An'cipa'on Inclusion
Reflexivity Responsiveness
Responsibleinnova'on
Limitstoresponsiveness
• Inflexibility,irreversibility,commitmentandotherformsoflock-in
• Lackofreflexivity• ‘Ins'tu'onalvoid’(Hajer)• Presump'onsofnoveltyactagainstlearning(Rayner)
Approachestosociallearning
1. Engineering–machineslearnaboutsociety2. Educa'onal–learninghappenssocially3. Ins'tu'onal– socialsystemsneedtolearn– AircrabvsNuclearpower(Perrow)
4. Poli'cal– learningtakesplaceinpublic– ‘Experimentalgovernment’(Deweyetal)
Socialisingmachinelearning
What’snew?
2015
GartnerHypeCycle2015
Levelsofautoma'on
“Howquicklycanwegetthisintopeople’shands?Ifyoureadthepapers,youseemaybeit’sthreeyears,maybeit’sthirtyyears.AndIamheretotellyouthathonestly,it’sabitofboth,”
ChrisUrmson(ex-Google),March2016
‘Solving’self-driving
Source:TheEconomist,2016
Ifsafetyistheques'on,areself-drivingcarstheanswer?
SeeinglikeaTesla
MobileyeandTeslapartcompany“Weneedtobethereonallaspectsofhowthetechnologyisbeingused,andnotsimplyprovidingtechnology”“Thisincidentinvolvedalaterallycrossingvehicle,whichcurrent-genera'onAEB[automa'cemergencybraking]systemsarenotdesignedtoactuateupon”
Mobileye
“AutonomousvehiclesisanAIcompu'ngproblem.It’snotadetec'onproblem”
Nvidia
Modesofautomo'velearningRule-following Rule-making
‘Autonomousvehicles’? Technicalandpoli'calentanglement
Technicalandpoli'cal‘autonomy’
Modesofmachinelearning
Formallogic End-to-endlearning
RelevantExperDse Domainexper'se,ergonomicsetc
Drivingasagame
Pedagogy Tradi'onal Autodidac'c
SocialconsDtuDon Favoursincumbents Upstartsandstart-ups
Strategy Incrementalinnova'on Disrup've,permissionlessinnova'on
Historicaltouchpoints Automatedhighways DARPAGrandChallenges
Escapingtheriskframe
Machinecontrolandsocialcontrol
Alterna'velearning
‘It'samazingtechnology…ButTeslashouldn'tmaketheircustomerstest
pilots… [Autopilot]allowsyoutothinkit'sworking.Andatahigherspeed,youcan'treact.It'salreadyhardenoughtoreact
whenyouarefocused,thatwhenyouarenot,anditerrs,it'sover.’
MarkMolthan,crashedAugust2016
Imagineddeficits
Thetyrannyofurgency
We“cannotwaitforperfect…Weshouldbedesperateforanythingwecanfindtosavepeople'slives,"
NHTSAAdministratorMarkRosekind
“Ifanybodylegislatesagainstit,theyshouldprobablygotojail”
EngineerAlainKornhauser
“Speedisoftheessencehere…We’reinaveryimportantraceagainstAsianandEuropeancountriesthataremovingforwardwiththistechnology”
SenatorGaryPeters
‘Dothemath’‘Ifanyonebotheredtodothemath(obviously,youdidnot)theywouldrealizethatoftheover1Mautodeathsperyearworldwide,approximatelyhalfamillionpeoplewouldhavebeensavediftheTeslaautopilotwasuniversallyavailable.Please,take5minsanddothebloodymathbeforeyouwriteanar'clethatmisleadsthepublic.’
ElonMusk,July2016‘If,inwri'ngsomear'clethat'snega've,you...dissuadepeoplefromusinganautonomousvehicle,you'rekillingpeople’
ElonMusk,October2016
Boundingtheexperiment?Broaderenvironment,economyandsociety
Landscape
Infrastructure
Otherroadusers
Otherdrivers
Self-drivingcaranduser
Ques'onsforsociallearning:aresearchagenda
• Transi'ons• An'cipa'ngtransportsystemsandcarusage– Whatmightbethepressuresonmachinereadableinfrastructure?
• Scale• Thegovernanceofwhat?– Sobware,hardware,system?
An'cipatorygovernance
Histories • Pre-history(Wetmore2003;Vinsel,forthcoming)
– Integra'ngvehiclesandinfrastructure• Automa'onofcarsystems
– Airbags– ABS/Trac'oncontrol– Cruisecontrol– ADAS
• 2004,2005,2007:Darpagrandchallenges• 2008àMachinelearning
– Neuralnetworksandimagerecogni'on– GPUs– Data
Thecomplexityoftransi'ons
• Closed,homogenoussystemsareeasytoimagine;Mixed,opensystemsarealmostimpossible
• Thehand-offproblem• Smarttechcanmakepeopledumb
outline• Teslaphoto
– Hardwareandsobware–poli'csofnovelty– Itisaworkinprogress-learninginpublic–andinnova'ng
• Crash• Emergingsociotechnicalsystem• Responsivenessandsociallearning• ImaginariesofAVs• Governance
– Stateasexperimenter,notjustlaboratory– Safety–limitstoresponsiveness?– Data-sharing
• DataisthecurrencyofAVinnova'on– Condi'onsforexperimenta'on– Sobwarevshardware–productliability– Democra'singimagina'on– Whobenefits?Whatarelikelytobethetransfersofpowerandmoney?Socialcons'tu'on?
Techaslegisla'on?
PerrowontheNTSB
• Whydoaeroplanesgetsaferwhilenuclearpowerdoesn’t?– Accidentsarenormal–notout-of-the-blue
• Learningascrucial– “pastaccidentsalsofailaswarningsifthewarningisavailabletoonlyonepartofthesystem”–andthatwasaproblemwithTMI
Star'ngwithtransport
• Infrastructure?• Condi'onsforacceptability?
U.S.AirCarrierSafetyIndices,1920-1975
‘NeverNeutral’
Ethicsandtrolleyproblems
An'cipa'on• Par'cipatory,notpredic've• Understandingexpecta'ons,promises
Inclusion• Publicengagement• User-driveninnova'on• Value-centreddesign
Reflexivity• 1stand2ndorder
Responsiveness• Answeringandreac'ng• Understandingthepoli'caleconomyofinnova'on
Responsible(researchand)Innova'on
Therulesoftheroad
Newlaboratoryrulesforself-drivingcars?
• Sobwarevshardware– Productliabilityrules
• Ethicalrules– GermanEthicsCommission
1. propertydamagebeforepersonalinjury.
2. noclassifica'onofpeoplebysize,ageetc.
3. “Ifsomethinghappens,themanufacturerisliable”
• AIredflags• Black-boxes
• Ini'alUSrules.NHTSA:“Theagencydoesnotbelievethatself-drivingvehiclesarecurrentlyreadytobedrivenonpublicroadsforpurposesotherthantes'ng.”…un'lTuesday
• Responsibilitymovingfromstatetofederallevel– 15-pointvoluntarychecklistformanufacturers
SeeinglikeaTesla
“Ifanything,Uber’sdriversaretheR&DforUber’sdriverlessfuture.Theyarespendingtheirlaborandcapitalinvestments(cars)ontheirownfutureunemployment.”–DouglasRushkoff
• NewCaliforniarules–talkinBerkeleyabouthowmisguidedthismoveis…• Thiscouldevenbeaseparatepaper–withHarrySurden?• Fornow,thepointisthatthingsareframedintermsofsafetyonly
– Goodtostartwiththis–butaswithTeslastory,askwhatwecanlearnabouttechgoingwrongforthegovernanceoftechgoingright.– Safetyframeisahugebarriertosociallearning– WerefusetodemocraDsethediscussionofbenefits
• • • TheCaliforniaDMVoriginallydrabedregula'onsthatdemandedsubstan'alcollabora'onbetweencompaniesandthestateonthetermsand
conductofthesocialexperiment.• The‘lawlag’isobenactuallyaperformance–‘gexngbehind’thetechnology,beforeitisdefined• A'ghteningofthesafetyframe• Makingdeploymentdefinedasusebymembersofthepublic)easier• AccidentsandDisengagements
– (Theserequirementsares'llintheregula'ons–don’tunderstandtherepor'ng)• Mademorepermissive–innova'onmorePermissionless
– ‘Embracing’thetechnologyactuallymeansdistancingoneselffromit–disengaging• Presumestechnologiesaredefinedascurrentlyimagined–level4and5,ratherthanawork-in-progress.Presumestechnologicalcompleteness
– Interes'ngdefini'onof“opera'onaldesigndomain”,asopposedtoreal-worldcondi'ons– Interes'ngthattrainingisrequiredforAVdrivers– Languageof‘accidents’hasbeenreplacedbythatof‘collisions’– Datarecorderincluded
• • CaliforniaannouncedinMarch2017theirinten'ontorelaxthesecontrols.Instead,theywouldputtheonusonmanufacturerstodeclarethattheir
carsweresafeandfully-insuredandtrustthatthelegalsystemwouldworkoutques'onsofliabilityandunintendedconsequences.Thethreatofbeingsuedisimaginedtobethestrongestlever.
Socialconstruc'onasperformance
Escapingtheriskframe