TDDD10AIProgrammingPuttingItAllTogether
CyrilleBerger
2/65
Lectures
1AIProgramming:Introduction2Introductionto3AgentsandAgents4Multi-Agentand5Multi-AgentDecision6CooperationAndCoordination7CooperationAndCoordination8MachineLearning9AutomatedPlanning10PuttingItAllTogether
3/65
Lecturegoals
Learnhowallthealgorithmsandconceptsfittogether
4/65
LecturesSummary
5/65
Lecturecontent
AgentArchitectureCommunicationRelayTrafficsurveillanceSensingStream-BasedReasoning
DetectingandTrackingcars
Multi-agentsensing
RescueoperationHumanDetectionandPlanningDelegation
BuildingScanningMasterthesis
AgentArchitecture
7
ArchitectureTheory
8
UAVArchitecture
CommunicationRelay
10
CommunicationRelay
Communicationisakeycomponentofmulti-agentsystems
Communicationcanbe
CommunicationnetworkscanbedisabledordestroyedForinstance,during9/11,cellularphoneswerenotworking
11
CommunicationRelay:multiplepath
12
CommunicationRelay:visibility
13
CommunicationRelayThegoalistofindachainofUAVthatminimizethenumberofThisisacomplicatedoptimizationGetworseifyouhavemultipleUseVoronoigraphandTreebased
Thisworkin2D,butwhatabout
14
Networkbuildingwithdelegation
NetworkbuildingmissionSequence
DeploybasestationSequence
Flyto Grab Flyto Grab
DeployrelaystationsConcurrent
Deployrelaystation1Sequence
...
15
Howtogetfullcoverage?
Findinganoptimalcoveragein3Disunreasonable
Instead,whenanagentisinanareawithlowcommunication,itshouldsendarequestforgettingarelaystationinstalled
Trafficsurveillance
17
TrafficsurveillanceScenario(1/2)
18
TrafficsurveillanceScenario(2/2)
Continuouslygatherinformationfrommanydifferentsources.
Selecttherelevantinformationforthecurrenttask.
Derivehigher-levelknowledgeabouttheenvironmentandtheUAV.Suchasdetectingmisbehavingdrivers
Correctlyinterpretwhatisgoingon.
CoordinationofUAVswithother
CoordinationofUAVswithotherhumanpoliceforces(calledmixinitiative)
Sensing Stream-BasedReasoning
21
TheSense-ReasoningGap
22
Stream-BasedReasoning
Autonomoussystemsproduceandprocesssequencesofvaluesincrementallycreatedatrun-time.
Thesesequencesarenaturaltomodelasstreams.Stream-basedreasoningisincrementalreasoningoverstreams.Stream-basedreasoningcapturesthecontinuousreasoningwithminimallatencynecessarytoreacttorapidchangesintheenvironment.
23
IncrementalEvaluationofTemporalLogicalFormulas
Thesemanticsoftheseformulasisdefinedoverinfinitestatesequences.Progressionisonetechniquetocheckwhetherthecurrentprefixissufficienttodeterminethetruthvalueofaformula.
24
StateStreams
DetectingandTrackingcars
26
TrafficMonitoring
27
Anchoring(1/3)
Theobjectiveoftheanchoringprocessistoconnectsymbolstosensordataoriginatinginthephysicalworldsothatthesymbolsrepresenttheobjectsintheworld
28
Anchoring(2/3)
33
DyKnowFederation
34
Cooperation
whathappenwhenanhelicopterneedtoleaveitspatrolareatogoinpursuitofanoffender?Either,needtofindareplacementforpatrollinghisareaOranotherhelicoptertopursuetheoffender
SurveillanceSequence
SurveillanceSequence
PursuitSequence
Rescueoperation
36
RescueoperationScenario(1/2)
37
RescueoperationScenario(2/2)
ExplorationtofindallvictimsDivisionofareaintoscanningareaTaskallocationDetectionofvictims
RescueofTaskallocation
HumanDetectionandApplication
39
VictimsDetection
40
Leashing
Planning
42
HowtoconstructaTask-SpecificationTree?
43
PlantoTask-SpecificationTree
AnautomatedplannertakesaproblemdescriptionObjectives
Availableresources,actions,…
…andgeneratesaplanthatachievestheobjectives
44
Naturaldisasterexample
Example:SupposetherehasbeenanaturaldisasterObjective:100peopleshouldhavefood,medicineandwaterWehaveasmallfleetofUAVsavailable
Howtodescribethistoaplanner?Howtogeneratea
45
Naturaldisasterexample:GeneralWorldDescription
High-leveldescriptionoftheworldweoperateinEntities:UAVs,crates,people,…
Properties:Entitieshavelocations,UAVshavecapabilities,…
Actionsthatcanbeperformed(:operator(deliver-crate?uav?crate?from?to)(:precond(and(has-capability?uavcarry-crates)(at?uav?from)(at?crate?from)…)(:phase:duration(flight-time?from?to):effects(:assign(location?crate)?to)(:assign(location?uav)?to)…)…)
46
Naturaldisasterexample:ProblemDescription
ThecurrentstateoftheworldLocationsofinjuredpeople,availabilityofcrates,…
Agoaltobeachieved(and(forall?person(has-crate?personfood))(has-crateperson2medicine)(has-crateperson7medicine)…)
47
Resultingplan
Theresultingplansshouldbeabletoexpress:Concurrency:sequentialplannersarenotapplicablePrecedence:uav7picksupcarrier2afteruav2loadscratesLackofprecedence:Onlywaitforotheragentswhenyouhaveto!Approximatetiming:(exactdurationsareunknown)
48
PlanningtoTask-SpecificationTree
Delegation
50
Delegation(1/2)
Missionconsistingofaflightaction+agoaltoachieve
51
Delegation(2/2)
Verifyexecutabilitythroughon-boardfunctionalitiesduringplanningMotionplannerScheduling,resourcereasoning,constraintreasoning
Infeasibleactionimmediatebacktracking!
Useofaconstraintsolver
52
DistributedPlanning
57
BuildingScanningpart2
58/65
LecturesSummary
Masterthesis
60
Masterthesis
InvariousdomainofArtificialIntelligencePlanningKnowledgeRepresentationMachineLearningRoboticSensing
61
GuidedExploration
Lookingforvictims,usingpriorknowledgeaswellasnewobservation
Prioritizedscanarea
62
Objectdetectionandrecognition
Objectrecongitionusingdeeplearning
Implementationwithagroundrobot
Evaluationonaerial
63
HumanRobotInterractions
VoicecommandsPoserecognition
Andmanymore...