1

Autonomous Foragingin a Simulated Environment

2

Introduction Autonomous foraging: the scenario

Robots from the Matrix, harvesting humans and avoiding mannequins

Multiple robots, laying pheromones to maximize harvesting efficiency

Dark post-industrial environment

3

Hybrid Architecture Mostly reactive

No internal map React immediately to sensor data Poses challenges for efficiency Difficult to fix minima

Partially deliberative Track relative offset from goal Some ordering exists Allows for some degree of predictability

4

Behavior Networkmain network

init forageloop*

* loops until all rewards found and delivered

forage

findobject

returnto goal

loop*

* loops until all rewards found and delivered

5

Behavior Network

avoidobstacle

wanderdetectreward

grabobject

find object

concurrent

triggers

6

Module Summary

7

Sensor Goals Link specific sensors to specific goals

Obstacle distance/detection Potential field measurement Object discrimination

Enable specific rules for architecture Sensors must facilitate reactive behavior

8

Sensors Laser range finder

Avoid obstacles in advance (less moves) Limited range due to fog, smog, and “gloom”

Thermal sensor Detect heat emitted by humans Inverse Euclidean distance from object center used

as gradient 2 color sensors

One for humans, one for mannequins Both are semi-active for goal (where all other

humans are stored)

9

Sensor Models Laser range finder (front)

Returns distance (integer) Assume accuracy for distances of 1 to 6

squares Used for obstacle detection

Thermal sensor (front) Returns 3 x 3 matrix (floating point) Assume 10% error per grid Used for object detection

2 color sensors (bottom) Returns wavelength intensity (floating point) Assume 2% error Used for object detection/discrimination

10

Reactive Components• sensor_read;• thermalsum = sum( sum( robot.thermal ) );• • % detect_reward • if robot.green >= 0.8 & robot.carrying == TYPE_BLANK• robot_grab;• end• • % detect_avoid• if robot.blue >= 0.8 & robot.carrying == TYPE_BLANK• disp(' - avoidance detected' );• end• • % detect_goal• if ( robot.blue >= .5 - .2 ) & ( robot.blue <= 0.5 + .2 )

& ( robot.green >= .5 - .2 ) & ( robot.green <= 0.5 + .2 )• • % drop_object• if robot.carrying ~= TYPE_BLANK• robot_drop;• end• end

11

Detect Object• if thermalsum > 0 & robot.carrying == TYPE_BLANK• • sumleft = sum( robot.thermal( 1:3, 1 ) );• sumcenter = sum( robot.thermal( 1:3, 2 ) );• sumright = sum( robot.thermal( 1:3, 3 ) );• • % detect_obstacle• if robot.laser == 1• • if randint( 0, 1 ) == 0• robot_moveleft;• else• robot_moveright;• end• • end

randomly moves left or right once to avoid obstacle

12

Detect Object• if sumcenter >= sumleft & sumcenter >= sumright• robot_move( robot.dir );• • elseif sumleft >= sumcenter & sumleft >= sumright• • if robot.thermal( 3, 1 ) > robot.thermal ( 1, 1 )• robot_turnleft;• • if randint( 0, 1 ) == 1• robot_turnleft;• end• • robot_move( robot.dir );• • else• robot_move( robot.dir );• robot_turnleft;• end handles movement towards heat source

13

Detect Object• else• if robot.thermal( 3, 3 ) > robot.thermal ( 1, 3 )• robot_turnright;• • if randint( 0, 1 ) == 1• robot_turnright;• end• • robot_move( robot.dir );• • else• robot_move( robot.dir );• robot_turnright;• end• end % sumcenter,left,right• • end

handles movement towards heat source

14

Move to Goal– if robot.laser >= 1

& robot.laser < 3 – – % perturb direction– if randint( 0, 1 ) == 0– robot_turnleft;– robot_turnleft;– else– robot_turnright;– robot_turnright;– end– – % move away from obstacle – robot_move( robot.dir );– robot_move( robot.dir );

– else– % move toward goal– if robot.x > robot.goal.x– robot_move( W );– elseif robot.x < robot.goal.x– robot_move( E );– end– – if robot.y > robot.goal.y– robot_move( S );– elseif robot.y < robot.goal.y– robot_move( N );– end– end

15

Wander Pattern

16

Wander Pattern• if thermalsum == 0• if robot.laser == 1• if wander_parity == 0– robot_turnleft;– robot_turnleft;– else– robot_turnright;– robot_turnright;– end– – sensor_read;– – % test if in corner– if robot.laser == 1– if wander_parity == 0– robot_turnleft;– robot_turnleft;– robot_turnleft;– robot_turnleft;– else– robot_turnright;– robot_turnright;– robot_turnright;– robot_turnright;– end

– else – for i = 1 : 5– robot_move( robot.dir );– end– – if wander_parity == 0– robot_turnleft;– robot_turnleft;– else– robot_turnright;– robot_turnright;– end– end % if robot.laser == 1– – wander_parity = ~wander_parity;– end % if robot_laser == 1– – robot_move( robot.dir ); – end % if thermalsum == 0

robot_turnleft and robot_turnright turn the robot left/right 45º

17

Original Plan Planned for multiple robots

Use implicit communication (stigmergy) through use of pheromones

Follow own pheromone trails back to sources of rewards, to goal, and around obstacles

Avoid other pheromone trails to prevent redundant searching

Ideally more robots could be added without changing any code

18

Actual Results Ran out of time!

Have single robot with no pheromones Inefficient obstacle avoidance Inefficient coverage of area

Some singularities may exist Rare looping around rewards (depends on

environment and robot direction) 3041 moves, 1232 turns (on average)

Taken from 10 simulation runs Moves ranged from 4633-2049 Turns ranged from 972-1470

19

Actual Results 25 separate MATLAB files

1225 lines of code total 193 lines for forage script 276 lines for display 5455 elements using 48694 bytes 7 scripts, 47 functions, 10 subfunctions

20

Actual Results

21

Future Work Add pheromone trails and multiple robots Enhance obstacle avoidance and detection Restructure reactive behavior code

Perhaps place code so that executed every time move or every time sensors are read

Force every action to update simulation figure

22

What We Learned Reactive approach demonstrates Braitenberg

teachings Designing simple rules to generate intended

emergence is tough Appearance of free will?

“the hobgoblin of philosophy” Inductive behavior analysis is tougher than

deductive Simple interactions may be the explanation

23

Simulation