Boston Elixir
Fun with Robots and Elixir
Presented by Jean-François Cloutier
At the Boston Elixir Meetup
January 21, 2016
Boston Elixir
Why program robots with Elixir?
Because robots are lots of fun and Elixir is the shiznit
And a robot's mind should be a “society of agents” (or, at least, a cacophony of agents)
Which is best implemented with processes, lots of processes, running concurrently
And Elixir (with OTP and the Erlang runtime system) totally shines here
Boston Elixir
Lego Mindstorms
The Greatest Toy Ever!
And it keeps getting better
1998 – RCX 2006 – NXT 2013 - EV3
EV3 brick● Faster CPU, more memory● Linux!● Bootable microSD card● USB port
● Daisy chain up to 4 bricks● Plug WiFi dongle
Boston Elixir
The goodies
The programmable brick
• 4 sensor input ports
• 4 motor output ports
• LEDs
• Sound
Sensors and actuators
• 2 large motors
• 1 medium motor
• 1 color sensor
• 1 IR sensor (with beacon/remote control)
• 1 touch sensor
• 1 ultrasonic sensor (extra)
• 1 gyro sensor (extra)
Boston Elixir
The goodies
And a whole bunch of
• Connectors
• Wheels, cogs, threads
• Etc.
Boston Elixir
Programming the EV3
The EV3 set comes with a customized version of LabView
Visual programming? Snazzy!
• If you don't mind coding everything as one big “sense and respond” loop
Boston Elixir
Blerch!
OK it's not that bad but we can do better!
Boston Elixir
ev3dev.org to the rescue
An updated LEGO Linux kernel (Debian)
• Bootable from a microSD card
• Can run most Debian packages● apt-get install …
EV3 software “rebuilt from the ground up”
• Motor, sensor and LED drivers use sysfs● Exposed as character files
● Sensing = reading files
● Controlling = writing to files
EV3 is now exposed to any and all programming languages running under Linux
Ergo, we can code the EV3 with Elixir!
Boston Elixir
Installing Elixir on the EV3
You will need to
• Connect the EV3 to the Internet (via WiFi dongle)
• Build the latest release of Erlang from sources
• Install Elixir from a precompiled zip
• Setup a user account with the right privileges
See details in my blog
• http://jfcloutier.github.io/robotex/
Boston Elixir
Meet the robot
Infrared sensor (to sense beacon
distance and direction)
Touch sensor(to detect collisions)
Large motor(to move and turn)
Medium motor (turned on when
“eating”)
LEDs(green when moving around, orange when finding food, red
when panicking)
Color sensor (to see the “food”)
Beacon(to simulate the smell of food)
Designed and built by my 12 year old son William
Ultrasonic sensor (to sense
proximity of obstacles)
Speaker(to emote audibly)
Powered by elixir
Boston Elixir
Boston Elixir
Perceptors
Perceptors generate high(er)-level percepts
Each Perceptor reacts to a new percept of interest
• and analyzes more or less recent memorized percepts
• to, possibly, create a new percept
• which it then notifies the CNS of
A “collision” Perceptor, for example, would analyze proximity and touch percepts
• Decreasing proximity = collision soon
• Close proximity in immediate past = collision imminent
• Recent collision imminent and touched = collision now
A “fear” Perceptor would analyze collisions and ambient light percepts
• It would create fear percepts when the robot collides in very low ambient light
Boston Elixir
Motivators
Motivators generate motives in response to new percepts
Motives are like emotions
• They trigger behaviors
A motive can inhibit other or all motives
• Hunger inhibits curiosity
• Fear inhibits all other motives
Motives are kept in memory for a while
Boston Elixir
Behaviors
Behaviors are finite-state machines triggered by motives
State transitions
• are caused by new percepts
• And typically generate intents
A behavior is frozen if all its triggering motives are inhibited
Curiosity
Roaming
CollisionNOW
CollisionIMMINENT
Avoid collision
Time elapsed
Started
Time elapsed
Behave this!Roam
Stuck
Back off
Boston Elixir
Actuators
Actuators translate intents into commands
Each actuator realizes one type of intent
• by executing a script that sends a sequence of commands to motors and LEDs
Different robots may require completely different scripts to realize the same intent
Boston Elixir
Memory
The Memory stores percepts, motives and intents
• For later recall by by Perceptors, Motivators and Behaviors
Expired, invalidated percepts are continuously removed from memory
• Percepts from different senses are retained for different amounts of time
Boston Elixir
The Central Nervous System
The CNS is an event dispatcher
Dispatched events include
• A percept, motive or intent was created
• An intent was realized
• An actuator is overwhelmed
The CNS connects the robot's mind via events
• Detectors, Memory, Perceptors, Motivators, Behaviors, Actuators... do not communicate directly
• They each listen to all events but only handle those of interest to them
Boston Elixir
CNS mediates all interactions
From percepts to actions to percepts
Detectors produce percepts when sensors detect changes
Perceptors generate higher-level percepts
Percepts cause motives via motivators
Motives trigger behaviors that are driven by percepts and generate intents
Intents are translated into commands by actuators
Commands make the robot move, generating new perceptions
• The passing of time is itself a periodic perception
Everything happens concurrently
• No single, big sense-respond loop
All interactions are via events dispatched by the CNS
Detector, Internal clock
Detector, Internal clock
PerceptorPerceptor
MotivatorMotivator
BehaviorBehavior
ActuatorActuatorActions cause measures to change
Actions cause measures to change
Boston Elixir
The urgency of now
A robot must deal with the “here and now”
• It can't afford to react to its situation as it was 10 seconds ago
When the robot is falling behind, it sees increasingly stale percepts, motives or intents
• At some point, they become dangerously out-of-sync with reality
One solution
• Ignore stale percepts, motives and intents● Strong intents take longer to become stale
• And make the robot faint (briefly)● To temporarily stop the production of Percepts by Detectors
and by the Internal Clock
● Giving the robot a chance to catch up
Boston Elixir
InspirationMarvin Minsky's society of mind
A core tenet of Minsky's philosophy is that "minds are what brains do".
The society of mind theory views the human mind and any other naturally evolved cognitive systems as a vast society of individually simple processes known as agents. These processes are the fundamental thinking entities from which minds are built, and together produce the many abilities we attribute to minds. The great power in viewing a mind as a society of agents, as opposed to the consequence of some basic principle or some simple formal system, is that different agents can be based on different types of processes with different purposes, ways of representing knowledge, and methods for producing results.
This idea is perhaps best summarized by the following quote:
What magical trick makes us intelligent? The trick is that there is no trick. The power of intelligence stems from our vast diversity, not from any single, perfect principle. —Marvin Minsky, The Society of Mind, p. 308
https://en.wikipedia.org/wiki/Society_of_Mind
Boston Elixir
InspirationRodney Brooks' subsumption architecture
Subsumption architecture is a reactive robotic architecture heavily associated with behavior-based robotics. The term was introduced by Rodney Brooks and colleagues in 1986.
Subsumption architecture is a control architecture that [decomposes] the complete behavior into sub-behaviors.
These sub-behaviors are organized into a hierarchy of layers. Each layer implements a particular level of behavioral competence, and higher levels are able to subsume lower levels in order to create viable behavior. For example, a robot's lowest layer could be "avoid an object". The second layer would be "wander around", which runs beneath the third layer "explore the world".
The layers, which all receive sensor-information, work in parallel and generate outputs. These outputs can be commands to actuators, or signals that suppress or inhibit other layers.
Each layer is made up by a set of processors that are augmented finite-state machines (AFSM), the augmentation being added instance variables to hold programmable data-structures. A layer is a module and is responsible for a single behavioral goal, such as "wander around." There is no central control within or between these behavioral modules. All AFSMs continuously and asynchronously receive input from the relevant sensors and send output to actuators (or other AFSMs). Input signals that are not read by the time a new one is delivered end up getting discarded. These discarded signals are common, and is useful for performance because it allows the system to work in real time by dealing with the most immediate information.
https://en.wikipedia.org/wiki/Subsumption_architecture
Boston Elixir
The robot's mind is constructed by defining its
• Perceptors
• Motivators
• Behaviors
• Actuators
Boston Elixir
Release the robot!
The robot starts hungry, so it forages first by...
• Trying to orient itself and move toward the beacon
• Looking for “blue” food, hopefully in well-lit areas (more plentiful)
• While avoiding obstacles or getting stuck
When not hungry, it expresses its innate curiosity by roaming
• Randomly going here and there
• And avoiding obstacles or getting stuck
If it collides into something, it panics
• And behaves like a headless chicken
• Until it calms down and resumes foraging or roaming
Boston Elixir
Test run
Beacon (as the smell of food)
“Food”
The robot starts up hungry and searches for food. It first heads in the wrong direction, gets stuck (“I'm stuck!”) but manages to free itself.
Then it smells the bacon beacon and makes a bee line toward the “food”. It slows down as it gets closer, adjusting its aim until it finds the food (blue paper). It then proceeds to eat it (“nom de nom de nom!”), turning on orange lights and running its small motor (the “mouth”).
When it becomes sated, curiosity takes over and the robot roams around, trying to avoid collisions. It sometimes panics (“I'm scared!”) when it runs into walls. The lights then turn red and the poor robot behaves like a headless chicken for a while until it calms down.
At some point, it gets hungry again and, this time, easily finds the food.
https://www.youtube.com/watch?v=NbquttnKuak
The beast
Boston Elixir
Coding the mind of a robot
I use Elixir file functions to write to and read from /sys/class/...
And take advantage of Elixir's functional programming
• Devices, percepts etc. as immutable data
• Detection, perception, behavior etc. as functional transformations
I made OTP my BFF
• The CNS and its event handlers are implemented with GenServer and GenEvent
• I use Agents for Detectors, Perceptors, Motivators, Behaviors and Actuators
• Memory is a singleton GenServer (creates a transaction boundary)
• Linked processes● Periodically remove stale memories
● Poll sensors
• Everything is supervised for fail-over
Boston Elixir
Accessing the EV3 via sysfs
Boston Elixir
Interacting with tacho-motors
ssh [email protected] /sys/class/tachomotorcd motor0cat driver_nameecho 50 > duty_cycle_specho run-forever > commandecho stop > command
Large motor Medium motor
Boston Elixir
Interacting with sensors
Touch sensor
cd /sys/class/legosensorcd sensor1cat driver_namecat value0cat modesecho IRSEEK > modecat value0cat value1
Infrared sensor and remote control Color sensor
Boston Elixir
Modeling EV3 devices
We can model the changing state of motors, sensors and LEDs
• As mutable global variables (the non-FP way)
● as embodied by /sys/class files
• Or as functions consuming and producing immutable data (the FP way)
● Done by wrapping /sys/class with pure functions that input and output device states
Device
Get connected
devices
Device
Device
Device
Alter controls
DeviceExecute
command
Get state
Device
Boston Elixir
Accessing EV3 devices the FP way
Devices are immutable data
Access to sensors and other devices
• Via pure functions
• Inputing and outputting devices as structs
Boston Elixir
A functional brain
The robot's state is modeled as a bunch of (immutable) data structures
• Percept, Motive, Behavior FSM, Intent
• Configurations for perception, motivation, behaviors and actuation
Memory queries are encoded as higher order functions and transformation pipelines
Functions-as-data encode
• Perception capabilities
• Motivation rules
• Elements of behavior
• Actuation scripts
Boston Elixir
OTP on the brain
In my implementation, I aimed for
• Maximum concurrency via processes ● A “society of mind” is inherently parallel
● Sequential processing happens only where absolutely necessary
• Maximum decoupling via events● Components don't know of each other
OTP is a spectacular fit
• Supervisor, GenServer, GenEvent, Agent etc. = high-level concurrency framework
• OTP makes it easy to implement a process-based design with predictable behavior● You don't need to be a genius to correctly interleave concurrent and sequential code
• The use of OTP favors economical and largely declarative code ● It says what it does and it does what it says
Boston Elixir
Processes in Elixir, a refresher
Processes are cheap (2.5K per process)
Preemptive scheduling (no CPU hogs) over multiple cores
Communication with processes:
• Via self-contained (deep-copied) messages
• Messages are sent to a process' mailbox (a queue)
• The receiving process handles one message a time
When a process creates another, it can...
• Link to it (it dies if the linked process dies abnormally)
• Monitor it (might receive a “wrongful death” message)
OTP is (among other things) a framework for easily and safely creating, interacting with and supervising processes
• Hides complexities and handles the corner cases
• Standardizes multi-process coding
ProcessProcess
ProcessProcess
linked
{:exit, pid, reason}
Boston Elixir
The Ev3 application'sprocess tree*
* Here running with mock motors and sensors
Boston Elixir
I OTP
The 'simple-one-for-one' supervisor strategy
• Supervised “worker” child processes are created on request and as needed
• Applied to (re)start Detectors, Perceptors, Motivators, Behaviors, Actuators
A singleton GenServer creates a transactional boundary
• N processes sending messages to one Genserver (transaction boundary!)
• Applied for Memory, CNS
GenServer
Process
Process
Process
Messages handled one at a time
Calls
Boston Elixir
I OTP
I use Agents (simplified GenServers) to implement
• Detectors, Perceptors, Motivators, Behaviors and Actuators
Agents are supervisable processes that
• Hold a non-shared state (while the process is alive)
Behaviors are finite state machines
• A behavior has states and state transitions
• I did not use OTP's gen_fsm but rolled my own FSM to fit my special needs
Boston Elixir
I OTP
The CNS is an event manager implemented as a GenServer that monitors its GenEvent handlers
• It notices and recovers from event handler failures
• See my Gist
I spawn processes to parallelize event dispatching
• Used by event handlers for Perceptors, and Actuators
Boston Elixir
Configuring a robot's mind
The particulars of a given robot's perception and execution are defined in these modules:
• Ev3.Perception● The ways by which the robot creates Percepts and interprets them into high-level
Percepts
• Ev3.Motivation● The rules by which the robot responds to Percepts by turning Motives on and off
• Ev3.Behaviors● The robot's finite state machines triggered by Motives and driven by Percepts to
generate Intents
• Ev3.Actuation● The rules by which the robot realizes Intents by sending scripted commands to its
motors and LEDs
Boston Elixir
Debugging a robot is painful
Debugging the real-world behavior of a physical being is not like debugging code
• The effects of commands are unpredictable (the robot hits a wall, a wheel slips)
• Sensors can lie (they sometimes see ghosts)
• Overall behavior is very sensitive to tuning (how much to turn, how far to move)
• The Heisenberg Principle is in full force (producing a debug trace drastically alters time-sensitive behavior causing new problems)
And debugging code is harder
• Some bugs only come out when dealing with real motors and sensors (vs mocks)
• The run-edit-deploy cycle on the EV3 is very long (it's a slooow 300MHz ARM9 single core computer)
Boston Elixir
Time to wrap it up
Functional programming FTW!
• I implemented a non-trivial robotics framework using FP
• I never once missed having classes, inheritance etc. (sorry OO, my old friend)
• Data structures + functional transformations make the code remarkably declarative and compact
OTP, how do I love thee?
• OTP is a treasure trove of battle-tested power tools for writing fault-tolerant, concurrent, event-driven code
• Elixir polishes it to a shine (minimal boilerplate, simple abstractions like Agent and Task)
Bottom line
• Coding the robot in Elixir/OTP is so much fun it at times verges on pure giddiness
Boston Elixir
Watch this space
http://jfcloutier.github.io/robotex/
Boston Elixir
Thank you all!
I organize the Portland (Maine) Erlang & Elixir meetup
• Twitter: jfcloutier
• Github: https://github.com/jfcloutier/ev3