&

P O R T L A N D

Robotics with Elixir

Meetup organized by Jean-François Cloutier

Held at Big Room Studios, Portland, ME, USA

February 23, 2015

Part 4 – Rise of the Machines

A dashboard just for me? Written in Elm? Shiny!

&

P O R T L A N D

Why am I doing this (still)?

I want to program Lego Mindstorms robots using Elixir

Because robots are super cool and Elixir is the shiznit

And a robot's mind should be a “society of agents”, not some big, giant sense-and-control loop

This is best implemented with processes, lots of processes running concurrently

And Elixir (with OTP and the Erlang runtime system) totally shines here

Oh and I'm learning lots while having way too much fun

&

P O R T L A N DInspiration

Marvin Minsky 1927-2016

“A core tenet of Minsky's philosophy is that 'minds are what brains do'.

The society of mind theory views the human mind and any other naturally evolved cognitive systems as a vast society of individually simple processes known as agents. These processes are the fundamental thinking entities from which minds are built, and together produce the many abilities we attribute to minds. The great power in viewing a mind as a society of agents, as opposed to the consequence of some basic principle or some simple formal system, is that different agents can be based on different types of processes with different purposes, ways of representing knowledge, and methods for producing results.

This idea is perhaps best summarized by the following quote:

What magical trick makes us intelligent? The trick is that there is no trick. The power of intelligence stems from our vast diversity, not from any single, perfect principle. —Marvin Minsky, The Society of Mind, p. 308”

From https://en.wikipedia.org/wiki/Society_of_Mind

&

P O R T L A N D

Part 1 recap

Lego Mindstorms EV3

• Has cool sensors and motors

• Is an open system

ev3dev.org provides a version of Linux customized for the EV3

• Boots from a microSD card

• Exposes sensors, motors and LEDs as files under /sys/class

&

P O R T L A N D

Part 1 recap

Elixir/OTP can be installed and run on the EV3 brick

I implemented a library to access EV3 sensors, motors and LEDs

• As pure functions

• The beginnings of an EV3 domain language

&

P O R T L A N D

Part 2 recap

I implemented portions of a home-brewed theory of robotic mind

• Detection, perception, memory

• and a CNS-as-dispatcher tying the pieces together

&

P O R T L A N D

Part 3 recap

I implemented the executive aspect of a robot's mind:

• Motivators turning on/off motives based on perception,

• Behaviors triggered by motives and generating intents

• Actuators interpreting intents into motors commands etc.

I demoed an autonomous robot that...

• Roams, forages, panics

&

P O R T L A N D

And now, part 4

I built a dashboard to peek into the mind of my robot

• It's an Elm-based app on the browser

• Talking to Phoenix running on the Ev3 brick

I also made a few improvements to the robot's mind

• It copes better with information overflow - by fainting

• Behaviors now include reflexes

&

P O R T L A N D

The robot dashboard

Pause/resume robot

The robot fainted

Latest value for each type of percepts

Current memory usage

Status of all motives

Green is on, red is off

Strike-through when inhibited

Current state of all behaviors

Green is triggered, red is terminated, yellow is

overwhelmedStrike-through when

inhibitedReflexes are in italics

Latest values of all realized intents

&

P O R T L A N D

Elm runtime

Dashboard – The moving parts

Router

Channel

CNS

ChannelsHandler

events

events

Web sockets(JSON)

View

HTTP(Rest) Controller

PortPort

App

UpdateUpdate

ModelModel

ViewView

signals

signals

Elixir/OTP

Phoenix

TaskTasksignals

&

P O R T L A N D

About

Elm is a functional programming language for the browser

• Elm transpiles to Javascript

• Intelligently typed

• Pure functions

Elm is reactive

• Signals in and signals out

• Actions modify the model which modifies the virtual DOM

● The real DOM is then automatically and minimally updated

● No cyclical side-effect cascades and other horrors

Elm is awesome

• Very declarative, very fast

• Marvelously informative compiler errors

• If it compiles, you know you won't see runtime errors

From http://staltz.com/unidirectional-user-interface-architectures.html

&

P O R T L A N D

Elm is small

Few concepts to master:

• Data● Numbers, strings, lists, tuples, records

• Functions● Named or anonymous

• Types● Implied or declared, composable, and never

ambiguous

• Signals● Time-varying values (reactivity!)

• Tasks● Deferred execution of functions

• Modules● To organize, hide and expose functions and types

• Ports● Sanitary access to and from the dirty, side-effect-

ridden outside worldSee http://elm-lang.org/docs/syntax

&

P O R T L A N D

Elm is functional

All functions all the time

All functions can be curried

Functions can operate on functions

• Mapping, filtering, folding...

From http://package.elm-lang.org/packages/elm-lang/core/3.0.0/List

List

From https://dennisreimann.de/articles/elm-functions.html

&

P O R T L A N D

Elm runtime (JS)

Elm is

Functions can not cause side-effects “within” Elm

• All side-effects happen “outside” of Elm

• As outgoing Tasks● Delayed

executions of functions

• Or as incoming Signals of Actions

● “To-do's” for updating the model

Elm app

updatefunction

port

immutablemodel

viewfunction

task

Causing side-effects:REST calls,

DB updates etc.

Signals

Strictlyfunctional

Reporting side-effects:

Socket pushes,user interactions...

&

P O R T L A N D

Elm is reactive

With signals, time becomes a 1st class citizen

• A signal is a stream of values representing a state changing over time (e.g. the position of the mouse)

The application reacts to streams of signals by...

• Constantly re-calculating from merged signals the aggregate state, from time zero to now

• Producing a signal of views on that aggregate state, causing display refreshes

• Creating signals of tasks (think JS

promises) that, when executed, cause external state changes (e.g. DB updates, REST calls etc.)

From http://www.elm-tutorial.org/effects/tasks.html

From http://elm-lang.org/guide/reactivity

&

P O R T L A N D

Elm is friendly

Excellent tooling

• REPL

• Build tool plugins (brunch, grunt...)

• Code editor support

• Easy install (nmp install ...)

World's best compiler error messages

No runtime errors. Evah!

&

P O R T L A N D

Integrating Elm and Phoenix

Elm is embedded in my Phoenix application

And integrated in the build (and automatic rebuild) process

• See package.json, brunch-config.js

Phoenix

RobotDashboard.js RobotDashboard.elmapp.js

brunch build

Adapted from http://www.cultivatehq.com/posts/phoenix-elm-2/

&

P O R T L A N D

The Dashboard components

AppStatus

Perception

Motivation

Comportment Actuation

&

P O R T L A N D

The Dashboard's Elm source

RobotDashboard.elm (the main module)

• StartApp (“main”), input signals and ports

The app module (under App) and component modules (under Status, Perception, Motivation, Comportment and Actuation)

• Model● The model types and functions for instantiation,

initialization

• Update● The update action types and the model update

functions

• View● The HTML generation functions

App structure and framework modeled after Elm-Hedley

&

P O R T L A N D

Models

Each component defines a model as a record

The payloads of signals from Phoenix are also records Status/Model.elm

&

P O R T L A N D

Models

The overall model is a record of records

App/Model.elm

&

P O R T L A N D

Updating

Actions are defined as union types

The update function responds to actions (from

signals) by

• transforming the model

• and potentially creating new effects

Status/Update.elm

&

P O R T L A N D

Updating

The App's update function

• Receives all actions

• Unpacks and dispatches each action to its target component

App/Update.elm

&

P O R T L A N D

Views

Views generate the HTML that presents the current state of the model

And sets up user triggered actions

Status/View.elm

&

P O R T L A N D

Views

The App view function displays the entire dashboard

• by delegating to the view function of each component

• and wrapping each component's “address” with its own distinguishing action tag

App/View.elm

&

P O R T L A N D

Elixir/OTP

Elm accesses Phoenix

Via a REST API

• Configured in Ev3.Router

• /robot/togglePaused (post)● Toggles the robot between

fainting and reviving

● Delegated to Ev3.RobotController

● Who sends a “toggle_paused” event to the CNS

• /robot/paused (get)● Ev3.RobotController asks the

CNS

● About the fainted vs. revived status of the robot

Elm

Phoenix Endpoint

Router

RobotController

CNS

Dashboard

Task

&

P O R T L A N D

Phoenix pushes events to Elm

A Phoenix channel (RobotChannel) can be joined by the Dashboard

CNS dispatches event notifications to its event handlers

• ChannelsHandler is one of the event handlers

• It converts events of interest into broadcasts to the channel the Dashboard has joined

&

P O R T L A N D

Phoenix pushes events to Elm

The main Elm module (RobotDashboard) defines ports for incoming signals

• Each port specifies a signal of a given type

Each type of Phoenix channel event (broadcasted from ChannelsHandler) is associated (in app.js) with an Elm port

• There's an event for reporting fainting, another event for new percepts, new motives etc.

• The payload of a broadcasted event matches the Elm data type of the targeted Elm port

• Data format translation (Elixir map to

JSON to Elm record) is automatic

Elixir/OTP

Elm

Phoenix UserSocket

RobotChannel

ChannelsHandler

CNS

Dashboard

Port

&

P O R T L A N D

Meanwhile, back at the robot factory

Bringing Phoenix “online” overtaxed the limited RAM of the EV3 brick (64M)

• RAM usage overflowed to swap on the SD card, slowing everything down

• The robot's mind suddenly had a much harder time keeping up with the flow of percepts

I decided to move to the BrickPi with a Raspberry Pi2

• 900 MHz 4-core, 1G RAM, oh my!

• But... the BrickPi firmware only supports NXT sensors and motors

So instead, I upgraded to the latest ev3dev image for the EV3 brick

• It uses zram to create a compressed RAM disk for swap (instead of using the SD card)

• The robot still faints quite a bit but at least the SD card won't wear out as much

Now with

ZRAM!

&

P O R T L A N D

The urgency of now – or why faint

A robot must deal with the “here and now”

• It can't afford to react to its situation as it was 10 seconds ago

When the robot is falling behind, it sees increasingly stale percepts, motives or intents

• At some point, they become dangerously out-of-sync with reality

One solution

• Ignore stale percepts, motives and intents● Strong intents take longer to become stale

• And make the robot faint (briefly)● CNS temporarily stops the production of Percepts by Detectors

and by the Internal Clock

● Giving the robot a chance to catch up

&

P O R T L A N D

Reflexes

The robot's behaviors were missing reflexes

• Simple percept-triggered behaviors that temporarily interrupt any active behavior

• To deal with urgent situations such as...

● Avoiding collisions

● Getting unstuck

&

P O R T L A N D

Reflexes in action

&

P O R T L A N D

Resources

Elm

• elm-lang.org

• Elm by Example

• The Pragmatic Studio online courses

● Elm: Building Reactive Web Apps

● Elm: Signals, Mailboxes & Ports

● Not free but worth every penny

Elm-Phoenix integration

• Cultivate's blog series

• Phoenix channels doc

&

P O R T L A N D

I blog about all this

http://jfcloutier.github.io/robotex/

&

P O R T L A N D

Thank you!

Next meetup Robotics with Elixir, part 5 – Short Circuit

• Two robots!

• Can societies of mind form a society of minds?