Building droids with JavaScript

Andrew Fisher @ajfisherMelbJS, 14 August 2013

Hi! My name is Andrew Fisher and I’m an interaction developer. Tonight I want to talk to you about playing with hardware and building droids with javascript for the next 15 minutes or so.

Atwood’s Law

We all laugh about the observation behind Atwood’s Law - mostly because we’ve all been culprits of it at one point or another and I’m no different - but the reason for this is that JS is becoming more and more capable as time goes on. Historically when it comes to hardware however that hasn’t been the case. So if you wanted to work with hardware you really had to go back to languages like C to work there. As someone who grew up writing C though, I never really had a problem with this, I’d just switch back and forth between languages depending on what part of the hardware stack I was working on.

JS all the REAL things

flickr (cc) Quasimondo

Recently though, some great work has been done in the node community getting JS to work with hardware - to the point where working with hardware using javascript is now extremely viable.

So tonight I want to talk to you about that and how all of you can all start working with hardware with JS and along the way we’ll bump into some robots.

Image: http://www.flickr.com/photos/quasimondo/5203908319

TODO

1. Different ways to interface with hardware2. The JS / hardware stack3. Applications and examples

So to do this, we’ll look at.

How we can interact with hardware.

Some code to show what a common JS hardware stack starts to look like

Then we’ll play with some examples.

Working with hardware

flickr (cc) Oskay

Let’s start by looking conceptually at how we develop with hardware. I think about this at sort of three levels.

At the metalAt the deviceAnd somewhere in between.

Image: http://www.flickr.com/photos/oskay/2310115216/

Working with the metal

flickr (cc) Wonderlane

When we work with the metal we are usually writing code specifically for a chip or board at a very low level. It usually means writing C or C++. You can get a bit of abstraction with hardware libraries but even with good libraries you need to write a lot of code.

How many of you have written C++? How many of you do for work or for “real”? So pretty much no one. Does anyone find it fun? No didn’t think so....

The other thing to consider here is you have insane limitations. It’s like going back to the 80s - an arduino for example has TWO KILOBYTES of RAM.

Image: http://www.flickr.com/photos/wonderlane/3198166347

Limitations{ "motor": 100, "id": 1, "dir": "forward", "period": 10, “position”: 78, “voltmax”: 5.3,}

This means that for example, a JSON string like this:

Will probably overrun a buffer in memory if you try and parse it as actual json object in a programme and not character by character. So there’s some serious limitations here at the metal.

However these sorts of constraints are really good for your coding discipline and making you think about performance at a very low level so I would thoroughly encourage you to play here as I know both my JS and Python have both improved by working with such extreme limitations.

Working low level

flickr (cc) lisovy

Playing at the chip level gives you masses of control but your iteration cycles take time and you’re constantly thinking about things like memory and garbage collection and not frying a chip. Debugging is also generally a nightmare.

Image: http://www.flickr.com/photos/lisovy/4677688431/

Device hacking

flickr (cc) Roo Reynolds

At the other end of the spectrum there are devices that are already made like this little printer from Berg. This is hardware with APIs - maybe they are via the network, maybe via an embedded web server or some sort of serial protocol.

However it works, you’re basically interacting with the hardware given to you as a service.

You can build clever ways of working with the service but fundamentally you can’t change the way the hardware works.

IMage: http://www.flickr.com/photos/rooreynolds/9350631793

AR.Drone

flickr (cc) neeravbhatt

A good example of this is the AR Drone - you can play with using a node library to interact with the copter directly to do some really interesting things but you can’t fundamentally change the hardware - you can’t suddenly change this helicopter and turn it into a submarine for example.

Having said that, people have done some fantastic things at this level with this device.

http://www.flickr.com/photos/neeravbhatt/6885424870

Blink(1)

flickr (cc) todbot

The Blink(1) is an interactive light which can be worked with via a library with various applications that you might want to create as well such as task or status indicators. These guys are doing a new version on kickstarter at the moment as well if you’re interested.

So there’s plenty of very interesting devices you can grab hold of and start playing about with really easily.

http://www.flickr.com/photos/todbot/8245679238

Hacking in comfort

flickr (cc) Zack Hoeken

In between these extremes though is a new category where you can work with and change the hardware but you can still use high level languages to do it. This is a great combo as it allows for rapid prototyping but also makes hardware more accessible to non-embedded application designers.

A raft of options over the last 12-18 months have appeared in this space such as:

Image: http://www.flickr.com/photos/hoeken/3519955473/

Raspberry PI

flickr (cc) GijsbertPeijs

The Raspberry PI which seems to have found a firm home within the python community.

image: http://www.flickr.com/photos/gijsbertpeijs/7988262046

Beaglebone black

flickr (cc) stfnix

The beagle bone which is still in its early days but seems to be aiming firmly at JS devs. It even ships with nodejs installed and running by default with Cloud9 IDE available to hack away on it directly. Hardware connectivity is a little light but once that is a bit better I think this may become the tool of choice for web devs crossing over to hardware.

Image: http://www.flickr.com/photos/stfnix/9176951660/sizes/o/in/photostream/

Arduino

But the device that has the best support with a huge community and masses of compatible hardware is the arduino using a protocol layer called Firmata which we’ll look at in more detail in a moment.

Sketching in hardware

flickr (cc) Camille Moussette

So these approaches give you the ability to prototype rapidly as well as work almost directly with the hardware. There are some limitations around some hardware but to start off with and for some prototyping that doesn’t matter - let’s make something.

http://www.flickr.com/photos/9225693@N08/6051548279

First steps

We’re going to focus on arduino and a tool called firmata.

This is an arduino - they cost about $30, they are awesome and come in many different forms for different applications from small to large. They run off USB or a battery so it’s hard to blow them up and even harder to electrocute yourself. There is also huge amounts of community information about them.

Firmata is a program or sketch in arduino-speak that you load onto the arduino which allows you to communicate with it over USB serial and tell it to do things like turn a pin on or off, take a reading etc. Firmata is a neat idea as it essentially exposes most of the features of the arduino but via a protocol so now you can control it from somewhere else.

How toGet an arduino (littlebitdelectronics.com.au)Install the arduino IDE (arduino.cc)Load the firmata firmware sketch (from the IDE)Install nodejs (nodejs.org)Install node firmata (npm install firmata)Write an application (vim something...)

So here’s a quick run through how we’ll make our first hardware application.

We’ll get an arduino, install the arduino IDE and load the firmata sketch onto the arduino.

On the node side, we install firmata so we can communicate with the arduino.

I won’t do these as I expect you can all buy things online and install software that’s well documented.

Once we have all of this we can get on with making something.

Hardware hello world

flickr (cc) pj_vanf

So we plug in an LED (this one is bigger than normal so you can see it easily) and then we’ll need to write some code.

I’ve got a little bit of scaffolding here so I don’t have to type everything. But you can see it just creates a board and connects it. Now it’s connected we can tell the board we want to assign a pin as an output then we’ll tell it to make the pin go HIGH which means send it some volts - we do that with the digitalWrite command. Then we can make it go LOW and as you can see as I do this it turns the LED on and off.

Code at http://github.com/ajfisher/nbs

http://www.flickr.com/photos/vanf/5210360116

Web thing hello world

http://github.com/ajfisher/nbsDemoed live

So we’re all web devs here so let’s not stray too far away from our comfort zone. Let’s wrap a web interface around this light so we can turn it on and off with a click of a button on a page.

We can’t do this interactively very easily so I’ll show you some code.

This is a bit of overkill but we’re going to use express and web sockets so it’s a bit more realtime but also it will lay the foundation for what I’ll show you next.

We set up the socket messages on the server to switch things on and off . On the client side all our HTML and JS is doing is just sending messages on click so nothing too interesting.

And there we go - button click to turn a light on and off via a web browser. I can hear the whirring of brains considering home interaction interfaces as I stand here.

The JS hardware stack

Arduino (Sensors and actuators)Firmata (Communications protocol)NodeJS (Application logic)WS/HTTP (Network & security protocols)Client (UI, Input, Visualisation)

So this is what the JS hardware stack looks like.

We’ve got an arduino with sensors and actuators. Firmata which provides the communications protocol. The NodeJS application gives us application logic and integration with other libraries such as johnny-five which allows us to turn hardware into objects.

Networking is provided over http or websockets and this can give us security and encryption.

Finally the web page for the client gives as input methods, a data viz layer as well as user interface.

Easy install

npm install johnny-five express socket.io

So this stack can be created with pretty much just this command plus an arduino with firmata on it in about 2 minutes.

Which is pretty cool. And it’s robust enough now that it’s most likely going to work the first time you try it.

Looking for droids?

flickr (cc) solo

So let’s look at an application of this.

I don’t have any flying robots due to space constraints and not wanting to take out any eyes but go get an ar-drone and check out nodecopter.org for some fun.

Here’s a ground based option however.

Image: http://www.flickr.com/photos/donsolo/3768623542/

Nodebot

This is a bot I built from scratch - The core was just some metal and some motors.

Originally last year I built it all in C. Then I layered a web interface onto it earlier this year for control via a browser and did that using node talking to my C code. Over the last couple of months I’ve ported it to be 100% javascript.

Nodebot architecture

RPiArduino

Battery

4WD motor control

Sensors Firmata

NodeJS- Express- Johnny Five- Open CV- Socket IO

WiFi

Camera

4WD motor control

Architecturally this is what it looks like:

The arduino deals with the hardware using firmata.

A raspberry pi runs a node server and communicates to the arduino over a serial connection.

The webcam provides a video feed using node-opencv as a stream

A web interface shows the video and provides input control to drive the bot using the device API and the gyro in your phone or tablet or keyboard on your desktop.

Nodebot code

var motor = new five.Motor(pins: {dir:11, pwm: 3});motor.forward(150);motor.stop();motor.reverse(150);

The code uses Johnny Five which is a node library on top of firmata that abstracts common hardware into objects to make it super simple to work with. For example you can define a motor then tell it to go forwards or backwards. Very very highly recommended.

Besides that, we set up the code, wait for someone to connect over web sockets when they do we send data from the sensors and the webcam to the web client. And we send control messages to the the motors from the gyro in the phone.

Let’s drive

http://github.com/ajfisher/ajnodebothttp://www.youtube.com/watch?v=Jkbcn1vA7ykDemoed live

In action then. Here we go.

Let’s drive

The audience at MelbJS

So this is pretty much 100% web tech driving this robot around.

Go and makeArduino.cc (start here for everything)Freetronics (support Aussie / Melbourne hardware designers)Make magazine(Great ideas and skill development)Johnny Five (Brilliant JS library to work with hardware)

So there’s plenty of ways all of you can start playing with hardware.

You can go low level with things like C or high level with already built devices. But you can also sit in the middle and prototype as well.

The stack I’ve talked about is very solid, it works really well and will allow you to experiment very efficiently. You can all do this and I’m sure build a robot much better than this one I’ve got here.

Don’t be worried about playing with hardware - it’s loads of fun - just stay away from mains electricity.

These links should get you started.

Nodebots Day Melbs

Saturday October 12, 2013Play with hardwareBuild web controlled things

Now if this has got you interested in playing with hardware I have some news for you.

A couple of weeks ago we ran a Nodebots day in Sydney with about 40 web devs who all came together to learn about and play with hardware. As it was really successful - you can see some pics there, we’re going to run one down here in Melbourne as well.

It is aimed at devs who haven’t had much hardware experience and will be a whole day event on Saturday October 12.

Thanks to Mark, Seek has provided us some space to use. I will have some more info soon so watch MelbJS and my twitter feeds for info over the next week around pricing and availability etc.

Building droids with JavaScript

Andrew Fisher @ajfisher (twitter, github, adn etc)MelbJS, 14 August 2013

So thanks a lot and if you have any questions now or after please let me know.