TWISTED AS AN IOTCONTROLLER.

BUILDING A 300+ SEAT IFE CONTROLLER IN TWISTED. / / David P. Novakovic @dpn dpn.name

QUICK OVERVIEWWTF is Twisted?IFE System DevicesIFE System requirementsCode examples

TWISTED THE SLAYER OF NOOBSSUPER POWERFUL

ALSO CONTROVERSIALSeems to have a bad rep with people who don't use it.

TWISTEDevent driven framework in pythonparticular focus on networking related thingsfrom web down to serial ports, tun/tap and udevasync io - single process can handle thousands of concurrentopen connections on a low end machineClean APIs for both callbacks and deferredsthread pool with deferred interfacelocking "primitives" - DeferredSemaphore, DeferredLock,DeferredQueue

MORE TWISTEDApplication frameworkAsync unit testingIntegration with most major event loops(wx/gtk/qt/gevent/etc)third party libsklein - sinatra/flask style web frameworkcyclone - tornado port to twistedtreq - "requests" style libampoule - nice enough (if slightly neglected) process poolimplementation

IFE STUFF.. YAY

TRADITIONAL"dumb" screens heavy use of streaming contentvery heavy requirements on serversvery expensive to upgrade

GLIDEHeavier clientssystem can be upgraded by replacing playersplane doesnt need rewiring etcserver can remain fairly unchangedmore importantly don't need to be upgraded in lockstep

GLIDE SOLUTIONplayers - embedded debianseat disconnect - custom embedded devicetriple redundant power supplies - switch + powercm controller - dual core atom running ubuntu LTS + lots ofembedded things attached

SYSTEM

SOME RELEVANT REQUIREMENTSmulticast commands out to embedded devices300+ seat updates over HTTP every secondlisten to audio stream over multicast (PA, pilot etc)low latency control of players ie. if pilot talks/decomptelneting into a streaming VLC process.some legacy - sync code needed to run in threadsrespond to and control hardware in the plane (overheadscreens etc)cabin crew inserting HDD with content (lock down usb)downloading content from the web (at gate)kiosk (lock down control keys/usb ports)manhole for debugging a running processssh reverse tunnel for remote access - conchtftp - firmware updates to players

MULTICASTfrom twisted.internet import reactor, task, protocol

class MulticastSeatControl(protocol.DatagramProtocol):

def startProtocol(self): self.transport.joinGroup("228.0.0.5")

def sendSeatControl(self, bytes): self.transport.write(bytes, ("228.0.0.5", 8005))

def datagramReceived(self, datagram, address): print "Datagram %s from %s" % (repr(datagram), repr(address))

HTTP INTERFACEimport jsonfrom klein import Klein

class DeviceAPI(object): app = Klein()

def __init__(self, cmc): self._cmc = {}

@app.route('/stat/<string:name>') def stat(self, request): body = json.loads(request.content.read()) self._cmc.logstat(body['something']) request.setHeader('Content-Type', 'application/json') return json.dumps({"status": "success"})</string:name>

SERIAL PORTfrom twisted.internet.serialport import SerialPortfrom twisted.internet import reactorfrom twisted.protocols.basic import LineReceiver

class SensorProtocol(LineReceiver): def connectionMade(self): print "Connected to serial port."

def lineReceived(self, line): print "Received line:", line

def send_our_command(self, command): self.sendLine("command:%s" % command)

def connect_serial_port(): return SerialPort(SensorProtocol(), "/dev/ttyACM0", reactor, 115200, rtscts=False, xonxoff=False, timeout=1)

DJANGO

WAIT.. WHAT?Yep, ORM is used a fair bit, as is admin.

DJANGOfrom django.core.handlers.wsgi import WSGIHandlerfrom twisted.python import threadpoolfrom twisted.internet import reactorfrom somewhere import Root

def gimme_some_django(): pool = threadpool.ThreadPool() wsgi_resource = wsgi.WSGIResource(reactor, pool, WSGIHandler()) r = Root(wsgi_resource) s = server.Site(r) pool.start() return internet.TCPServer(8082, s)

eg. django admin available at http://localhost:8082/admin/

https://github.com/clemesha/twisted-wsgi-django

YET MOREclass VLCRemoteControlProtocol(LineReceiver): ...

class UDevMonitor(abstract.FileDescriptor): ...

AND TELNET INTO YOUR OWN PROCESS..import twisted.manhole.telnetfrom twisted.internet.endpoints import TCP4ServerEndpoint

def start_manhole(service): f = twisted.manhole.telnet.ShellFactory() f.username = "b" f.password = "b" f.namespace["var1"] = service endpoint = TCP4ServerEndpoint(reactor, 7777) endpoint.listen(f)

TIE IT ALL TOGETHERApplication frameworkUnit Testing

APPLICATION FRAMEWORKTWISTD

Daemonises optionallypidfilelog filesError catchallhttp://twistedmatrix.com/documents/13.2.0/core/howto/basics.html

CREATE SERVICESclass IOTCServer(Service): def startService(self): self.serialport = connect_serial_port() p = MulticastSeatControl() reactor.listenMulticast(8005, p) start_manhole(self) self.mc_looper = task.LoopingCall(p.sendSeatControl, "s1:on,s2:on") self.mc_looper.start(1)

def stopService(self): self.serialport.transport.loseConnection() self.mc_looper.stop()

def get_http_service(iots_instance): device_api = DeviceAPI(iots_instance) server.Site(device.api.app.resource()) return internet.TCPServer(8082)

TWISTD PLUGINclass Options(usage.Options): optParameters = [["serialport", "s", "/dev/ttyACM0", "Serial port."]

class ServiceMaker(object): tapname = "iotc" description = "The IOTC Server" options = Options

def makeService(self, options): service_parent = service.MultiService() iots = IOTCServer() iots.setServiceParent(service_parent) http_service = get_http_service() http_service.setServiceParent(service_parent) django_service = gimme_some_django() django_service.setServiceParent(service_parent) return service_parentserviceMaker = ServiceMaker()

yourproject/twisted/plugins/iotc_plugin.py$ twistd -r epoll -n iotc --serialport=/dev/ttyACM0

UNIT TESTINGTwisted TrialWrapper around pyunit "unittest"Allows tests to return deferred responsessubclass twisted.trial.unittest.TestCaserun tests: trial yourpackageSome helpers to let you mock wire-level connectionshttp://twistedmatrix.com/documents/11.1.0/core/howto/trial.html

BUILD AWESOME THINGS!DIGECOR IS HIRING

If you enjoy working on this kind of stuff, let me know and I'llforward your details onto digEcor.

@dpn

THE ENDDAVID NOVAKOVIC - DPN.NAME