LECTURE 16 Twisted

A SIMPLE TCP SERVER

To begin our introduction to Twisted, we’ll build a simple TCP server.

REACTOR BASICS

At the core of any Twisted application is the reactor loop. The reactor loop works by waiting for events to occur and then calls the relevant event handler (“dispatches the event”). The simplest Twisted program is the following, which accesses the reactor object and starts it.

from twisted.internet import reactorreactor.run()

REACTOR BASICS

At the core of any Twisted application is the reactor loop. The reactor loop works by waiting events to occur and then calls the relevant event handler (“dispatches the event”). The simplest Twisted program is the following, which accesses the reactor object and starts it.

The reactor loop runs in the main and only thread. Once we run the reactor, it assumes control of the program. If there is nothing to do (as is the case here), the reactor loop just sits idle, not consuming CPU resources.

from twisted.internet import reactor # Not created explicitly! Just import it. reactor.run()

REACTOR BASICS

The reactor provides basic interfaces to a number of services, including network communications, threading, and event dispatching. There are several reactor options, each specialized for a particular application. The default reactor on most systems is a Select-based reactor.

To use a different reactor, you must “install” it.

from twisted.internet import pollreactorpollreactor.install() # When reactor is imported, it will be a pollreactor

from twisted.internet import reactor # Always import reactor right before you run itreactor.run()

REACTOR BASICS

The core required functionality of a reactor is defined in twisted.internet.interfaces.IReactorCore. These include:

• run() – move the reactor to a “running” state. Starts the main loop.

• stop() – shutdown the reactor.

• callWhenRunning() – call a function when the reactor is running.

• addSystemEventTrigger() -- add a function to be called when a system event occurs.

• fireSystemEvent() -- fire a system-wide event.

There are additional methods defined in IReactorProcess, IReactorTCP, IReactorSocket, IReactorThreads, IReactorSSL, etc…The choice of reactor determine which/how these methods are implemented.

REACTOR BASICS

Here’s an example Twisted program which registers a function call to be made when the reactor starts running.

Twisted has a basic logging mechanism defined in twisted.python.log.

import sysfrom twisted.python import loglog.startLogging(sys.stdout)

def func(x):log.msg("In func as event handler")log.msg(str(x))log.msg(“Shutting down now!")reactor.stop()

from twisted.internet import reactorreactor.callWhenRunning(func, “Hello!")reactor.run()

REACTOR BASICS

$ python twist.py 2015-02-17 11:46:16-0500 [-] Log opened.2015-02-17 11:46:16-0500 [-] In func as event handler2015-02-17 11:46:16-0500 [-] Hello!2015-02-17 11:46:16-0500 [-] Shutting down now!2015-02-17 11:46:16-0500 [-] Main loop terminated

import sysfrom twisted.python import loglog.startLogging(sys.stdout)

def func(x):log.msg("In func as event handler")log.msg(str(x))log.msg(“Shutting down now!")reactor.stop()

from twisted.internet import reactorreactor.callWhenRunning(func, “Hello!")reactor.run()

REACTOR BASICS

import sys, timefrom twisted.python import loglog.startLogging(sys.stdout)

def func(x):log.msg(str(x))now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S", now)))log.msg(“Shutting down now!")reactor.stop()

now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S", now)))from twisted.internet import reactorreactor.callLater(5, func, “Hello after 5 seconds!”)reactor.run()

Another example using the callLater scheduling method defined in IReactorTime.

REACTOR BASICS

import sys, timefrom twisted.python import loglog.startLogging(sys.stdout)

def func(x):log.msg(str(x))now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S", now)))log.msg(“Shutting down now!")reactor.stop()

now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S", now)))from twisted.internet import reactorreactor.callLater(5, func, “Hello after 5 seconds!”)reactor.run()

$ python twist.py

2015-02-17 11:49:10-0500 [-] Log opened.

2015-02-17 11:49:10-0500 [-] 15/02/17 11:49:10

2015-02-17 11:49:15-0500 [-] Hello after 5 seconds!

2015-02-17 11:49:15-0500 [-] 15/02/17 11:49:15

2015-02-17 11:49:15-0500 [-] Shutting down now!

2015-02-17 11:49:15-0500 [-] Main loop terminated.

REACTOR BASICS

import sys, timefrom twisted.python import loglog.startLogging(sys.stdout)

def func(x):log.msg(str(x))now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))log.msg("Fire FireworkEvent after 3 seconds…")reactor.callLater(3, reactor.fireSystemEvent, ‘FireworkEvent')

def Firework_CustomEventHandler():now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))log.msg(“BABY YOU’RE A FIIIIIIIIREWOOOOOORK")reactor.stop()

now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))from twisted.internet import reactorreactor.callLater(5, func, "func called after 5 sec")reactor.addSystemEventTrigger('during', ‘FireworkEvent', Firework_CustomEventHandler)reactor.run()

Let’s create a custom self-esteem boosting event.

REACTOR BASICS

import sys, timefrom twisted.python import loglog.startLogging(sys.stdout)

def func(x):log.msg(str(x))now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))log.msg("Fire FireworkEvent after 3 seconds…")reactor.callLater(3, reactor.fireSystemEvent, ‘FireworkEvent')

def Firework_CustomEventHandler():now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))log.msg(“BABY YOU’RE A FIIIIIIIIREWOOOOOORK")reactor.stop()

now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))from twisted.internet import reactorreactor.callLater(5, func, "func called after 5 sec")reactor.addSystemEventTrigger('during', ‘FireworkEvent', Firework_CustomEventHandler)reactor.run()

Let’s create a custom self-esteem boosting event.

addSystemEventTrigger can be done

‘before’, ‘during’, or ‘after’ event.

REACTOR BASICS

import sys, timefrom twisted.python import loglog.startLogging(sys.stdout)

def func(x):log.msg(str(x))now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))log.msg("Fire FireworkEvent after 3 seconds…")reactor.callLater(3, reactor.fireSystemEvent, ‘FireworkEvent')

def Firework_CustomEventHandler():now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))log.msg(“BABY YOU’RE A FIIIIIIIIREWOOOOOORK")reactor.stop()

now = time.localtime(time.time())log.msg(str(time.strftime("%y/%m/%d %H:%M:%S",now)))from twisted.internet import reactorreactor.callLater(5, func, "func called after 5 sec")reactor.addSystemEventTrigger('during', ‘FireworkEvent', Firework_CustomEventHandler)reactor.run()

$ python twist.py

2015-02-17 11:53:43-0500 [-] Log opened.

2015-02-17 11:53:43-0500 [-] 15/02/17 11:53:43

2015-02-17 11:53:48-0500 [-] func called after 5 sec

2015-02-17 11:53:48-0500 [-] 15/02/17 11:53:48

2015-02-17 11:53:48-0500 [-] Fire FireworkEvent after 3 seconds...

2015-02-17 11:53:51-0500 [-] 15/02/17 11:53:51

2015-02-17 11:53:51-0500 [-] BABY YOU'RE A FIIIIIIIIREWOOOOOORK

2015-02-17 11:53:51-0500 [-] Main loop terminated.

REACTOR BASICS

Let’s back up for a second and make sure we understand what’s going on.

• Our callback code runs in the same thread as the Twisted loop.

• When our callbacks are running, the Twisted loop is not running. And vice versa.

• The reactor loop resumes when our callback returns.

Our code blocks the reactor loop so it should return asap.

Before we continue writing our TCP server, we need to learn about a couple of abstractions.

TRANSPORT BASICS

The Transport abstraction is defined in twisted.internet.interfaces.ITransport.

A Twisted Transport represents a single connection that can send/receive data.

The Transport abstraction represents any such connection and handles the details of asynchronous I/O for whatever sort of connection it represents. The methods defined in Itransport are:

• write(data) – send some data.

• writeSequence(list_of_data) – send a sequence of data.

• loseConnection() – close connection.

• getPeer() – get remote address of other side of connection.

• getHost() – get address of this side of connection.

TRANSPORT BASICS

You may have noticed that there are no methods for reading data. The Transport object will make a callback when it receives data – we don’t have to explicitly make it happen.

Also, note that these methods are really just suggestions. Remember, Twisted is in control, not us. So when we tell the Transport to write some data, we’re really asking it to write some data whenever it is able to.

PROTOCOLS

The Protocol abstraction is defined in twisted.internet.interfaces.IProtocol.

Protocols implement protocols. This could be one of the built-in protocols in twisted.protocols.basic or one of your own design.

Strictly speaking, a Protocol instance implements the protocol for a single connection. This connection is represented by a Transport object.

So every connection requires its own Protocol instance (and therefore, Protocol is a good candidate for storing stateful information about the connection).

PROTOCOLS

Methods of the IProtocol class include:

• dataReceived(data) – called whenever data is received (transport’s callback!)

• connectionLost(reason) – called when connection is shut down.

• makeConnection(transport) – associates transport with protocol instance to make connection.

• connectionMade() – called when a connection is made.

PROTOCOL FACTORIES

The Protocol Factory object is defined in twisted.internet.interfaces.IProtocolFactory.

Protocol Factories simply create Protocol instances for each individual connection. Of interest is just one method:

• buildProtocol(addr): The buildProtocol method is supposed to return a new Protocol instance for the connection to addr. Twisted will call this method to establish Protocols for connections.

SIMPLE TCP SERVER

from twisted.internet.protocol import Protocol, Factory

class Echo(Protocol): # reactor will call makeConnection with current transport instance.

def dataReceived(self, data): # when transport executes callback with data, just echo

self.transport.write(data)

f = Factory()f.protocol = Echo # f.buildProtocol() will create an instance of f.protocol

from twisted.internet import reactorreactor.listenTCP(9000, f) # listenTCP defined in IReactorTCP, which the default reactor inherits

reactor.run()

SIMPLE TCP SERVER

from twisted.internet.protocol import Protocol, Factory

class Echo(Protocol):def dataReceived(self, data):

self.transport.write(data)

f = Factory()f.protocol = Echofrom twisted.internet import reactorreactor.listenTCP(9000, f) reactor.run()

Try “telnet localhost 9000”.

Anything you send will be echoed back to you.

The only thing we’re doing explicitly is running

the server. Everything that is executed from that

point forward is handled and scheduled by Twisted.