filtered_websocket DocumentationRelease 0.4.1

Morgan Phillips

July 29, 2014

Contents

1 Install 3

2 Getting Started 5

3 Using Redis 7

4 Custom Modules 9

5 Configuration Files 115.1 Default Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115.2 Writing Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125.3 A Closer Look At Redis Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135.4 Supported Default Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145.5 Using The Redis Storage Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145.6 Writing Modules For Redis PubSub . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

6 Indices and tables 17

Python Module Index 19

i

ii

filtered_websocket Documentation, Release 0.4.1

Filtered WebSocket is a straight forward server framework which allows you to compose complex behaviors withoutusing callbacks or subclassing any protocols. It also features redis integration for remote storage and message passingvia pubsub.

Contents 1

filtered_websocket Documentation, Release 0.4.1

2 Contents

CHAPTER 1

Install

pip install filtered_websocket

or

git clone https://github.com/mrrrgn/filtered_websocket && cdfiltered_websocket && python setup.py install

3

filtered_websocket Documentation, Release 0.4.1

4 Chapter 1. Install

CHAPTER 2

Getting Started

After installing filtered_websocket you’ll have access to the server launcher via fws_server. Runningfws_server -h will give you a list of configuration options while simply running fws_server will start aserver running the default broadcast_messages and stdout_messages modules on port 9000.

filtered_websocket is bundled with several useful modules which may be attached to a server via the ‘-f’ option likeso:

# This will start a server which prints rawdata and broadcasts messages to users who have set a "token"$ fws_server -f filtered_websocket.filters.stdout_rawdata filtered_websocket.filters.broadcast_messages_by_token

5

filtered_websocket Documentation, Release 0.4.1

6 Chapter 2. Getting Started

CHAPTER 3

Using Redis

Redis integration is activated by setting an environment variable STORAGE_OBJECT_MODULE tofiltered_websocket.storage_objects.redis. When the redis storage object is activatedfws_server -h will display new argument options for configuring your redis instance connection and channelsubscriptions.

The following would start a server which broadcasts messages received from a redis channel named global to allconnected clients:

$ export STORAGE_OBJECT_MODULE="filtered_websocket.storage_objects.redis"$ fws_server -f filtered_websocket.filters.broadcast_pubsub

This message, published via a redis client, connected to the same redis instance, would be sent to all connectedWebSocket clients:

>>> import redis>>> r = redis.Redis() # connections to localhost on default port>>> r.publish("global", "hello from the server!") # Connected WebSocket clients receive this message

To start a server which broadcasts messages both between clients and from remote, server side, processes to clients:

$ export STORAGE_OBJECT_MODULE="filtered_websocket.storage_objects.redis"$ fws_server -f filtered_websocket.filters.broadcast_messages filtered_websocket.filters.broadcast_pubsub

7

filtered_websocket Documentation, Release 0.4.1

8 Chapter 3. Using Redis

CHAPTER 4

Custom Modules

Writing servers with filtered_websocket is extremely terse. Here is an entire broadcast/chat server:

# saved as broadcast.pyClass Broadcast(WebSocketMessageFilter):@classmethodfilter(cls, web_socket_instance, data):

for user_id, user_instance in web_socket_instance.users.items():user_instance.sendMessage(bytes(data))

To run the broadcast server written above simply run: fws_server -f broadcast

9

filtered_websocket Documentation, Release 0.4.1

10 Chapter 4. Custom Modules

CHAPTER 5

Configuration Files

Any argument from the help menu may be used to create a json formatted config file. Simply use the long form (–prefixed) option names as keys and pass in arguments as values. Any filters specified should be added within a list:

# config.json{

"port": "9999","filters": ["filtered_websocket.filters.broadcast_messages_by_token", "filtered_websocket.filters.stdout_messages"]

}

# Passing it in creates a broadcast by token server which prints all messages to stdout$ fws_server -c config.json

Contents:

5.1 Default Modules

Any or all of these may be used to compose a custom server via: fws_server -f

class filtered_websocket.filters.broadcast_messages.BroadcastMessagesFilterBroadcasts messages to all connected clients.

class filtered_websocket.filters.broadcast_messages_by_token.TokenMessageFilterAllows clients to register a token group by prefixing an id with “token:” Once registered all messages sent bythe client will only be received by other clients registered under the same key.

class filtered_websocket.filters.broadcast_messages_by_token.TokenDisconnectFilterRemove ids of clients no longer connected.

class filtered_websocket.filters.broadcast_pubsub.BroadcastPubSubFilterWill broadcast all consumed events to connected websocket clients.

class filtered_websocket.filters.stdout_rawdata.StdoutRawDataFilterPrint any client data received to the server console’s stdout.

class filtered_websocket.filters.stdout_messages.StdoutMessageFilterPrint any client messages received to the server console’s stdout.

class filtered_websocket.filters.stdout_pubsub.StdoutPubSubFilterPrint any consumed queue data to the server console’s stdout.

11

filtered_websocket Documentation, Release 0.4.1

5.2 Writing Modules

5.2.1 Introduction

In Filtered WebSocket, instead of using callbacks, new behaviors are added by writing modules which inherit frombase WebSocketFilter types. Each base filter class represents an entry point into a chain of modules which will run, inthe order in which they were added, whenever some event happens. FilterBase and FilterMeta allow for the simplecreation of a filter chains. Any class that inherits from a child of FilterBase and FilterMeta will have its filter methodcalled upon run being executed from its parent class. Here class A constructs a new filter chain, which B and C becomemembers of.

>>> class A(FilterBase, metaclass=FilterMeta)>>> pass>>>>>> class B(A):>>> @classmethod>>> def filter(cls, web_socket_instance, data):>>> print("foo")>>>>>> class C(A):>>> @classmethod>>> def filter(cls, web_socket_instance, data):>>> print("bar")>>>>>> A.run(web_socket_instance, None)foobar

Each filter has access to a server instance with a dictionary of user ids mapped to user objects, a storage object, and anoptional token object for storing credentials. It also receives payload data in the form of a bytearray().

Filter Module Types

There are four filter chains, each corresponding to a callback.

class filtered_websocket.filters.base.WebSocketDataFilterRuns whenever a web socket server instance receives any data from a client.

class filtered_websocket.filters.base.WebSocketMessageFilterRuns whenever a web socket server instance receives a full data frame.

class filtered_websocket.filters.base.WebSocketDisconnectFilterRuns whenever a user disconnects from a web socket server instance (passes in no data).

class filtered_websocket.filters.base.WebSocketConsumerFilterRuns whenever data is popped off of a web socket server instance’s queue.

Example Module

To write a module that responds to some sort of event just make it a subclass of the corresponding parent and override“filter” as either a classmethod or a static method. Let’s write a rawdata module that responds to my name:

# Saved as hi_morgan.pyclass HiMorgan(WebSocketDataFilter):

@classmethodfilter(cls, web_socket_instance, data):

12 Chapter 5. Configuration Files

filtered_websocket Documentation, Release 0.4.1

payload = bytes(data)if payload.lower().strip() == b’morgan’:

print(’Hi Mrrrgn!’)

To inject the module into the server import it via the ‘-f’ option:

$ fws_server -f hi_morgan

$ echo "Morgan" | nc localhost 9000$ Hi Mrrrgn!

Building Pipelines via Data Mutability & Run Order

Filter modules run in the order that they are imported and are passed mutable object refernces, meaning, they may becomposed into pipelines. The filter below pops the last character off of any data sent, when combined with a defaultfilter module which prints client messages it demonstrates that data has indeed been manipulated en route:

# Saved as popchar.pyclass PopChar(WebSocketMessageFilter):

@classmethodfilter(cls, web_socket_instance, data):

data.pop() # remove last character from the bytearray

# Server console$ fws_server -f popchar filtered_websocket.filters.stdout_messages

# From a browser JS console>>> var ws = new WebSocket("ws://localhost:9000");>>> ws.send("Test");

# Server console$ Tes

Now, if we reverse the order of the modules the data is printed before the character is popped:

# Server console$ fws_server -f filtered_websocket.filters.stdout_messages popchar

# From a browser JS console>>> var ws = new WebSocket("ws://localhost:9000");>>> ws.send("Test");

# Server console$ Test

Additional Examples

For additional module examples please see the default modules.

5.3 A Closer Look At Redis Integration

Along with filter modules, filtered_websocket also supports pluggable back end storage via the StorageObject andPubSubStorageObject interfaces. To use redis as a distributed storage object and message broker simply set theenvironment variable “STORAGE_OBJECT_MODULE”

5.3. A Closer Look At Redis Integration 13

filtered_websocket Documentation, Release 0.4.1

export STORAGE_OBJECT_MODULE=filtered_websocket.storage_objects.redis

If you look at fws_server you’ll now notice some new options for managing your redis connection:

fws_server -husage: fws_server [-h] [-p PORT] [-c CONFIG] [-f [FILTERS [FILTERS ...]]]

[-key KEY] [-cert CERT] [--redis_host REDIS_HOST][--redis_port REDIS_PORT][--redis_channels [REDIS_CHANNELS [REDIS_CHANNELS ...]]][--redis_key REDIS_KEY]

optional arguments:-h, --help show this help message and exit-p PORT, --port PORT The listening port.-c CONFIG, --config CONFIG

A JSON config file.-f [FILTERS [FILTERS ...]], --filters [FILTERS [FILTERS ...]]

Filters to import at runtime.-key KEY A key file (ssl).-cert CERT A certificate file (ssl).--redis_host REDIS_HOST--redis_port REDIS_PORT--redis_channels [REDIS_CHANNELS [REDIS_CHANNELS ...]]

pubsub channels to subscribe to.--redis_key REDIS_KEY

A key prefix.

5.4 Supported Default Modules

By default the redis storage object listens to a channel called “global” and will attach any messages it receives to theserver instances’s queue. As such messages from redis may be handled by the consumer filter. As an example examinethe default stdout_pubsub filter module:

‘‘fws_server -f filtered_websocket.filters.stdout_pubsub‘‘

class filtered_websocket.filters.stdout_pubsub.StdoutPubSubFilterPrint any consumed queue data to the server console’s stdout.

Importing the stdout_pubsub filter will print all redis messages captured by the server to your console:

import redisredis_client = redis.Redis()redis_client.publish("global", "hello from the server!")

To send redis messages to connected clients use the broadcast_pubsub filter:

fws_sever -f filtered_websocket.filters.broadcast_pubsub

class filtered_websocket.filters.broadcast_pubsub.BroadcastPubSubFilterWill broadcast all consumed events to connected websocket clients.

5.5 Using The Redis Storage Object

Inside your modules any data written to web_socket_instance.storage_object will go to redis. Thestorage object interface is defined as:

14 Chapter 5. Configuration Files

filtered_websocket Documentation, Release 0.4.1

web_socket_instance.storage_object.get(key) # Try to grab some data from redisweb_socket_instance.storage_object.add(key, value) # Add some data to redisweb_socket_instance.remove(key, value) # delete something in a fashion akin to storage_object[key].remove(value)

5.6 Writing Modules For Redis PubSub

All data received from pubsub subscriptions will be attached to the server’s queue and later consumed. So, moduleswhich wish to interact with these events should inherit from:

class filtered_websocket.filters.base.WebSocketConsumerFilterRuns whenever data is popped off of a web socket server instance’s queue.

5.6. Writing Modules For Redis PubSub 15

filtered_websocket Documentation, Release 0.4.1

16 Chapter 5. Configuration Files

CHAPTER 6

Indices and tables

• genindex

• modindex

• search

17

filtered_websocket Documentation, Release 0.4.1

18 Chapter 6. Indices and tables

Python Module Index

ffiltered_websocket.filters.base, 12

19