Pico Documentation

Pico DocumentationRelease 2.0.4

Fergal Walsh

Nov 10, 2017

Contents

1 Features 3

2 Installation 5

3 The User Guide 73.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3 Decorators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.4 The Javascript Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.5 Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.6 Development Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.7 Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.8 The PicoApp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4 Acknowledgements 19

Python Module Index 21

i

ii

Pico Documentation, Release 2.0.4

Release v2.0.4. (Installation)

Pico is a very small HTTP API framework for Python. It enables you to write HTTP APIs with minimal fuss, overheadand boilerplate code.

Listing 1: example.py

import picofrom pico import PicoApp

@pico.expose()def hello(who="world"):

s = "hello %s!" % whoreturn s

@pico.expose()def goodbye(who):

s = "goodbye %s!" % whoreturn s

app = PicoApp()app.register_module(__name__)

Start the development server:

python -m pico.server example

Call your http api functions from with any http client:

curl http://localhost:4242/example/hello/curl http://localhost:4242/example/hello/?who="fergal"curl http://localhost:4242/example/goodbye/?who="fergal"

Use the Javascript client.

Listing 2: index.html

<!DOCTYPE HTML><html><head><title>Pico Example</title>

<script src="/example.js"></script></head><body><p id="message"></p><script>example.hello("Fergal").then(function(response){document.getElementById('message').innerHTML = response;

});</script>

</body></html>

Contents 1


Use the Python client.

Listing 3: client.py

import pico.client

example = pico.client.load('http://localhost:4242/example')example.hello('World')

2 Contents

CHAPTER 1

Features

• Automatic URL routing

• Decorators

• Automatic JSON serialisation

• Automatic argument parsing & passing

• Simple streaming responses

• Development server with debugger

• WSGI Compliant

• Built upon Werkzeug

• Python Client

• Javascript Client

3

http://werkzeug.pocoo.org/


4 Chapter 1. Features

CHAPTER 2

Installation

Simply install with pip:

pip install -U pico

5


6 Chapter 2. Installation

CHAPTER 3

The User Guide

An introduction to Pico and a guide to how to use it.

3.1 Introduction

Pico is designed to help you build a HTTP API with the minimal amount of interference and maximal relief frommundane tasks. It enables you to focus on the application logic while it takes care of all the lower level details ofURL routing, serialisation and argument handling. I believe that writing a HTTP API handler should be no different towriting a normal Python module function. Instead of writing functions that take Request objects and extract argumentsfrom GET or POST data structures we should be writing functions that take the arguments we need to use. Insteadof serialising our result objects and returning Response objects we should simply return our result objects. Theframework can and should take care of those details for us. It is not just a case of being lazy. It is more about writingclean, maintainable, reusable and testable code.

3.1.1 URL Routing

With Pico you name your API endpoint once when you write your handler function. The name of the function isused to name the endpoint. If you have a function called profile in a module called users then its URL will be/users/profile. You have no choice over the URL so it is one less thing to think about. It is also always clearwhich module a function is defined in when you look at the URL.

When Pico handles a request it finds the appropriate handler function by looking up a dictionary mapping URLs tofunctions.

3.1.2 Argument Passing

Pico creates a keyword argument dictionary, kwargs, from the GET and POST parameters in the request object andpasses this to the handler function as handler(**kwargs). The expected parameters for an endpoint are definedby the arguments in the function signature. The usual Python semantics apply for default arguments. It is an error torequest an endpoint with parameters it does not expect or to not supply a value for an argument with no default value:

7


@pico.expose()def hello(who='world'):

return 'Hello %s' % who

curl http://localhost:4242/example/hello/>>> "Hello world"

curl http://localhost:4242/example/hello/?who=Fergal>>> "Hello Fergal"

curl http://localhost:4242/example/hello/?spam=foo>>> 500 INTERNAL SERVER ERROR>>> ....>>> TypeError: hello() got an unexpected keyword argument 'spam'

3.1.3 Request Arguments

Most web application backends require access to some properties of the Request object sooner or later for uses otherthan accessing GET or POST data. You may need the client IP address, headers, cookies, or some arbitrary value setby some other WSGI middleware. Most frameworks usually provide access to the Request object as an argument toevery handler, a property of the handler class, a global, or via a module level function. Pico takes a different approach.

Any handler function may accept any property of the request object as an argument. The author indicates this to Picoby using the @request_args decorator to specify which arguments should be mapped to which properties. Whenthe handler function is called by Pico these arguments are populated with the appropriate values from the Requestobject. In this example we need the users IP address:

@pico.expose()@request_args(ip='remote_addr')def list_movies(ip):

client_country = lookup_ip(ip)movies = fetch_movies(client_country)...

The Request object in Pico is an instance of werkzeug.wrappers.Request so you can refer to its documentation tosee all available attributes.

Note: If the HTTP client passes a value for an argument mapped with @request_args it will be ignored. Forexample the following would not give you a list of movies in South Korea:

curl http://localhost:4242/example/list_movies/?ip="42.42.42.42"

In another situation we may want to get the username of the currently logged in user. We could pass the cookies headerand the authentication token header and basic auth header and check each inside our function to see if there is a loggeduser. This would be quite messy though, especially when we need this value in many different functions. Instead wecan use @request_args with a helper function to return a computed property of the Request object:

def current_user(request):# check basic_authif request.authorization:

auth = request.authorizationif not check_password(auth.username, auth.password):

raise Unauthorized("Incorrect username or password!")return auth.username

8 Chapter 3. The User Guide

http://werkzeug.pocoo.org/docs/0.11/wrappers/#werkzeug.wrappers.Request


# check for session cookieelif 'session_id' in request.cookies:

user = sessions.get(request.cookies['session_id'])return user['username']

else:...

@pico.expose()@request_args(user=current_user)def profile(user):

return Profiles.get(user=user)

@pico.expose()@request_args(user=current_user)def save_post(user, post):

pass

By explicitly specifying which properties of the Request object we want to use we keep the code cleaner and easierto understand and maintain. It also allows us to continue to use the functions from other code without having to passa request object. If our function needs an IP address then we simply pass a string IP address, not a Request objectcontaining an IP address. The same applies for testing. We don’t need to mock the Request object for most tests.We write tests for our API in the same way as any other library.:

class TestMoviesList(unittest.TestCase):

def test_movies_ireland(self):movies = example.list_movies('86.45.123.136')self.assertEqual(movies, movies_list['ie'])

As you can see this is a normal (contrived) unit test without mocked request objects. We simply test the public interfaceof our module.

The only exceptions of course are helper functions like get_user above which operate directly on the Requestobject. They should be properly tested with a mock Request object. There should be very few such functions in atypical application however.

Note: The arguments specified with @request_args are only populated when the function is called by Pico. Ifthe function is called directly (inside another function, in a script, in the console, etc) this decorator is a nop.

3.1.4 Protectors

There are other situations where you may need to access properties of the Request object to check if the functionmay be called with the used HTTP method, by the current user or from the remote IP address, for example. Thesechecks are part of your application logic but are usually not specific to an individual function and not necessarilyrelated to the actual function being called. For example imagine we have a function to delete posts:

@pico.expose()def delete_post(id):

# delete the post

We want to restrict this endpoint to admin users. We could do the following:

3.1. Introduction 9


@pico.expose()@request_args(user=current_user)def delete_post(id, user):

if user in admin_users:# delete the post

else:raise Forbidden

This works but now we have made our function dependant on a user even though the actual user isn’t relevant to thereal logic of the function. If we want to use this function elsewhere in our code we need to pass a admin user as aparameter just to pass the check. Pico provides another decorator to help with this common situation: @protected:

def is_admin(request, wrapped, args, kwargs):user = current_user(request)if user not in admin_users:

raise Forbidden

@pico.expose()@protected(is_admin)def delete_post(id):

# delete the post

If the protector function (is_admin) doesn’t return False or raise an exception then the function is executed asnormal. As you can see from the protector’s signature it can use any of the request object, function object, args andkwargs in its decision to pass or raise.

Note: Just like @request_args, @protected is only active when Pico calls the function. If it is called directlyelsewhere the decorator is a nop.

3.2 Installation

Simply install with pip:

pip install -U pico

3.3 Decorators

Pico includes a number of useful decorators. These decorators all use the request object in some way. They are onlyactive when the decorated function is called from Pico. If the the function is imported and called normally like anyother Python function then the decorator is a nop.

@pico.expose(*args, **kwargs)Exposes the decorated function via the HTTP API.

Note: This decorator must be the final decorator applied to a function (it must be on top).

@pico.prehandle(*args, **kwargs)Used to decorate a function of the form f(request, kwargs) which is called before the handler function is called.

This can be used to modify the request object (e.g. for setting the .user attribute based on cookies or headers)or the kwargs dictionary passed to the the handler function (e.g. to pop out and check a common token queryparameter sent with every request):



@pico.prehandle()def set_user(request, kwargs):

# check basic_authif request.authorization:

auth = request.authorizationif not check_password(auth.username, auth.password):

raise Unauthorized("Incorrect username or password!")request.user = auth.username

# check for session cookieelif 'session_id' in request.cookies:

user = sessions.get(request.cookies['session_id'])request.user = user['username']

elif 'token' in kwargs:token = kwargs.pop('token')user = sessions.get(token)request.user = user['username']

else:...

@pico.decorators.request_args(*args, **kwargs)Passes the request object or attribute(s) of the request object to the decorated function. It has 3 different forms;a single argument, string keyword arguments, and functional keyword arguments.

To pass the request object specify the argument name:

@pico.expose()@request_args('req')def foo(req, something):

return req.remote_addr

To pass an attribute of the request object specify the argument and attribute as a keyword argument pair:

@pico.expose()@request_args(ip='remote_addr')def foo(ip, something):

return ip

To pass a value computed from the request object specify a keyword argument with a function:

def get_curent_user(request):# do somethingreturn request.user

@pico.expose()@request_args(user=get_curent_user)def foo(user, something):

pass

@pico.decorators.protected(protector)Protects a function by preventing its execution in certain circumstances.

Parameters protector (function) – A function of the form protector(request, wrapped, args,kwargs) which raises an exception or returns False when the decorated function should not beexecuted.

An example of a function that can only be called via POST:

3.3. Decorators 11


def post_only(request, wrapped, args, kwargs):if not request.method == 'POST':

raise MethodNotAllowed()

@pico.expose()@protected(post_only)def foo():

pass

@pico.decorators.require_method(method)Requires that a specific HTTP method is used to call this function.

Parameters method (str) – ‘GET’ or ‘POST’

Raises MethodNotAllowed – if the method is not correct.

The same example as above:

@pico.expose()@require_method('POST')def foo():

pass

@pico.decorators.stream(*args, **kwargs)Marks the decorated function as a streaming response. The function should be a generator that yield its response.The response is transmitted in the Event Stream format.

An example of a streaming generator that yields messages from pubsub:

@pico.expose()@stream()def subscribe(channels):

pubsub = redis.pubsub()pubsub.subscribe(channels)while True:

message = pubsub.get_message()yield message

3.4 The Javascript Client

The Pico Javascript client makes it simple to call your API from a web application. The client automatically generatesproxy module objects and functions for your API so you can call your API functions just like any other libraryfunctions. All serialisation of arguments and deserialisation of responses is taken care of by the client so you can focuson your application logic.

The only thing you need to be aware of is that all function calls are asynchronous returning promises.

3.4.1 Basic Structure

The basic structure of a web app using the Pico Javascript client is as follows:

1 <!DOCTYPE HTML>2 <html>3 <head>4 <title>Pico Example</title>


https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#Event_stream_format


5 8 <script src="/example.js"></script>9 </head>

10 <body>11 <p id="message"></p>12 <script>13 var example = pico.importModule('example')14

15 example.hello('Fergal')16 .then(function(response){17 document.getElementById('message').innerHTML = response;18 });19 </script>20 </body>21 </html>

• Line 6: We include Pico’s pico.js library inside the head of the document.

• Line 8: We load our example module definition as JavaScript in the head.

• Line 13: In a script element in the body we import our example module assigning it to the example variable.

• Line 15: We use our hello function.

• Line 16-18: We assign a callback to the promise.

The order and position of these elements within the document is important. pico.js must always be loaded before themodule is loaded and these both must be in the head of the document to ensure they have completed by the time theyare used later.

3.4.2 Promises

The proxy functions generated by the client are asynchronous, meaning that they will not wait for the result beforereturning. This is due to the nature of how HTTP requests work in the browser. Instead they immediately return apromise which later resolves and calls a callback with the result as a parameter. The promise object has twomethods of interest: then and catch.

var p = example.hello('world')p.then(function(result){

console.log(result)})p.catch(function(err){

console.error(err)})

The callback function passed to .then is called when the promise resolves successfully. If an error occurs then thefunction passed to .catch is called. If you don’t set a catch callback any errors are ignored.

The error object passed to catch callback contains a .message and .code property which describe the excep-tion that occurred on the Python side and the relevant HTTP status code.

3.4. The Javascript Client 13


3.4.3 API

Asynchronous functions

Each of these functions is asynchronous, so they return a Promise.

pico.loadAsync(module)

Arguments

• module (string) – The name of the module to load.

Load the Python module named module. The module proxy will be passed to the promise callback.

Submodules may be loaded by using dotted notation e.g. module.sub_module.

pico.reload(module_proxy)

Arguments

• module_proxy (object) – The module to reload.

Reload the module definition and recreate the module proxy for the supplied module_proxy object. Note thatmodule_proxy is a module proxy object, not a string.

Synchronous functions

pico.importModule(module)

Arguments

• module (string) – The name of the module to import.

Returns the proxy module object.

Note the module definition must have been previously loaded using pico.loadAsync or by loading /<module_name>.js in a script tag in the head of the document.

pico.loadModuleDefinition(module_definition)

Arguments

• module_definition (object) – An object representing the definition of the module.

Returns the proxy module object.

This function creates a proxy module from the given definition and stores it in the internal module registry forlater import with pico.importModule. It also returns the proxy module directly.

This function is called internally by the /<module_name>.js loading mechanism.

pico.help(proxy_object)

Arguments

• proxy_object (object) – The function or module proxy you want help for.

Returns the docstring of a proxy module or function.



3.5 Deployment

While Pico includes a Development Server it should only be used for development. To deploy a Pico app for Internetwide access you should always use a WSGI server together with a HTTP server.

Pico is based on the standard WSGI interface so any compliant WSGI server is capable running a Pico app.

Note: When other WSGI related documentation refers to the WSGI application or WSGI callable this isthe instance of the PicoApp, usually called app.

If you have a favourite WSGI/HTTP server combo then go ahead and use that as you normally do. If not, I recommenduWSGI & nginx.

3.5.1 uWSGI

There are many configuration options for uWSGI. The most simple way to run a Pico app with uWSGI is like this:

uwsgi -s /tmp/uwsgi.sock --plugins python --module=api:app

Where api is the name of the module with a PicoApp instance called app.

3.5.2 nginx

To setup nginx to proxy requests to this app we do the following:

server {listen 80;server_name example.com;

location / {include uwsgi_params;uwsgi_pass unix:/tmp/uwsgi.sock;

}

}

If we want only requests under a certain path (/myapp/) to go to our app we do this:


location /myapp/ {include uwsgi_params;uwsgi_param HTTP_X_SCRIPT_NAME /myapp/;uwsgi_pass unix:/tmp/uwsgi.sock;

}

}

Pico uses the HTTP_X_SCRIPT_NAME variable to correct the path. If you don’t include this and make a requestto example.com/myapp/api/ it will try to call a function called api from a module called myapp. SettingHTTP_X_SCRIPT_NAME correctly tells Pico to strip this part of the path before processing.

3.5. Deployment 15


If we have static files (html, js, css, images, etc) in a static folder we can do this:


location /myapp/ {root path/to/static/;try_files $uri @app;

}

location @app {uwsgi_param HTTP_X_SCRIPT_NAME /myapp/;include uwsgi_params;uwsgi_pass unix:/tmp/uwsgi.sock;

}

}

For more general information about deploying Python WSGI applications with uWSGI and nginx please see the officialQuickstart for Python/WSGI applications

3.6 Development Server

Pico includes a development server, based on Werkzeug’s amazing development server. It auto reloads code when itchanges and includes and interactive debugger.

To start a Pico application with the development server simply run:

$ python -m pico.server myapp

* Running on http://127.0.0.1:4242/ (Press CTRL+C to quit)

* Restarting with fsevents reloader

This assumes your PicoApp object instance is called app in a module called myapp. If your app instance is namedsomething else like foo then you need to specify it as follows:

python -m pico.server myapp:foo

The server chooses the first available port starting with 4242.

Warning: This server is for development only. It is inefficient and insecure compared to productionweb/application servers. Please read the Deployment guide for details on how to deploy a Pico app in produc-tion.

3.7 Error Handling

Pico catches all exceptions raised in your app and returns an appropriate response in JSON format in all cases. If theexception is a subclass of werkzeug.exceptions.HTTPException then the exception’s code and name are used for thestatus of the Response. The body will be JSON as follows:

{"name": "Forbidden",


http://uwsgi-docs.readthedocs.io/en/latest/WSGIquickstart.html


"code": 403,"message": "You don't have the permission to access the requested resource. It is

→˓either read-protected or not readable by the server."}

For all other exceptions a generic 500 Internal Server Error will be returned with the following body:

{"name": "Internal Server Error","code": 500,"message": "'The server encountered an internal error and was unable to complete

→˓your request. Either the server is overloaded or there is an error in the→˓application."}

If PicoApp.debug == True an extra key __debug__ will contain details of the internal error:

{"name": "Internal Server Error","code": 500,"message": "'The server encountered an internal error and was unable to complete

→˓your request. Either the server is overloaded or there is an error in the→˓application.",

"__debug__": {"stack_trace": [

"./api.py:25 in fail: raise Exception('fail!')"],"message": "fail!","name": "Exception"

}}

Note: The debug information is only provided to aid manual debugging. It should not be programatically accessedand should never be enabled on public services as it could easily expose sensitive information.

3.7.1 Sentry

Sentry is a popular exception reporting system for Python and many other languages. It is so useful and popular inPython web API applications that a Sentry ‘mixin’ is included in Pico.

To enable it just do the following:

from raven import Client # sentry client library

from pico import PicoAppfrom pico.extras.sentry import SentryMixin

class App(SentryMixin, PicoApp):sentry_client = Client() # uses SENTRY_DSN from environment by default

app = App()app.register_module(__name__)...

3.7. Error Handling 17

https://sentry.io/


The SentryMixin provides custom handle_exception and prehandle methods that take care of capturing exceptions andcontext and sending these to Sentry. When an exception is captured by Sentry a identifier string called sentry_id isadded to the exception JSON response. This can be used to quickly find the related issue in the Sentry web UI.

For more details please see the source code. It is very simple and can easily be overridden or used as a model for othersimilar services if you wish.

3.8 The PicoApp

The PicoApp object is the actual WSGI application that gets called by the web server. It is the initial handler androuter of all requests and acts as a registry of exposed modules and functions. Despite its central role in the executionof your application it has a very small public API. Many applications will simply create a PicoApp object and registermodules.

Occasionally we need to override some of the functionality of PicoApp to implement app-wide custom logic. Themost commonly overridden methods are prehandle and handle_exception.

class pico.PicoApp

register_module(module, alias=None)Imports and registers the exposed functions of the specified module.

If an alias is provided this is used as the namespace for the exposed functions, otherwise the module nameis used.

prehandle(request, kwargs)Called just before every execution of an exposed function.

This method does nothing by default but may be overridden to apply modifications to the request objecton every request. This is useful for performing authentication and setting a user attribute on the requestobject for example.

posthandle(request, response)Called after every execution of an exposed function and after any error handling.

This method does nothing by default but may be overridden to apply modifications to the response objecton every request, to clear a request cache, close database connections, etc.

handle_exception(exception, request, **kwargs)Called when any uncaught exception occurs.

By default this returns a JsonErrorResponse with a 500 Internal Server Error status. This can be overrid-den to customise error handling.

json_load(value)This method is used internally to load JSON. It can be overridden if you require custom JSON logic.

json_dump(value)This method is used internally to dump JSON. It can be overridden if you require custom JSON logic.


https://github.com/fergalwalsh/pico/blob/master/pico/extras/sentry.py

CHAPTER 4

Acknowledgements

Pico development is kindly supported by Hipo.

19

http://hipolabs.com


20 Chapter 4. Acknowledgements

Python Module Index

ppico, 18pico.decorators, 11

21


22 Python Module Index

Index

Eexpose() (in module pico), 10

Hhandle_exception() (pico.PicoApp method), 18

Jjson_dump() (pico.PicoApp method), 18json_load() (pico.PicoApp method), 18

Ppico (module), 10, 18pico.decorators (module), 11pico.help() (pico method), 14pico.importModule() (pico method), 14pico.loadAsync() (pico method), 14pico.loadModuleDefinition() (pico method), 14pico.reload() (pico method), 14PicoApp (class in pico), 18posthandle() (pico.PicoApp method), 18prehandle() (in module pico), 10prehandle() (pico.PicoApp method), 18protected() (in module pico.decorators), 11

Rregister_module() (pico.PicoApp method), 18request_args() (in module pico.decorators), 11require_method() (in module pico.decorators), 12

Sstream() (in module pico.decorators), 12

23

Date post:	12-Mar-2022
Category:	Documents
Upload:	others
View:	7 times
Download:	0 times

Pico Documentation

Documents