ﬂower Documentation · ﬂower Documentation, 0.9.1 format_task...

flower Documentation0.9.1

BeginMan

2015 04 28

Contents

1 31.1 Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

i

ii

flower Documentation, 0.9.1

Flowerweb[Celery](http://www.celeryproject.org/).

Contents 1

http://www.celeryproject.org/


2 Contents

CHAPTER 1

• Celery

–

– arguments, start time, runtime

–

• – worker

– worker

– (autoscale settings)

– worker

– tasks

– (ETA()/)

–

– (Configuration viewer)

–

• Broker monitoring

– Celery

–

• HTTP API

• Basic Auth and Google OpenID authentication

1.1 Contents

1.1.1

• Celery

–

– arguments, start time, runtime

–

3


• – worker

– worker

– (autoscale settings)

– worker

– tasks

– (ETA()/)

–

– (Configuration viewer)

–

• Broker monitoring

– Celery

–

• HTTP API

• Basic Auth and Google OpenID authentication

1.1.2 Screenshots

Worker dashboard:

Task dashboard:

4 Chapter 1.


Worker tasks:

Graphs:

1.1. Contents 5


Worker info:

6 Chapter 1.


1.1. Contents 7


Task info:

Configuration viewer:

8 Chapter 1.


1.1.3

pip flower

$ pip install flower

easy_install

$ easy_install flower

http://localhost:5555:

$ flower --port=5555

Celery:

$ celery flower --address=127.0.0.1 --port=5555

Broker URLCelery:

$ celery flower --broker=amqp://guest:guest@localhost:5672//

1.1.4

flower:

1.1. Contents 9

http://www.pip-installer.org/

http://pypi.python.org/pypi/setuptools


$ flower --auto_refresh=False

flowerconfig.py :

# Broker settingsBROKER_URL = ’amqp://guest:guest@localhost:5672//’

# RabbitMQ management apibroker_api = ’http://guest:guest@localhost:15672/api/’

# Enable debug logginglogging = ’DEBUG’

flower‘FLOWER_‘:

$ export FLOWER_BASIC_AUTH=foo:bar

conf .

Celery,‘Celery Configuration reference‘_

CeleryFlower‘–broker‘broker:

$ flower -A proj --broker=amqp://guest:guest@localhost:5672//

:

$ celery --help

• address• auth• auto_refresh• basic_auth• broker_api• certfile• conf• db• debug• enable_events• format_task• inspect_timeout• keyfile• max_tasks• natural_time• persistent• port• xheaders• tasks_columns

address

http

10 Chapter 1.


auth

OpenIDauth:ref:google-openid

auto_refresh

(auto_refresh=True)

basic_auth

HTTPbasic_auth‘:‘username:passworrd,:ref:basic-auth.

broker_api

Flower RabbitMQ Managment Plugin broker_apiRabbitMQ HTTP API.

$ flower -A proj --broker_api=http://username:password@rabbitmq-server-name:15672/api/

: managment plugin:

$ rabbitmq-plugins enable rabbitmq_management

: The port number for RabbitMQ versions prior to 3.0 is 55672.

certfile

SSL

conf

(by default, flowerconfig.py)

db

(by default, db=flower)

debug

(by default, debug=False)

enable_events

Celery events‘enable_events‘,‘enable_event=True‘

1.1. Contents 11

https://www.rabbitmq.com/management.html


format_task

task‘format_task‘flowerconfig.py tasktask

‘format_task‘

:

from flower.utils.template import humanize

def format_task(task):task.args = humanize(task.args, length=10)task.kwargs.pop(’credit_card_number’)task.result = humanize(task.result, length=20)return task

inspect_timeout

workerinspect_timeout=10000

keyfile

SSL key

max_tasks

taskmax_tasks=10000

natural_time

‘natural_time=True‘

persistent

Flowerpersistent=False

port

HTTP‘port=5555‘

xheaders

X-Real-Ip X-Scheme headersxheaders=False

tasks_columns

/tasks/Avalible slugs: name, uuid, state, args, kwargs, result, received, started, runtime

12 Chapter 1.


1.1.5 API Reference

GET /api/workers

workers

Example request:

GET /api/workers HTTP/1.1Host: localhost:5555

Example response:

HTTP/1.1 200 OKContent-Length: 119Content-Type: application/json; charset=UTF-8

{"celery@worker1": {

"completed_tasks": 0,"concurrency": 4,"queues": [

"celery"],"running_tasks": 0,"status": true

},"celery@worker2": {

"completed_tasks": 0,"concurrency": 4,"queues": [],"running_tasks": 0,"status": false}}

wroker.json[workers.json](ApiReference/workers.json)

## POST /api/worker/shutdown/(.+)

worker

Example request:

POST /api/worker/shutdown/celery@worker2 HTTP/1.1Content-Length: 0Host: localhost:5555

Example response:


{"message": "Shutting down!"}

1.1. Contents 13


POST /api/worker/pool/restart/(.+)

worker

Example request:

POST /api/worker/pool/restart/celery@worker2 HTTP/1.1Content-Length: 0Host: localhost:5555

Example response:


{"message": "Restarting ’celery@worker2’ worker’s pool"}

POST /api/worker/pool/grow/(.+)

worker

Example request:

POST /api/worker/pool/grow/celery@worker2?n=3 HTTP/1.1Content-Length: 0Host: localhost:5555

Example response:


{"message": "Growing ’celery@worker2’ worker’s pool by 3"}

Query Parameters:

n – number of pool processes to grow, default is 1

10..."pool": {

"processes": [3303,3304,3305,3306,3307,3308,3309,3310,3311,3312

],

14 Chapter 1.


"max-concurrency": 10,"timeouts": [

0,0

],....

POST /api/worker/pool/shrink/(.+)

worker’s pool

Example request:

POST /api/worker/pool/shrink/celery@worker2 HTTP/1.1Content-Length: 0Host: localhost:5555

Example response:


{"message": "Shrinking ’celery@worker2’ worker’s pool by 1"

}

Query Parameters:

n – number of pool processes to shrink, default is 1

POST /api/worker/pool/autoscale/(.+)

worker pool

Example request:

POST /api/worker/pool/autoscale/celery@worker2?min=3&max=10 HTTP/1.1Content-Length: 0Content-Type: application/x-www-form-urlencoded; charset=utf-8Host: localhost:5555

Example response:


{"message": "Autoscaling ’celery@worker2’ worker (min=3, max=10)"}

Query Parameters:

min - minimum number of pool processesmax - maximum number of pool processes

1.1. Contents 15


POST /api/worker/queue/add-consumer/(.+)

Example request:

POST /api/worker/queue/add-consumer/celery@worker2?queue=sample-queueContent-Length: 0Content-Type: application/x-www-form-urlencoded; charset=utf-8Host: localhost:5555

Example response:


{"message": "add consumer sample-queue"

}

Query Parameters:

queue – the name of a new queue

POST /api/worker/queue/cancel-consumer/(.+)

Example request:

POST /api/worker/queue/cancel-consumer/celery@worker2?queue=sample-queueContent-Length: 0Content-Type: application/x-www-form-urlencoded; charset=utf-8Host: localhost:5555

Example response:


{"message": "no longer consuming from sample-queue"

}

Query Parameters:

queue – the name of queue

GET /api/tasks

tasks

Example request:

GET /api/tasks HTTP/1.1Host: localhost:5555User-Agent: HTTPie/0.8.0

Example response:

16 Chapter 1.


HTTP/1.1 200 OKContent-Length: 1109Content-Type: application/json; charset=UTF-8Etag: "b2478118015c8b825f7b88ce6b660e5449746c37"Server: TornadoServer/3.1.1

{"e42ceb2d-8730-47b5-8b4d-8e0d2a1ef7c9": {

"args": "[3, 4]","client": null,"clock": 1079,"eta": null,"exception": null,"exchange": null,"expires": null,"failed": null,"kwargs": "{}","name": "tasks.add","received": 1398505411.107885,"result": "’7’","retried": null,"retries": 0,"revoked": null,"routing_key": null,"runtime": 0.01610181899741292,"sent": null,"started": 1398505411.108985,"state": "SUCCESS","succeeded": 1398505411.124802,"timestamp": 1398505411.124802,"traceback": null,"uuid": "e42ceb2d-8730-47b5-8b4d-8e0d2a1ef7c9"

},"f67ea225-ae9e-42a8-90b0-5de0b24507e0": {

"args": "[1, 2]","client": null,"clock": 1042,"eta": null,"exception": null,"exchange": null,"expires": null,"failed": null,"kwargs": "{}","name": "tasks.add","received": 1398505395.327208,"result": "’3’","retried": null,"retries": 0,"revoked": null,"routing_key": null,"runtime": 0.012884548006695695,"sent": null,"started": 1398505395.3289,"state": "SUCCESS","succeeded": 1398505395.341089,"timestamp": 1398505395.341089,"traceback": null,"uuid": "f67ea225-ae9e-42a8-90b0-5de0b24507e0"

1.1. Contents 17


}}

Query Parameters:

• limit – maximum number of tasks

• workername – filter task by workername

• taskname – filter tasks by taskname

• state – filter tasks by state

GET /api/task/types

List (seen) task types

Example request:

GET /api/task/types HTTP/1.1Host: localhost:5555

Example response:


{"task-types": [

"tasks.add","tasks.sleep"

]}

GET /api/task/info/(.*)

task

Example request:

GET /api/task/info/91396550-c228-4111-9da4-9d88cfd5ddc6 HTTP/1.1Accept: */*Accept-Encoding: gzip, deflate, compressHost: localhost:5555

Example response:


{"args": "[2, 2]","client": null,"clock": 25,"eta": null,"exception": null,"exchange": null,

18 Chapter 1.


"expires": null,"failed": null,"kwargs": "{}","name": "tasks.add","received": 1400806241.970742,"result": "’4’","retried": null,"retries": null,"revoked": null,"routing_key": null,"runtime": 2.0037889280356467,"sent": null,"started": 1400806241.972624,"state": "SUCCESS","succeeded": 1400806243.975336,"task-id": "91396550-c228-4111-9da4-9d88cfd5ddc6","timestamp": 1400806243.975336,"traceback": null,"worker": "celery@worker1"

}

POST /api/task/async-apply/(.+)

task

Example request:

POST /api/task/async-apply/tasks.add HTTP/1.1Accept: application/jsonAccept-Encoding: gzip, deflate, compressContent-Length: 16Content-Type: application/json; charset=utf-8Host: localhost:5555

{"args": [1, 2]

}

Example response:

HTTP/1.1 200 OKContent-Length: 71Content-Type: application/json; charset=UTF-8Date: Sun, 13 Apr 2014 15:55:00 GMT

{"state": "PENDING","task-id": "abc300c7-2922-4069-97b6-a635cc2ac47c"}

Query Parameters:

• args – a list of arguments

• kwargs – a dictionary of arguments

• options – a dictionary of apply_async keyword arguments

1.1. Contents 19


POST /api/task/send-task/(.+)

task

Example request:

POST /api/task/send-task/tasks.add HTTP/1.1Accept: application/jsonAccept-Encoding: gzip, deflate, compressContent-Length: 16Content-Type: application/json; charset=utf-8Host: localhost:5555

{"args": [1, 2]

}

Example response:


{"state": "SUCCESS",

"task-id": "c60be250-fe52-48df-befb-ac66174076e6"}

Query Parameters:

• args – a list of arguments

• kwargs – a dictionary of arguments

GET /api/task/result/(.+)

task

Example request:

GET /api/task/result/c60be250-fe52-48df-befb-ac66174076e6 HTTP/1.1Host: localhost:5555

Example response:


{"result": 3,"state": "SUCCESS","task-id": "c60be250-fe52-48df-befb-ac66174076e6"}

POST /api/task/timeout/(.+)

Change soft and hard time limits for a task

20 Chapter 1.


Example request:

POST /api/task/timeout/tasks.sleep HTTP/1.1Content-Length: 44Content-Type: application/x-www-form-urlencoded; charset=utf-8Host: localhost:5555

soft=30&hard=100&workername=celery%40worker1

Example response:


{"message": "new rate limit set successfully"}

Query Parameters:

workername – worker name

POST /api/task/rate-limit/(.+)

Change rate limit for a task

Example request:

POST /api/task/rate-limit/tasks.sleep HTTP/1.1Content-Length: 41Content-Type: application/x-www-form-urlencoded; charset=utf-8Host: localhost:5555

ratelimit=200&workername=celery%40worker1

Example response:


{"message": "Revoked ’1480b55c-b8b2-462c-985e-24af3e9158f9’"}

Query Parameters:

workername – worker name

POST /api/task/revoke/(.+)

Revoke a task

Example request:

POST /api/task/revoke/1480b55c-b8b2-462c-985e-24af3e9158f9?terminate=trueContent-Length: 0Content-Type: application/x-www-form-urlencoded; charset=utf-8Host: localhost:5555

1.1. Contents 21


Example response:


{"message": "Revoked ’1480b55c-b8b2-462c-985e-24af3e9158f9’"}

Query Parameters:

terminate – terminate the task if it is running

1.1.6

Flower

HTTP Basic Authentication

Flower Basic Authentication .

–basic_auth user:password . Flower :

$ celery flower --basic_auth=user1:password1,user2:password2

See also

Google OAuth 2.0

Flower Google OAuth 2.0. google. Google OAuth 2.0 –auth, –oauth2_key, –oauth2_secret and –oauth2_redirect_uri.

–auth . –oauth2_key –oauth2_secret Google Developer Console. –oauth2_redirect_uri is there to specify what is theredirect_uri associated to you key and secret

For instance, if you want to grant access to [email protected] and [email protected]:

$ celery flower --auth="[email protected]|[email protected]" --oauth2_key=... --oauth2_secret=... --oauth2_redirect_uri=http://flower.example.com/login

:

$ export FLOWER_GOOGLE_OAUTH2_KEY=...$ export FLOWER_GOOGLE_OAUTH2_SECRET=...$ export FLOWER_GOOGLE_OAUTH2_REDIRECT_URI=http://flower.example.com/login$ celery flower --auth=.*@example\.com

1.1.7

nginx :

server {listen 80;server_name flower.example.com;charset utf-8;

22 Chapter 1.

https://console.developers.google.com


location / {proxy_pass http://localhost:5555;proxy_set_header Host $host;proxy_redirect off;proxy_http_version 1.1;proxy_set_header Upgrade $http_upgrade;proxy_set_header Connection "upgrade";

}}

‘htpasswd‘‘nginx‘

auth_basic "Restricted";auth_basic_user_file htpasswd;

1.1.8 Docker

, build flower from the Dockerfile, start the container and open http://localhost:49555

$ docker build -t "flower" .$ docker run -d -p=49555:5555 flower flower --port=5555

For more information about running with Docker see http://docs.docker.io/en/latest/

Flower is Open Source and licensed under the BSD License.

1.1. Contents 23

http://localhost:49555

http://docs.docker.io/en/latest/

http://www.opensource.org/licenses/BSD-3-Clause

Date post:	19-Mar-2020
Category:	Documents
Upload:	others
View:	23 times
Download:	0 times

ﬂower Documentation · ﬂower Documentation, 0.9.1 format_task...

Documents