celery

What's a Task Queue?

Task queues are used as a mechanism to distribute work across threads or machines.

A task queue's input is a unit of work called a task. Dedicated worker processes constantly monitor task queues for new work to perform.

Celery communicates via messages, usually using a broker to mediate between clients adn workers. To initiate a task the client adds a message to the queue, the broker then dilivers that message to a worker.

A Celery system can consist of multiple workers and brokers, giving way to high avaliability and horizontal scaling.

Features

• Monitoring

• Scheduling

• Work-flows

• Resource Leak Protection

• Time & Rate Limits

• User Components

First Steps with Celery

pip install celery
pip install redis

# RabbitMQ
sudo docker run -it -d --name rabbitmq  -p 5672:5672 rabbitmq
# Redis
sudo docker run -it -d --name redis -p 6379:6379 redis
# Docker 查看端口
sudo docker port rabbitmq
# Docker 查看日志
sudo docker logs rabbitmq
# 进入 Docker
sudo docker exec -it rabbitmq bash
# RabbitMQ 运维
rabbitmqctl --help

# foo.py
from celery import Celery
app = Celery('foo', broker='pyamqp://guest@127.0.0.1//', backend='redis://127.0.0.1')

@app.task
def add(x, y):
    return x + y
    
# 启动worker服务
celery worker -A foo --loglevel=info

# calling
from foo import add
ret = add.deley(1,2)