This document describes the current stable version of Celery (4.1). For development docs, go here.
Internals: The worker¶
Introduction¶
The worker consists of 4 main components: the consumer, the scheduler, the mediator and the task pool. All these components runs in parallel working with two data structures: the ready queue and the ETA schedule.
Components¶
Consumer¶
Receives messages from the broker using Kombu.
When a message is received it’s converted into a
celery.worker.request.Request
object.
Tasks with an ETA, or rate-limit are entered into the timer, messages that can be immediately processed are sent to the execution pool.
ETA and rate-limit are 2 incompatible parameters, and the ETA is overriding the rate-limit by default. A task with both will follow its ETA and ignore its rate-limit.