Celery 1.0.6 (stable) documentation

This Page

Configuration and defaults

This document describes the configuration options available.

If you’re using celery in a Django project these settings should be defined in the project’s settings.py file.

In a regular Python environment, that is using the default loader, you must create the celeryconfig.py module and make sure it is available on the Python path.

Example configuration file

This is an example configuration file to get you started. It should contain all you need to run a basic celery set-up.

DATABASE_NAME = "mydatabase.db"

BROKER_HOST = "localhost"
BROKER_USER = "guest"

## If you're doing mostly I/O you can have more processes,
## but if mostly spending CPU, try to keep it close to the
## number of CPUs on your machine. If not set, the number of CPUs/cores
## available will be used.

# CELERYD_LOG_FILE = "celeryd.log"

Concurrency settings


    The number of concurrent worker processes, executing tasks simultaneously.

    Defaults to the number of CPUs/cores available.


    How many messages to prefetch at a time multiplied by the number of concurrent processes. The default is 4 (four messages for each process). The default setting seems pretty good here. However, if you have very long running tasks waiting in the queue and you have to start the workers, note that the first worker to start will receive four times the number of messages initially. Thus the tasks may not be fairly balanced among the workers.

Task result backend settings


    The backend used to store task results (tombstones). Can be one of the following:

    • database (default)

      Use a relational database supported by the Django ORM.

    • cache

      Use memcached to store the results.

    • mongodb

      Use MongoDB to store the results.

    • redis

      Use Redis to store the results.

    • tyrant

      Use Tokyo Tyrant to store the results.

    • amqp

      Send results back as AMQP messages (WARNING While very fast, you must make sure you only receive the result once. See Executing Tasks).

Database backend settings

Please see the Django ORM database settings documentation: http://docs.djangoproject.com/en/dev/ref/settings/#database-engine

If you use this backend, make sure to initialize the database tables after configuration. When using celery with a Django project this means executing:

$ python manage.py syncdb

When using celery in a regular Python environment you have to execute:

$ celeryinit

Example configuration

DATABASE_USER = "myusername"
DATABASE_PASSWORD = "mypassword"
DATABASE_NAME = "mydatabase"
DATABASE_HOST = "localhost"

AMQP backend settings

The AMQP backend does not have any settings yet.

Example configuration


Cache backend settings

Please see the documentation for the Django cache framework settings: http://docs.djangoproject.com/en/dev/topics/cache/#memcached

To use a custom cache backend for Celery, while using another for Django, you should use the CELERY_CACHE_BACKEND setting instead of the regular django CACHE_BACKEND setting.

Example configuration

Using a single memcached server:

CACHE_BACKEND = 'memcached://'

Using multiple memcached servers:

CACHE_BACKEND = 'memcached://;'

Tokyo Tyrant backend settings

NOTE The Tokyo Tyrant backend requires the pytyrant library:

This backend requires the following configuration directives to be set:


    Hostname of the Tokyo Tyrant server.


    The port the Tokyo Tyrant server is listening to.

Example configuration

TT_HOST = "localhost"
TT_PORT = 1978

Redis backend settings

NOTE The Redis backend requires the redis library:

To install the redis package use pip or easy_install:

$ pip install redis

This backend requires the following configuration directives to be set:


    Hostname of the Redis database server. e.g. "localhost".


    Port to the Redis database server. e.g. 6379.

Also, the following optional configuration directives are available:


    Name of the database to use. Default is celery_results.


    Password used to connect to the database.

Example configuration

REDIS_HOST = "localhost"
REDIS_DATABASE = "celery_results"

MongoDB backend settings

NOTE The MongoDB backend requires the pymongo library:

    This is a dict supporting the following keys:

    • host

      Hostname of the MongoDB server. Defaults to “localhost”.

    • port

      The port the MongoDB server is listening to. Defaults to 27017.

    • user

      User name to authenticate to the MongoDB server as (optional).

    • password

      Password to authenticate to the MongoDB server (optional).

    • database

      The database name to connect to. Defaults to “celery”.

    • taskmeta_collection

      The collection name to store task meta data. Defaults to “celery_taskmeta”.

Example configuration

    "host": "",
    "port": 30000,
    "database": "mydb",
    "taskmeta_collection": "my_taskmeta_collection",

Messaging settings


  • CELERY_QUEUES The mapping of queues the worker consumes from. This is a dictionary of queue name/options. See Routing Tasks for more information.

    The default is a queue/exchange/binding key of "celery", with exchange type direct.

    You don’t have to care about this unless you want custom routing facilities.


    The queue used by default, if no custom queue is specified. This queue must be listed in CELERY_QUEUES. The default is: celery.


    Name of the default exchange to use when no custom exchange is specified. The default is: celery.


    Default exchange type used when no custom exchange is specified. The default is: direct.


    The default routing key used when sending tasks. The default is: celery.



    The timeout in seconds before we give up establishing a connection to the AMQP server. Default is 4 seconds.


    Automatically try to re-establish the connection to the AMQP broker if it’s lost.

    The time between retries is increased for each retry, and is not exhausted before CELERY_BROKER_CONNECTION_MAX_RETRIES is exceeded.

    This behavior is on by default.


    Maximum number of retries before we give up re-establishing a connection to the AMQP broker.

    If this is set to 0 or None, we will retry forever.

    Default is 100 retries.

Task execution settings


    If this is True, all tasks will be executed locally by blocking until it is finished. apply_async and Task.delay will return a celery.result.EagerResult which emulates the behavior of celery.result.AsyncResult, except the result has already been evaluated.

    Tasks will never be sent to the queue, but executed locally instead.


    Whether to store the task return values or not (tombstones). If you still want to store errors, just not successful return values, you can set CELERY_STORE_ERRORS_EVEN_IF_IGNORED.


    Time (in seconds, or a datetime.timedelta object) for when after stored task tombstones are deleted.

    NOTE: For the moment this only works with the database, cache and MongoDB


    If True the task will report its status as “started” when the task is executed by a worker. The default value is False as the normal behaviour is to not report that level of granularity. Tasks are either pending, finished, or waiting to be retried. Having a “started” status can be useful for when there are long running tasks and there is a need to report which task is currently running. backends.


    A string identifying the default serialization method to use. Can be pickle (default), json, yaml, or any custom serialization methods that have been registered with carrot.serialization.registry.

    Default is pickle.


    The global default rate limit for tasks.

    This value is used for tasks that does not have a custom rate limit The default is no rate limit.


    Disable all rate limits, even if tasks has explicit rate limits set.

Worker: celeryd


    A sequence of modules to import when the celery daemon starts. This is useful to add tasks if you are not using django or cannot use task auto-discovery.


    Send events so the worker can be monitored by tools like celerymon.


    If set to True, errors in tasks will be sent to admins by e-mail. If unset, it will send the e-mails if settings.DEBUG is False.


    If set, the worker stores all task errors in the result store even if Task.ignore_result is on.



    The default file name the worker daemon logs messages to, can be overridden using the –logfile` option to celeryd.

    The default is None (stderr) Can also be set via the --logfile argument.


    Worker log level, can be any of DEBUG, INFO, WARNING, ERROR, CRITICAL.

    Can also be set via the --loglevel argument.

    See the logging module for more information.


    The format to use for log messages. Can be overridden using the --loglevel option to celeryd.

    Default is [%(asctime)s: %(levelname)s/%(processName)s] %(message)s

    See the Python logging module for more information about log formats.

Periodic Task Server: celerybeat


    Name of the file celerybeat stores the current schedule in. Can be a relative or absolute path, but be aware that the suffix .db will be appended to the file name.

    Can also be set via the --schedule argument.


    The maximum number of seconds celerybeat can sleep between checking the schedule. Default is 300 seconds (5 minutes).


    The default file name to log messages to, can be overridden using the –logfile` option.

    The default is None (stderr). Can also be set via the --logfile argument.


    Logging level. Can be any of DEBUG, INFO, WARNING, ERROR, or CRITICAL.

    Can also be set via the --loglevel argument.

    See the logging module for more information.

Monitor Server: celerymon


    The default file name to log messages to, can be overridden using the –logfile` option.

    The default is None (stderr) Can also be set via the --logfile argument.


    Logging level. Can be any of DEBUG, INFO, WARNING, ERROR, or CRITICAL.

    See the logging module for more information.