This document is for pytest_celery's development version, which can be significantly different from previous releases. Get the stable docs here: 1.0.
Introduction to pytest-celery¶
- Release:
1.1
- Date:
Sep 30, 2024
What is pytest-celery?¶
It is an essential pytest plugin designed for Celery application developers. It enables dynamic orchestration of Celery environments for testing tasks in isolated conditions, leveraging Docker & pytest-docker-tools for environment simulation.
The plugin is designed to be highly configurable, and can be used to test a wide range of Celery architectures. It encapsulate the complexity of setting up a Celery environment, and provides a simple interface to access the Celery architecture components in a test case.
The plugin functions as the backbone of Celery’s testing infrastructure, and is used to test Celery itself. The pytest-celery plugin is considered the official Celery testing infrastructure, internally and externally.
What do I need?¶
The pytest-celery plugin is using docker containers to define a Celery setup for a test. This means the testing environment must have a working Docker installation. The plugin provides built-in containers, but it is expected to be configured with the architecture and requirements of the target project.
Get Started¶
Sometimes jumping into the deep water is the best way to learn how to swim. A good place to start are the Examples. They can demonstrates basic common use cases and provide a good starting point for understanding how to use the plugin.
If you wish a more structured approach, start with the First Steps with pytest-celery section.
Other useful use cases are the smoke tests of the Celery repository. The Celery smoke tests are the official production environment for pytest-celery, and may be insightful for understanding how to use the plugin from a production perspective.
Note
The Celery smoke tests were firstly introduced in Celery v5.4.0’s development cycle. The tests requirements have defined the MVP scope for version 1.0.0 of pytest-celery. They use most of the plugin’s features, and vary in complexity and use cases.
It is recommended to familiarize yourself with Celery & pytest-celery before diving deep into the smoke tests.
Essential Resources¶
The pytest-celery plugin is using core pytest features to encapsulate the complexity of setting up a dockerized Celery environment into a simple interface. It is highly recommended to familiarize yourself with the following resources to get the most out of the plugin.
Fixtures Reference: Detailed guide to pytest fixtures. Extremely useful for understanding how to use the plugin effectively.
Pytest Parametrization: Guide for pytest parametrization. Useful for understanding how the Celery architecture matrix is generated.
pytest_docker_tools: The engine that powers up the dockerized environment. Useful for understanding how the plugin manages the docker containers.
Additional resources can be found in the Useful Resources section.
pytest-celery is…¶
Features¶
Quick Jump¶
Installation¶
The pytest-celery plugin can be easily installed via the Python Package Index (PyPI) using pip.
Installing the pytest-celery package¶
To install the latest version of pytest-celery, run the following command:
pip install -U pytest-celery
This command installs pytest-celery along with its required dependencies.
This will include:
Installing pytest-celery vendors¶
The plugin detects which vendor dependencies are installed in the test environment to configure the default configurations automatically. This means that by just installing the matching dependencies, the plugin will allow extending the default configurations, up to the supported built-in Vendors.
Warning
If you don’t install any vendor (e.g. no extras and no manual installation), the plugin will result in an empty Test Setup Matrix and might not be fully functional.
To install the vendors, you may either install all of the dependencies manually, or use the following extras:
all
: Installs all vendors.redis
: Installs Redis vendor, providing broker and result backend components.memcached
: Installs Memcached vendor, providing a result backend component.sqs
: Installs Localstack vendor, providing an SQS broker component.
The following extra is installed by default:
rabbitmq
: Installs RabbitMQ vendor, providing a broker component.
To install pytest-celery with the built-in Vendors, replace <extra>
with the name of the vendor.
pip install -U "pytest-celery[<extra>]"
RabbitMQ & Redis combo¶
pip install -U "pytest-celery[redis]"
This will configure the plugin to generate all possible setups using only RabbitMQ and Redis vendors.
SQS & Redis combo¶
pip install -U "pytest-celery[redis,sqs]"
This will configure the plugin to generate all possible setups using only Localstack and Redis vendors.
All vendors¶
pip install -U "pytest-celery[all]"
This will configure the plugin to generate all possible setups.
This approach allows you to tailor the installation to your project’s specific needs by including only the necessary optional vendors.
Tip
See Advanced Installation for more advanced installation options.