This document describes Celery 2.4. For development docs, go here.
celery.concurrency.processes¶
-
class
celery.concurrency.processes.
TaskPool
(limit=None, putlocks=True, logger=None, **options)¶ Process Pool for processing tasks in parallel.
Parameters: - processes – see
processes
. - logger – see
logger
.
-
limit
¶ The number of processes that can run simultaneously.
-
logger
¶ The logger used for debugging.
-
class
Pool
(processes=None, initializer=None, initargs=(), maxtasksperchild=None, timeout=None, soft_timeout=None)¶ Class which supports an async version of the apply() builtin
-
class
Process
(group=None, target=None, name=None, args=(), kwargs={})¶ Process objects represent activity that is run in a separate process
The class is analagous to threading.Thread
-
authkey
¶
-
daemon
¶ Return whether process is a daemon
-
exitcode
¶ Return exit code of process or None if it has yet to stop
-
ident
¶ Return identifier (PID) of process or None if it has yet to start
-
is_alive
()¶ Return whether process is alive
-
join
(timeout=None)¶ Wait until child process terminates
-
name
¶
-
pid
¶ Return identifier (PID) of process or None if it has yet to start
-
run
()¶ Method to be run in sub-process; can be overridden in sub-class
-
start
()¶ Start child process
-
terminate
()¶ Terminate process; sends SIGTERM signal or uses TerminateProcess()
-
-
class
TaskPool.Pool.
ResultHandler
(outqueue, get, cache, poll, join_exited_workers, putlock)¶ -
body
()¶
-
-
exception
TaskPool.Pool.
SoftTimeLimitExceeded
¶ The soft time limit has been exceeded. This exception is raised to give the task a chance to clean up.
-
TaskPool.Pool.
apply
(func, args=(), kwds={})¶ Equivalent of apply() builtin
-
TaskPool.Pool.
apply_async
(func, args=(), kwds={}, callback=None, accept_callback=None, timeout_callback=None, waitforslot=False, error_callback=None, soft_timeout=None, timeout=None)¶ Asynchronous equivalent of apply() builtin.
Callback is called when the functions return value is ready. The accept callback is called when the job is accepted to be executed.
Simplified the flow is like this:
>>> if accept_callback: ... accept_callback() >>> retval = func(*args, **kwds) >>> if callback: ... callback(retval)
-
TaskPool.Pool.
close
()¶
-
TaskPool.Pool.
grow
(n=1)¶
-
TaskPool.Pool.
imap
(func, iterable, chunksize=1)¶ Equivalent of itertools.imap() – can be MUCH slower than Pool.map()
-
TaskPool.Pool.
imap_unordered
(func, iterable, chunksize=1)¶ Like imap() method but ordering of results is arbitrary
-
TaskPool.Pool.
join
()¶
-
TaskPool.Pool.
map
(func, iterable, chunksize=None)¶ Equivalent of map() builtin
-
TaskPool.Pool.
map_async
(func, iterable, chunksize=None, callback=None)¶ Asynchronous equivalent of map() builtin
-
TaskPool.Pool.
shrink
(n=1)¶
-
TaskPool.Pool.
terminate
()¶
-
class
-
TaskPool.
grow
(n=1)¶
-
TaskPool.
num_processes
¶
-
TaskPool.
on_start
()¶ Run the task pool.
Will pre-fork all workers so they’re ready to accept tasks.
-
TaskPool.
on_stop
()¶ Gracefully stop the pool.
-
TaskPool.
on_terminate
()¶ Force terminate the pool.
-
TaskPool.
shrink
(n=1)¶
-
TaskPool.
terminate_job
(pid, signal=None)¶
- processes – see