This document describes an older version of Celery (2.1). For the latest stable version please go here.

Datastructures - celery.datastructures

class celery.datastructures.AttributeDict

Dict subclass with attribute access.

class celery.datastructures.ExceptionInfo(exc_info)

Exception wrapping an exception and its traceback.

Parameters:exc_info – The exception tuple info as returned by traceback.format_exception().
exception

The original exception.

traceback

A traceback from the point when exception was raised.

class celery.datastructures.LimitedSet(maxlen=None, expires=None)

Kind-of Set with limitations.

Good for when you need to test for membership (a in set), but the list might become to big, so you want to limit it so it doesn’t consume too much resources.

Parameters:
  • maxlen – Maximum number of members before we start deleting expired members.
  • expires – Time in seconds, before a membership expires.
add(value)

Add a new member.

as_dict()
chronologically
clear()

Remove all members

first

Get the oldest member.

pop_value(value)

Remove membership by finding value.

update(other)
class celery.datastructures.LocalCache(limit=None)

Dictionary with a finite number of keys.

Older items expires first.

class celery.datastructures.PositionQueue(length)

A positional queue of a specific length, with slots that are either filled or unfilled. When all of the positions are filled, the queue is considered full().

Parameters:length – see length.
length

The number of items required for the queue to be considered full.

class UnfilledPosition(position)

Describes an unfilled slot.

PositionQueue.filled

Returns the filled slots as a list.

PositionQueue.full()

Returns True if all of the slots has been filled.

class celery.datastructures.SharedCounter(initial_value)

Thread-safe counter.

Please note that the final value is not synchronized, this means that you should not update the value by using a previous value, the only reliable operations are increment and decrement.

Example

>>> max_clients = SharedCounter(initial_value=10)

# Thread one >>> max_clients += 1 # OK (safe)

# Thread two >>> max_clients -= 3 # OK (safe)

# Main thread >>> if client >= int(max_clients): # Max clients now at 8 ... wait()

>>> max_client = max_clients + 10 # NOT OK (unsafe)
decrement(n=1)

Decrement value.

increment(n=1)

Increment value.

class celery.datastructures.TokenBucket(fill_rate, capacity=1)

Token Bucket Algorithm.

See http://en.wikipedia.org/wiki/Token_Bucket Most of this code was stolen from an entry in the ASPN Python Cookbook: http://code.activestate.com/recipes/511490/

Parameters:
fill_rate

The rate in tokens/second that the bucket will be refilled.

capacity

Maximum number of tokens in the bucket. Default is 1.

timestamp

Timestamp of the last time a token was taken out of the bucket.

can_consume(tokens=1)
expected_time(tokens=1)

Returns the expected time in seconds when a new token should be available. Note: consumes a token from the bucket

celery.datastructures.consume_queue(queue)

Iterator yielding all immediately available items in a Queue.Queue.

The iterator stops as soon as the queue raises Queue.Empty.

Example

>>> q = Queue()
>>> map(q.put, range(4))
>>> list(consume_queue(q))
[0, 1, 2, 3]
>>> list(consume_queue(q))
[]

Previous topic

Serialization Tools - celery.serialization

Next topic

Message Routers - celery.routes

This Page