Datastructures - celery.datastructures

class celery.datastructures.ExceptionInfo(exc_info)

Exception wrapping an exception and its traceback.

Parameters:exc_info – The exception tuple info as returned by traceback.format_exception().

exception

The original exception.

traceback

A traceback from the point when exception was raised.

class celery.datastructures.LimitedSet(maxlen=None, expires=None)

Kind-of Set with limitations.

Good for when you need to test for membership (a in set), but the list might become to big, so you want to limit it so it doesn’t consume too much resources.

Parameters:

• maxlen – Maximum number of members before we start deleting expired members.
• expires – Time in seconds, before a membership expires.

add(value)

Add a new member.

first

Get the oldest member.

pop_value(value)

Remove membership by finding value.

class celery.datastructures.LocalCache(limit=None)

Dictionary with a finite number of keys.

Older items expires first.

class celery.datastructures.PositionQueue(length)

A positional queue of a specific length, with slots that are either filled or unfilled. When all of the positions are filled, the queue is considered full().

Parameters:length – see length.

length

The number of items required for the queue to be considered full.

class UnfilledPosition(position)

Describes an unfilled slot.

PositionQueue.filled

Returns the filled slots as a list.

PositionQueue.full()

Returns True if all of the slots has been filled.

class celery.datastructures.SharedCounter(initial_value)

Thread-safe counter.

Please note that the final value is not synchronized, this means that you should not update the value by using a previous value, the only reliable operations are increment and decrement.

Example

>>> max_clients = SharedCounter(initial_value=10)

# Thread one >>> max_clients += 1 # OK (safe)

# Thread two >>> max_clients -= 3 # OK (safe)

# Main thread >>> if client >= int(max_clients): # Max clients now at 8 ... wait()

>>> max_client = max_clients + 10 # NOT OK (unsafe)

decrement(n=1)

Decrement value.

increment(n=1)

Increment value.

celery.datastructures.consume_queue(queue)

Iterator yielding all immediately available items in a Queue.Queue.

The iterator stops as soon as the queue raises Queue.Empty.

Example

>>> q = Queue()
>>> map(q.put, range(4))
>>> list(consume_queue(q))
[0, 1, 2, 3]
>>> list(consume_queue(q))
[]