funcx.queues package

Submodules

funcx.queues.base module

class funcx.queues.base.FuncxQueue

Bases: object

Queue interface required by the Forwarder

This is a metaclass that only enforces concrete implementations of functionality by the child classes.

connect(*args, **kwargs)

Connects and creates the queue. The queue is not active until this is called

get(*args, **kwargs)

Get an item from the Queue

is_connected

Returns the connected status of the queue.

Returns:
Return type:Bool
put(*args, **kwargs)

Put an item into the Queue

exception funcx.queues.base.NotConnected(queue)

Bases: funcx.errors.FuncxError

Queue is not connected/active

Module contents

class funcx.queues.RedisQueue(prefix, hostname, port=6379)

Bases: funcx.queues.base.FuncxQueue

A basic redis queue

The queue only connects when the connect method is called to avoid issues with passing an object across processes.

Parameters:
  • hostname (str) – Hostname of the redis server
  • port (int) – Port at which the redis server can be reached. Default: 6379
connect()

Connects to the Redis server

get(timeout=1)

Get an item from the redis queue

Parameters:timeout (int) – Timeout for the blocking get in seconds
is_connected

Returns the connected status of the queue.

Returns:
Return type:Bool
put(key, payload)

Put’s the key:payload into a dict and pushes the key onto a queue :param key: The task_id to be pushed :type key: str :param payload: Dict of task information to be stored :type payload: dict