funcx.serialize package

Submodules

funcx.serialize.base module

exception funcx.serialize.base.DeserializationError(reason)

Bases: Exception

Base class for all deserialization errors

class funcx.serialize.base.RemoteExceptionWrapper(e_type, e_value, traceback)

Bases: object

reraise()
class funcx.serialize.base.SerializerError(reason)

Bases: object

class funcx.serialize.base.fxPicker_enforcer

Bases: object

Ensure that any concrete class will have the serialize and deserialize methods

deserialize(payload)
serialize(data)
class funcx.serialize.base.fxPicker_shared

Bases: object

Adds shared functionality for all serializer implementations

check(payload)
chomp(payload)

If the payload starts with the identifier, return the remaining block

Parameters:payload (str) – Payload blob
identifier

Get the identifier of the serialization method

Returns:identifier
Return type:str

funcx.serialize.concretes module

funcx.serialize.concretes.bar(x, y={'a': 3})
class funcx.serialize.concretes.code_pickle

Bases: funcx.serialize.base.fxPicker_shared

deserialize(payload)
serialize(data)
class funcx.serialize.concretes.code_text_dill

Bases: funcx.serialize.base.fxPicker_shared

We use dill to get the source code out of the function object and then exec the function body to load it in. The function object is then returned by name.

deserialize(payload)
serialize(data)
class funcx.serialize.concretes.code_text_inspect

Bases: funcx.serialize.base.fxPicker_shared

We use dill to get the source code out of the function object and then exec the function body to load it in. The function object is then returned by name.

deserialize(payload)
serialize(data)
class funcx.serialize.concretes.json_base64

Bases: funcx.serialize.base.fxPicker_shared

deserialize(payload)
serialize(data)
class funcx.serialize.concretes.pickle_base64

Bases: funcx.serialize.base.fxPicker_shared

deserialize(payload)
serialize(data)

funcx.serialize.facade module

class funcx.serialize.facade.FuncXSerializer

Bases: object

Information that we want to be able to ship around:

  • Function run information ->
  • Function id <—> Container id ? (is there a 1-1 mapping here?)
  • Container id
  • Endpoint id
  • Function body
  • Potentially in a byte compiled form ?
  • All parameters ->
  • Args + Kwargs

From the client side. At function invocation we need to capture

deserialize(payload)
Parameters:payload (str) – Payload object to be deserialized
pack_buffers(buffers)
buffers : list of

terminated strings

serialize(data)
unpack_and_deserialize(packed_buffer)

Unpacks a packed buffer and returns the deserialized contents :param packed_buffers: :type packed_buffers: packed buffer as string

unpack_buffers(packed_buffer)
Parameters:packed_buffers (packed buffer as string) –

Module contents