funcx.sdk package

Submodules

funcx.sdk.client module

class funcx.sdk.client.FuncXClient(http_timeout=None, funcx_home='~/.funcx', force_login=False, fx_authorizer=None, funcx_service_address='https://dev.funcx.org/api/v1', **kwargs)

Bases: funcx.sdk.utils.throttling.ThrottledBaseClient

Main class for interacting with the funcX service

Holds helper operations for performing common tasks with the funcX service.

CLIENT_ID = '4cf29807-cf21-49ec-9443-ff9a3fb9f81c'
TOKEN_DIR = '/home/docs/.funcx/credentials'
TOKEN_FILENAME = 'funcx_sdk_tokens.json'
add_to_whitelist(endpoint_id, function_ids)

Adds the function to the endpoint’s whitelist

Parameters:
  • endpoint_id (str) – The uuid of the endpoint
  • function_ids (list) – A list of function id’s to be whitelisted
Returns:

The response of the request

Return type:

json

batch_run(batch)

Initiate a batch of tasks to funcX

Parameters:batch (a Batch object) –
Returns:task_ids
Return type:a list of UUID strings that identify the tasks
create_batch()

Create a Batch instance to handle batch submission in funcX

Returns:Status block containing “status” key.
Return type:Batch instance
delete_from_whitelist(endpoint_id, function_ids)

List the endpoint’s whitelist

Parameters:
  • endpoint_id (str) – The uuid of the endpoint
  • function_ids (list) – A list of function id’s to be whitelisted
Returns:

The response of the request

Return type:

json

get_batch_result(task_id_list)

Request results for a batch of task_ids

get_batch_status(task_id_list)

Request status for a batch of task_ids

get_container(container_uuid, container_type)

Get the details of a container for staging it locally.

Parameters:
  • container_uuid (str) – UUID of the container in question
  • container_type (str) – The type of containers that will be used (Singularity, Shifter, Docker)
Returns:

The details of the containers to deploy

Return type:

dict

get_containers(name, description=None)

Register a DLHub endpoint with the funcX service and get the containers to launch.

Parameters:
  • name (str) – Name of the endpoint
  • description (str) – Description of the endpoint
Returns:

The port to connect to and a list of containers

Return type:

int

get_endpoint_status(endpoint_uuid)

Get the status reports for an endpoint.

Parameters:endpoint_uuid (str) – UUID of the endpoint in question
Returns:The details of the endpoint’s stats
Return type:dict
get_result(task_id)

Get the result of a funcX task

Parameters:task_id (str) – UUID of the task
Returns:Result obj
Return type:If task completed
Raises:Exception obj: Exception due to which the task failed
get_task_status(task_id)

Get the status of a funcX task.

Parameters:task_id (str) – UUID of the task
Returns:Status block containing “status” key.
Return type:dict
get_whitelist(endpoint_id)

List the endpoint’s whitelist

Parameters:endpoint_id (str) – The uuid of the endpoint
Returns:The response of the request
Return type:json
logout()

Remove credentials from your local system

map_run(*args, endpoint_id=None, function_id=None, asynchronous=False, **kwargs)

Initiate an invocation

Parameters:
  • *args (Any) – Args as specified by the function signature
  • endpoint_id (uuid str) – Endpoint UUID string. Required
  • function_id (uuid str) – Function UUID string. Required
  • asynchronous (bool) – Whether or not to run the function asynchronously
Returns:

  • task_id (str)
  • UUID string that identifies the task

register_container(location, container_type, name='', description='')

Register a container with the funcX service.

Parameters:
  • location (str) – The location of the container (e.g., its docker url). Required
  • container_type (str) – The type of containers that will be used (Singularity, Shifter, Docker). Required
  • name (str) – A name for the container. Default = ‘’
  • description (str) – A description to associate with the container. Default = ‘’
Returns:

The id of the container

Return type:

str

register_endpoint(name, endpoint_uuid, description=None)

Register an endpoint with the funcX service.

Parameters:
  • name (str) – Name of the endpoint
  • endpoint_uuid (str) – The uuid of the endpoint
  • description (str) – Description of the endpoint
Returns:

{‘endopoint_id’ : <>,

’address’ : <>, ‘client_ports’: <>}

Return type:

A dict

register_function(function, function_name=None, container_uuid=None, description=None, public=False, group=None)

Register a function code with the funcX service.

Parameters:
  • function (Python Function) – The function to be registered for remote execution
  • function_name (str) – The entry point (function name) of the function. Default: None
  • container_uuid (str) – Container UUID from registration with funcX
  • description (str) – Description of the file
  • public (bool) – Whether or not the function is publicly accessible. Default = False
  • group (str) – A globus group uuid to share this function with
Returns:

function uuid – UUID identifier for the registered function

Return type:

str

run(*args, endpoint_id=None, function_id=None, **kwargs)

Initiate an invocation

Parameters:
  • *args (Any) – Args as specified by the function signature
  • endpoint_id (uuid str) – Endpoint UUID string. Required
  • function_id (uuid str) – Function UUID string. Required
  • asynchronous (bool) – Whether or not to run the function asynchronously
Returns:

  • task_id (str)
  • UUID string that identifies the task

update_table(return_msg, task_id)

Parses the return message from the service and updates the internal func_tables

Parameters:
  • return_msg (str) – Return message received from the funcx service
  • task_id (str) – task id string

funcx.sdk.config module

funcx.sdk.config.write_option(option, value)

Write an option to disk – doesn’t handle config reloading

funcx.sdk.config.lookup_option(option)
funcx.sdk.config.remove_option(option)
funcx.sdk.config.internal_auth_client()

Get the globus native app client.

Returns:
funcx.sdk.config.check_logged_in()

Check if the user is already logged in.

Returns:
funcx.sdk.config.safeprint(s)

Catch print errors.

Parameters:s
Returns:
funcx.sdk.config.format_output(dataobject)

Use safe print to make sure jobs are correctly printed.

Parameters:dataobject
Returns:

funcx.sdk.version module

Module contents