The Globus Compute Client

class globus_compute_sdk.Client(http_timeout=None, funcx_home=None, environment: Optional[str] = None, task_group_id: Union[None, uuid.UUID, str] = None, funcx_service_address: Optional[str] = None, do_version_check: bool = True, *, code_serialization_strategy: Optional[globus_compute_sdk.serialize.base.SerializationStrategy] = None, data_serialization_strategy: Optional[globus_compute_sdk.serialize.base.SerializationStrategy] = None, login_manager: Optional[globus_compute_sdk.sdk.login_manager.protocol.LoginManagerProtocol] = None, **kwargs)

Main class for interacting with the Globus Compute service

Holds helper operations for performing common tasks with the Globus Compute service.

Initialize the client

Parameters

  • http_timeout (int) –

    Timeout for any call to service in seconds. Default is no timeout

    DEPRECATED - see self.web_client

  • funcx_home (any) – DEPRECATED - was never used

  • environment (str) – For internal use only. The name of the environment to use. Sets funcx_service_address appropriately, unless already set.

  • task_group_id (str|uuid.UUID) –

    Set the TaskGroup ID (a UUID) for this Client instance. Typically, one uses this to submit new tasks to an existing session or to reestablish Executor futures. Default: None (will be auto generated)

    DEPRECATED - use create_batch or the executor instead

  • funcx_service_address (str) – For internal use only. The address of the web service.

  • do_version_check (bool) – Set to False to skip the version compatibility check on client initialization Default: True

  • code_serialization_strategy (SerializationStrategy) – Strategy to use when serializing function code. If None, globus_compute_sdk.serialize.DEFAULT_STRATEGY_CODE will be used.

  • data_serialization_strategy (SerializationStrategy) – Strategy to use when serializing function arguments. If None, globus_compute_sdk.serialize.DEFAULT_STRATEGY_DATA will be used.

  • login_manager (LoginManagerProtocol) – Allows login logic to be overridden for specific use cases. If None, a LoginManager will be used.

version_check(endpoint_version: Optional[str] = None) → None

Check this client version meets the service’s minimum supported version.

Raises a VersionMismatch error on failure.

logout()

Remove credentials from your local system

get_task(task_id)

Get a Globus Compute task.

Parameters

task_id (str) – UUID of the task

Returns

Task block containing “status” key.

Return type

dict

get_result(task_id)

Get the result of a Globus Compute task

Parameters

task_id (str) – UUID of the task

Returns

Result obj

Return type

If task completed

Raises

Exception obj – Exception due to which the task failed:

get_batch_result(task_id_list)

Request status for a batch of task_ids

run(*args, endpoint_id: Union[uuid.UUID, str], function_id: Union[uuid.UUID, str], **kwargs) → str

Initiate an invocation

Parameters

  • *args (Any) – Args as specified by the function signature

  • endpoint_id (uuid str) – Endpoint UUID string. Required

  • function_id (uuid str) – Function UUID string. Required

Returns

  • task_id (str)

  • UUID string that identifies the task

create_batch(task_group_id: Union[None, uuid.UUID, str] = None, user_endpoint_config: Optional[dict[str, Any]] = None, create_websocket_queue: bool = False) → globus_compute_sdk.sdk.batch.Batch

Create a Batch instance to handle batch submission in Globus Compute

Parameters

  • endpoint_id (UUID-like) – ID of the endpoint where the tasks in this batch will be executed

  • task_group_id (UUID-like (optional)) – Associate this batch with a pre-existing Task Group. If there is no Task Group associated with the given ID, or the user is not authorized to use it, the services will respond with an error. If task_group_id is not specified, the services will create a Task Group.

  • user_endpoint_config (dict (optional)) – User endpoint configuration values as described and allowed by endpoint administrators

  • create_websocket_queue (bool) – Whether to create a websocket queue for the task_group_id if it isn’t already created

Return type

Batch instance

batch_run(endpoint_id: Union[uuid.UUID, str], batch: globus_compute_sdk.sdk.batch.Batch) → dict[str, str | dict[str, list[str]]]

Initiate a batch of tasks to Globus Compute

Parameters

  • endpoint_id – The endpoint identifier to which to send the batch

  • batch – a Batch object

Returns

a dictionary with the following keys:

  • tasks: a mapping of function IDs to lists of task IDs

  • request_id: arbitrary unique string the web-service assigns this request (only intended for help with support requests)

  • task_group_id: the task group identifier associated with the submitted tasks

  • endpoint_id: the endpoint the tasks were submitted to

register_endpoint(name, endpoint_id: Optional[Union[uuid.UUID, str]], metadata: dict | None = None, multi_user: bool | None = None, display_name: str | None = None, allowed_functions: list[Union[uuid.UUID, str]] | None = None, auth_policy: Optional[Union[uuid.UUID, str]] = None)

Register an endpoint with the Globus Compute service.

Parameters

  • name (str) – Name of the endpoint

  • endpoint_id (str | UUID | None) – The uuid of the endpoint

  • metadata (dict | None) – Endpoint metadata

  • multi_user (bool | None) – Whether the endpoint supports multiple users

  • display_name (str | None) – The display name of the endpoint

  • allowed_functions (list[str | UUID] | None) – List of functions that are allowed to be run on the endpoint

  • auth_policy (str | UUID | None) – Endpoint users are evaluated against this Globus authentication policy

Returns

{‘endpoint_id’<>,

’address’ : <>, ‘client_ports’: <>}

Return type

dict

get_containers(name, description=None)

Register a DLHub endpoint with the Globus Compute service and get the containers to launch.

Parameters

  • name (str) – Name of the endpoint

  • description (str) – Description of the endpoint

Returns

The port to connect to and a list of containers

Return type

int

get_container(container_uuid, container_type)

Get the details of a container for staging it locally.

Parameters

  • container_uuid (str) – UUID of the container in question

  • container_type (str) – The type of containers that will be used (Singularity, Shifter, Docker)

Returns

The details of the containers to deploy

Return type

dict

get_endpoint_status(endpoint_uuid)

Get the status reports for an endpoint.

Parameters

endpoint_uuid (str) – UUID of the endpoint in question

Returns

The details of the endpoint’s stats

Return type

dict

get_endpoint_metadata(endpoint_uuid)

Get the metadata for an endpoint.

Parameters

endpoint_uuid (str) – UUID of the endpoint in question

Returns

Informational fields about the metadata, such as IP, hostname, and configuration values. If there were any issues deserializing this data, may also include an “errors” key.

Return type

dict

get_endpoints()

Get a list of all endpoints owned by the current user across all systems.

Returns

A list of dictionaries which contain endpoint info

Return type

list

register_function(function, function_name=None, container_uuid=None, description=None, metadata: dict | None = None, public=False, group=None, searchable=None) → str

Register a function code with the Globus Compute service.

Parameters

  • function (Python Function) – The function to be registered for remote execution

  • function_name (str) – The entry point (function name) of the function. Default: None

  • container_uuid (str) – Container UUID from registration with Globus Compute

  • description (str) – Description of the file

  • metadata (dict) – Function metadata (E.g., Python version used when serializing the function)

  • public (bool) – Whether or not the function is publicly accessible. Default = False

  • group (str) – A globus group uuid to share this function with

  • searchable (bool) –

    If true, the function will be indexed into globus search with the appropriate permissions

    DEPRECATED - ingesting functions to Globus Search is not currently supported

Returns

function uuid – UUID identifier for the registered function

Return type

str

get_function(function_id: Union[uuid.UUID, str])

Submit a request for details about a registered function.

Parameters

function_id (UUID | str) – UUID of the registered function

Returns

Information about the registered function, such as name, description, serialized source code, python version, etc.

Return type

dict

register_container(location, container_type, name='', description='')

Register a container with the Globus Compute service.

Parameters

  • location (str) – The location of the container (e.g., its docker url). Required

  • container_type (str) – The type of containers that will be used (Singularity, Shifter, Docker). Required

  • name (str) – A name for the container. Default = ‘’

  • description (str) – A description to associate with the container. Default = ‘’

Returns

The id of the container

Return type

str

build_container(container_spec)

Submit a request to build a docker image based on a container spec. This container build service is based on repo2docker, so the spec reflects features supported by it.

Only members of a managed globus group are allowed to use this service at present. This call will throw a ContainerBuildForbidden exception if you are not a member of this group.

Parameters

container_spec (globus_compute_sdk.sdk.container_spec.ContainerSpec) – Complete specification of what goes into the container

Returns

UUID of the container which can be used to register your function

Return type

str

Raises

ContainerBuildForbidden – User is not in the globus group that protects the build

add_to_whitelist(endpoint_id, function_ids)

Adds the function to the endpoint’s whitelist

Parameters

  • endpoint_id (str) – The uuid of the endpoint

  • function_ids (list) – A list of function id’s to be whitelisted

Returns

The response of the request

Return type

json

get_whitelist(endpoint_id)

List the endpoint’s whitelist

Parameters

endpoint_id (str) – The uuid of the endpoint

Returns

The response of the request

Return type

json

delete_from_whitelist(endpoint_id, function_ids)

List the endpoint’s whitelist

Parameters

  • endpoint_id (str) – The uuid of the endpoint

  • function_ids (list) – A list of function id’s to be whitelisted

Returns

The response of the request

Return type

json

stop_endpoint(endpoint_id: str)

Stop an endpoint by dropping it’s active connections.

Parameters

endpoint_id (str) – The uuid of the endpoint

Returns

The response of the request

Return type

json

delete_endpoint(endpoint_id: str)

Delete an endpoint

Parameters

endpoint_id (str) – The uuid of the endpoint

Returns

The response of the request

Return type

json

delete_function(function_id: str)

Delete a function

Parameters

function_id (str) – The UUID of the function

Returns

The response of the request

Return type

json

class globus_compute_sdk.sdk.container_spec.ContainerSpec(name: Optional[str] = None, description: Optional[str] = None, apt: Optional[list[str]] = None, pip: Optional[list[str]] = None, conda: Optional[list[str]] = None, python_version: str = '3.7', payload_url: Optional[str] = None)

Construct a container spec to pass to service for build operation.

Parameters

  • name (str) – Name of this container to be used inside Globus Compute

  • description (str) – Description of the container inside Globus Compute

  • apt (List[str]) – List of Ubuntu library packages to install in container

  • pip (List[str]) – List of Python libraries to install from pypi

  • conda (List[str]) – List of Conda packages to install

  • python_version (str) – Version of Python to build into image

  • payload_url (str) – GitHub repo or publicly readable zip file to copy into container