The Globus Compute Client#

class globus_compute_sdk.Client(http_timeout=None, funcx_home=None, environment: str | None = None, task_group_id: None | UUID | str = None, local_compute_services: bool = False, do_version_check: bool = True, *, code_serialization_strategy: SerializationStrategy | None = None, data_serialization_strategy: SerializationStrategy | None = None, login_manager: LoginManagerProtocol | None = None, **kwargs)#

Main class for interacting with the Globus Compute service

Holds helper operations for performing common tasks with the Globus Compute service.

Initialize the client

Parameters:
  • http_timeout (int) –

    Timeout for any call to service in seconds. Default is no timeout

    DEPRECATED - see self.web_client

  • funcx_home (any) – DEPRECATED - was never used

  • environment (str) – For internal use only. The name of the environment to use.

  • task_group_id (str|uuid.UUID) –

    Set the TaskGroup ID (a UUID) for this Client instance. Typically, one uses this to submit new tasks to an existing session or to reestablish Executor futures. Default: None (will be auto generated)

    DEPRECATED - use create_batch or the executor instead

  • local_compute_services (str) – For internal use only. TODO

  • do_version_check (bool) – Set to False to skip the version compatibility check on client initialization Default: True

  • code_serialization_strategy (SerializationStrategy) – Strategy to use when serializing function code. If None, globus_compute_sdk.serialize.DEFAULT_STRATEGY_CODE will be used.

  • data_serialization_strategy (SerializationStrategy) – Strategy to use when serializing function arguments. If None, globus_compute_sdk.serialize.DEFAULT_STRATEGY_DATA will be used.

  • login_manager (LoginManagerProtocol) – Allows login logic to be overridden for specific use cases. If None, a LoginManager will be used.

version_check(endpoint_version: str | None = None) None#

Check this client version meets the service’s minimum supported version.

Raises a VersionMismatch error on failure.

logout()#

Remove credentials from your local system

get_task(task_id)#

Get a Globus Compute task.

Parameters:

task_id (str) – UUID of the task

Returns:

Task block containing “status” key.

Return type:

dict

get_result(task_id)#

Get the result of a Globus Compute task

Parameters:

task_id (str) – UUID of the task

Returns:

Result obj

Return type:

If task completed

Raises:

Exception obj – Exception due to which the task failed:

get_batch_result(task_id_list)#

Request status for a batch of task_ids

run(*args, endpoint_id: UUID | str, function_id: UUID | str, **kwargs) str#

Initiate an invocation

Parameters:
  • *args (Any) – Args as specified by the function signature

  • endpoint_id (uuid str) – Endpoint UUID string. Required

  • function_id (uuid str) – Function UUID string. Required

Returns:

  • task_id (str)

  • UUID string that identifies the task

create_batch(task_group_id: None | UUID | str = None, user_endpoint_config: dict[str, Any] | None = None, create_websocket_queue: bool = False) Batch#

Create a Batch instance to handle batch submission in Globus Compute

Parameters:
  • endpoint_id (UUID-like) – ID of the endpoint where the tasks in this batch will be executed

  • task_group_id (UUID-like (optional)) – Associate this batch with a pre-existing Task Group. If there is no Task Group associated with the given ID, or the user is not authorized to use it, the services will respond with an error. If task_group_id is not specified, the services will create a Task Group.

  • user_endpoint_config (dict (optional)) – User endpoint configuration values as described and allowed by endpoint administrators

  • create_websocket_queue (bool) – Whether to create a websocket queue for the task_group_id if it isn’t already created

Return type:

Batch instance

batch_run(endpoint_id: UUID | str, batch: Batch) dict[str, str | dict[str, list[str]]]#

Initiate a batch of tasks to Globus Compute

Parameters:
  • endpoint_id – The endpoint identifier to which to send the batch

  • batch – a Batch object

Returns:

a dictionary with the following keys:

  • tasks: a mapping of function IDs to lists of task IDs

  • request_id: arbitrary unique string the web-service assigns this request (only intended for help with support requests)

  • task_group_id: the task group identifier associated with the submitted tasks

  • endpoint_id: the endpoint the tasks were submitted to

register_endpoint(name, endpoint_id: UUID | str | None, metadata: dict | None = None, multi_user: bool | None = None, display_name: str | None = None, allowed_functions: list[UUID | str] | None = None, auth_policy: UUID | str | None = None, subscription_id: UUID | str | None = None, public: bool | None = None)#

Register an endpoint with the Globus Compute service.

Parameters:
  • name (str) – Name of the endpoint

  • endpoint_id (str | UUID | None) – The uuid of the endpoint

  • metadata (dict | None) – Endpoint metadata

  • multi_user (bool | None) – Whether the endpoint supports multiple users

  • display_name (str | None) – The display name of the endpoint

  • allowed_functions (list[str | UUID] | None) – List of functions that are allowed to be run on the endpoint

  • auth_policy (str | UUID | None) – Endpoint users are evaluated against this Globus authentication policy

  • subscription_id (str | UUID | None) – Subscription ID to associate endpoint with

  • public (bool | None) – Indicates if all users can discover the multi-user endpoint.

Returns:

{‘endpoint_id’<>,

’address’ : <>, ‘client_ports’: <>}

Return type:

dict

get_containers(name, description=None)#

Register a DLHub endpoint with the Globus Compute service and get the containers to launch.

Parameters:
  • name (str) – Name of the endpoint

  • description (str) – Description of the endpoint

Returns:

The port to connect to and a list of containers

Return type:

int

get_container(container_uuid, container_type)#

Get the details of a container for staging it locally.

Parameters:
  • container_uuid (str) – UUID of the container in question

  • container_type (str) – The type of containers that will be used (Singularity, Shifter, Docker)

Returns:

The details of the containers to deploy

Return type:

dict

get_endpoint_status(endpoint_uuid)#

Get the status reports for an endpoint.

Parameters:

endpoint_uuid (str) – UUID of the endpoint in question

Returns:

The details of the endpoint’s stats

Return type:

dict

get_endpoint_metadata(endpoint_uuid)#

Get the metadata for an endpoint.

Parameters:

endpoint_uuid (str) – UUID of the endpoint in question

Returns:

Informational fields about the metadata, such as IP, hostname, and configuration values. If there were any issues deserializing this data, may also include an “errors” key.

Return type:

dict

get_endpoints()#

Get a list of all endpoints owned by the current user across all systems.

Returns:

A list of dictionaries which contain endpoint info

Return type:

list

register_function(function, function_name=None, container_uuid=None, description=None, metadata: dict | None = None, public=False, group=None, searchable=None) str#

Register a function code with the Globus Compute service.

Parameters:
  • function (Python Function) – The function to be registered for remote execution

  • function_name (str) – The entry point (function name) of the function. Default: None

  • container_uuid (str) – Container UUID from registration with Globus Compute

  • description (str) – Description of the file

  • metadata (dict) – Function metadata (E.g., Python version used when serializing the function)

  • public (bool) – Whether or not the function is publicly accessible. Default = False

  • group (str) – A globus group uuid to share this function with

  • searchable (bool) –

    If true, the function will be indexed into globus search with the appropriate permissions

    DEPRECATED - ingesting functions to Globus Search is not currently supported

Returns:

function uuid – UUID identifier for the registered function

Return type:

str

get_function(function_id: UUID | str)#

Submit a request for details about a registered function.

Parameters:

function_id (UUID | str) – UUID of the registered function

Returns:

Information about the registered function, such as name, description, serialized source code, python version, etc.

Return type:

dict

register_container(location, container_type, name='', description='')#

Register a container with the Globus Compute service.

Parameters:
  • location (str) – The location of the container (e.g., its docker url). Required

  • container_type (str) – The type of containers that will be used (Singularity, Shifter, Docker). Required

  • name (str) – A name for the container. Default = ‘’

  • description (str) – A description to associate with the container. Default = ‘’

Returns:

The id of the container

Return type:

str

build_container(container_spec)#

Submit a request to build a docker image based on a container spec. This container build service is based on repo2docker, so the spec reflects features supported by it.

Only members of a managed globus group are allowed to use this service at present. This call will throw a ContainerBuildForbidden exception if you are not a member of this group.

Parameters:

container_spec (globus_compute_sdk.sdk.container_spec.ContainerSpec) – Complete specification of what goes into the container

Returns:

UUID of the container which can be used to register your function

Return type:

str

Raises:

ContainerBuildForbidden – User is not in the globus group that protects the build

add_to_whitelist(endpoint_id, function_ids)#

Adds the function to the endpoint’s whitelist

Parameters:
  • endpoint_id (str) – The uuid of the endpoint

  • function_ids (list) – A list of function id’s to be whitelisted

Returns:

The response of the request

Return type:

json

get_whitelist(endpoint_id)#

List the endpoint’s whitelist

Parameters:

endpoint_id (str) – The uuid of the endpoint

Returns:

The response of the request

Return type:

json

delete_from_whitelist(endpoint_id, function_ids)#

List the endpoint’s whitelist

Parameters:
  • endpoint_id (str) – The uuid of the endpoint

  • function_ids (list) – A list of function id’s to be whitelisted

Returns:

The response of the request

Return type:

json

stop_endpoint(endpoint_id: str)#

Stop an endpoint by dropping it’s active connections.

Parameters:

endpoint_id (str) – The uuid of the endpoint

Returns:

The response of the request

Return type:

json

delete_endpoint(endpoint_id: str)#

Delete an endpoint

Parameters:

endpoint_id (str) – The uuid of the endpoint

Returns:

The response of the request

Return type:

json

delete_function(function_id: str)#

Delete a function

Parameters:

function_id (str) – The UUID of the function

Returns:

The response of the request

Return type:

json

get_worker_hardware_details(endpoint_id: UUID | str) str#

Run a function to get hardware details. Returns a task ID; when that task is finished, the result of the run will be a string containing hardware information on the nodes that the endpoint workers are running on. For example:

from globus_compute_sdk import Client
gcc = Client()
task_id = gcc.get_worker_hardware_details(ep_uuid)
# wait some time...
print(gcc.get_result(task_id))
class globus_compute_sdk.sdk.container_spec.ContainerSpec(name: str | None = None, description: str | None = None, apt: list[str] | None = None, pip: list[str] | None = None, conda: list[str] | None = None, python_version: str = '3.7', payload_url: str | None = None)#

Construct a container spec to pass to service for build operation.

Parameters:
  • name (str) – Name of this container to be used inside Globus Compute

  • description (str) – Description of the container inside Globus Compute

  • apt (List[str]) – List of Ubuntu library packages to install in container

  • pip (List[str]) – List of Python libraries to install from pypi

  • conda (List[str]) – List of Conda packages to install

  • python_version (str) – Version of Python to build into image

  • payload_url (str) – GitHub repo or publicly readable zip file to copy into container