The Globus Compute Client¶
- class globus_compute_sdk.Client(http_timeout=None, funcx_home=None, environment: Optional[str] = None, task_group_id: Union[None, uuid.UUID, str] = None, funcx_service_address: Optional[str] = None, do_version_check: bool = True, *, code_serialization_strategy: Optional[globus_compute_sdk.serialize.base.SerializationStrategy] = None, data_serialization_strategy: Optional[globus_compute_sdk.serialize.base.SerializationStrategy] = None, login_manager: Optional[globus_compute_sdk.sdk.login_manager.protocol.LoginManagerProtocol] = None, **kwargs)¶
Main class for interacting with the Globus Compute service
Holds helper operations for performing common tasks with the Globus Compute service.
Initialize the client
- Parameters
http_timeout (int) –
Timeout for any call to service in seconds. Default is no timeout
DEPRECATED - see self.web_client
funcx_home (any) – DEPRECATED - was never used
environment (str) – For internal use only. The name of the environment to use. Sets funcx_service_address appropriately, unless already set.
task_group_id (str|uuid.UUID) –
Set the TaskGroup ID (a UUID) for this Client instance. Typically, one uses this to submit new tasks to an existing session or to reestablish Executor futures. Default: None (will be auto generated)
DEPRECATED - use create_batch or the executor instead
funcx_service_address (str) – For internal use only. The address of the web service.
do_version_check (bool) – Set to
False
to skip the version compatibility check on client initialization Default: Truecode_serialization_strategy (SerializationStrategy) – Strategy to use when serializing function code. If None, globus_compute_sdk.serialize.DEFAULT_STRATEGY_CODE will be used.
data_serialization_strategy (SerializationStrategy) – Strategy to use when serializing function arguments. If None, globus_compute_sdk.serialize.DEFAULT_STRATEGY_DATA will be used.
login_manager (LoginManagerProtocol) – Allows login logic to be overridden for specific use cases. If None, a LoginManager will be used.
- version_check(endpoint_version: Optional[str] = None) None ¶
Check this client version meets the service’s minimum supported version.
Raises a VersionMismatch error on failure.
- logout()¶
Remove credentials from your local system
- get_task(task_id)¶
Get a Globus Compute task.
- get_result(task_id)¶
Get the result of a Globus Compute task
- Parameters
task_id (str) – UUID of the task
- Returns
Result obj
- Return type
If task completed
- Raises
Exception obj – Exception due to which the task failed:
- get_batch_result(task_id_list)¶
Request status for a batch of task_ids
- run(*args, endpoint_id: Union[uuid.UUID, str], function_id: Union[uuid.UUID, str], **kwargs) str ¶
Initiate an invocation
- Parameters
*args (Any) – Args as specified by the function signature
endpoint_id (uuid str) – Endpoint UUID string. Required
function_id (uuid str) – Function UUID string. Required
- Returns
task_id (str)
UUID string that identifies the task
- create_batch(task_group_id: Union[None, uuid.UUID, str] = None, user_endpoint_config: Optional[dict[str, Any]] = None, create_websocket_queue: bool = False) globus_compute_sdk.sdk.batch.Batch ¶
Create a Batch instance to handle batch submission in Globus Compute
- Parameters
endpoint_id (UUID-like) – ID of the endpoint where the tasks in this batch will be executed
task_group_id (UUID-like (optional)) – Associate this batch with a pre-existing Task Group. If there is no Task Group associated with the given ID, or the user is not authorized to use it, the services will respond with an error. If task_group_id is not specified, the services will create a Task Group.
user_endpoint_config (dict (optional)) – User endpoint configuration values as described and allowed by endpoint administrators
create_websocket_queue (bool) – Whether to create a websocket queue for the task_group_id if it isn’t already created
- Return type
Batch instance
- batch_run(endpoint_id: Union[uuid.UUID, str], batch: globus_compute_sdk.sdk.batch.Batch) dict[str, str | dict[str, list[str]]] ¶
Initiate a batch of tasks to Globus Compute
- Parameters
endpoint_id – The endpoint identifier to which to send the batch
batch – a Batch object
- Returns
a dictionary with the following keys:
tasks: a mapping of function IDs to lists of task IDs
request_id: arbitrary unique string the web-service assigns this request (only intended for help with support requests)
task_group_id: the task group identifier associated with the submitted tasks
endpoint_id: the endpoint the tasks were submitted to
- register_endpoint(name, endpoint_id: Optional[Union[uuid.UUID, str]], metadata: dict | None = None, multi_user: bool | None = None, display_name: str | None = None, allowed_functions: list[Union[uuid.UUID, str]] | None = None, auth_policy: Optional[Union[uuid.UUID, str]] = None)¶
Register an endpoint with the Globus Compute service.
- Parameters
name (str) – Name of the endpoint
endpoint_id (str | UUID | None) – The uuid of the endpoint
metadata (dict | None) – Endpoint metadata
multi_user (bool | None) – Whether the endpoint supports multiple users
display_name (str | None) – The display name of the endpoint
allowed_functions (list[str | UUID] | None) – List of functions that are allowed to be run on the endpoint
auth_policy (str | UUID | None) – Endpoint users are evaluated against this Globus authentication policy
- Returns
- {‘endpoint_id’<>,
’address’ : <>, ‘client_ports’: <>}
- Return type
- get_containers(name, description=None)¶
Register a DLHub endpoint with the Globus Compute service and get the containers to launch.
- get_container(container_uuid, container_type)¶
Get the details of a container for staging it locally.
- get_endpoint_status(endpoint_uuid)¶
Get the status reports for an endpoint.
- get_endpoint_metadata(endpoint_uuid)¶
Get the metadata for an endpoint.
- get_endpoints()¶
Get a list of all endpoints owned by the current user across all systems.
- Returns
A list of dictionaries which contain endpoint info
- Return type
- register_function(function, function_name=None, container_uuid=None, description=None, metadata: dict | None = None, public=False, group=None, searchable=None) str ¶
Register a function code with the Globus Compute service.
- Parameters
function (Python Function) – The function to be registered for remote execution
function_name (str) – The entry point (function name) of the function. Default: None
container_uuid (str) – Container UUID from registration with Globus Compute
description (str) – Description of the file
metadata (dict) – Function metadata (E.g., Python version used when serializing the function)
public (bool) – Whether or not the function is publicly accessible. Default = False
group (str) – A globus group uuid to share this function with
searchable (bool) –
If true, the function will be indexed into globus search with the appropriate permissions
DEPRECATED - ingesting functions to Globus Search is not currently supported
- Returns
function uuid – UUID identifier for the registered function
- Return type
- get_function(function_id: Union[uuid.UUID, str])¶
Submit a request for details about a registered function.
- register_container(location, container_type, name='', description='')¶
Register a container with the Globus Compute service.
- Parameters
location (str) – The location of the container (e.g., its docker url). Required
container_type (str) – The type of containers that will be used (Singularity, Shifter, Docker). Required
name (str) – A name for the container. Default = ‘’
description (str) – A description to associate with the container. Default = ‘’
- Returns
The id of the container
- Return type
- build_container(container_spec)¶
Submit a request to build a docker image based on a container spec. This container build service is based on repo2docker, so the spec reflects features supported by it.
Only members of a managed globus group are allowed to use this service at present. This call will throw a ContainerBuildForbidden exception if you are not a member of this group.
- Parameters
container_spec (globus_compute_sdk.sdk.container_spec.ContainerSpec) – Complete specification of what goes into the container
- Returns
UUID of the container which can be used to register your function
- Return type
- Raises
ContainerBuildForbidden – User is not in the globus group that protects the build
- add_to_whitelist(endpoint_id, function_ids)¶
Adds the function to the endpoint’s whitelist
- get_whitelist(endpoint_id)¶
List the endpoint’s whitelist
- Parameters
endpoint_id (str) – The uuid of the endpoint
- Returns
The response of the request
- Return type
json
- delete_from_whitelist(endpoint_id, function_ids)¶
List the endpoint’s whitelist
- stop_endpoint(endpoint_id: str)¶
Stop an endpoint by dropping it’s active connections.
- Parameters
endpoint_id (str) – The uuid of the endpoint
- Returns
The response of the request
- Return type
json
- class globus_compute_sdk.sdk.container_spec.ContainerSpec(name: Optional[str] = None, description: Optional[str] = None, apt: Optional[list[str]] = None, pip: Optional[list[str]] = None, conda: Optional[list[str]] = None, python_version: str = '3.7', payload_url: Optional[str] = None)¶
Construct a container spec to pass to service for build operation.
- Parameters
name (str) – Name of this container to be used inside Globus Compute
description (str) – Description of the container inside Globus Compute
apt (List[str]) – List of Ubuntu library packages to install in container
pip (List[str]) – List of Python libraries to install from pypi
conda (List[str]) – List of Conda packages to install
python_version (str) – Version of Python to build into image
payload_url (str) – GitHub repo or publicly readable zip file to copy into container