The Globus Compute Engine#

class globus_compute_endpoint.engines.GlobusComputeEngine(*args, label: str = 'GlobusComputeEngine', max_retries_on_system_failure: int = 0, strategy: ~globus_compute_endpoint.strategies.simple.SimpleStrategy | None = <globus_compute_endpoint.strategies.simple.SimpleStrategy object>, executor: ~parsl.executors.high_throughput.executor.HighThroughputExecutor | None = None, container_type: ~typing.Literal['docker', 'singularity', 'apptainer', 'custom', None] | None = None, container_uri: str | None = None, container_cmd_options: str | None = None, encrypted: bool = True, **kwargs)#

GlobusComputeEngine is a wrapper over Parsl’s HighThroughputExecutor

The GlobusComputeEngine is a shim over Parsl’s HighThroughputExecutor, almost all of arguments are passed along, unfettered. Consequently, please reference Parsl’s HighThroughputExecutor documentation for a complete list of arguments; we list below only the arguments specific to the GlobusComputeEngine.

Parameters:

  • label (str) – Label used to name engine log directories and batch jobs default: “GlobusComputeEngine”

  • max_retries_on_system_failure (int) – Set the number of retries for functions that fail due to system failures such as node failure/loss. Since functions can fail after partial runs, consider additional cleanup logic before enabling this functionality default: 0

  • strategy (Stategy object) – Specify scaling strategy. default: SimpleStrategy

  • encrypted (bool) – Flag to enable/disable encryption (CurveZMQ). Default is True.

containerized_launch_cmd() → str#

Recompose executor’s launch_cmd to launch with containers

Return type:

str launch_cmd

get_connected_managers() → List[Dict[str, Any]]#
Return type:

List of dicts containing info for all connected managers

get_total_managers(managers: List[Dict[str, Any]]) → int#
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Total number of managers

get_total_active_managers(managers: List[Dict[str, Any]]) → int#
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Number of managers that have capacity for new tasks

get_outstanding_breakdown(managers: List[Dict[str, Any]] | None = None) → List[Tuple[str, int, bool]]#
Parameters:

managers (list[dict[str, Any]] | None) – List of dicts containing info for all connected managers

Return type:

List of tuples of the form (component, # of tasks on component, active?)

get_total_tasks_outstanding() → dict#
Returns:

Dict of type {str_task_type

Return type:

count_tasks}

get_total_tasks_pending(managers: List[Dict[str, Any]]) → int#
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Total number of pending tasks

get_total_live_workers(managers: List[Dict[str, Any]] | None = None) → int#
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Total number of live workers

get_total_idle_workers(managers: List[Dict[str, Any]]) → int#
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Total number of workers that are not actively running tasks

property scaling_enabled: bool#

Indicates whether scaling is possible

get_status_report() → EPStatusReport#
Return type:

Object containing info on the current status of the endpoint