The Globus Compute Engine

class globus_compute_endpoint.engines.GlobusComputeEngine(*args, label: str = 'GlobusComputeEngine', max_retries_on_system_failure: int = 0, executor: HighThroughputExecutor | None = None, container_type: Literal['docker', 'singularity', 'apptainer', 'custom', None] | None = None, container_uri: str | None = None, container_cmd_options: str | None = None, encrypted: bool = True, strategy: str | None = None, job_status_kwargs: JobStatusPollerKwargs | None = None, **kwargs)

GlobusComputeEngine is a wrapper over Parsl’s HighThroughputExecutor

The GlobusComputeEngine is a shim over Parsl’s HighThroughputExecutor, almost all of arguments are passed along, unfettered. Consequently, please reference Parsl’s HighThroughputExecutor documentation for a complete list of arguments; we list below only the arguments specific to the GlobusComputeEngine.

Parameters:

  • label (str) – Label used to name engine log directories and batch jobs default: “GlobusComputeEngine”

  • max_retries_on_system_failure (int) – Set the number of retries for functions that fail due to system failures such as node failure/loss. Since functions can fail after partial runs, consider additional cleanup logic before enabling this functionality default: 0

  • strategy (str | None) – Specify which scaling strategy to use; this is eventually given to Parsl’s Strategy. [Deprecated; use job_status_kwargs]

  • job_status_kwargs (dict | None) – Keyword arguments to be passed through to Parsl’s JobStatusPoller class that drives strategy to do auto-scaling.

  • encrypted (bool) – Flag to enable/disable encryption (CurveZMQ). Default is True.

containerized_launch_cmd() → str

Recompose executor’s launch_cmd to launch with containers

Return type:

str launch_cmd

get_connected_managers() → List[Dict[str, Any]]
Return type:

List of dicts containing info for all connected managers

get_total_managers(managers: List[Dict[str, Any]]) → int
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Total number of managers

get_total_active_managers(managers: List[Dict[str, Any]]) → int
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Number of managers that have capacity for new tasks

get_outstanding_breakdown(managers: List[Dict[str, Any]] | None = None) → List[Tuple[str, int, bool]]
Parameters:

managers (list[dict[str, Any]] | None) – List of dicts containing info for all connected managers

Return type:

List of tuples of the form (component, # of tasks on component, active?)

get_total_tasks_outstanding() → dict
Returns:

Dict of type {str_task_type

Return type:

count_tasks}

get_total_tasks_pending(managers: List[Dict[str, Any]]) → int
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Total number of pending tasks

get_total_live_workers(managers: List[Dict[str, Any]] | None = None) → int
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Total number of live workers

get_total_idle_workers(managers: List[Dict[str, Any]]) → int
Parameters:

managers (list[dict[str, Any]]) – List of dicts containing info for all connected managers

Return type:

Total number of workers that are not actively running tasks

property scaling_enabled: bool

Indicates whether scaling is possible

get_status_report() → EPStatusReport
Return type:

Object containing info on the current status of the endpoint