funcX is a Function-as-a-Service (FaaS) platform for science that enables you to convert almost any computing resource into a high-performance function serving device. To do this, you deploy a funcX endpoint agent on the resource, which integrates it into the function serving fabric, allowing you to dynamically send, monitor, and receive results from function invocations. funcX is built on top of Parsl, enabling a funcX endpoint to use large compute resources via traditional batch queues, where funcX will dynamically provision, use, and release resources on-demand to fulfill function requests. The function service fabric, which is run centrally as a service, is hosted in AWS.
Here we provide an example of using funcX to register a function and run it on a publicly available tutorial endpoint.
We start by instantiating a funcX client as a programmatic means of communicating with the function service fabric. The client allows you to: - Register functions - Register containers and execution environments - Launch registered functions against endpoints - Check the status of launched functions - Retrieve outputs from functions
Instantiating a client will force an authentication flow where you will be asked to authenticate with Globus Auth. Every interaction with funcX is authenticated to allow us to enforce access control on both functions and endpoints. As part of the authentication process we request access to your identity information (to retrieve your email address) and Globus Groups management access. We require Groups access in order to facilitate sharing.
from funcx.sdk.client import FuncXClient fxc = FuncXClient()
Next we define a Python function, which we will later register with funcX. This function simply sums its input.
When defining a function you can specify *args and **kwargs as inputs.
Note: any dependencies for a funcX function must be specified inside the function body.¶
def funcx_sum(items): return sum(items)
Registering a function¶
To use a function with funcX, you must first register it with the service, using
register_function. You can optionally include a description of the function.
The registration process will serialize the function body and transmit it to the funcX function service fabric.
Registering a function returns a UUID for the function, which can then be used to invoke it.
func_uuid = fxc.register_function(funcx_sum, description="A summation function") print(func_uuid)
Running a function¶
To invoke (perform) a function, you must provide the function’s UUID, returned from the registration process, and an
endpoint_id. Note: here we use the funcX public tutorial endpoint, which is running on AWS.
run function will serialize any *args and **kwargs, and pass them to the function when invoking it. Therefore, as our example function simply takes an arg input (items), we can specify an input arg and it will be used by the function. Here we define a small list of integers for our function to sum.
The Web service will return the UUID for the invokation of the function, which we call a task. This UUID can be used to check the status of the task and retrieve the result.
endpoint_uuid = '4b116d3c-1703-4f8f-9f6f-39921e5864df' # Public tutorial endpoint items = [1, 2, 3, 4, 5] res = fxc.run(items, endpoint_id=endpoint_uuid, function_id=func_uuid) print(res)
You can now retrieve the result of the invocation using
get_result() on the UUID of the task.