Cross-Platform Workflows
To utilize the system, users (or other Executors) submit function specifications to a Colonies Server, either through a JSON file or by using an SDK. Upon submission, these specifications are transformed into a so-called meta-process and are then registered in a database. An Executor connected to the Colonies Server may be assigned a meta-process, provided it meets the requirements outlined in the function specification.
It’s also possible to create a workflow, which is essentially a collection of function specifications with conditions dictating the order in which they may be executed. For example, the first function specification might be designated to run only on a specific type of Executor, such as a smartphone. The subsequent function specification could be set to execute on an Edge server, and a third may be restricted to an Executor located within an HPC system. When a workflow is submitted, all the function specifications within that workflow are transformed into meta-processes and registered in the database for future assignment. Because Executors can reside on different platforms and may be geographically dispersed, this architecture enables the implementation of cross-platform workflows that integrate seamlessly with various platforms.
It’s worth noting that meta-processes within the same workflow can utilize the output from previous meta-processes. This capability enables the implementation of complex computational patterns like map-reduce, as well as the execution of cross-platform parallel algorithms. This feature adds an extra layer of flexibility and power to the workflow, allowing for more advanced and integrated computational tasks.