Compute Brokering
To utilize ColonyOS, users or other Executors submit function specifications or workflows to a Colonies Server. These specifications, typically in JSON format, serve as high-level blueprints that outline future computations. Once submitted, these specifications are transformed into meta-processes and registered in a database that functions as a queue. The Colonies Server, acting as a broker, then assigns an Executor that meets the conditions specified in the function description to the meta-process. Once assigned, the Executor proceeds to execute the meta-process, effectively executing the functions detailed in the original specifications.
By adopting a brokering model, the system effectively separates the submission of function specifications from the real-time allocation of computing resources (Executors). This architectural separation adds a layer of adaptability and offers several strategic advantages. For instance, users can set conditions requiring that a meta-process to only be executed by Executors powered by green energy. Note that users can submit function specifications even when no green-energy Executors are available. In this case, the meta-process will simply remain in the queue until a suitable green-energy Executor becomes available.
Additionally, a brokering model makes the system easy to scale. One could start with a limited number of Executors operating on-premises and then, during times of increased demand, seamlessly transition to cloud-based resources by deploying additional Executors. This built-in flexibility enables a decoupled system that can adapt to changing operational requirements, thereby optimizing both resource utilization and overall operational efficiency.