Skip to content

Elastic computing - Dynamically allocate compute workers #2740

Open

Description

@Didayolo

I create this post-it to keep track of the discussion about elastic computing.

The observartion

Dynamically allocation of computing resources can be crucial for competition organizers, and is currently a blind spot of the software.

Most of the time, competitions only receive submissions sparsely. Keeping all VMs up and running while waiting for submissions is not optimized, and can be costly for organizers renting cloud computing. In the other hand, limiting too much the computation can get the queue stuck during submissions rushes.

Manually allocating the resources is annoying, not very precise, or even impossible if the goal is to have 0 VMs running and only creating them when receiving a submission.

The solution

The solution is yet to be discussed.

Where the scheduler should be?

Is it a separate orchestrator that can communicate with Codabench?

Are there existing solutions to be configured, or at least explained in the documentation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Post-itInternal ideas

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions