Skip to content

Allow concurrency for tasks that does not collide. Fixes #365 #366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from Aug 19, 2017
Merged

Allow concurrency for tasks that does not collide. Fixes #365 #366

merged 1 commit into from Aug 19, 2017

Conversation

Natanande
Copy link
Contributor

Two different concurrency modes are implemented, and is enabled by
setting "concurrency_mode" in the config file to either "project" or "node".

When "project" concurrency is enabled, tasks will run in parallel if and
only if they do not share the same project id, with no regard to the
nodes/hosts that are affected.

When "node" concurrency is enabled, a task will run in parallel if and
only if the hosts affected by tasks already running does not intersect
with the hosts that would be affected by the task in question.

If "concurrency_mode" is not specified, no task will start before the
previous one has finished.

The collision check is based on the output from the "--list-hosts"
argument to ansible, which uses the hosts specified in the inventory.
Thus, if two different hostnames are used that points to the same node,
such as "127.0.0.1" and "localhost", there will be no collision and two
tasks may connect to the same node concurrently. If this behaviour is
not desired, one should make sure to not include aliases for their hosts
in their inventories when enabling concurrency mode.

To restrict the amount of parallel tasks that runs at the same time, one
can add the "max_parallel_tasks" to the config file. This defaults to a
humble 10 if not specified.

Two different concurrency modes are implemented, and is enabled by
setting "concurrency_mode" in the config file to either "project" or "node".

When "project" concurrency is enabled, tasks will run in parallel if and
only if they do not share the same project id, with no regard to the
nodes/hosts that are affected.

When "node" concurrency is enabled, a task will run in parallel if and
only if the hosts affected by tasks already running does not intersect
with the hosts that would be affected by the task in question.

If "concurrency_mode" is not specified, no task will start before the
previous one has finished.

The collision check is based on the output from the "--list-hosts"
argument to ansible, which uses the hosts specified in the inventory.
Thus, if two different hostnames are used that points to the same node,
such as "127.0.0.1" and "localhost", there will be no collision and two
tasks may connect to the same node concurrently. If this behaviour is
not desired, one should make sure to not include aliases for their hosts
in their inventories when enabling concurrency mode.

To restrict the amount of parallel tasks that runs at the same time, one
can add the "max_parallel_tasks" to the config file. This defaults to a
humble 10 if not specified.
@matejkramny
Copy link
Contributor

👍 will check this out & test

Thanks!

@Natanande
Copy link
Contributor Author

Also provides partial solution to #237

@matejkramny
Copy link
Contributor

Many thanks for this. I'm reviewing if it breaks anything & will be merged shortly after.

My apologies for the long delay.

@matejkramny matejkramny merged commit 8bc1b7f into semaphoreui:develop Aug 19, 2017
@Echobob Echobob mentioned this pull request Aug 28, 2017
@jfloodnet
Copy link

Would be nice to have a 3rd option to allow the same task to deploy concurrently. I've been using semaphore to provision VM's on my proxmox cluster.

There's no reason I couldn't run the same task concurrently in this scenario and it would have been nice if the option was there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants