[manunicholasjacob]

Description

In codecarbon/core/util.py, count_cpus() currently returns psutil.cpu_count() in the non-SLURM case. This reports the total number of logical CPUs on the host, even when the process has access to only a subset of them.

There are two constraints that need to be considered:

1. CPU affinity: restrictions imposed by taskset, Docker cpusets, or cpuset cgroups can limit which CPUs a process can use. On Linux, os.sched_getaffinity(0) provides the CPUs available to the current process.
2. CPU quota: container limits such as Docker's --cpus or Kubernetes CPU limits can restrict CPU usage independently of affinity. These limits can also be fractional.

The CPU count is used when estimating CPU power consumption. If the CPU model is not available in CodeCarbon's registry, the estimate is calculated using the number of threads multiplied by DEFAULT_POWER_PER_CORE (4 W). For example, a container limited to two CPUs running on a 64-core host could be reported as using 256 W instead of approximately 8 W. This significantly overestimates power consumption and affects the resulting energy and CO2 calculations.

Related Issue

No existing issue or PR covers this (checked before opening).

Motivation and Context

Update count_cpus() to report the effective number of CPUs available to the process rather than the total host count:

• Use os.sched_getaffinity(0) when available to determine the number of CPUs allowed by the affinity mask.
• Read the cgroup CPU quota and use the lower of the quota and affinity counts.
• Reuse the existing _read_cgroup_cpu_limit / _cgroup_cpu_limit helpers, which already support both cgroup v1 and v2 and return None when no CPU limit is configured.
• Preserve the existing psutil.cpu_count() behaviour when neither affinity nor a CPU quota imposes a restriction.
• Handle unsupported platforms and missing cgroup files gracefully so that the function does not fail when these mechanisms are unavailable.

CodeCarbon already does allocation-aware counting on SLURM (parsing NumCPUs=); this extends the same intent to the common container and cpuset case so the power model and metadata are not computed against cores the process cannot use.

How Has This Been Tested?

tests/test_core_util.py, run locally:

• a 2-CPU affinity mask yields 2;
• a 2.0-CPU cgroup quota caps an 8-CPU affinity to 2;
• affinity unavailable falls back to psutil.cpu_count();
• cgroup v2 cpu.max and v1 cfs_quota/cfs_period parsing (no root, no real cgroup);
• existing SLURM tests unchanged and passing.

black, isort (--profile black) and flake8 (--config=.flake8) are clean on the two changed files.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

AI Usage Disclosure

• 🟥 AI-vibecoded: You cannot explain the logic. Car analogy : the car drive by itself, you are outside it and just tell it where to go.
• 🟠 AI-generated: Car analogy : the car drive by itself, you are inside and give instructions.
• ⭐ AI-assisted. Car analogy : you drive the car, AI help you find your way.
• ♻️ No AI used. Car analogy : you drive the car.

I wrote this implementation, reusing my own cgroup-limit technique from numexpr PR #584 (_read_cgroup_cpu_limit / _cgroup_cpu_limit and the sched_getaffinity preference). An AI assistant produced an initial draft, which I rewrote in my own style, and the test scaffolding, which I reviewed.

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the docs/how-to/contributing.md document.
• I have added tests to cover my changes.
• All new and existing tests passed.

[SaboniAmine]

Thanks for this contribution, which will unlock some use cases on Kubernetes which we couldn't adress, due to our lack of knowledge of the way cgroups work.
Would it be possible for you to run some benchmarks, to show us how it behaves live in this kind of deployments ? This would be a great learning opportunity for us.

[manunicholasjacob]

glad this is useful

I tested the branch on a Raspberry Pi 5 (4 cores, aarch64, cgroup v2) to check how it handles both CPU affinity and cgroup quotas.

Here are the results from the actual runs using count_cpus() on this branch (commit 977df8b), compared with the previous psutil.cpu_count() implementation.

Scenario	New count_cpus()	Old psutil.cpu_count()	CPU affinity	Cgroup quota
Baseline, no limits	4	4	4	None
CPU pinning (taskset -c 0,1)	2	4	2	None
CFS quota (2.0 cores)	2	4	4	2.0
1.5-core quota with 3-core affinity	2	4	3	1.5

A few observations from these results:

The baseline remains unchanged, so there is no impact when running without CPU restrictions.

With CPU pinning, the process is restricted to two cores. os.sched_getaffinity(0) correctly reports those two cores, and the updated implementation reflects that. Previously, it reported all four.

With a CFS quota, affinity still reports all four cores, but the cpu.max setting (200000 100000) limits CPU usage to two cores. The updated implementation correctly accounts for this.

When both restrictions are present, the implementation uses the lower of the two limits.

This also affects the downstream power estimates. When the CPU model is not available in the registry, CodeCarbon uses threads * DEFAULT_POWER_PER_CORE, with a default of 4 W per core.

For example, on the Raspberry Pi, a two-core limit results in an estimated 8 W instead of 16 W. On a typical 64-core node, the same two-core container would previously have been estimated at 256 W instead of approximately 8 W. This also affects the resulting energy and CO2 estimates.

I also verified that the quota test was running with an actual cgroup limit (cpu.max = 200000 100000 under the transient scope), rather than relying on an assumed configuration.

I'd be happy to add the benchmark script to the repository or put together a short documentation note explaining the cgroup behaviour if that would be helpful.