Skip to content

Commit 29e4eda

Browse files
committed
GHA/windows: improve build perf with cmake unity batches
Default curl unity builds make a single unit for each target. It means all target sources are batched together and built in a single compiler invocation. With multi-core CPUs this doesn't always result in the best possible performance. This patch enables smaller batches for jobs where this resulted in shorter build times. These jobs are Cygwin, MSYS2, MinGW, running on the Windows runners. Use batch of 30 (meaning 30 sources batched into units), and 32 for Cygwin/MSYS2 to avoid a unity fallout that's subject to a different PR. (CMake allows to set the number of sources per unit, not the number of units, though the latter may be more practical to max out CPU cores.) Also override to not batch the `curlu` target because batching lost a little bit of time there, due to the already existing parallelism when building the `testdeps` targets. For jobs on the macOS and Linux runners jobs were already mostly single digit or below teen seconds, and batching didn't improve on them noticeably. On VM jobs, the virtual CPUs are limited, so I didn't make a try. In AppVeyor and GHA vcpkg jobs (using msbuild), batching didn't result in conclusive or any gains. Build times in seconds (curl + testdeps): Job | Before | After w curlu=0 | Gain :--------------------| :-------------- | :-------------- | :--- cygwin, CM | 19 + 32 = 51 | 12 + 32 = 44 | 7 msys2, CM | 7 + 15 = 22 | 5 + 14 = 19 | 3 mingw gcc U, CM | 19 + 30 = 49 | 13 + 32 = 45 | 4 mingw ucrt, CM | 32 + 42 = 74 | 15 + 43 = 58 | 16 mingw clang, CM | 15 + 21 = 36 | 8 + 21 = 29 | 7 mingw uwp, CM | 30 + 40 = 70 | 14 + 40 = 54 | 16 mingw gcc, CM | 20 + 31 = 51 | 12 + 31 = 43 | 8 mingw x86, CM | 35 + 40 = 75 | 15 + 38 = 53 | 22 dl-mingw, CM 9.5.0 | 88 + 99 = 187 | 42 + 101 = 143 | 44 dl-mingw, CM 7.3.0 U | 24 + 32 = 56 | 17 + 35 = 52 | 4 Total | | | 131 Total gain per GHA/windows workflow runs: 2m11s Runs: Before: https://github.com/curl/curl/actions/runs/13220256084/job/36904342259 After: https://github.com/curl/curl/actions/runs/13220383702/job/36904602981 https://github.com/curl/curl/actions/runs/13220613141/job/36905170104 https://github.com/curl/curl/actions/runs/13222019443/job/36908358550 With curlu tweak: https://github.com/curl/curl/actions/runs/13222239255/job/36908782462 Ref: 116950a curl#16265 Closes curl#16272
1 parent 48df83a commit 29e4eda

File tree

2 files changed

+6
-3
lines changed

2 files changed

+6
-3
lines changed

.github/workflows/windows.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ jobs:
8686
if [ '${{ matrix.build }}' = 'cmake' ]; then
8787
PATH="/usr/bin:$(cygpath "${SYSTEMROOT}")/System32"
8888
cmake -B bld -G Ninja ${options} \
89-
-DCMAKE_UNITY_BUILD=ON -DCURL_TEST_BUNDLES=ON \
89+
-DCMAKE_UNITY_BUILD=ON -DCMAKE_UNITY_BUILD_BATCH_SIZE=32 -DCURL_TEST_BUNDLES=ON \
9090
-DCURL_WERROR=ON \
9191
${{ matrix.config }}
9292
else
@@ -252,7 +252,7 @@ jobs:
252252
cmake -B bld -G Ninja ${options} \
253253
-DCMAKE_C_FLAGS="${{ matrix.cflags }} ${CFLAGS_CMAKE} ${CPPFLAGS}" \
254254
-DCMAKE_BUILD_TYPE='${{ matrix.type }}' \
255-
-DCMAKE_UNITY_BUILD=ON -DCURL_TEST_BUNDLES=ON \
255+
-DCMAKE_UNITY_BUILD=ON -DCMAKE_UNITY_BUILD_BATCH_SIZE=32 -DCURL_TEST_BUNDLES=ON \
256256
-DCURL_WERROR=ON \
257257
${{ matrix.config }}
258258
else
@@ -438,7 +438,7 @@ jobs:
438438
cmake -B bld -G 'MSYS Makefiles' ${options} \
439439
-DCMAKE_C_COMPILER=gcc \
440440
-DCMAKE_BUILD_TYPE='${{ matrix.type }}' \
441-
-DCMAKE_UNITY_BUILD=ON -DCURL_TEST_BUNDLES=ON \
441+
-DCMAKE_UNITY_BUILD=ON -DCMAKE_UNITY_BUILD_BATCH_SIZE=30 -DCURL_TEST_BUNDLES=ON \
442442
-DCURL_WERROR=ON \
443443
-DCURL_USE_LIBPSL=OFF \
444444
${{ matrix.config }}

lib/CMakeLists.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,9 @@ if(CURL_BUILD_TESTING)
5555
)
5656
target_compile_definitions(curlu PUBLIC "UNITTESTS" "CURL_STATICLIB")
5757
target_link_libraries(curlu PRIVATE ${CURL_LIBS})
58+
# There is plenty of parallelism when building the testdeps target.
59+
# Override the curlu batch size with the maximum to optimize performance.
60+
set_target_properties(curlu PROPERTIES UNITY_BUILD_BATCH_SIZE 0)
5861
endif()
5962

6063
if(ENABLE_CURLDEBUG)

0 commit comments

Comments
 (0)