Skip to content

[ci] Remove label-based test configuration - #27416

Open
rolfbjarne wants to merge 3 commits into
mainfrom
dev/rolf/remove-label-test-skips
Open

rolfbjarne wants to merge 3 commits into
mainfrom
dev/rolf/remove-label-test-skips

Conversation

@rolfbjarne

Copy link
Copy Markdown
Member

Remove GitHub-label controls from xharness and Azure Pipelines, including test, signing, packaging, and API-diff decisions. Keep the command-line --label selectors used to partition test matrices. Also fix PR-number output when the lookup returns a single ID and update the CI documentation and fixtures.

🤖 Pull request created by Copilot

rolfbjarne and others added 3 commits November 20, 2026 17:10
Ignore pull request labels that disable tests and remove the obsolete pipeline skip paths. Keep command-line test selection and labels that enable tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c67666e9-65ac-4e6f-bb7a-9acefd611319
Match the current configuration output, including Xcode settings and the removal of Tags.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c67666e9-65ac-4e6f-bb7a-9acefd611319
Stop loading external labels in xharness and remove obsolete Azure Pipelines label configuration. Preserve the internal test matrix selectors and full PR numbers for manual builds.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c67666e9-65ac-4e6f-bb7a-9acefd611319
Copilot AI balanced review requested due to automatic review settings November 20, 2026 17:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the removal of GitHub-label controls from CI and xharness.

No open issues 🎉

What changed in this PR

Removes GitHub-label controls from the Apple-platform CI pipelines and xharness while preserving command-line --label selectors for test-matrix partitioning.

Changes:

  • Removes label-based test, packaging, signing, and API-diff configuration.
  • Renames configuration-step output references and fixes PR-number handling when a lookup returns a single ID.
  • Updates CI documentation, regression coverage, and fixtures; deletes unused GitHub helpers.
File Description
tools/​devops/​README.md Documents CI configuration without PR-label controls.
tools/​devops/​automation/​templates/​tests/​stage.yml Removes label-based test gating and updates PR output references.
tools/​devops/​automation/​templates/​tests/​publish-results.yml Removes the label-triggered skipped-tests reporting job.
tools/​devops/​automation/​templates/​tests-stage.yml Removes label variables and updates PR output references.
tools/​devops/​automation/​templates/​mac/​stage.yml Uses the renamed configuration output for PR IDs.
tools/​devops/​automation/​templates/​common/​load_configuration.yml Renames the configuration step and updates its description.
tools/​devops/​automation/​templates/​common/​configure.yml Renames the configuration step and updates its description.
tools/​devops/​automation/​templates/​build/​build.yml Replaces label parsing with direct build-directory setup.
tools/​devops/​automation/​templates/​build/​build-stage.yml Removes label variables and updates PR output references.
tools/​devops/​automation/​templates/​build/​build-pkgs.yml Removes the label-derived NuGet build condition.
tools/​devops/​automation/​templates/​build/​build-mac-tests-stage.yml Uses the renamed configuration output for PR IDs.
tools/​devops/​automation/​templates/​build/​api-diff-stage.yml Removes label forwarding and updates the PR output reference.
tools/​devops/​automation/​scripts/​VSTS.Tests.ps1 Updates configuration expectations and tests full PR-number output.
tools/​devops/​automation/​scripts/​VSTS.psm1 Removes label/tag plumbing and normalizes PR lookup results to an array.
tools/​devops/​automation/​scripts/​TestResults.Tests.ps1 Aligns fixtures with the revised pipeline outputs.
tools/​devops/​automation/​scripts/​parse_pr_labels.ps1 Deletes the label-based build configuration script.
tools/​devops/​automation/​scripts/​GitHub.Tests.ps1 Removes tests for the deleted PR-information helper.
tools/​devops/​automation/​scripts/​GitHub.psm1 Removes the unused PR-information helper and export.
tools/​devops/​automation/​scripts/​bash/​configure-decisions.sh Bases test decisions on enabled platforms rather than labels.
tools/​devops/​automation/​run-nightly-codeql.yml Removes the label-derived NuGet build condition.
tests/​xharness/​Jenkins/​TestSelector.cs Restricts label input to command-line selectors.
tests/​xharness/​Jenkins/​Jenkins.cs Constructs the test selector without a GitHub dependency.
tests/​xharness/​GitHub.cs Deletes the unused xharness GitHub integration.

Review effort: Balanced

@MauiBot

MauiBot commented Nov 20, 2026

Copy link
Copy Markdown

✅ API diff for current PR / commit

NET (empty diffs)

✅ API diff vs stable

NET (empty diffs)

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: b01aa788c05bc23fdd7af1c52e5b965ea0e0b69a [PR build]

@MauiBot

MauiBot commented Nov 20, 2026

Copy link
Copy Markdown

🔥 [CI Build #b01aa78] Test results 🔥

Test results

❌ Tests failed on VSTS: test results

0 tests crashed, 2 tests failed, 195 tests passed.

Failures

❌ introspection tests

1 tests failed, 3 tests passed.

Failed tests

  • introspection/macOS/Debug: Failed (Test run crashed (exit code: 134).
    Tests run: 34 Passed: 34 Inconclusive: 0 Failed: 0 Ignored: 0)

Html Report (VSDrops) Download

❌ Tests on macOS Golden Gate (27) tests

1 tests failed, 4 tests passed.

Failed tests

  • macOS/osx-arm64 introspection: Failed (exit code 134)
    • No test failure details available. Output tail:
      • 38 libcoreclr.dylib 0x0000000105b2d278 _ZN8Assembly17ExecuteMainMethodEPP8PtrArrayb + 320
      • 39 libcoreclr.dylib 0x0000000105b4fd30 _ZN8CorHost215ExecuteAssemblyEjPKDsiPS1_Pj + 748
      • 40 libcoreclr.dylib 0x0000000105cfc628 coreclr_execute_assembly + 244
      • 41 introspection 0x0000000104586e90 mono_jit_exec + 196
      • 42 introspection 0x000000010458bae0 xamarin_main + 932
      • 43 introspection 0x000000010458d244 main + 64
      • 44 dyld 0x0000000186520d7c start + 6652
      • )
      • libc++abi: terminating due to uncaught exception of type NSException
      • Execution completed with exit code 134

Html Report (VSDrops) Download

Successes

✅ assembly-processing: All 1 tests passed. Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (iOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (MacCatalyst): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (macOS): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (Multiple platforms): All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests (tvOS): All 1 tests passed. Html Report (VSDrops) Download
✅ framework: All 2 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 4 tests passed. Html Report (VSDrops) Download
✅ generator: All 1 tests passed. Html Report (VSDrops) Download
✅ interdependent-binding-projects: All 4 tests passed. Html Report (VSDrops) Download
✅ linker (iOS): All 17 tests passed. Html Report (VSDrops) Download
✅ linker (MacCatalyst): All 17 tests passed. Html Report (VSDrops) Download
✅ linker (macOS): All 17 tests passed. Html Report (VSDrops) Download
✅ linker (tvOS): All 17 tests passed. Html Report (VSDrops) Download
✅ monotouch (iOS): All 19 tests passed. Html Report (VSDrops) Download
✅ monotouch (MacCatalyst): All 19 tests passed. Html Report (VSDrops) Download
✅ monotouch (macOS): All 19 tests passed. Html Report (VSDrops) Download
✅ monotouch (tvOS): All 19 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ sharpie: All 1 tests passed. Html Report (VSDrops) Download
✅ windows: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 4 tests passed. Html Report (VSDrops) Download
✅ xtro: All 1 tests passed. Html Report (VSDrops) Download

macOS tests

✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Sequoia (15): All 5 tests passed. Html Report (VSDrops) Download
✅ Tests on macOS Tahoe (26): All 5 tests passed. Html Report (VSDrops) Download

Linux Build Verification

✅ Linux build succeeded

Pipeline on Agent
Hash: b01aa788c05bc23fdd7af1c52e5b965ea0e0b69a [PR build]

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants