fix: extend MODEL_NOT_SUPPORTED_PATTERN to catch 404 standalone "Model not found"#41792
Merged
Merged
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
… not found shape The AIC api-proxy returns "404 Not Found: Model not found" when a model snapshot is unavailable. The existing MODEL_NOT_SUPPORTED_PATTERN required a model name token between "model" and "not found", so the standalone phrase was silently missed (isInvalidModelError=false) and the harness burned all 3×5 retries before giving up. Add a new alternative `404\b[^\n]*\bModel\s+not\s+found` that matches any line containing a 404 status code followed by the bare "Model not found" phrase, making the error non-retryable on first detection. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Stop pinning unstable Codex snapshot and improve error handling
fix: extend MODEL_NOT_SUPPORTED_PATTERN to catch 404 standalone "Model not found"
Jun 26, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves agent error classification in gh-aw’s JS harness tooling by extending the invalid-model detection regex to recognize an AIC api-proxy “404 … Model not found” error shape, preventing wasted retries on a non-retryable configuration/error condition.
Changes:
- Extend
MODEL_NOT_SUPPORTED_PATTERNto match404 … Model not foundlines without an inline model identifier. - Add regression tests in both the pattern unit tests and the codex harness classifier tests.
- Update the generated
pr-triage-agent.lock.ymlworkflow metadata.
Show a summary per file
| File | Description |
|---|---|
| actions/setup/js/detect_agent_errors.cjs | Extends the invalid-model regex to catch standalone 404 … Model not found errors. |
| actions/setup/js/detect_agent_errors.test.cjs | Adds test coverage for the new 404 “Model not found” variants (standalone + embedded in logs). |
| actions/setup/js/codex_harness.test.cjs | Verifies the codex harness’s isInvalidModelError returns true for the new 404 shape. |
| .github/workflows/pr-triage-agent.lock.yml | Updates generated workflow lock metadata (body_hash). |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 2
- Review effort level: Low
| // - "Model not found" (standalone, e.g. AIC api-proxy 404: "404 Not Found: Model not found") | ||
| const MODEL_NOT_SUPPORTED_PATTERN = | ||
| /(?:The requested model is not supported|invalid model(?:\s+name)?\s+['"`]?[a-z0-9._:/@-]+['"`]?(?=(?:\s*$|\s*[\n\r.,;:!?)]))|unknown model\s+['"`]?[a-z0-9._:/@-]+['"`]?(?=(?:\s*$|\s*[\n\r.,;:!?)]))|model(?:\s+name)?\s+['"`]?[a-z0-9._:/@-]+['"`]?\s+(?:is\s+)?(?:not found|does not exist|not supported|not available|unavailable))/i; | ||
| /(?:The requested model is not supported|invalid model(?:\s+name)?\s+['"`]?[a-z0-9._:/@-]+['"`]?(?=(?:\s*$|\s*[\n\r.,;:!?)]))|unknown model\s+['"`]?[a-z0-9._:/@-]+['"`]?(?=(?:\s*$|\s*[\n\r.,;:!?)]))|model(?:\s+name)?\s+['"`]?[a-z0-9._:/@-]+['"`]?\s+(?:is\s+)?(?:not found|does not exist|not supported|not available|unavailable)|404\b[^\n]*\bModel\s+not\s+found)/i; |
| @@ -1,4 +1,4 @@ | |||
| # gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"98a744d3dc9efdd12bf821f81911e181d26ee211ece147180a6021797cd8bbe1","body_hash":"f7ed7fcaf7cb3e6043d11f79b63a516194a755016f6bbfd6875ce1c9efb20a86","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.65","copilot-sdk":"1.0.4"}} | |||
| # gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"98a744d3dc9efdd12bf821f81911e181d26ee211ece147180a6021797cd8bbe1","body_hash":"8d505062b276bc9c6fda2b79316117cb30c88ae04cab333ef78d113bf91c6aec","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.65","copilot-sdk":"1.0.4"}} | |||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The AIC api-proxy returns
404 Not Found: Model not found(no inline model name) when a snapshot is unavailable.MODEL_NOT_SUPPORTED_PATTERNonly matched forms with a model identifier token before "not found", so the harness loggedisInvalidModelError=falseand burned all 15 retries (3 attempts × 5 internal) on zero useful work.Changes
detect_agent_errors.cjs— adds a new alternative toMODEL_NOT_SUPPORTED_PATTERN:Matches any log line with a
404status code followed by the bareModel not foundphrase. Sincecodex_harness.cjsimports this asINVALID_MODEL_ERROR_PATTERN, both the real-time retry classifier and the post-rundetect-agent-errorsstep are fixed by a single change.detect_agent_errors.test.cjs/codex_harness.test.cjs— adds test cases for404 Not Found: Model not found,ResponseError: 404 Not Found: Model not found, and the embedded-in-log variant.