Repository navigation
Automatically Generated Security Regression Tests - #43
Conversation
[Medium impact] The PR adds an optional generated-test workflow to scans and saved reports. Fix the disabled coverage-plugin failure and normalize reserved test paths before merging.
|
| def available(rel: str, stored_code: Optional[str] = None) -> bool: | ||
| path = repo_path(rel) | ||
| if vulnerable.exists(path): | ||
| return False | ||
| if remediated is not None and remediated.exists(path): | ||
| # The developer may have committed our pending test alongside | ||
| # the fix; re-validating that identical file is fine. | ||
| existing = (remediated.root / path).read_text(encoding="utf-8", errors="replace") | ||
| return stored_code is not None and existing == stored_code | ||
| return True | ||
|
|
||
| suggested = adapter.suggest_test_path(profile, invariant, lambda rel: not available(rel)) |
There was a problem hiding this comment.
Accepted test paths are not reserved across findings. Two IDOR findings for the same resource get the same suggested filename because available() checks only the original revisions. Both tests can be accepted, but their patches both add the same file, so users cannot apply both. Track accepted repository paths and reject or rename later collisions.
Prompt To Fix With AI
This is a comment left during a code review.
Path: openhack/regression/pipeline.py
Line: 476-487
Comment:
**Generated patches conflict**
Accepted test paths are not reserved across findings. Two IDOR findings for the same resource get the same suggested filename because `available()` checks only the original revisions. Both tests can be accepted, but their patches both add the same file, so users cannot apply both. Track accepted repository paths and reject or rename later collisions.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| # --no-cov is added (only when pytest-cov is installed, as it is | ||
| # otherwise an unknown option). | ||
| script = ( | ||
| 'if python -c "import pytest_cov" 2>/dev/null; then set -- "$1" --no-cov; fi; ' |
There was a problem hiding this comment.
Disabled coverage stops pytest
If a project installs pytest-cov but disables it with addopts = -p no:cov, importing pytest_cov still succeeds. This command then adds --no-cov, but the disabled plugin cannot register that option. Pytest exits before running any tests, so otherwise valid regression tests cannot be validated. Check whether the plugin is enabled before adding its option.
Prompt To Fix With AI
This is a comment left during a code review.
Path: openhack/regression/frameworks.py
Line: 704
Comment:
**Disabled coverage stops pytest**
If a project installs `pytest-cov` but disables it with `addopts = -p no:cov`, importing `pytest_cov` still succeeds. This command then adds `--no-cov`, but the disabled plugin cannot register that option. Pytest exits before running any tests, so otherwise valid regression tests cannot be validated. Check whether the plugin is enabled before adding its option.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
Summary
Implements autonomous security regression test generation for verified vulnerabilities, helping prevent previously discovered and remediated security issues from being reintroduced. Solves (#41).
Changes
Motivation
Extends OpenHack's vulnerability discovery and remediation capabilities into persistent security regression prevention, enabling developers to catch recurring vulnerabilities through CI/CD without requiring repeated AI-powered scans.