chore(ci): add goodegg contributor scoring action#8091
Conversation
✅ Deploy Preview for tiptap-embed ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
📝 WalkthroughAdds a GitHub Action that scores contributors based on their contribution history. It runs when pull requests are opened, updated, or reopened, helping identify consistent contributors and possible contribution farming. WalkthroughAdds a GitHub Actions workflow that runs ChangesGood Egg workflow
Estimated code review effort: 1 (Trivial) | ~2 minutes Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
🥚 Diet Egg: HIGH TrustScore: 81% Score Breakdown
Top Contributions
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/good-egg.yml:
- Around line 11-13: Update the good-egg action reference in the workflow to an
immutable full-length commit SHA instead of the mutable v2.0.0 tag, while
preserving the existing github-token input. Also pin the action’s transitive
setup-python and good-egg dependency references to immutable versions where they
are controlled by this workflow.
- Around line 2-6: Update the workflow trigger and permissions around the
pull_request configuration to explicitly handle forked pull requests: either
provide a trusted reporting path that can post Good Egg comments for external
contributors, or clearly document that fork PRs are unsupported. Do not switch
to pull_request_target while executing untrusted action code.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 66da5dff-44a3-4b3e-94ac-9419dae0f64e
📒 Files selected for processing (1)
.github/workflows/good-egg.yml
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened] | ||
| permissions: | ||
| pull-requests: write |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== workflow =="
git ls-files .github/workflows/good-egg.yml docs/github-action.md
echo
echo "== workflow contents =="
cat -n .github/workflows/good-egg.yml
echo
echo "== fork-related docs search =="
rg -n -i "fork|pull_request_target|pull-request|permissions|GITHUB_TOKEN|comment" docs/github-action.md .github/workflows/good-egg.yml || true
echo
echo "== docs excerpt =="
cat -n docs/github-action.md | sed -n '1,260p'Repository: ueberdosis/tiptap
Length of output: 945
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== workflow file =="
cat -n .github/workflows/good-egg.yml
echo
echo "== docs file search =="
rg -n -i "fork|pull_request_target|GITHUB_TOKEN|pull-requests: write|comment|unsupported" docs/github-action.md .github/workflows/good-egg.yml || true
echo
echo "== docs excerpt around matches =="
python3 - <<'PY'
from pathlib import Path
path = Path("docs/github-action.md")
lines = path.read_text().splitlines()
terms = ["fork", "pull_request_target", "GITHUB_TOKEN", "pull-requests: write", "comment", "unsupported"]
hits = []
for i, line in enumerate(lines, 1):
if any(t.lower() in line.lower() for t in terms):
hits.append(i)
for i in hits[:20]:
start = max(1, i-3)
end = min(len(lines), i+3)
print(f"\n-- lines {start}-{end} --")
for j in range(start, end+1):
print(f"{j}: {lines[j-1]}")
PYRepository: ueberdosis/tiptap
Length of output: 1349
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf 'Reading files only, no execution.\n'
for f in .github/workflows/good-egg.yml docs/github-action.md; do
echo "=== $f ==="
if [ -f "$f" ]; then
nl -ba "$f" | sed -n '1,220p'
else
echo "missing"
fi
doneRepository: ueberdosis/tiptap
Length of output: 268
🌐 Web query:
GitHub Actions pull_request from fork GITHUB_TOKEN read-only permissions pull-requests write documentation
💡 Result:
When a GitHub Actions workflow is triggered by a pull_request event originating from a forked repository, the GITHUB_TOKEN is automatically granted read-only permissions by default, regardless of the permissions configured in the workflow file [1][2][3]. This is a security measure designed to prevent untrusted code from a fork from accessing or modifying the base repository with write privileges [4][3]. To manage or circumvent this restriction, you have the following options: 1. Repository Settings: If you are an administrator of the repository, you can navigate to Settings > Actions > General > Workflow permissions and enable the option Send write tokens to workflows from pull requests [5][2][3]. Enabling this allows pull requests from forks to use a GITHUB_TOKEN with write permissions [5]. 2. Use pull_request_target: If you need to perform actions that require write access (such as adding labels, posting comments, or updating status checks), you can use the pull_request_target event [4][3]. Unlike the standard pull_request event, pull_request_target runs in the context of the base repository's default branch, not the commit from the fork [4][3]. Because it executes trusted code from the base repository, it is granted the repository's standard GITHUB_TOKEN permissions [4][2]. Warning: Since pull_request_target runs with elevated trust, you must exercise extreme caution. Do not use it to check out and execute code directly from the untrusted pull request, as this could lead to security vulnerabilities [4][3]. 3. Use a Personal Access Token (PAT): If you do not wish to use pull_request_target, or if you need permissions that go beyond the scope of GITHUB_TOKEN, you can use a PAT or a GitHub App token stored as a repository secret [3]. This allows the workflow to authenticate with specific, controlled permissions independent of the GITHUB_TOKEN limitations [3].
Citations:
- 1: Clarify Permissions Calculation for Workflow Jobs in Documentation github/docs#33566
- 2: https://help.github.com/en/enterprise-server@3.5/actions/security-guides/automatic-token-authentication
- 3: https://devopsil.com/articles/2026-03-30-github-actions-resource-not-accessible-fix
- 4: https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target
- 5: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository
Handle fork PRs explicitly. pull_request on forked PRs gets a read-only GITHUB_TOKEN, so Good Egg won’t be able to post comments for external contributors unless you add a trusted reporting path or clearly document that fork PRs are unsupported. Avoid switching this to pull_request_target with an untrusted action.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/good-egg.yml around lines 2 - 6, Update the workflow
trigger and permissions around the pull_request configuration to explicitly
handle forked pull requests: either provide a trusted reporting path that can
post Good Egg comments for external contributors, or clearly document that fork
PRs are unsupported. Do not switch to pull_request_target while executing
untrusted action code.
| - uses: 2ndSetAI/good-egg@v2.0.0 | ||
| with: | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the workflow file around the referenced lines.
git ls-files .github/workflows/good-egg.yml
cat -n .github/workflows/good-egg.yml | sed -n '1,80p'Repository: ueberdosis/tiptap
Length of output: 549
🏁 Script executed:
#!/bin/bash
set -euo pipefail
repo=2ndSetAI/good-egg
tag=v2.0.0
echo "== tag ref =="
curl -fsSL "https://api.github.com/repos/$repo/git/refs/tags/$tag" | jq '{object_type:.object.type, object_sha:.object.sha, ref:.ref}'
echo "== tag object (if annotated) =="
tag_obj_sha="$(curl -fsSL "https://api.github.com/repos/$repo/git/refs/tags/$tag" | jq -r '.object.sha')"
tag_obj_type="$(curl -fsSL "https://api.github.com/repos/$repo/git/refs/tags/$tag" | jq -r '.object.type')"
if [ "$tag_obj_type" = "tag" ]; then
curl -fsSL "https://api.github.com/repos/$repo/git/tags/$tag_obj_sha" | jq '{type:.object.type, commit_sha:.object.sha, tag:.tag}'
else
echo "lightweight tag -> commit $tag_obj_sha"
fi
echo "== action.yml at tag =="
commit_sha="$(curl -fsSL "https://api.github.com/repos/$repo/git/refs/tags/$tag" | jq -r '.object.sha')"
if [ "$(curl -fsSL "https://api.github.com/repos/$repo/git/refs/tags/$tag" | jq -r '.object.type')" = "tag" ]; then
commit_sha="$(curl -fsSL "https://api.github.com/repos/$repo/git/tags/$commit_sha" | jq -r '.object.sha')"
fi
curl -fsSL "https://raw.githubusercontent.com/$repo/$commit_sha/action.yml" | cat -n
echo "== package metadata / workflow references in repo tree =="
curl -fsSL "https://api.github.com/repos/$repo/contents?ref=$commit_sha" | jq -r '.[].name'Repository: ueberdosis/tiptap
Length of output: 220
🏁 Script executed:
#!/bin/bash
set -euo pipefail
repo_url=https://github.com/2ndSetAI/good-egg.git
tag=v2.0.0
echo "== git refs =="
git ls-remote --tags "$repo_url" "refs/tags/$tag" "refs/tags/$tag^{}" || true
echo "== action.yml at tag =="
curl -fsSL "https://raw.githubusercontent.com/2ndSetAI/good-egg/$tag/action.yml" | cat -n | sed -n '1,220p'
echo "== package.json at tag (if present) =="
curl -fsSL "https://raw.githubusercontent.com/2ndSetAI/good-egg/$tag/package.json" | cat -n | sed -n '1,220p' || true
echo "== repo root listing at tag =="
curl -fsSL "https://api.github.com/repos/2ndSetAI/good-egg/contents?ref=$tag" | jq -r '.[].name' || trueRepository: ueberdosis/tiptap
Length of output: 3287
Pin this action to an immutable commit
2ndSetAI/good-egg@v2.0.0 is still a mutable tag, and this workflow runs with pull-requests: write. The composite action also pulls actions/setup-python@v5 and pip install 'good-egg>=2,<3', so the supply chain is still open-ended.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/good-egg.yml around lines 11 - 13, Update the good-egg
action reference in the workflow to an immutable full-length commit SHA instead
of the mutable v2.0.0 tag, while preserving the existing github-token input.
Also pin the action’s transitive setup-python and good-egg dependency references
to immutable versions where they are controlled by this workflow.
Fixes
Changes and Review
Recently, we’ve seen an increase in AI-written pull requests. AI itself is not the problem, but a high number of PRs from the same contributors can come across less as genuine interest in the project and more as contribution farming.
For the long-term health of the project, I think we should prioritize contributors who are trustworthy, read and follow the guidelines, and take the time to understand an issue before submitting a quick AI-generated fix for something they were not assigned to.
Goodegg is a GitHub Action that scores contributors based on their contribution history. It looks at metrics such as how many pull requests they opened within a given timeframe, how many repositories those PRs were spread across, and how many were merged or closed.
I hope this action can help us quickly identifying if a user is just here to farm contributions or genuinely wants to improve the project.
Open for discussion.
Checklist
Responsibility