awesome-quant/docs/superpowers/specs/2026-03-22-pr-review-agent-design.md

PR Review Agent — Design Spec

Date: 2026-03-22 Status: Approved

Overview

An automated GitHub Actions workflow that evaluates every new pull request against the rules in CONTRIBUTING.md. Compliant PRs are auto-merged to main. Non-compliant PRs are left untouched for manual review.

Goals

• Auto-merge PRs that fully comply with all contributing rules
• Zero false positives — only merge when every check passes
• No action on non-compliant PRs (no comments, no closing, no labels)
• Notifications via GitHub's built-in notification system

Note

: CONTRIBUTING.md states that certain PRs "will be closed" automatically. This agent deliberately deviates from that — non-compliant PRs are left open for manual review instead. The maintainer handles rejections personally.

New Files

File	Purpose
.github/workflows/pr-review.yml	Workflow triggered on PR open/update
scripts/review_pr.py	Validation script implementing all CONTRIBUTING.md rules

Component 1: GitHub Actions Workflow (pr-review.yml)

Trigger: pull_request_target events (types: opened, synchronize, reopened, edited) targeting main.

Why pull_request_target? Most contributions come from forks. The pull_request event gives a read-only GITHUB_TOKEN for fork PRs, which cannot merge. pull_request_target runs in the context of the base repo and has write access. This is safe because the validation script never executes code from the PR — it only reads the diff via the GitHub API.

Steps:

1. Checkout the base branch (not the PR branch — security measure with pull_request_target)
2. Set up Python + uv
3. Run the validation script: uv run python scripts/review_pr.py
4. On exit 0: squash-merge the PR directly (no approval step needed)
5. On non-zero exit: do nothing

Why no approval step? The default GITHUB_TOKEN cannot submit an approving review on a PR triggered by the same workflow context. Since the repo does not require branch protection approvals, the workflow merges directly.

Permissions: contents: write, pull-requests: write.

Concurrency: No special concurrency controls needed — each PR runs independently.

Component 2: Validation Script (scripts/review_pr.py)

A single Python script using PyGithub (already a project dependency) and urllib.request from the standard library (for URL reachability checks).

Input

Environment variables set by the workflow:

• GITHUB_TOKEN — GitHub Actions token
• PR_NUMBER — the pull request number
• GITHUB_REPOSITORY — owner/repo string

Validation Pipeline

All checks must pass for auto-merge. The script exits 0 only if every check passes, non-zero otherwise.

1. PR Description Check

• Reject if the PR body is empty or whitespace-only.

2. Files Changed Check

• Only README.md should be modified.
• Any other file changed → skip (exit non-zero).

3. Extract Added Lines

• From the PR diff via the GitHub API (not from a local checkout of the PR branch).
• Extract only lines added to README.md.
• Ignore blank lines, section headers (##, ###), and other non-entry lines.

4. Single Entry Check

• Exactly one new entry line must be added.
• Zero or more than one → skip.

5. Format Validation

The entry must match the regex used by parse.py:

^\s*- \[(.*)\]\((.*)\) - (.*)$

Additional checks:

• Description ends with a period (.) before the optional [GitHub] link.
• All URLs use https://.
• [GitHub](url) link, if present, uses the exact format [GitHub](https://github.com/owner/repo).

6. Duplicate Check

• Search the current README.md (from the base branch) for the project name (case-insensitive).
• Search for all URLs found in the entry.
• Query the GitHub API for open and recently closed PRs that contain the same project name or URL.
• If any match is found → skip.

7. Section Placement Check

• Verify the entry is placed under a valid ## (language) and ### (category) heading.
• The entry must not be floating outside a recognized section.

8. Activity Check

For any GitHub URL found in the entry (primary URL or [GitHub](url) link):

• Fetch the repo via GitHub API.
• archived == true → skip.
• Last push date older than 365 days from the workflow run date → skip.
• No README file in the repo → skip.

If the entry has no GitHub URL at all (e.g., a CRAN-only or PyPI-only entry without a [GitHub] link), skip auto-merge and leave for manual review. Activity cannot be verified without a GitHub repo.

9. URL Reachability

• Verify the primary URL returns a non-error HTTP status (2xx or 3xx).
• Uses urllib.request with a 10-second timeout.

Output

• Prints a structured summary to stdout (pass/fail per check) for workflow logs.
• Exit code 0 = all passed, non-zero = at least one check failed.

Component 3: Merge & Notification Flow

On All Checks Pass (exit 0)

1. The workflow squash-merges the PR to main with commit message: Add [Project Name] (#PR_NUMBER).
2. A merge comment is posted: "All checks passed. Thanks for the contribution!"
3. This triggers the existing build.yml workflow (push to main with README.md change), so the site auto-updates.

On Any Check Fails (non-zero exit)

• The workflow does nothing.
• The PR remains open for manual review.

Notifications

GitHub's built-in notifications cover all cases:

• New PR opened → notified as repo owner.
• PR auto-merged → notified via merge event.
• PR left open → visible in open PR list.

No additional notification infrastructure required.

Dependencies

• PyGithub — already in pyproject.toml
• urllib.request — standard library, no installation needed
• No new dependencies

Security Considerations

• The GITHUB_TOKEN provided by GitHub Actions has scoped permissions (only contents: write and pull-requests: write).
• Uses pull_request_target for fork write access, but the workflow checks out the base branch only — never the PR branch. The validation script reads the PR diff via the GitHub API, never executing PR code locally.
• URL reachability checks use HEAD/GET requests only via urllib.request.