codex-review

Codex Review

Automated code review for GitHub/GitLab with AI (OpenAI/Anthropic), ESLint, npm audit, and basic security checks. Runs locally or in CI, posts a summary comment and optional inline file comments on PRs/MRs.

How it works

• Clones the repository into a temporary workdir.
• Checks out a branch/commit or fetches the PR/MR head.
• Analyzes the code:
  • ESLint (JavaScript, and optionally TypeScript)
  • npm audit (dependency vulnerabilities)
  • Simple security scan (regex checks for risky patterns)
  • Best practices heuristics
• Optionally calls an AI provider with a prompt that includes recent diffs and dependencies to generate extra feedback.
• Aggregates results into a JSON object:
  • Counts by category and total
  • Issues array (lint, security, dependency)
  • AI review text (optional)
  • Best practice findings
• Posts a summary comment on the PR/MR. For GitHub, it can also post inline comments on files for issues that include file and line numbers.

Supported repositories

• Best results: JavaScript/TypeScript Node.js repositories that include a package.json.
• ESLint: Works out-of-the-box for JS. For TypeScript inline linting, install @typescript-eslint/parser and @typescript-eslint/eslint-plugin in this tool’s environment.
• npm audit: Requires npm to be in PATH. It reads package.json without needing to install dependencies.
• Non-Node repos: Security scan and AI summary still work, but dependency audit may be skipped and ESLint may find few or no files.

Requirements

• Node.js 18+
• Network access (clone repo and optionally call AI/provider APIs)
• Tokens for posting comments:
  • GitHub: GITHUB_TOKEN with Pull Requests write permission
  • GitLab: GITLAB_TOKEN with API scope
• Optional AI keys:
  • OpenAI: OPENAI_API_KEY (default model gpt-4o-mini)
  • Anthropic: ANTHROPIC_API_KEY (default model claude-3-5-sonnet-latest)

Install

npm install
# optional: create .env for defaults
cp .env.example .env

Configuration

Environment variables can be set inline or via .env.

• AI configuration
  • AI_PROVIDER: openai | anthropic | none
  • OPENAI_API_KEY, OPENAI_MODEL (default: gpt-4o-mini)
  • ANTHROPIC_API_KEY, ANTHROPIC_MODEL (default: claude-3-5-sonnet-latest)
• Providers
  • GITHUB_TOKEN: Personal Access Token or GitHub Actions GITHUB_TOKEN
  • GITLAB_TOKEN: GitLab Personal Access Token with API scope
• Workdir
  • WORKDIR: default .codex-workdir

Local usage (CLI-like)

Review a branch (prints JSON only):

node examples/review-branch.js https://github.com/owner/repo.git main

Review a PR and post comments on GitHub (summary + inline):

AI_PROVIDER=openai OPENAI_API_KEY=... GITHUB_TOKEN=... \
  node examples/post-github-comment.js https://github.com/owner/repo.git 123

Notes:

• To speed up testing, set AI_PROVIDER=none.
• Inline comments are posted for issues that include both filePath and line (primarily ESLint results). Summary comment includes all categories and AI text.

Library API

const {
  reviewRepository,
  formatResultAsMarkdown,
  postFeedbackToPlatform,
  postInlineFeedbackToGithub
} = require('codex-review');

async function run(repoUrl, prNumber) {
  const { result } = await reviewRepository({ repoUrl, refType: 'pr', ref: prNumber, ai: true });
  const summaryComment = await postFeedbackToPlatform({ repoUrl, prNumber, result });
  const inline = await postInlineFeedbackToGithub({ repoUrl, prNumber, result, maxComments: 15 });
  return { summaryComment, inline };
}

GitHub Actions (CI)

name: Codex Review
on:
  pull_request:
    types: [opened, synchronize, reopened]

permissions:
  contents: read
  pull-requests: write

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 18
      - run: npm ci
      - run: node bin/review-ci.js
        env:
          AI_PROVIDER: openai # or anthropic or none
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

GitLab CI

stages: [review]

codex_review:
  stage: review
  image: node:18
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
  script:
    - npm ci
    - node bin/review-ci.js
  variables:
    AI_PROVIDER: "openai" # or "anthropic" or "none"
    OPENAI_API_KEY: "$OPENAI_API_KEY"
    ANTHROPIC_API_KEY: "$ANTHROPIC_API_KEY"
    GITLAB_TOKEN: "$GITLAB_TOKEN"

Troubleshooting

• Only “token test” comment appears: You likely ran the curl check. Use examples/post-github-comment.js to run analysis and posting.
• No inline comments appear: ESLint may not find files or issues lack line numbers. For TypeScript, install @typescript-eslint/parser and @typescript-eslint/eslint-plugin in this tool’s environment.
• ESLint “No files matching … were found”: The repo might be pure TS and TS parser is not installed; the run will continue and post the summary.
• npm audit fails or exits non-zero: We parse its JSON even on non-zero exit; results still included.
• Forked PRs: GitHub permissions may block commenting. Ensure the token has rights on the base repo, or use pull_request_target carefully.

Limitations

• Monorepos: Only the checked-out root is analyzed. You can extend the tool to iterate packages.
• Non-JS/TS repos: Security scan and AI summary still run; lint/dependency checks may be minimal.
• Inline comments: Currently posted for ESLint results; security/dependency/AI surfaced in the summary comment.

JSON shape (simplified)

{
  "summary": { "counts": { "lint": 0, "security": 0, "dependency": 0 }, "total": 0 },
  "issues": [
    { "type": "lint", "filePath": "...", "severity": "warning", "message": "...", "line": 1 }
  ],
  "bestPractices": [ { "ruleId": "no-console", "filePath": "..." } ],
  "aiReview": "string (AI-provided JSON or text)"
}

Notes

• Requires Node.js >= 18.
• npm should be available in PATH for npm audit.
• AI review is optional; set AI_PROVIDER=none to disable.