OpenReview

OpenReview is an open-source, self-hostable, repository-aware AI pull request reviewer. It uses a GitHub App, Convex for durable application state and jobs, Next.js for the dashboard, and your model credentials. The same TypeScript review engine powers GitHub and the local CLI.

Status: implemented with passing local unit/integration, production browser, web build, and self-hosted Docker checks. Live GitHub installation and model quality require credentials and separate verification. This project makes no Greptile parity claims.

Documentation is published at /docs (Markdown in docs/public/ plus selected guides), and the changelog at /changelog. Internal QA and acceptance logs live in docs/internal/.

Features

Each feature below has automated tests. The evidence level for each one (tests with fakes, verified locally, or verified live on a provider) is listed in the acceptance matrix and the parity report.

Reviews

• API guidance packs: built-in and repository packs (.openreview/guides/*.md) load when a change imports a matching SDK and add review guidance only.
• Knowledge base entries about the changed files are supplied to reviews as labelled background (never evidence) and listed in the summary's coverage.
• Repository-aware reviews. A tree-sitter code graph covers 12 languages with resolved cross-file calls and imports. Optional 1,536-dimension chunk embeddings add hybrid ranking.
• Three engines. The default is a specialist pipeline. An optional tool-using agent investigates the codebase across steps and resumes after interruptions, and an optional hypothesis engine plans concrete bug hypotheses and investigates each one in parallel with its own budget.
• Security checks: deterministic pattern checks on added lines (secrets, SQL concatenation, disabled TLS verification, dynamic evaluation, shell commands, unsafe deserialization) and OSV.dev advisories for changed npm, PyPI, Go, crates.io and Maven dependencies; feedback cannot suppress them.
• Experimental routing for agent-authored pull requests: explicit signals (agent bot authors, Co-Authored-By trailers, branch prefixes, description markers) can send them to a dedicated review model; the reason is recorded.
• A requirements check for pull requests that link Linear, Jira or Notion tickets: each acceptance criterion is quoted from the ticket and marked met, not met or unclear with code evidence; grounded "not met" items become findings and block automatic approval.
• Opt-in generated reproduction tests: for up to three verified findings OpenReview drafts a test, runs it in the isolated runtime sandbox and attaches the recorded outcome to the finding (never committed).
• Grounded findings. Each finding quotes exact lines from the reviewed revision and passes a second verification before it is posted.
• Committable suggestion blocks for small fixes, and multi-line comments within a hunk.
• One summary comment, edited in place on every push. It opens with a verdict computed from the score, findings and coverage, then has an overview, files changed, issues, a confidence score, skipped files and an optional Mermaid sequence diagram.
• Priority badges (🔴 P0 to 🔵 P3) and a 🔒 Security tag on comments, the summary, the dashboard, notifications, the CLI and the API, as text and emoji only.
• Findings outside the changed code: a defect in an untouched caller is kept when its evidence quotes the changed line that causes it, and is listed in the summary instead of inline.
• A summary footer that numbers each review on the pull request, names the provider's re-run command and can carry one rate-limited feature tip (on by default for the hosted cloud only).
• Conversations in finding threads: replies to OpenReview's inline findings are answered without an @openreview mention (on by default, rate-limited, never for bots), and findings can ask the author one clarifying question.
• Incremental re-reviews that cover only new commits, and automatic resolution of fixed findings' threads.
• GitHub check runs, used only when the app has Checks write permission. The conclusion is neutral by default, or fails at a severity you choose.
• Configurable triggers:
  • Automatic reviews, label-only reviews, or reviews on @openreview mention.
  • Excluded authors and branches, draft handling and size limits.
  • Strictness presets and opt-in comment categories.
• Data controls: workspaces are private by default (opt-in aggregate counts only), configurable retention for reviews and transcripts, and owner-only workspace deletion with a grace period and a complete, resumable purge. See docs/data-controls.md.
• Per-workspace single sign-on with SAML 2.0 or OpenID Connect: DNS-verified domains, optional just-in-time membership and an optional SSO requirement (owners keep a way in). See docs/sso.md.
• Optional auto-enable for newly installed repositories, per workspace or team, and suggested cross-repository clusters from package references, shared dependencies and repository names. Suggestions wait for an admin's approval.
• Custom rules scoped to the workspace, a team, repositories matching a name pattern (such as acme/*-service) or one repository; the most specific same-titled rule wins.
• Claude Code and Codex plugins (plugins/openreview) with the OpenReview MCP server and a resolve loop that fixes findings, commits locally, asks before pushing (never force-pushes) and re-reviews until clean; openreview resolve runs the same loop from the terminal. Install in Claude Code with /plugin install openreview --marketplace <owner>/<repo>; see agent plugins.
• A rule assistant that drafts rules from a description, tightens vague rules, flags conflicting or duplicate rules, and suggests rules from findings fixed on several pull requests and from approved conventions. Every suggestion cites its sources and waits for an admin's approval.
• Per-pull-request effort: comment @openreview review fast|standard|deep or add an openreview:<tier> label. An admin can cap the tier, and agent and hypothesis turn and tool-call budgets are configurable within operator hard caps (AGENT_MAX_TURNS, AGENT_MAX_TOOL_CALLS).
• Configuration as code in .openreview.json and cascading .openreview/ folders (per-directory categories, strictness, thresholds and ignore/include globs; the highest tier among touched directories), read from the base branch, with admin locks that a pull request cannot override.

Learning and memory

• 👍/👎 reactions and dashboard feedback become learned preferences. Reactions are polled on GitHub and arrive by webhook on GitLab.
• Learning from commits: findings that a follow-up commit fixed, and findings merged unaddressed, feed a bounded per-category preference that never narrows security review. The Analytics page shows the fixed-before-merge rate over time.
• Conventions learned from human reviewers' comments on merged pull requests. Each one cites its evidence and is applied only after an admin approves it, unless auto-approve is on.
• A Memory page where admins can inspect, edit, scope (repository, team or workspace), enable, disable and delete learned rules. Every change is audited, and nothing learned can weaken security review.

Fixes

• @openreview fix this replies with a validated suggestion. … in a new PR opens a fix pull request from an openreview/fix-* branch. This is opt-in, needs Contents write, and never writes to the PR branch.
• Fix handoff to Codex, Claude Code, Cursor Agent or Conductor: through the paired local bridge, a copied prompt, or the agents' documented "open with prompt" links. Optional Send to Devin uses your own Devin API key.

Ask your codebase

• Dashboard chat with streamed investigation steps. Every citation is checked against the indexed source.
• A public API with read, write and admin key scopes: POST /api/query (JSON or SSE), POST /api/search and repository enable/status.
• A remote MCP server at /api/mcp (Streamable HTTP, OAuth sign-in or API keys) and a local stdio MCP server, both with the same 16 tools, including triggering a review, listing your unresolved comments, review status, adding custom context for approval, and reports.
• Slack: review and high-severity notifications, @OpenReview questions in threads, and /openreview status. Email notifications through Resend.

Context and integrations

• Requirements from Linear, Jira, Confluence and Notion pages linked in the pull request, with read-only scopes chosen by an admin.

Providers

• GitHub and GitHub Enterprise, GitLab, Azure DevOps (PAT or Entra service principal), Bitbucket Cloud and Data Center, Gitea and Perforce.
• Self-managed GitLab behind corporate networks: an internal API URL, a private CA bundle and an operator-approved HTTPS proxy, following only same-host redirects. See docs/gitlab.md.

Workspaces

• Sign-in with GitHub, GitLab or Google, or by email link. Identities link only through verified email.
• Server-side sessions with revoke and sign-out-everywhere, and email invitations.
• Teams, roles and audit logs.
• Analytics trends: reviews, findings, addressed rate and estimated time saved.
• A diff viewer with inline findings.
• Optional Stripe seat billing with review quotas. It is off unless BILLING_ENABLED=true.
• An optional referral program for the hosted cloud: personal links, a Checkout discount, commissions with refund and dispute clawback, workspace credit, and optional cash payouts through Stripe Connect. It is off unless REFERRALS_ENABLED=true and billing is on. See docs/billing.md.
• Self-hosting kit: a Helm chart, AWS Terraform, an offline mode (OPENREVIEW_OFFLINE=true) that only contacts your model, embedding, code-host and SSO endpoints, and a checksummed tree-sitter grammar bundle. See docs/self-hosting-kit.md.

Command line

• openreview onboard connects a checkout to a workspace, prints the install link, enables repositories and imports rules from CLAUDE.md, AGENTS.md and similar files as suggestions for approval.
• openreview review --inline prints findings next to the code, and --resume continues an interrupted agentic review on the same commit.

Architecture

GitHub App → signed Convex HTTP webhook → durable delivery → scheduled review
                                                        ↓
                 incremental blob index → code graph → ranked context
                                                        ↓
                  specialty reviewers → evidence checks → verifier
                                                        ↓
                    stored findings/usage → GitHub batch review
                                                        ↓
                     Next.js dashboard / authenticated API / MCP

Local Git branch → CLI → the same reviewChange engine

Repository content and model output are untrusted. Findings require exact source evidence and a changed-line location before a second model verification pass. Duplicate delivery IDs and review head SHAs are persisted. Reviews record stages and retries; publishing resumes stored results after failures.

Local quickstart

Use Node.js 24 and pnpm 11.

pnpm install
CONVEX_AGENT_MODE=anonymous pnpm exec convex dev --once
pnpm setup:local
pnpm backend
# In another terminal:
pnpm dev

Open http://localhost:3000. Local setup generates independent session, service, and webhook secrets and synchronizes them to the local backend without printing their values. .env.local and .env.backend are ignored by Git. For provider/GitHub settings, edit .env.local, then run pnpm setup:local again. Backend changes must be deployed with pnpm backend or convex dev --once.

To test billing and referrals locally, point the Stripe CLI at a Stripe sandbox (stripe switch, never live mode), and run stripe listen --print-secret once for the webhook signing secret. Put the sandbox values from docs/billing.md in .env.local: STRIPE_SECRET_KEY (rk_test_…), STRIPE_WEBHOOK_SECRET, STRIPE_SEAT_PRICE_ID and BILLING_ENABLED=true, plus REFERRALS_ENABLED=true and STRIPE_REFERRAL_COUPON_ID for referrals. Then run pnpm setup:local, which copies the billing settings that have a value to the local backend, replacing the ones it had, so a new sandbox key or webhook secret takes effect. Keep pnpm stripe:listen running to forward sandbox webhooks to the backend that NEXT_PUBLIC_CONVEX_SITE_URL in .env.local names. Claude Code copies .env.local into each new worktree (.worktreeinclude), so agents working there can use the same sandbox. In a worktree created earlier, copy only the STRIPE_*, BILLING_* and REFERRAL* lines into its .env.local, because the other lines name that worktree's own backend and secrets, then run pnpm setup:local there.

GitHub App setup

Register your own GitHub App in GitHub Settings → Developer settings → GitHub Apps.

• Homepage: your APP_URL.
• User authorization callback: APP_URL/api/auth/callback.
• Setup URL: APP_URL/api/github/callback.
• Webhook URL: NEXT_PUBLIC_CONVEX_SITE_URL/github/webhook.
• Webhook secret: your GITHUB_WEBHOOK_SECRET.
• Repository permissions: Contents read, Metadata read, Pull requests read/write, Issues read, Checks read. No code-write permission is required; Contents read/write is optional and only enables fix pull requests (see below).
• Subscribe to Installation, Installation repositories, Pull request, Pull request review, Pull request review comment, Issue comment, Push, and Repository events. Installation events are provided automatically where GitHub requires them.
• Leave Request user authorization (OAuth) during installation off. With it on, GitHub skips the Setup URL and the installation is not linked to a workspace. The App's client ID/secret are still used for sign-in.
• Turn on Redirect on update, so changing an installation's repository access on GitHub returns to OpenReview and refreshes the repository list.
• Generate a private key. Convert GitHub's PKCS#1 key to PKCS#8 with openssl pkcs8 -topk8 -nocrypt -in github-key.pem -out app-key.pem.

Set GITHUB_APP_ID, GITHUB_APP_SLUG, GITHUB_APP_PRIVATE_KEY (PKCS#8, literal \n accepted), GITHUB_WEBHOOK_SECRET, GITHUB_CLIENT_ID, and GITHUB_CLIENT_SECRET. Synchronize App credentials to Convex. Keep the App private key on server infrastructure only. Installation tokens are minted as needed and never persisted.

Sign in → create workspace → install App → choose authorized repositories in GitHub → save repository selection in workspace setup → configure behavior → enable selected repositories and start indexing → open a PR. Return to /dashboard/onboarding to resume saved selection and inspect progress, skipped files, revisions or retryable failures. Existing enabled repositories remain enabled when setup selection changes. An installation started from OpenReview can take up to 30 minutes on GitHub; if the GitHub sign-in token expired meanwhile, OpenReview re-authorizes once and finishes linking. If an installation exists on GitHub but is not linked (it was installed from github.com, or took longer), use Link an existing installation on the setup, Repositories or Code providers page. GitHub webhooks need a public HTTPS backend URL; a localhost backend cannot receive public GitHub deliveries without a tunnel.

Supported follow-ups from repository collaborators with write/admin/maintain permission: @openreview explain this, @openreview suggest a fix, @openreview what code depends on this, @openreview review again, and @openreview review security. @openreview ignore this pattern in a finding thread saves an inspectable repository preference. @openreview fix this in a finding thread replies with a committable suggestion, checked against the current head before it is published, plus the Fix with your agent link. @openreview fix this in a new PR also opens a fix pull request from a new openreview/fix-* branch targeting the pull request's branch. This only happens when a workspace admin has enabled Fix pull requests for the repository and the GitHub App has Contents: Read and write. The pull request's own branch is never written, and changes from forks are skipped. Dashboard feedback/settings can reset preferences.

Follow-up answers retrieve current changed files and indexed dependencies/callers, then validate exact quoted source lines before publication. Source links pin the inspected revision. Historical reviews are unverified context; test source does not establish execution. Retrieval is bounded to 20 files and 40,000 characters, with generated/binary/secret files excluded. Invalid citations fail without posting and use the existing bounded recovery path. Model failures now record their usage as well as successful calls.

Dashboard pull request actions (Merge, Approve, Request changes, Comment, Close/Reopen, Convert to draft/Ready for review, Update branch) on GitHub.com repositories are sent with the clicking developer's own GitHub user token, so GitHub enforces their permissions and branch protection and shows them as the actor. The token comes from the App's sign-in (a silent re-authorization when the 10-minute sign-in token has lapsed), is checked against the GitHub account linked to the OpenReview user, and is never stored. A user token can do only what both the user and the App may do: Merge and Update branch need the App's Contents read/write; the rest work with the required Pull requests read/write. Each action is recorded in the workspace audit log.

Optional Checks: Read and write lets OpenReview create an "OpenReview" check run, which is in_progress while reviewing and then completed with a short summary and a dashboard link. The GitHub check run setting keeps the conclusion neutral by default, or fails the check when verified findings reach a chosen severity (and passes otherwise). Installations that only grant Checks: Read keep working and simply skip check runs. PR comments use Pull requests write; the Issues read permission is required for the issue-comment webhook subscription. See the comment endpoint permissions.

Official setup references: GitHub App permissions, GitHub App webhooks.

Models

Configure LLM_PROVIDER=openai|anthropic|openrouter|compatible, LLM_MODEL, and LLM_API_KEY. A compatible endpoint may run without a key if your server allows it. Set LLM_BASE_URL to its API base (for example https://models.example.com/v1). Responses must support JSON objects and validate against Zod schemas. Anthropic uses its native Messages API; other adapters use OpenAI-compatible chat completions. With LLM_PROVIDER=anthropic and no LLM_MODEL, reviews, verification and conversations use claude-sonnet-5-5 and summaries and knowledge use claude-haiku-5-5.

The OpenAI and Anthropic adapters request native JSON-schema output (LLM_STRUCTURED_OUTPUT=false disables it); every response is still validated with Zod and repaired with bounded retries, and an endpoint that rejects the schema falls back to prompt-only JSON once. The shared review payload is sent as a stable prompt prefix: Anthropic receives cache_control breakpoints and OpenAI's automatic prefix caching applies, and cache reads and writes are recorded per model call. Output limits default to 6000 tokens, or 16000 for Claude and OpenAI reasoning models (which receive max_completion_tokens); override them with LLM_MAX_OUTPUT_TOKENS or LLM_MAX_OUTPUT_TOKENS_<TASK>. The engine also exposes native tool calling (runTools), which powers the experimental agentic review engine: set Review engine to agentic to let a read-only, tenant-scoped tool loop follow callers and definitions across files within per-mode budgets, with every finding passing the same grounding and verification gates as the default specialist pipeline.

LLM_ALLOW_LOCAL=true explicitly permits private/local endpoints, including local HTTP. This is an operator setting; end users cannot supply arbitrary model URLs. Cost estimates come from a built-in price table of current OpenAI and Anthropic models, or from LLM_INPUT_COST_PER_MILLION and LLM_OUTPUT_COST_PER_MILLION when set; a zero estimate for an unknown model means unknown pricing, not free inference. When an inference budget is active, only models with a built-in price on the official endpoint or an operator entry in INFERENCE_PRICES may run; everything else fails closed before any request is sent.

Task overrides include LLM_MODEL_REVIEW, LLM_MODEL_VERIFY, LLM_MODEL_SUMMARY, LLM_MODEL_KNOWLEDGE, and LLM_MODEL_CONVERSATION. Review depth adjusts context budgets: Fast 8k, Standard 16k, Deep 40k estimated tokens. Two simultaneous specialist calls bound inference concurrency. Provider rate limits and transient failures receive bounded backoff. Review cancellation stops between persisted stages; an already-running HTTP model call may finish.

Code graph: TypeScript/JavaScript use the TypeScript compiler, and Python, Go, Java, Kotlin, Rust, Ruby, C#, PHP, C/C++, Swift and Elixir use tree-sitter grammars. Import and call edges resolve to repository files, and reviews and the agentic engine use a ranked two-hop neighbourhood of callers and callees. See docs/code-graph.md.

Review summary: each pull request (or GitLab merge request) has exactly one OpenReview summary comment. It is found by a hidden marker and edited in place on every run, while inline findings are posted as review comments; GitHub creates no review when there are no inline findings. Sections can be toggled per scope under Review formatting:

• overview
• a per-file table (file and one-line change description)
• a Mermaid sequence diagram, generated only when the change alters an interaction between at least two components; every arrow is backed by an exact source quote, OpenReview renders the Mermaid itself, and the diagram is omitted if validation fails
• an issues table (severity, linked location, title)
• a confidence score n/5 with a one-sentence rationale, capped by the most severe verified finding
• risk assessment
• resolved findings
• skipped files
• test source

Test execution, coverage and a visible Last reviewed commit link always appear. Gitea, Bitbucket, Perforce and Origin keep posting one summary per run.

Reaction feedback: 👍 and 👎 on OpenReview finding comments count as feedback. A person who only gives 👍 marks the finding useful, and one who only gives 👎 marks it not useful. Each person's vote is recorded once and feeds the learned-preference system that dashboard feedback already uses. Bots never count. GitHub has no reaction webhooks, so reactions are polled when a pull request closes or merges and hourly for recently reviewed open pull requests. GitLab reactions arrive through award-emoji webhooks (enable "Emoji events" on the project hook). Analytics shows reaction counts per period.

Learned team conventions:

• Ingestion: each day OpenReview reads, incrementally and with bounds, the human review comments on newly merged GitHub pull requests in each enabled repository. Bots and trivial remarks are excluded.
• Suggestion: a model call (task knowledge:conventions, counted against the inference budget) extracts recurring conventions. Similar ones are clustered and stored as suggested rules with links to the supporting comments and a frequency count.
• Approval: administrators approve, reject or disable suggestions on the repository settings page, and every decision is audited. Auto-approval can be turned on per scope.
• Use: approved conventions reach reviews only as review guidance.
• Guardrails: text that would suppress findings, approve changes, touch permissions, secrets or budgets, or looks like prompt injection is rejected.

Memory management (Dashboard → Memory → Preferences): every learned preference and convention is listed with its provenance (source review, finding, feedback, developer request, or supporting comments), plus search and status filters. Workspace admins can edit the text, set the scope (this repository, its team, or the whole workspace), enable or disable, approve or reject suggestions, and delete. Every change is audit-logged. Viewers have read-only access. A team- or workspace-scoped entry applies to reviews in every repository in that scope, and disabling it takes effect everywhere at once. The guardrails apply to edits too, and a preference can never suppress security findings.

Codebase chat (Dashboard → Chat):

• Asking: ask questions across up to five indexed repositories you can access.
• Investigation: OpenReview uses the same read-only code tools as the agentic reviewer (search, read file, definitions, references, related files), bound to each repository's indexed revision. Investigation steps stream to the page as they happen.
• Answers: an answer is shown only after every [n] claim marker maps to a citation whose quote exactly matches lines OpenReview read at that revision. Each citation links to the provider pinned to that commit.
• Threads: threads are private to the member who started them, and follow-ups keep the thread's earlier turns as unverified context. If you lose access to a repository, answers that used it are hidden.
• Limits and cost: questions are rate-limited (6 per member and 30 per workspace per minute), count against the inference budget, and are recorded as chat usage.

Notifications (Organization Settings → Notifications, see docs/notifications.md): connect a Slack workspace, then link channels for review-completed and high-severity notifications. Optionally turn on Q&A, which answers @OpenReview questions about up to five linked repositories with the same cited-answer engine as Chat. /openreview status reports a channel's links. Members can opt in to email for their own pull requests or for every repository they can access; only verified addresses receive mail. Nothing is sent until Slack or email is configured.

Requirements context from Notion: connect a Notion workspace in Integrations. When a pull request links Notion pages that were shared with OpenReview, their text is read as read-only requirements, alongside Linear issues, Jira issues and Confluence pages. See docs/notion.md. The Confluence space and Jira project pickers group resources by site, can be filtered, and support select all/none per site. They also flag selections that are no longer authorized.

Billing (cloud mode only, see docs/billing.md): when BILLING_ENABLED=true, new workspaces choose Starter or Pro (Stripe per-seat subscriptions with included usage and a 14-day trial; Pro adds deep reviews, generated tests and fix pull requests) while workspaces that are already on Free keep its monthly usage allowance, with grace for past-due payments and Organization Settings → Billing & Usage with Checkout and the Customer Portal. Self-hosted deployments leave it off and have no quotas.

Error tracking and analytics (hosted deployments only, see docs/observability.md): Sentry for errors and PostHog for page views and signed-in usage, each off unless its public key is set at build time. No session replays, cookies, request bodies or secret URL parameters are sent, and signed-in pages send page views only.

Sign-in and accounts (see docs/identity.md): GitHub, GitLab, Google or an emailed magic link. Accounts link only when both sides have the same verified email. Server-side sessions can be revoked individually or all at once. Workspace removal signs the member out and revokes their API keys and bridges for that workspace. Admins can also invite by email with single-use, expiring links.

Azure DevOps (see docs/azure-devops.md): connect an organization with a PAT or a Microsoft Entra service principal, and add service hooks secured by a shared-secret header. OpenReview computes diffs from the base and head blobs, posts findings as positioned comment threads, keeps one closed summary thread, and reports an openreview pull request status. Follow-up commands are not yet supported on Azure DevOps.

Repository configuration as code: .openreview.json, .openreview/config.json, .openreview/rules.md and nested per-directory .openreview.json files are read from the pull request's base revision. They are applied after workspace, team and repository dashboard settings, except keys an administrator locks. Invalid files produce visible warnings without blocking the review. See docs/repo-config.md.

Triggers and filters (GitHub, configurable per workspace, team or repository):

• With Automatic reviews off, reviews run only on @openreview review (or a bare @openreview) or a trigger label.
• Trigger labels start a review on pull_request.labeled and keep re-reviewing labelled pull requests.
• Excluded authors are globs such as dependabot[bot] or *-bot.
• Workspaces created through guided setup start with Re-review new revisions off and dependabot[bot] and renovate[bot] excluded; existing workspaces keep their settings. Branch globs ignore case. An explicit @openreview review also reviews drafts.
• Changed-file and changed-line limits skip oversized automatic reviews, with a visible reason; an explicit @openreview review overrides the limits.
• Optional style and syntax categories are off by default.
• Strictness presets 1–3 set the confidence and severity thresholds: 1 = 0.95/high, 2 = 0.85/medium, 3 = 0.75/low.

Other providers keep their existing automatic-review filters.

Incremental review and thread hygiene (GitHub and GitHub Enterprise): when a push builds on the last reviewed commit (checked with the compare API), only the new commits' changes are reviewed. The full pull request stays available as context, and findings must still anchor on lines the pull request changes. Force-pushes, rebases and comparisons too large to complete fall back to a full review, and the summary's Coverage section says which mode ran. When a later review verifies that an earlier finding is fixed, OpenReview replies in its own thread and resolves it with the GraphQL resolveReviewThread mutation. Threads started by people are never resolved, and retries never reply twice.

Committable suggestions: a finding can carry a small exact replacement for its own line range. OpenReview publishes it only if the range lies inside the changed lines and the original text still matches the reviewed head revision; otherwise the suggestion is dropped (recorded in diagnostics) and the finding is kept. On GitHub and GitHub Enterprise, findings whose range lies inside one diff hunk are posted as multi-line comments (start_line…line); any other range falls back to its first line. GitHub and GitHub Enterprise render a ```suggestion block when the comment covers the whole suggestion range, and GitLab renders ```suggestion:-N+M. Gitea, Bitbucket, Perforce and Origin show the finding and Fix handoff without a suggestion block, because they have no native apply-suggestion support.

Optional semantic retrieval: set EMBEDDING_MODEL with an OpenAI-compatible embedding endpoint and key (EMBEDDING_BASE_URL, EMBEDDING_API_KEY) that supports exactly 1,536 dimensions. Indexing embeds definition-level chunks (function, class or method boundaries from the code graph, with line windows as fallback). Reviews and the agentic search_code tool rank context with a hybrid of vector similarity, full-text relevance and code-graph proximity. Re-indexing backfills idempotently: unchanged files already embedded with the current model are skipped, and a model change re-embeds them. Embedding tokens and estimated cost are recorded as usage (EMBEDDING_COST_PER_MILLION, or the built-in price table). The repository settings page shows whether semantic retrieval is on. Without embeddings, full-text and graph retrieval still run.

Environment

See .env.example for the full variable list. Web and Convex must share SESSION_SECRET and SERVICE_SECRET. GitHub private key/webhook secret and model keys must exist in Convex's environment; setting them only in the web process is insufficient.

Convex local environment:

pnpm exec convex env set --from-file .env.backend

Self-hosted production environment:

CONVEX_SELF_HOSTED_URL=https://backend.example.com \
CONVEX_SELF_HOSTED_ADMIN_KEY='<admin-key>' \
pnpm exec convex env set --from-file .env.backend

Avoid printing secrets in logs, terminal transcripts, or CI output. Rotate session/service secrets together on both services.

CLI

npm install -g openreview-cli
openreview login
openreview review --agent

Signed in, openreview review runs on your workspace with its model, rules and plan, and the usage shows in the dashboard; --local (or signing out) reviews on your machine with your own model key. See CLI. From this checkout:

pnpm cli --help
pnpm cli review --base main
pnpm cli review --json
pnpm cli review --agent
pnpm cli review --security --mode deep
pnpm cli review --inline
pnpm cli review --local --engine agentic --resume
pnpm cli onboard --repositories all
pnpm cli resolve --pr 42 --fix-agent claude

To install the openreview command from this checkout, run npm link (Node.js 24); pnpm build:cli builds the openreview-cli npm package in packages/cli. Review compares committed HEAD with the merge base; commit your working changes first. Configure model credentials through the environment or .env.local in that repository. Local indexing respects ignored paths and limits eligible source to 5,000 files / 20 MB; it does not execute repository code.

Optional .openreview.json:

{"rules":[{"id":"membership","instructions":"Every organization API must validate membership.","severity":"high","category":"security","globs":["src/api/**"],"enabled":true}]}

openreview login signs in through the browser; for CI, generate an API key in dashboard Settings and run openreview login --url https://review.example.com --token <key>. openreview status shows the server and repositories. CLI configuration is saved with owner-only filesystem permissions. Local review works independently of server login.

For local agent fixes, install the selected Codex, Claude Code, or Cursor Agent CLI and sign in to it. Pair and register the checkout:

openreview bridge install
openreview bridge pair --url https://review.example.com
# Approve the displayed code in Settings → Personal agent.
openreview bridge register /absolute/path/to/repository
openreview bridge status

bridge status checks local agent availability even before pairing or while the server is unavailable. It also shows registered checkout counts when the local configuration is readable. An unpaired, unreadable, disconnected or rejected bridge retains a nonzero exit status with recovery instructions; this diagnostic command does not replace credentials or modify registered checkouts.

If dispatch says the agent started but its server status is unconfirmed, keep the terminal open and check its output before creating another Fix request. The bridge continues tracking the agent and retries launch reporting after successful completion. A reporting error does not establish agent failure or finding resolution; review the local diff and test output.

Cursor setup supports the official standalone cursor-agent CLI and falls back to the desktop cursor agent command. Keep the installed command on your terminal PATH; bridge status verifies its help command without authenticating or requesting inference.

Click Fix on an eligible finding or Fix All on the latest completed review. The registered checkout must match the repository and reviewed revision; existing uncommitted changes are preserved. Before releasing fix context, OpenReview checks the provider’s current open revision and rechecks workspace/repository authorization. A changed or closed PR, unavailable provider, revoked access, expired request or replay blocks dispatch. Re-review changed revisions before retrying. The portal shows launch, successful process exit, or failure, including nonzero exits and termination signals. A successful agent exit does not prove a correct fix or passing tests. Inspect the diff and test output, then commit and push yourself; the next review assesses resolution. Pairing credentials expire after 90 days and can be revoked in personal settings. Real macOS CLI dispatch, agent changes and regression checks have been verified for Codex, Claude Code and Cursor Agent; browser/protocol click acceptance is verified for Codex only. The remaining agent/OS acceptance matrix is still outstanding. Agents retain their native permission prompts and use their own authenticated accounts.

Without the bridge, every finding and the Fix page offer Copy fix prompt plus four links: Open in Cursor, Open in Claude Code, Open in Codex and Open in Conductor.

• Each link uses that agent's officially documented prompt deep link: cursor://anysphere.cursor-deeplink/prompt, claude-cli://open, codex://new and conductor://prompt=.
• A link only pre-fills the prompt (with the repository when the agent supports it), and nothing runs until you send it. Conductor creates nothing until you click Create.
• The Conductor link leaves out the optional local repository path and branch, because the server does not know where your checkout is.
• Prompts longer than an agent's link limit fall back to copy.

Send to Devin appears only after you save your own Devin API key under Personal agents. Devin documents no deep link, only its API. The key is encrypted on the server (CREDENTIAL_ENCRYPTION_KEY) and used only when you click the button, which starts an unlisted Devin session with the same prompt and opens it.

Workspace owners/admins set the default coding agent in Personal agents. Personal preferences inherit it until explicitly overridden; choose “Use workspace default” to reset an override. Review detail, individual findings, Fix All, and authenticated Fix links use the same effective preference. Viewers cannot pair bridges or launch fixes.

API and MCP

API keys are hashed at rest, scoped to one workspace, and revocable. Use Authorization: Bearer or_.... Each key has a scope:

• read: query, search, repositories and reviews.
• write: adds requesting reviews, feedback and enabling repositories.
• admin: adds administrator endpoints.

The key owner's role still applies, and keys created before scopes keep full access. The full reference is docs/openapi.yaml.

The Claude Code plugin bundles the remote MCP server with OAuth sign-in and the resolve-loop skills. Install it from the repository that holds this marketplace (Codex steps are in agent plugins):

/plugin install openreview --marketplace <owner>/<repo>

• POST /api/query with {question,repositoryIds,threadId?,stream?}: a codebase answer with sources that exactly match lines at the indexed revision. Returns JSON, or Server-Sent Events when stream is true. GET /api/query/:messageId fetches a result later.
• POST /api/search with {query,repositoryIds?,limit?}: hybrid full-text and semantic search, returning ranked snippets.
• POST /api/repositories with {repositoryId} (write scope): enables an installed repository and indexes it. GET /api/repositories/:id returns its index status.
• GET /api/repositories
• POST /api/p4/index — admin-scoped metadata upload for manual submitted-revision indexing; see Perforce setup and openreview p4 upload --help.
• GET /api/reviews?cursor=...
• POST /api/reviews with {repositoryId,prNumber,headSha}
• GET /api/reviews/:id
• POST /api/findings/:id/feedback with {value,comment}
• GET /api/context?repositoryId=...&query=...
• GET /api/context?repositoryId=...&path=...

Remote MCP (Streamable HTTP, no local install): point your client at https://review.example.com/api/mcp with the header Authorization: Bearer or_.... The endpoint is stateless and returns JSON responses. It rejects requests without a valid key (401 with WWW-Authenticate: Bearer). Each tool call runs through the public API, so the key's scope and workspace apply.

{"mcpServers":{"openreview":{"type":"http","url":"https://review.example.com/api/mcp","headers":{"Authorization":"Bearer or_..."}}}}

Local stdio MCP:

{"mcpServers":{"openreview":{"command":"pnpm","args":["--dir","/absolute/path/to/openreview","--silent","mcp"],"env":{"OPENREVIEW_URL":"https://review.example.com","OPENREVIEW_API_KEY":"or_..."}}}}

Both servers expose the same tools:

• Reviews and findings: get_review, get_findings, get_finding.
• Repository context: get_repository_context, search_codebase (paths and symbols), get_related_files, get_custom_rules, list_repositories.
• Codebase questions and search: ask_codebase (cited answers, as in POST /api/query) and search_code (hybrid semantic search, as in POST /api/search).
• Feedback: mark_finding_resolved (needs a write-scoped key). Keep keys in your client's secret configuration. Portable fix prompts support Codex, Claude Code, Cursor, and Fix All; they contain evidence and commit coordinates, and the dashboard's "Open in" links use only each agent's documented prompt deep link.

Self-hosting

See deployment documentation for Docker Compose, and the self-hosting kit for the Helm chart, the AWS Terraform module, offline mode (OPENREVIEW_OFFLINE=true, which limits outbound traffic to your model, embedding, code-host and SSO endpoints), the egress list and the bundled tree-sitter grammars. Both web and Convex backend run on your infrastructure. Convex's open-source backend provides persistent scheduling and file storage. Source files use Convex storage blobs; file metadata, symbols, and relationships are indexed in tables.

Local demo without GitHub

Configure the same model environment used by the CLI, then run pnpm demo --fixture cross-file. This simulates a known PR using the real review engine and writes actual results to reports/demo.json and reports/demo.md, including evidence, retrieved context, usage, and a portable fix prompt. pnpm demo --list lists the ten fixtures. No findings are seeded; zero findings remains a valid model result. The demo does not create production repository records or require a GitHub App.

Optional runtime validation

From the committed target repository, an operator can run:

openreview runtime --image 'trusted-image@sha256:<digest>' --command 'npm test' --timeout 120 > runtime.json

Use a prebuilt image with offline dependencies. Network access is disabled. The helper sends the committed checkout into bounded container tmpfs, captures commit-bound JSON evidence, and exposes no host mounts/application secrets. Save its output to runtime.json, then use openreview review --json --runtime-evidence runtime.json. The CLI evidence file is operator-supplied local context and must match the committed HEAD. Hosted POST /api/reviews and direct review-request mutations reject caller-supplied runtime evidence; hosted results must come from a paired runtime worker with an authorized exact-revision job. Logs are redacted before persistence/inference and shown in review details. Execution is explicitly operator initiated or enabled through trusted administrator runtime policy, never triggered by repository instructions.

Checks and evaluation

pnpm lint
pnpm typecheck
pnpm test
pnpm build
pnpm test:e2e # requires running local web and Convex services
pnpm eval

pnpm test runs unit and Convex integration tests. Browser checks authenticate a dedicated local fixture identity, visit onboarding/all dashboard sections, and save real rules/settings. They do not simulate successful GitHub OAuth or live inference. pnpm eval requires a real model and writes reports/evaluation.json; without credentials it exits with a blocked report, without inventing scores. pnpm eval --validate checks fixture identities, source/diff presence and expected finding locations without making model calls. The suite includes ten seeded defects and two behavior-preserving changes; clean fixtures contribute true negatives or false positives rather than missed bugs. Use --fixture=<id> for a targeted run, and --engine=agentic to evaluate the tool-using engine. Agentic results record the engine that actually ran (a fallback to the pipeline is reported as a failure), a one-line trace of tool calls, and the agent's transcript, with each tool result cut to 2,000 characters. Reports therefore contain fixture source and model output. Evaluation scores use location/keyword matching and require human adjudication before quality claims. Provider/schema failures are unscored, return a nonzero exit code, and are reported separately from missed bugs.

Troubleshooting

• Sign-in unavailable: configure GitHub client ID/secret and exact callback URL.
• Invalid installation state: sign in again and restart installation within ten minutes.
• No reviews: verify App permissions, webhook URL/secret, repository enabled status, draft behavior, branch filters, and automatic reviews.
• Model failures: ensure model env variables exist in Convex, verify endpoint/model support JSON output, and inspect persisted review errors.
• Failed indexing: inspect the repository error and retry indexing. Source-size limits are explicit; truncated GitHub trees are walked per subtree. Very large pull requests are reviewed risk-first within the mode's budget, and every skipped file is listed with its reason in the summary and review diagnostics.
• Unknown cost: set provider/model token prices.

See the implementation and verification report, architecture, database, security policy, and contributing.

For read-only issue requirements in PR reviews, see Linear connection and resource scope and Atlassian Jira/Confluence setup.