Juror open-sources a multi-model alternative to Greptile
A Show HN post introduces Juror, an open-source alternative to the commercial PR-review tool Greptile. Instead of a single model, Juror sends the diff to several frontier models in parallel, each running through its own native coding-agent harness, on the user's own GitHub Actions runner. Each model gets a private scratch directory and read-only access to a clean, detached checkout; nothing but the model API call itself leaves that runner, and there is no prebuilt repository index or SaaS backend to maintain, since the supported agent harnesses already ship repository read and search tools.
Setup is three steps that the project describes as taking about two minutes: add a workflow file at .github/workflows/juror.yml, add at least one provider API key (OpenAI, Anthropic, xAI, or Fireworks) as a GitHub secret, and open a pull request. Juror posts a sticky "Juror is reviewing..." comment immediately, then replaces it in place with the findings, a merge score, and a cost receipt once the run finishes; failed runs replace the spinner with a terminal error state instead of leaving it hanging forever. One key is enough for a working single-model review; every additional key adds another model to the jury, and any key left out is simply skipped with a note in the receipt. The same binary and code path also run locally via npx or a global npm install, so a PR can be reviewed and printed to the terminal, or posted with --post, before committing a workflow file at all.
On the project's own benchmark, a manually adjudicated set of defects seeded from a single production pull request (case #10359), Juror Fast found 4 of 6 (66.7%) P0-P2 defects at 100% precision, while Greptile found 1 of 6 (16.7%) at 50% precision. The project is explicit that this is a one-PR seed, not a statistically sufficient replacement benchmark, and provides both the corpus and a reproduction command (juror benchmark --file benchmarks/platform-10359.json) so the comparison can be checked or rerun.
Findings from the different models pass through five merge stages before anything is published: an anchor stage snaps each finding to a line the diff actually touches (findings outside the diff are kept, not dropped); a block stage groups findings by file and overlapping line window; an exact-collapse stage merges normalized identical reports for free; a similarity-and-referee stage uses a small model call to merge only findings that share the same faulty mechanism and fix; and a coverage-audit stage proves every raw finding maps to exactly one published or explicitly suppressed result, falling back to reporting everything separately if that accounting ever fails. By default, publish_mode is "all": every unique finding at or above the default severity floor of P3 is shown, and model agreement is displayed but never hides a finding. A stricter "consensus" mode instead requires the configured level of agreement; at the unanimous default, publication requires literal unanimity, with no exception for a P0 or P1 finding raised by just one model. That exception, letting a lone-model P0 or P1 finding publish if it survives an adversarial refutation pass, applies only when agreement is deliberately loosened to majority or a numeric threshold. A merge score from 1 to 5 is computed deterministically as the median of each model's self-reported confidence, capped by a penalty for confirmed P0, P1, and P2 findings, so models cannot vote away a confirmed blocker.
Cost figures are, per the project, never fabricated: each number is labeled as either provider-reported or estimated from tokens times list price, and a harness that supplies neither shows as "unknown" with the total marked as a lower bound. Juror also accounts for billing details that a flat per-token estimate would miss: GPT-5.6 Sol reprices an entire request at double the input rate above 272k tokens, and Grok 4.5 does the same above 200k tokens; prompt cache writes on GPT-5.6 and later bill at 1.25 times the uncached input rate. A sample receipt shown in the post priced one review at $0.91 across 3 models in 2 minutes 14 seconds. The project describes Juror as explicitly not an autofix bot, not a linter, and not a chat interface, only a diff reviewer that posts findings; the source names no company, funding, or pricing for the tool itself beyond these example inference costs, and does not say how the single-PR benchmark result would generalize to other repositories.
Key facts
- On the project's bundled single-PR benchmark, Juror Fast found 4/6 (66.7%) P0-P2 defects at 100% precision, versus Greptile's 1/6 (16.7%) at 50% precision; the project calls this a manually adjudicated one-PR seed, not a statistically sufficient replacement benchmark.
- Setup is three steps, about two minutes: add a GitHub Actions workflow file, add at least one provider API key (OpenAI, Anthropic, xAI, or Fireworks), and open a pull request.
- Findings from multiple models pass through five merge stages, anchor, block, exact collapse, similarity plus referee, and coverage audit, to deduplicate reports of the same defect without silently dropping any.
- Default publish_mode 'all' shows every unique finding at or above severity floor P3; the stricter 'consensus' mode requires unanimous model agreement (min_agreement: all) by default, with no exception for a single-model P0/P1 finding under that default; the refutation-based exception applies only if agreement is loosened to majority or a numeric threshold.
- Juror accounts for long-context billing cliffs itself: GPT-5.6 Sol doubles input pricing above 272k tokens and Grok 4.5 above 200k tokens, and cache writes on GPT-5.6 and later bill at 1.25x the uncached input rate.
Why it matters
Single-model PR review bots have three costs the post calls out directly: blind spots, since every model misses different bugs; duplication, since multiple reviewers often describe the same defect in different words; and opacity, since teams pay per seat or per PR without seeing what the inference actually cost. Juror's pitch is to run several models as a jury instead of one reviewer, merge their findings through a deterministic pipeline rather than a second model guessing at duplicates, and make both the merge score and the dollar cost auditable instead of hidden behind a subscription. It also drops the SaaS layer other PR bots use: no account to create, no repository index to build or keep fresh, and no code leaving the user's own GitHub Actions runner beyond the model API calls themselves.
Who it affects
Development teams already running GitHub Actions on their pull requests, and anyone currently paying per-seat or per-PR for a commercial review bot like Greptile who wants to see the actual inference cost instead. Using it requires at least one API key from OpenAI, Anthropic, xAI, or Fireworks; additional keys add more models to the jury. The source names no company, team, or individual behind the project beyond the GitHub organization juror-ai.
How to use it
Setup is three steps the project times at about two minutes: drop a workflow file at .github/workflows/juror.yml, add at least one provider key as a GitHub secret (a prefixed key such as JUROR_OPENAI_API_KEY takes priority, but an existing unprefixed OPENAI_API_KEY still works as a fallback), and open a pull request. Every Action input besides the GitHub token is optional, including preset, models, config, cost-target-usd, and whether to post at all. A local .juror.yml file is optional too; its defaults include severity_floor: P3 (every severity shown by default), max_inline_comments: 15, and a target_cost_usd_per_pr of $5.00, described as a planning target rather than an enforced cap, with actual spend always shown in the receipt. The tool can also run outside CI via npx juror-ai review --pr
How solid is it
The only performance evidence in the source is a single benchmark: on a manually adjudicated, bundled production-PR seed (case #10359), Juror Fast found 4 of 6 (66.7%) P0-P2 defects at 100% precision, against Greptile's 1 of 6 (16.7%) at 50% precision. The project states directly that this is a one-PR seed, not a statistically sufficient replacement benchmark, and publishes both the corpus and a command to reproduce it (juror benchmark --file benchmarks/platform-10359.json), alongside a stated protocol covering P0-P2 recall, overall recall, precision, duplicate rate, cost, and latency for every reviewer compared.
Risks and caveats
The headline benchmark numbers rest on one adjudicated pull request, a sample size the project itself disclaims as statistically insufficient; nothing in the source says whether the same gap would hold on other repositories or PR types. The source names no company, funding, or team behind Juror beyond the GitHub organization juror-ai, and states no price or license terms for the tool itself. Running it means managing your own provider API keys and their spend: the target_cost_usd_per_pr figure is only a planning target, not an enforced cap, and the stricter consensus publish mode requires unanimous model agreement by default, which could suppress a real defect that only one model catches; the P0/P1 refutation exception does not soften that at the unanimous default, only if agreement is loosened to majority or a numeric threshold. The provided text also cuts off mid-sentence describing how cache-read costs are handled after the first model sees a diff, so that detail is not available here.
“Not an autofix bot. Not a linter (yours is better and free). Not a chat interface. It reviews a diff and posts findings.”
— Juror project documentation