JevBench finds classifier.dev's fast tier is just Jev 1.13.0

Key facts

• classifier.dev's fast tier scores 83.6 on JevBench, higher than Jev 1.13.0's own #1 score of 74.4, but is excluded from ranking because it runs Jev directly (confirmed by classifier.dev's own pages and its API's "model": "jev-1.13.0" response).
• classifier.dev's smart tier escalates only the Jev answers under 0.7 confidence to a reasoning model, gemini-3.8-flash; JevBench never tested this tier, so it has no score.
• classifier.dev's Pro plan ($20/month for 200,000 fast classifications a day) works out to about $0.0033 per 1,000 decisions at full use and $0.033 per 1,000 at a tenth of that usage; the free tier, 20,000 classifications a day, is what JevBench's own run used and cost nothing.
• Small score gaps between classifier.dev's fast tier and Jev directly (97.3% vs 94.5% on the judge tier, 70.5% vs 74.1% on the hard tier) are attributed by classifier.dev to batching, and measured as noise on the vendor's own test sets.
• JevBench version v1.3.0 runs 534 decisions (220 hard) through an MIT-licensed harness; the highest-ranked fully open entrants are SemIf (73.1), djev (73.0), Winnow-12B Q8 (71.2) and reflex 4B (70.3).

Why it matters

JevBench's finding is a case study in what a benchmark can catch that a marketing page will not: a hosted product's headline score belonged to the model underneath it, not to anything classifier.dev built. Confirming that took nothing exotic, just reading classifier.dev's own benchmark page against its own API response and matching model versions. That is the kind of check any leaderboard-driven buying decision skips by default.

Who it affects

Anyone picking a decision-model API by benchmark score rather than by what runs underneath it, and specifically teams evaluating classifier.dev, Jev or comparable typed-classification services. It also matters to developers who want a self-hosted or EU-hosted option, since JevBench's board lists which entrants publish code or weights at all.

How to use it

Check the JevBench board itself and read the fine print on any entrant scoring near the top: an unranked honorable-mention position, as classifier.dev has here, is the signal that the score belongs to a different model. For self-hosting or GDPR-sensitive work, open entrants like SemIf, djev, Winnow-12B Q8 and reflex 4B publish code or weights and can run on infrastructure of the user's choosing, including in the EU, though JevBench is explicit that open weights and EU hosting are not compliance by themselves. classifier.dev itself is usable free without an account, or on its $20/month Pro plan for higher volume; the free tier's 20,000 fast classifications a day is what JevBench's own test used.

How solid is it

The methodology is disclosed in detail: a frozen harness, 534 decisions with 220 marked hard, a geometric mean across four equally-weighted subscores, and an MIT-licensed, publicly inspectable evaluation set. The claim that classifier.dev's fast tier is Jev is not JevBench's inference; it is classifier.dev's own stated positioning plus its API's own model identifier, both cited directly. JevBench also discloses that its authors separately run jev-router.com, a self-hosted decision-model offering, and states plainly that it receives no scoring advantage and is not itself a ranked entrant.

Risks and caveats

The price comparisons are estimates built on assumed usage levels against a published flat-rate plan, not classifier.dev's disclosed unit economics, since classifier.dev does not say how that flat rate is funded. The smart tier's escalation to a reasoning model was never run in this benchmark, so its accuracy, cost or speed are unmeasured here. And while JevBench's conflict-of-interest disclosure about jev-router.com is upfront, it does not remove the fact that the benchmark's authors compete in the same market as some of the systems it evaluates.

“The fast tier is Jev, TypeSafe's decision model”

— classifier.dev, on its own benchmark page