OpenAI is well positioned to fast-follow TypeSafe's Jev

TypeSafe's product Jev is a new kind of LLM-based classifier that, according to Vercel, was adopted faster than any other model in the history of its AI Gateway. In a blog post, John Berryman, who previously worked at GitHub on Copilot, argues that OpenAI is well placed to copy Jev and then go further by building the same capability directly into its own models and agents. His central claim is that OpenAI has effectively been running LLMs as implicit classifiers for years without packaging that as a product. He traces this back to tool calling: internally, a chat session is organized with ChatML delimiter tokens, and the very first token an OpenAI assistant model predicts after a message start is either a newline, which continues a normal reply, or a special sequence that signals a tool call. That one token choice is itself a binary classifier, and every following token, from which tool to call to when the response is finished, works the same way. Berryman illustrates the fragility of this token-level control with an anecdote from his time on Copilot, where an early GPT-4 API build could not stop itself from generating without special end-of-message tokens and kept appending closing pleasantries until it hit the length limit. Berryman assumes Jev itself is close to a conventional LLM, citing a report from Latent Space that many early Jev clones are LLM-based, and that Jev works by reading the model's token probabilities: for a yes/no question it compares only the true and false tokens and normalizes them into a single probability, and for multiple-choice questions it compares the probabilities of the candidate answer tokens. He argues TypeSafe's real moat, if it has one, is not the architecture but the training data and calibration process behind Jev. He quotes TypeSafe cofounder Diogo Almeida describing the kind of dataset he would build: large numbers of examples with a known real-world outcome, such as support tickets and how they were actually routed, resumes and whether the candidate was actually hired, product reviews and their actual star ratings, moderation queues and their actual verdicts, and prediction markets and how they actually resolved, each paired with a question whose true answer is already known, so the model generalizes classification across very different domains. Berryman is less certain about the reinforcement-learning side of Jev's training, saying only that he does not know what it involves. He proposes that instead of merely cloning Jev as a separate model type, OpenAI could fold this capability into a conventional LLM by introducing a new tag, which he calls , that a model could insert into its own reasoning whenever it needs a quick calibrated judgment. In his hypothetical example, a model reasoning about whether someone likes the user based on the comment "nice haircut" would emit a claim and a probability, such as 0.04, entirely inside its own chain of thought, without handing control to an external tool the way ordinary tool calls do. Mechanically, he describes this as reading the logits for just the true and false tokens at that one decoding position, normalizing them into a probability, and writing that number back into the text as if the model had generated it normally, the same kind of guided decoding that constrained-output libraries already use for grammar. He suggests a mixture-of-experts model could plausibly dedicate a fine-tuned expert to this calibrated snap-judgment skill without disturbing the rest of the model's normal text generation. Berryman says he is rooting for TypeSafe, but also notes he has already found domains where Jev's own probabilities do not hold up, and he stresses that OpenAI has not announced anything: the entire scenario of OpenAI copying or absorbing Jev is his own speculation about the company's next move, not a reported plan.

Key facts

• TypeSafe's Jev was, according to Vercel, adopted faster than any other model in the history of its AI Gateway.
• Blogger John Berryman, a former GitHub Copilot engineer, argues OpenAI has used LLMs as implicit classifiers since introducing tool calling, where a single token decides whether and which tool to invoke.
• He says Jev's likely moat is not its architecture but its training data and calibration process, quoting TypeSafe cofounder Diogo Almeida on building datasets from support tickets, hiring outcomes, product ratings, moderation verdicts and prediction markets with known real outcomes.
• He proposes OpenAI could embed Jev-style judgments directly into a model via a hypothetical tag that reads the logits for just the true and false tokens at one decoding step and writes back a probability, without an external tool-call handoff.
• Berryman states plainly that OpenAI has made no announcement: the whole scenario is his own speculation, and he has already found domains where Jev's probabilities do not hold up.

Why it matters

If Berryman's thesis holds, the interesting story is not whether OpenAI can clone a fast-adopted product but whether it needs to: he argues the underlying skill, treating single tokens as calibrated classifiers, already exists inside OpenAI's models via tool calling. Folding a general classification capability into a model directly, rather than shipping it as a separate product like Jev, could make routing, guardrails and quick judgments cheaper and faster because they would happen inside the same forward pass instead of a separate tool round trip.

Who it affects

TypeSafe and Jev, whose competitive position depends on whether their training data and calibration process are hard to reproduce. OpenAI, which the author argues is structurally positioned to absorb this capability. Developers building agents or classification pipelines on top of LLM-based judgments, who would gain a built-in alternative to a standalone classifier product if OpenAI or others ship one.

How to use it

There is nothing to adopt yet: no OpenAI feature exists, and Jev itself is a third-party product this piece analyzes rather than sells. Berryman's own contribution is illustrative rather than production advice: a hypothetical tag and worked example showing how a model could read the logits for two tokens at a chosen decoding position and write a normalized probability back into its own generated text, the same guided-decoding trick constrained-output libraries already use for grammar.

How solid is it

This is one engineer's technical analysis and speculation, not a report of anything OpenAI has said or done. Berryman backs his tool-calling claims with his own earlier writing and a firsthand anecdote from working on GitHub Copilot with an early GPT-4 API, and he cites Vercel for Jev's adoption claim and Latent Space for early clones being LLM-based. The forward-looking part, that OpenAI will copy or absorb Jev, rests entirely on his own reasoning about incentives and capability, not on any statement from OpenAI or TypeSafe.

Risks and caveats

No benchmark, accuracy figure or launch date for Jev is given anywhere in the source. OpenAI has made no public statement about copying or integrating Jev-style classification, and the entire discussion of OpenAI's next move is explicitly the author's own guess. The author also says he has already found domains where Jev's stated probabilities do not hold up, and he admits he does not know what TypeSafe's reinforcement-learning training actually involves.

“Jev was adopted faster than any other model in AI Gateway history.”

— Vercel