Glonce

Laya launches as an open, 6-8x faster rival to TypeSafe's Jev

modelsThe Glonce editorsHacker News🔥78Glonce rating78Impact70Novelty45Relevance100Surprise50Players40Credibility80

The post's author says he started working on non-autoregressive decision models in March 2025: instead of generating text, the architecture outputs calibrated probabilities over structured schemas in a single forward pass. He backed the work with two arXiv papers (arXiv:2503.23303 in March 2025 and arXiv:2510.01237, published in September 2025, which formalized the framework for schema-based decisions guided by reinforcement learning), open model weights and a dataset on Hugging Face, a PyPI package, and a Reddit writeup. He describes reinforcement learning, specifically PPO over sequence representations producing turn-by-turn conversion probabilities in sales conversations, as the core of the approach from the start.

In September 2026, TypeSafe AI, which the post says was founded by Diogo Almeida, a co-inventor of ChatGPT at OpenAI, launched a product called Jev built on what it calls RLCD (Reinforcement Learning for Calibrated Decisions). Jev outputs confidence distributions and schema choices in parallel rather than turn by turn. According to the post, Jev shipped without technical papers, open weights, or open training datasets, priced at $0.042 per million input tokens with typical response times around 150 ms; the author attributes Jev's benchmark numbers to third-party studies (credited as AbdelStark and nibzard) and to TypeSafe AI itself, not to independent verification of his own claims.

The author's response is Laya, a fully open-source (Apache 2.0) family of bidirectional-encoder decision models. He reports 32.8 ms latency for a single forward pass on one GPU, dropping to 7.2 ms per question when batched, which he calls 6 to 8 times faster than Jev. Laya supports three typed primitives: choice (pick one option from a set, with a probability distribution over all options), score (place an input on an ordinal rubric), and noul (a calibrated true/false probability). Because the output is always a probability or a number, the author argues the model cannot hallucinate or produce malformed output. Laya ships as three checkpoints, an English model, a multilingual model covering over 100 languages, and a typed-decisions model, bundled in one Hugging Face repository so users can download only the subfolder they need (about 808 MB for English versus roughly 647 MB for the multilingual checkpoint alone, against a combined 2.5 GB for all three).

A large part of the post is a routing argument. In a 51-language sweep on the MASSIVE benchmark (20 options, random baseline 0.050), the English checkpoint fails outside Latin script while staying confident: 0.000 accuracy at 0.952 mean confidence on Khmer, 0.050 accuracy (exact coin flip) at 0.885 confidence on Armenian, 0.060 at 0.964 on Hebrew, 0.080 at 0.945 on Bengali, and 0.100 at 0.941 on Hindi. Across all 51 languages the English checkpoint's mean confidence never drops below 0.885 regardless of whether its actual accuracy is 82% or 0%, which the author uses to argue that a model's own confidence score cannot be trusted to flag a script mismatch; the choice of checkpoint has to be made before the forward pass. Laya ships a built-in router that inspects Unicode scripts across 22 alphabets before dispatching to the right checkpoint, adding 0.09 ms overhead for English text, 0.54 ms for Devanagari or other Indic scripts, and 0.73 ms for large 200-row nested JSON, against a roughly 33 ms forward pass; preloading all checkpoints removes a 7 to 10 second cold-swap penalty when traffic switches languages.

On head-to-head benchmarks the author frames as measured for Laya and sourced from third parties or TypeSafe AI for Jev, Laya scores 0.993 accuracy on Enron email spam filtering, 0.980 on phishing detection, 0.755 to 0.762 on held-out ToxicChat jailbreak detection (rising to 0.931 at 50% selective coverage), 0.657 on RAG passage relevance filtering, and 0.522 on 10-way support ticket routing. The author is explicit about where Laya loses: on a Banking77 stress test with 77 choice labels, Laya scores 0.425 against Jev's 0.870, which he attributes to a fixed 192-256 token budget shared across all candidate options, leaving only about 3 to 4 tokens per candidate at that many choices; his stated fix is to keep choice schemas under 20 options or use a coarse-to-fine two-step hierarchy. He also notes Laya's zero-shot score on the typed-decisions benchmark is about 0.35, near random, and reaches 0.766 only after fine-tuning on that benchmark's training split, and that fitting a single scalar temperature per question type cuts expected calibration error from 0.466 to 0.081. The post closes by describing Laya as installable via pip (laya>=0.3.3), with code, a router, benchmark harnesses, a live demo space, and a Kaggle notebook for fine-tuning a custom model on free GPUs in about 4 hours.

Key facts

  • The author says he built a non-autoregressive, RL-guided decision-model architecture starting in March 2025, publishing two arXiv papers (2503.23303 and 2510.01237), open weights, a dataset, and a PyPI package before TypeSafe AI's Jev launched in September 2026.
  • Jev, from TypeSafe AI (founded by Diogo Almeida, a co-inventor of ChatGPT at OpenAI per the post), shipped closed: no papers, no open weights, no open datasets, priced at $0.042 per million input tokens with about 150 ms response time.
  • Laya, the author's open-source (Apache 2.0) answer, runs in 32.8 ms on a single GPU (7.2 ms per question batched), which the post calls 6 to 8 times faster than Jev, and supports over 100 languages.
  • On a 51-language MASSIVE benchmark sweep, the English checkpoint's confidence never drops below 0.885 even when its accuracy on non-Latin scripts falls to 0% to 10% (for example 0.000 accuracy at 0.952 confidence on Khmer), which is why Laya routes by detected script before running inference.
  • Laya trails Jev on Banking77's 77-option choice task (0.425 versus 0.870) because of a fixed per-schema token budget, and its zero-shot score on the typed-decisions benchmark (about 0.35) only reaches 0.766 after fine-tuning.

Why it matters

The post argues most production AI pipelines waste a generative LLM on decisions that only need a label, a score, or a probability, such as routing a support ticket or flagging phishing. It frames this as a System 1 versus System 2 problem: reflex-speed structured decisions do not need an autoregressive model that streams tokens for 500 ms to 2 seconds and then has to be parsed back into a label. The post is also a public priority dispute: the author says he shipped the same non-autoregressive, RL-guided idea with open papers and weights over a year before TypeSafe AI's closed Jev launched under the same premise.

Who it affects

Teams building high-volume classification, routing, guardrail, or triage systems, the kind of workload the post lists directly (department routing for support tickets, phishing versus spam detection, prompt-injection screening, urgency scoring, RAG passage filtering) where a generative LLM call is currently doing a job the author argues a sub-35ms model could do instead.

How to use it

Laya installs via pip (laya>=0.3.3) and is Apache 2.0 licensed with no API subscription cost. Three checkpoints, English, multilingual (100+ languages), and typed-decisions, live in one Hugging Face repository (convaiinnovations/laya); Hugging Face's allow_patterns lets the SDK download only the needed subfolder rather than the full 2.5 GB bundle. A built-in Router detects the input's Unicode script and dispatches to the right checkpoint automatically, and can preload all checkpoints into memory to avoid a multi-second cold-swap delay when traffic alternates languages. The three typed primitives, choice, score, and noul, cover picking one option, placing input on an ordinal scale, and answering a calibrated true/false question. A live demo space and a Kaggle notebook for fine-tuning a custom model in about 4 hours are also linked from the post.

How solid is it

This is a self-published post, not an independent review: the author states plainly that every Laya number in the head-to-head comparison is his own measurement, while Jev's numbers come from third-party studies (credited as AbdelStark and nibzard) and from TypeSafe AI's own materials, not from the author's independent testing of Jev. The priority claim, that the author's March 2025 work predates and matches TypeSafe AI's September 2026 Jev, rests entirely on his own account; the post contains no response or comment from TypeSafe AI or Diogo Almeida.

Risks and caveats

The author is upfront about where Laya falls short: on a 77-option choice stress test (Banking77) it scores 0.425 against Jev's reported 0.870, because the model's token budget for candidate options is fixed regardless of how many there are, so accuracy degrades past roughly 20 choices. Out of the box, before fine-tuning, Laya's score on the typed-decisions benchmark is only about 0.35, near random, and its base weights ship with uncalibrated temperature logits (expected calibration error of 0.466) until a user fits a temperature scalar per question type. The confidence-versus-script finding cuts both ways: an unrouted English checkpoint reports high confidence even when it is reading a script it cannot handle, so the model's own output cannot be trusted as a safety check on its own applicability.

“When an LLM outputs "confidence: 0.95," it is just predicting tokens that sound confident. There is zero mathematical calibration behind it.”

— the post's author, announcing Laya