jevchat builds a chatbot by asking Jev one symbol at a time
jevchat is a small open-source project that wraps a chat interface around a base model called Jev, which the source never further identifies. At every step the tool asks Jev a single question: given the user's question and the reply written so far, which symbol comes next? Jev returns a probability for every symbol in a chosen alphabet plus an option to stop. A sampler draws the next symbol from that normalised distribution, the symbol is appended to the reply, and the process repeats until the stop option is drawn.
Several alphabets are available, including truncated token lists, and the alphabet can be paired with different sampling strategies. choice asks one question covering the entire alphabet in a single request; an --ensemble option can send several re-orderings of that same question in parallel to cancel out any position bias in how Jev reads the option list. bisect instead asks a series of earlier/later yes-no questions that narrow the alphabet down to a small group, by default 20 symbols, before a final choice question picks inside that group. buckets splits the alphabet across many separate questions, each with an escape option for "none of these"; it is the only strategy that can handle an alphabet of more than 255 symbols, which rules out choice and bisect for the larger token-based alphabets. refine chains three passes: buckets first, then a question over the leading candidates, then a rescored shortlist; the author states this roughly doubles the probability mass landing on the right symbol and resolves about 19 times the vocabulary, compared to running buckets alone.
A second, separate choice is how each question is presented to Jev, independent of the alphabet and strategy above it. Under symbol presentation, the options Jev sees are the bare symbols themselves, and Jev has to mentally append each option to the reply before judging it; the project's own documentation used to instruct it explicitly to "judge grammar and spelling on the concatenation, not on the option on its own." Under hypothesis presentation, the append is done for the model: the options it is shown are the full candidate strings that would result from each choice, so Jev only has to rank already-finished text. The author calls this switch the single largest improvement in the project: on character-level alphabets it roughly triples top-1 accuracy and doubles the probability mass landing on the correct symbol, while using fewer input tokens than presenting each symbol option with its own description.
jevchat also supports a beam-search mode that keeps multiple candidate replies alive at once (-b 3 keeps three) instead of committing to one symbol per step; above a beam width of 1, temperature, top_p and top_k no longer apply, since beams are then ranked purely by probability rather than sampled from a distribution. The CLI, installed with poetry install and configured with a Jev key placed in a git-ignored .env file (accepting api_key, JEV_API_KEY or TYPESAFE_API_KEY, with .env values overriding any exported ones), offers an interactive jevchat chat mode, a one-shot jevchat ask "<question>", a jevchat alphabets listing, and a jevchat bench command that runs every mode against a fixed set of questions for comparison. The reply streams into a panel showing the live generation rate in symbols and characters per second, the milliseconds per API call, and the top few symbols Jev scored at the last step. A first Ctrl-C stops generation once the in-flight request returns, keeping the partial reply; a second Ctrl-C aborts immediately, and the ask command exits with code 130 when cancelled.
The author describes the project as a "Claude accelerated experiment": he designed the sampling algorithms and strategies, and Claude implemented them. The repository ships 158 tests, all run offline against a scripted fake client and a mocked HTTP layer, so the test suite needs no API key or network access; only the bench command actually calls the live Jev API. The author is upfront that the idea is for fun, the cost of generating text one symbol-question at a time is "somewhat impractical," and the chatbot itself, per the project's own title, is "lousy."
Key facts
- At every step jevchat asks Jev a single question, given the conversation so far, which symbol comes next from a chosen alphabet plus a stop option, then samples the next symbol from the probability distribution Jev returns.
- Four sampling strategies are available: choice (one question over the whole alphabet), bisect (binary search down to groups of 20 before a final choice), buckets (the only strategy that can handle more than 255 symbols), and refine (buckets, then a question over the winners, then a rescored shortlist).
- Switching how the question is presented, from ranking bare symbols to ranking full candidate strings (hypothesis presentation), is called the single largest improvement in the project: on character alphabets it roughly triples top-1 accuracy and doubles the probability mass on the right symbol, using fewer input tokens.
- A beam-search mode can keep multiple candidate replies alive at once (for example, 3), but above beam width 1 temperature, top_p and top_k stop applying because beams are then ranked purely by probability.
- The project has 158 offline tests using a fake client and a mocked HTTP layer, needing no API key or network; only the bench command hits the real API, and the author calls the build a Claude accelerated experiment, the cost somewhat impractical, and the results hilarious.
Why it matters
jevchat is a demonstration of pushing a ranking-style model well past its natural interface: instead of generating text directly, Jev only ever answers single questions about which symbol should come next or which of two option groups is more likely, and jevchat turns a long sequence of those narrow answers into a full chatbot reply. The most interesting result is not the chatbot itself but the presentation experiment inside it: rephrasing the same underlying question, ranking finished candidate strings instead of bare next-symbol options, is reported to roughly triple accuracy on its own, with no change to the model or the alphabet. That is a case study in how much a model's output depends on how a question is framed, separate from what the model itself can do.
Who it affects
The audience is people building on ranking or classification-style model APIs and curious how far such an interface can be stretched toward open-ended generation, plus anyone interested in sampling strategies (beam search, bisection, bucketed multiple-choice) as a general technique. It is not aimed at end users looking for a usable chatbot: the author calls the results "lousy" and "hilarious," and the cost of running it "somewhat impractical."
How to use it
Install with poetry install and place a Jev API key in a git-ignored .env file next to pyproject.toml (as api_key, or the accepted aliases JEV_API_KEY / TYPESAFE_API_KEY; .env values override any exported environment variables). poetry run jevchat opens an interactive chat, jevchat ask "<question>" answers once, jevchat alphabets lists what can be sampled from, and jevchat bench runs every mode against the same questions for comparison. Strategy (-s/--strategy: choice, bisect, buckets, refine) and alphabet (-a/--alphabet) are independently swappable, and presentation (-p/--presentation: symbol or hypothesis) is a third, separate switch. In-chat commands include /help, /alphabet [name], /temp <v>, /stop-bias <v>, /reset, /stats and /exit. A first Ctrl-C stops generation after the current request returns and keeps the partial reply; a second Ctrl-C aborts immediately, and ask exits with code 130 when cancelled. No pricing is given beyond the author's own description of the cost as "somewhat impractical."
How solid is it
The repository ships 158 tests, all run offline against a scripted fake client for the generation loop and an httpx mock transport for the HTTP layer, so the suite needs neither an API key nor network access; only the separate bench command actually calls the live Jev API. The performance claims for the hypothesis presentation (roughly triples top-1 accuracy, doubles the probability mass on the right symbol) and for the refine strategy (twice the probability on the right symbol, about 19 times the vocabulary resolved, compared to buckets alone) are the author's own stated comparisons in the project's documentation, not figures from an independent benchmark. The author frames the whole project as an exploratory, for-fun build rather than a finished or production tool.
Risks and caveats
The source never says what Jev actually is or which underlying model it wraps, and gives no concrete cost numbers beyond calling per-query generation "somewhat impractical." No example chat transcript is quoted anywhere, despite the project's own title calling the chatbot "lousy" and the text calling its results "hilarious," so the quality claims cannot be checked against an actual conversation. The implementation was written by Claude from the author's description of the algorithms, which the author discloses openly, but that means the code's correctness rests on the 158 offline tests rather than on independent review beyond them.
“judge grammar and spelling on the concatenation, not on the option on its own.”
— jevchat's documentation, on the instructions given to Jev under symbol presentation (since revised)