Ollaya runs TypeSafe Jev decision models locally, Ollama-style
Ollaya is a new open-source, self-hosted runtime for what its site calls "decision models": models that answer typed questions (choice, score or yes/no) about a block of text or JSON in a single forward pass, with no token-by-token generation. It is built to be a drop-in replacement for TypeSafe's hosted Jev API: Ollaya serves the same /v1/systemone and /v1/models endpoints with TypeSafe's request and response shapes, and the site states that the official TypeSafe Python SDK 0.7.1 works unchanged against a local Ollaya server, just by pointing its base URL, API key and default model at localhost instead of TypeSafe's cloud. A sample request shows a customer message being classified for intent (invoice, refund or other) and returned with a confidence score and per-option probabilities.
On speed, the site reports that a five-question request to its own laya model, run locally on an RTX 4090, takes 8 to 10 milliseconds end to end through the HTTP API, compared with 236 to 276 milliseconds for a median request to TypeSafe Jev's hosted API. Ollaya ships eight families of open-weight decision models that can be pulled and run locally: laya (from Convai Innovations, calibrated answers in English and over 100 other languages), decider (decoder models by Mapika on a Qwen3.5 base, whose 2b variant scores 0.591 on typed decisions), nli (zero-shot classifiers by Moritz Laurer), gliclass (an instruction-following zero-shot classifier by Knowledgator), qwen3guard (a safety guard from the Qwen team that screens text for safe, controversial or unsafe content in 119 languages), decision (fully fine-tuned Qwen3.5-backbone models from the vLLM Semantic Router contributors, trained on 16k-token rows), kev (LoRA-on-Qwen3.5 models by Jared Palmer with their own calibration), and von (a ModernBERT-large model by Victor Hugo Panisa that reads up to 8,192 tokens of context).
The pitch is privacy and cost: all inference runs locally through ONNX Runtime on CPU or an NVIDIA GPU, the server listens on 127.0.0.1 by default, and there is no metering or API bill since decisions are limited only by local hardware. Model weights are pulled directly from each model's author's Hugging Face repository, pinned to a specific commit and checked against a sha256 hash; Ollaya says it never re-hosts the weights itself, and the runtime is released under Apache-2.0. NVIDIA GPUs need driver R580 or newer, with install scripts fetching CUDA libraries only when a suitable driver is detected; on Mac, only the laya and nli models run on the Apple GPU through MLX, while other models and AMD/Intel GPUs fall back to CPU. Ollaya ships as a desktop app and command-line tool for macOS, Windows and Linux, plus a Docker image for servers, with setup reduced to a single command: "ollaya run laya".
Key facts
- Ollaya is a local, open-source runtime for typed "decision models" that answer choice, score or yes/no questions about text or JSON in a single forward pass.
- It serves an API compatible with TypeSafe's hosted Jev service; the official TypeSafe Python SDK 0.7.1 works unchanged against a local Ollaya server.
- On an RTX 4090, a five-question request through the laya model takes 8 to 10 milliseconds end to end, versus 236 to 276 milliseconds for a median request to TypeSafe Jev's hosted API.
- Eight open-weight model families are available to pull, from Convai Innovations (laya), Mapika (decider, scoring 0.591 on typed decisions), Moritz Laurer (nli), Knowledgator (gliclass), the Qwen team (qwen3guard), vLLM Semantic Router contributors (decision), Jared Palmer (kev) and Victor Hugo Panisa (von, reading up to 8,192 tokens).
- Weights are pulled from each author's Hugging Face repository, pinned to a commit and checked against sha256; the Apache-2.0 runtime never re-hosts them, and NVIDIA GPUs need driver R580 or newer for acceleration.
Why it matters
Ollaya positions itself as an Ollama-style local alternative to a hosted classification API, cutting reported latency from a few hundred milliseconds to single-digit milliseconds and removing per-token fees, while keeping the data being classified on the user's own machine.
Who it affects
Developers currently using TypeSafe's hosted Jev API for tasks like intent detection or ticket routing, and anyone who wants private, self-hosted typed-classification inference over sensitive text such as tickets, emails or user messages.
How to use it
Ollaya installs as one binary with a single command, "ollaya run laya", and is distributed as a desktop app and CLI for macOS, Windows and Linux plus a Docker image for servers, all under the Apache-2.0 licence with no per-token fees. Existing TypeSafe SDK code can point at a local server by setting its base URL, API key and default model to localhost, or callers can hit the /v1/systemone endpoint directly; the server listens on 127.0.0.1 by default.
How solid is it
The account comes from Ollaya's own site, including the headline latency figures for both the local laya model and TypeSafe Jev's hosted API; no independent benchmark methodology is given beyond the hardware used (RTX 4090, five-question request), and the site does not name a company or team behind Ollaya itself.
Risks and caveats
The 8 to 10ms versus 236 to 276ms comparison is self-reported and the two figures are described in different terms (a specific GPU test versus a general hosted-API median), so they are not a controlled benchmark. GPU acceleration is uneven: NVIDIA needs driver R580 or newer, and on Mac only the laya and nli models use the Apple GPU via MLX, with other models and AMD or Intel GPUs falling back to CPU.