Janas-LLM runs an 80B model on a 32 GB laptop by streaming experts from SSD
Janas is a family of tools for AI, written in C with no dependencies beyond the C library, designed and written by Maurizio "camauri" Cammalleri. Its first component, Janas-LLM, runs large mixture-of-experts language models on ordinary computers, with or without a GPU. The weights stay on disk, the experts each token needs are streamed from an NVMe SSD, and the ones that keep coming back stay in memory. The README's headline claim is that an 80-billion parameter model runs, and answers, on a laptop with 32 GB of RAM.
The numbers come from one laptop: Debian 13 Linux, an Intel Core Ultra 9 185H with its integrated Arc GPU, 32 GB of DDR5 and a 1 TB NVMe SSD that reads at about 5.7 GB/s. On Qwen3-Next-80B, a 48 GB model whose file is larger than the laptop's 30 GB, Janas-LLM writes 37 tok/s with drafts and 24 tok/s without. llama.cpp manages 2.8 tok/s writing on the CPU alone (2.0 with the GPU). Reading a prompt, Janas-LLM reaches 62 tok/s against 16 for llama.cpp. Janas-LLM held the most-used experts in a cache of about 19 GB and read the rest from disk. The README says Janas-LLM was faster than llama.cpp on every model compared on the same laptop, measured on 9-11 October 2026 against llama.cpp build 2b18470 of 18 September 2026 with its Vulkan backend.
There is one stated exception. The 3-bit Qwen3-30B reads a prompt slower than llama.cpp, because its Q3_K weights do not yet go to the GPU's fast prompt path, though it writes three times as fast.
The llama.cpp side of the 80B comparison needed special handling. With its usual build, llama.cpp ran out of memory as soon as the GPU took part, even with the experts left on the CPU. The author ran it with a build without repacking (GGML_CPU_REPACK=OFF) plus --no-host 1 and -lm mmap, which leave the experts in the file and read them through the page cache; those produced its 16 tok/s reading and 2.8 tok/s writing. In that row Qwen3-Coder-Next stood in as Qwen3-Next-80B.
On Windows 10 (1903 or later) and 11 on x86-64 there is a build too. On the same laptop under Windows 11 (12 October 2026) Janas-LLM wrote as fast as on Linux (Qwen3-4B 25.6 tok/s, Qwen3-30B-A3B 34.2). It read a prompt as fast with the mixture (Qwen3-30B-A3B 132 tok/s) and 1.8 times as fast with the dense model (Qwen3-4B 346 tok/s against 193), whose whole blocks go to the GPU. Separately, in the Debian 13 Linux janas-bench run, Qwen3-4B without the GPU (JANAS_GPU=0) reads a 256-token prompt at 85 tok/s and writes at 27.
The drafts are a cheap guess at the next tokens that the model then checks: the Qwen3.5 and Qwen3.6 models' and Qwen3-Next's own prediction block, or Gemma 4's small assistant model. On an English chat reply the drafts survived 74-89% of the time on the Qwen models and Gemma's two larger ones, and 63-72% on Gemma's E2B and E4B. On Italian prose Qwen3.5-2B kept about a third and gained next to nothing. Qwen3-30B-A3B and the dense Qwen3 models have no such block. The author also notes the draft column in the benchmark table was wrong until 26 September 2026, because it had measured a paragraph that repeated itself.
Beyond the chat tool, the repository includes janas-server, an HTTP API that follows OpenAI's plus llama.cpp's /infill for code completion in VSCodium and VS Code, and janas-mcp, which serves the model as tools to MCP clients such as Claude Code. In code completion the window sent at every keystroke is kept from moving, giving 1.5-2.2 s a proposal on a laptop, where a window slid by a line was read again in 17 s. The project is licensed under GNU GPL, version 3 or later, and the README states plainly: "This is an experiment, not a product."
Key facts
- Janas-LLM, a dependency-free C engine, streams the experts each token needs from an NVMe SSD and keeps frequently reused ones in memory, so an 80B mixture-of-experts model runs on a 32 GB laptop.
- On Qwen3-Next-80B the author reports 24 tok/s writing without drafts and 37 with drafts, against 2.8 tok/s for llama.cpp on the CPU alone; prompt reading is 62 tok/s against 16.
- Janas-LLM beat llama.cpp on every model compared except prompt reading on the 3-bit Qwen3-30B, where it still wrote three times as fast.
- All figures are the author's own, from one Intel Core Ultra 9 185H laptop; other CPUs, AMD, discrete GPUs and slower disks have never been tried.
- Linux x86-64 and Windows builds are available under GNU GPL v3 or later; the README calls the project an experiment, not a product.
Why it matters
Large mixture-of-experts models usually need their whole file in memory. Janas-LLM takes a different route: the weights stay on disk, only the experts each token needs are streamed from an NVMe SSD, and the ones that keep coming back stay cached. That lets a 48 GB model run on a laptop with 32 GB of RAM at a usable pace, 24 tok/s without drafts. In the same comparison llama.cpp, the usual reference, managed 2.8 tok/s writing on the 80B model, and only with a special build. The claim is notable but self-reported: it comes from the engine's own author on a single machine.
Who it affects
People with a laptop or desktop with an NVMe SSD and a moderate amount of RAM who want to try large mixture-of-experts models locally, on Linux x86-64 (Ubuntu 22.04, Debian 12 and later) or Windows 10 (1903 or later) and 11. The hardware list is narrow: the engine was written and measured on one Intel laptop, and the author says the most useful thing owners of other machines can do is send back a janas-bench report. Developers can also use it from C or FreeBASIC as a library, through MCP clients, or as an editor code-completion backend.
How to use it
On Linux x86-64, download janas-linux-x86_64.tar.gz from the project's GitHub releases, unpack it, and run ./janas-chat; it lists the models it knows, and the chosen one is downloaded and checked. On Windows, download janas-windows-x86_64.zip, unpack it, and run .\janas-chat in a terminal inside the janas folder. Models come already converted from Hugging Face; building from source uses ./build.sh with build-essential, and optional Vulkan packages for the GPU. Ctrl-C stops a reply and Ctrl-D leaves the chat. --cache
How solid is it
The evidence is the README's own benchmarks on one laptop (Intel Core Ultra 9 185H, integrated Arc GPU, 32 GB DDR5, 1 TB NVMe SSD at about 5.7 GB/s, Debian 13), with no independent verification or third-party results given. The main janas-bench table uses the mean of three rounds, not the best round, on a prompt of 256 tokens and 64 written, measured 10 October 2026. In the llama.cpp comparison, Janas-LLM's figures for the largest model are one round each, with llama.cpp at its best configuration and Janas-LLM at its own choice. The author states the two tools do not measure in exactly the same way, and that it is one machine. Both the Windows 11 and Linux numbers come from that same laptop.
Risks and caveats
The author calls it an experiment, not a product. Anything beyond the tested laptop is unknown: other CPUs (especially without AVX-VNNI, or AMD), discrete GPUs, slower disks, and more or less memory have never been tried. The 37 tok/s figure depends on drafts, which pay off unevenly: about a third kept and almost no gain for Qwen3.5-2B on Italian prose, and a tenth either way between runs on a single 64-token round. The 80B result rests on a lot of disk reading, so it depends on a fast SSD and an idle machine: with a browser and an editor open, the same model kept about three quarters of its speed and waited five times longer on the disk. The 3-bit Qwen3-30B reads prompts slower than llama.cpp for now. The 2.8 tok/s llama.cpp figure came from a special build, because the usual one ran out of memory.
“This is an experiment, not a product.”
— Janas README