Glonce

TERMy skips LLMs, matching patterns for terminal commands instead

agentsThe Glonce editorsHacker News🔥60Glonce rating60Impact50Novelty60Relevance85Surprise60Players25Credibility80
TERMy skips LLMs, matching patterns for terminal commands instead

A Hacker News user posting as gioscarab, who says he created the PJON network protocol (recently implemented in silicon by ETH Zurich through the research of Pius Sieber), spent two free months starting in early July building a terminal assistant that answers natural-language commands without any embeddings, machine learning or LLM involved. He first tried to train a generative model from scratch on a decade-old PC (an i7-4790K at 4.0GHz with 8 cores, an NVIDIA GTX 1050 Ti with 4GB VRAM, and 16GB of RAM), starting from a NanoGPT-like transformer with 100 to 200 million parameters and later adding flash attention and Mamba-style architectures. The outputs were incoherent and repetitive, and he estimated a proper training run would need at least a month of continuous compute, so he abandoned the approach. He then tried routing commands through local open-weight models via Ollama, but models such as ornith:9b, mistral:7b and cogito:14b were too slow on his 4GB of VRAM to feel responsive. His final approach, built after recalling the blockchain hype cycle and wanting to avoid a similar overreach, imposed three constraints: no embeddings, no machine learning, no LLMs. He designed a dataset format he calls NDF (NPC-Forge Dataset Format), where each JSON entry pairs example input sentences with a response, tool calls and a permission level (such as "ask" for potentially destructive commands), plus a template system that tags reusable concepts like "<||vocab_create||>" with lists of synonyms and extracts variables such as filenames through regular expressions. On top of this he wrote two classes, FlintParser and FlintNPC, in parallel Python and JavaScript implementations totaling around 1000 lines of code, running a pipeline that strips filler words, runs sentiment analysis, then tries exact matching, template matching and finally probabilistic matching using inverse document frequency, bag-of-words and IDF-weighted Levenshtein distance for typo tolerance. He named the resulting assistant TERMy and the underlying framework NPC-Forge, which also includes an OpenAI-compatible API server, NPC management and diagnostics tooling, and says it can run on constrained hardware such as a Raspberry Pi Zero. He has connected TERMy to Copilot to handle a subset of prompts he previously sent to a large model like Claude, and argues that AI harnesses such as Copilot and Pi should default to deterministic agents like TERMy and fall back to a full LLM only when necessary, calling it wasteful and irresponsible to spend compute and electricity on trivial requests. He describes the release as an early, first Python project that is not yet production-ready, expects the chain of commands TERMy sometimes emits to draw criticism, and says more work is needed on the dataset and code before it is safe for general use.

Key facts

  • Gioscarab, who says he created the PJON protocol later implemented in silicon by ETH Zurich via Pius Sieber's research, built TERMy and NPC-Forge over two free months starting in early July.
  • He first tried training a from-scratch transformer (100 to 200 million parameters, NanoGPT-like, later with flash attention and Mamba) on a decade-old PC with a GTX 1050 Ti (4GB VRAM), 16GB RAM and an i7-4790K, but outputs looped and rambled, and a full run would have needed at least a month of continuous training.
  • A follow-up attempt routing commands through local Ollama models (ornith:9b, mistral:7b, cogito:14b) was too slow on 4GB of VRAM for general use.
  • The final design uses no embeddings, machine learning or LLMs: a rule-based pipeline of noise stripping, sentiment analysis, exact match, template match and IDF-weighted probabilistic match, implemented in about 1000 lines of Python and JavaScript code.
  • The author connected TERMy to Copilot to offload some prompts previously sent to Claude, arguing AI harnesses should default to deterministic agents and treat a full LLM as a last resort.

Why it matters

The project is a direct reaction to rising subscription costs for AI coding assistants: the author says Copilot's price hikes pushed him to stop paying for a large model to handle trivial commands like activating a virtual environment. It argues, by example, that a fixed, rule-based NLU pipeline can replace an LLM for a narrow but common class of requests, and that harnesses should route only the harder cases to a heavy model.

Who it affects

Developers who currently pay for AI copilots to run simple, repetitive terminal commands, and anyone building an agent harness that has to decide when a request needs a full LLM versus a cheaper deterministic path. It also targets a hobbyist and open-source audience: the author frames NPC-Forge as a way for anyone to build and share small conversational agents.

How to use it

NPC-Forge is released as an open-source framework combining an OpenAI-compatible API server, NPC management and diagnostics tooling, with parallel Python and JavaScript implementations so the same assistant can run locally or client-side in a browser or Node.js. New capabilities are added by dropping a dataset JSON file, following the NDF format, into a dataset directory; the author says the resulting NPCs can run on constrained hardware such as a Raspberry Pi Zero. No release date, version number or license terms are given.

How solid is it

This is a single author's account of his own project, with no independent benchmarks: the only performance claim is that TERMy responds "instantaneously" compared with waiting on a local LLM, without latency or accuracy figures. The author calls it his first Python project and says it is not yet production-ready, so the design has not been tested or reviewed beyond his own use.

Risks and caveats

The author expects the chain of commands TERMy sometimes emits to draw criticism from experienced Linux users, and says a lot of work remains on the dataset and source code before it reaches production-grade safety. He notes that requiring explicit "ask" permission for potentially destructive commands makes the tool inherently safer, but acknowledges that bugs in the implementation or in a dataset could still cause harm.

“I believe that harnesses like Copilot and Pi should rely on deterministic NPCs like TERMy and route to a heavy LLM only as a last resort.”

— gioscarab, creator of TERMy and NPC-Forge