Off-grid AI is a hallucination risk for survival use, The Register argues

In an opinion column for The Register, Systems Editor Tobias Mann argues that small, locally run AI models marketed as off-grid survival assistants are a bad idea for anything beyond harmless roleplay. His hook is a hypothetical zombie apocalypse: a language model running on cheap hardware like a Raspberry Pi, with no internet connection, pitched as a bug-out-bag companion. Mann notes that one startup already sells an assembled off-grid AI device built on an Nvidia Jetson Orin Nano board with a battery enclosure, priced at 7 times the bare Jetson board's own MSRP, and that the pitch trades on a real technical fact: local models are remarkably information dense. The entire English text of Wikipedia compresses to about 24.7 GB, he writes, yet a four-billion-parameter model that fits in just 2 GB of memory can hold more than that, especially once vision, translation, and speech features get layered on.

Mann's objection is that density is not the same as reliability. Even the largest LLMs still hallucinate, confidently inventing information and presenting it as fact, because although they are trained on tens of trillions of words, or tokens, of data, retrieving that data accurately is imperfect: the models are predicting the statistically likeliest next word, not verifying truth. Cloud chatbots from OpenAI, Anthropic, or Google partly compensate by pulling in live web data mid-answer, effectively researching a query rather than answering from memory alone, though Mann stresses this still offers no guarantee the retrieved information is accurate. An off-grid model has neither option: no internet to fall back on, and, in his view, almost certainly none of the guardrails that major commercial chatbots build in.

He grounds the medical-advice risk in a number and a direct source. OpenAI says more than 230 million people globally ask ChatGPT health and wellness questions every week. Medical professionals at Duke University School of Medicine, Mann writes, warn that LLMs cannot read between the lines the way a clinician does and tend toward sycophancy, telling people what they want to hear. Ayman Ali, a fourth-year surgical resident at Duke Health, made the comparison directly in a blog post earlier this year: clinicians read between the lines to understand what a patient is really asking and are trained to interrogate the broader context, while large language models, in his words, 'just don't redirect people that way.'

Mann's conclusion draws a line between the fictional framing and real use. Roleplaying a zombie apocalypse with a local model is harmless, he writes, because the risk in a contrived scenario is minimal. The danger starts when someone treats the same off-grid model as a genuine field tool, for instance asking it whether the berries in a photo are safe to eat. Without live web access or a vendor's guardrails to catch a wrong answer, Mann calls that specific move 'playing Russian roulette with a probability engine.'

Key facts

Tobias Mann, Systems Editor at The Register, argues in an opinion column that off-grid AI models marketed as survival tools are riskier than useful outside harmless zombie-apocalypse roleplay.
One startup already sells an assembled off-grid AI device, an Nvidia Jetson Orin Nano board with a battery enclosure, for 7 times the bare Jetson board's own MSRP.
Local models are extremely information dense: a four-billion-parameter model that fits in 2 GB of memory holds more than the roughly 24.7 GB of compressed text that makes up all of English Wikipedia, yet LLMs still hallucinate because retrieving accurate answers from tens of trillions of training tokens is imperfect.
OpenAI says more than 230 million people globally ask ChatGPT health and wellness questions every week; medical professionals at Duke University School of Medicine say LLMs cannot read between the lines the way a clinician does and tend toward sycophancy.
Off-grid, an LLM has no live web to fall back on and, Mann writes, almost certainly lacks the guardrails of cloud chatbots, so he singles out asking a model whether pictured berries are edible as 'playing Russian roulette with a probability engine.'

Why it matters

Off-grid AI pitches are starting to sell the promise of a survival companion: a language model running locally on cheap hardware, no signal required, doubling as a guide when cell service and cloud AI are unavailable. Mann's column takes that pitch seriously enough to push back on it, because the underlying hardware trend is real: small local models are already remarkably information dense, and layering on vision, translation, and speech features makes the pitch sound complete. The catch, in his telling, is that a model built for total offline recall inherits the same hallucination problem users have experienced for roughly the past four years, but without the live web lookups and safety guardrails that cloud services use to catch bad answers.

Who it affects

People shopping for bug-out-bag gear or off-grid hiking equipment are the direct audience for this kind of pitch, including buyers of the Jetson Orin Nano based device Mann cites, sold with a battery enclosure for 7 times the bare board's own MSRP. More broadly, the piece points to a much larger population already leaning on cloud chatbots for sensitive advice: OpenAI says more than 230 million people globally ask ChatGPT health and wellness questions every week. Medical professionals at Duke University School of Medicine are already warning against that reliance, a concern Duke Health surgical resident Ayman Ali voiced directly in a blog post.

How to use it

Mann frames the concept as a small language model running on inexpensive hardware such as a Raspberry Pi, then layered with vision, translation, and text-to-speech or speech-to-text add-ons to round out a standalone assistant. As a separate, concrete example, he cites one startup that already sells an assembled off-grid AI device built on an Nvidia Jetson Orin Nano board with a battery enclosure, priced at 7 times the bare Jetson board's own MSRP. The piece names neither the startup nor any specific off-grid model, benchmark, or Raspberry Pi variant, so there is no bill of materials or setup guide here, only the outline of the product category.

How solid is it

This is a labeled OPINION column, not a report on a new study, product launch, or documented incident, and Mann builds his case from established facts rather than new data: the Wikipedia-size and model-density figures illustrate a general point about information density, not a claim about any specific off-grid device. The medical-advice section rests on one attributed statistic, OpenAI's weekly figure of more than 230 million ChatGPT health queries, and one named source, Ayman Ali of Duke Health, quoted from a blog post rather than a fresh interview. No real-world case of an off-grid model giving bad medical advice or misidentifying a plant is cited; both risks are presented as illustrative, not documented.

Risks and caveats

Mann draws a sharp line between fiction and real use. Roleplaying a zombie apocalypse with a local model carries minimal risk because the scenario itself is not real. The risk he actually flags is a genuine field question, such as asking an off-grid model whether pictured berries are safe to eat, with no internet connection to double check the answer and, in his view, none of the guardrails that OpenAI, Anthropic, or Google build into their cloud chatbots. He calls that specific move 'playing Russian roulette with a probability engine.' The piece does not name the startup behind the Jetson-based device or the demonstration video that inspired the column, and it does not identify which off-grid model, if any, actually powers either one.

“When a patient comes to us with a question, we read between the lines to understand what they're really asking. We're trained to interrogate the broader context. Large language models just don't redirect people that way.”

— Ayman Ali, fourth-year surgical resident at Duke Health