Glonce

Dev runs TinyStories LLM on a $10 ESP32 microcontroller

infrastructureThe Glonce editorsThe Register🔥60Glonce rating60Impact55Novelty45Relevance90Surprise50Players18Credibility65
Dev runs TinyStories LLM on a $10 ESP32 microcontroller

A developer who goes by the handle SlvDev has gotten a small language model running locally on an ESP32-S3 microcontroller that costs less than $10, a chip normally used for remote sensors and other IoT tasks rather than generative AI. SlvDev documented the process on GitHub and it was showcased on the Better Stack YouTube channel; The Register, whose Systems Editor Tobias Mann wrote up the project, published its account on 4 August 2026. The ESP32-S3 has only 520 KB of SRAM and 8 MB of pseudo SRAM (PSRAM), far too little for a large model like DeepSeek V4 Flash, so SlvDev picked TinyStories instead, a 28.9 million-parameter model originally developed by Microsoft Research, about 10,000 times smaller than a typical large language model. Even TinyStories needs about 60 MB of memory at 16-bit precision, more than the chip has. SlvDev first applied quantization, cutting weight precision and shrinking the memory footprint by 75 percent, from about 60 MB down to 14.9 MB. That still did not fit, so SlvDev borrowed a technique called per-layer-embedding (PLE) from Google's Gemma model family to offload about 25 million parameters, roughly 12 MB, to the chip's flash storage, which the ESP32-S3 can carry up to 16 MB of. Because PLE accesses those offloaded weights sparingly, it avoids the severe slowdown that normally comes from parking model weights on slow storage. With PLE, the model needs only about 2 MB of active memory: the output head, embeddings and KV cache stay in PSRAM, while activations run in the 520 KB of SRAM. The setup reached 9.88 tokens a second, faster than the average person reads. TinyStories can generate short, fairly coherent stories but, per the article, cannot power a chatbot, generate code or run an agent. A separate small model called Barista answers questions about twice as fast as TinyStories, but only on espresso-related topics. For comparison, Google's Gemma 4-E2B-it, a 5.1 billion-parameter vision language model launched in April, uses the same quantization and PLE techniques to fit into just over a gigabyte of memory at 4-bit weights, or around 500 MB with heavier quantization and offloading; on hardware with more headroom than an ESP32, such as a Raspberry Pi or a smartphone, it is capable enough to power local chatbots and orchestrate local agents such as calendar management, reducing reliance on OpenAI or Anthropic, provided the user tolerates occasional hallucinations.

Key facts

  • Developer SlvDev ran a 28.9 million-parameter TinyStories model, originally from Microsoft Research, on an ESP32-S3 microcontroller costing under $10.
  • Quantization cut the model's memory footprint by 75 percent, from about 60 MB to 14.9 MB; per-layer-embedding (PLE) offload to flash then shrank active memory needs to about 2 MB.
  • The setup hit 9.88 tokens a second on a chip with just 520 KB of SRAM and 8 MB of PSRAM.
  • TinyStories can only generate short stories, not power a chatbot, write code or run an agent; a sibling model called Barista answers espresso-related questions at roughly twice the speed.
  • Google's 5.1 billion-parameter Gemma 4-E2B-it uses the same techniques to run in about a gigabyte (or around 500 MB with heavier quantization) on devices like a Raspberry Pi or smartphone, capable enough for local chatbots and agents.

Why it matters

The project is a proof of concept for how far model compression has come: quantization plus per-layer-embedding offload, techniques already used to squeeze billion-parameter models onto phones, also work on a microcontroller with just 520 KB of SRAM and no dedicated AI hardware. It shows the same toolkit that shrinks advanced models like Gemma 4-E2B-it scales down to the cheapest embedded chips available, at the cost of nearly all capability.

Who it affects

Mainly hobbyist and embedded developers exploring how small a language model can go, plus anyone tracking edge AI deployment techniques. It is not aimed at end users: TinyStories on an ESP32 has no practical application beyond demonstrating that it runs at all. Developers working with more capable edge hardware, such as a Raspberry Pi or a smartphone, are the ones who stand to actually use the same compression techniques, via a model like Gemma 4-E2B-it.

How to use it

SlvDev's process is documented on GitHub and walked through on the Better Stack YouTube channel, so it can be reproduced on similar ESP32-S3 hardware. TinyStories itself is limited to generating short, reasonably coherent stories; a related model called Barista answers questions about twice as fast but only on espresso topics. For anything resembling a usable local assistant, chatbot, code generation, or an agent that can manage a calendar, the article points instead to Gemma 4-E2B-it running on hardware with more memory, such as a Raspberry Pi or smartphone.

How solid is it

The figures come from a single developer's own writeup and video demo rather than independent benchmarking, and The Register's account relays SlvDev's reported numbers, including the 9.88 tokens a second inference speed, without an outside verification pass. The techniques described, quantization and per-layer-embedding offload, are established methods already used in Google's Gemma models, which lends the approach credibility even though this specific ESP32 implementation has not been independently reproduced or reviewed.

Risks and caveats

TinyStories at 28.9 million parameters is roughly 10,000 times smaller than a large language model and, per the article, cannot build a chatbot, generate code or power an agent, so the result should not be read as evidence that microcontrollers can run general-purpose AI. Even the far more capable Gemma 4-E2B-it, which does support local chatbots and agents on beefier edge hardware, still produces occasional hallucinations. No ESP32-S3 board vendor or specific product is named, so the sub-$10 cost is an approximate figure rather than a price tied to one listing.

“You aren't going to build a chatbot with it, generate code, or power an agent.”

— Tobias Mann, The Register