Glonce

TurboFieldfare runs Gemma 4 26B in 2GB of RAM on 8GB Apple Silicon Macs

toolsThe Glonce editorsHacker News🔥80Glonce rating80Impact60Novelty65Relevance95Surprise70Players25Credibility85

Andrey Mikhaylov, an iOS and Metal engineer, released TurboFieldfare, an open-source inference engine written in Swift and Metal that runs Google's instruction-tuned Gemma 4 26B-A4B model in about 2GB of RAM. The full model is 14.3GB, but TurboFieldfare never loads it whole. It keeps a 1.35GB shared core and the FP16 key-value cache resident in memory, then streams only the experts a given token actually needs from SSD. That design is what lets a 26-billion-parameter mixture-of-experts model run on Macs with as little as 8GB of RAM; the validated target machine is an 8GB M2 MacBook Air.

Unlike wrappers built on top of MLX or llama.cpp, TurboFieldfare is written specifically for this model. Installing it downloads about 15GB through bounded Hugging Face range requests from a pinned checkpoint and repacks it directly into the project's own .gturbo format, occupying about 14.3GB on disk once the manifest and file hashes validate; the installer never stages a second full copy of the checkpoint. The project ships three interfaces that share the same model directory: a native SwiftUI/AppKit Mac app, a command-line tool, and an experimental OpenAI-compatible loopback server that listens on 127.0.0.1:8080, supports streaming chat completions and function-tool declarations, but leaves the client responsible for authorizing and executing any tool call the model produces; the server carries no authentication or TLS.

Under the hood, each transformer layer has Metal compute attention and the expert router from resident weights while the CPU uses the router's top-8 expert choices against a 16-slot LFU cache per layer, issuing bounded parallel reads for cache misses. Weights use 4-bit MLX affine quantization for embeddings, attention, and both shared and routed experts, with an 8-bit router, run through custom Metal kernels for quantized GEMV, attention, mixture-of-experts routing, normalization, RoPE, and sampling. The KV cache stores FP16 values in bounded circular buffers for 25 sliding-window layers and linear storage for 5 full-attention layers, and prompt prefill runs in chunks of up to 128 tokens. Mikhaylov backs the design with 103 measured results spanning kernels, caching, I/O, prefill, and decode, published as part of the project's documentation. Requirements are strict: an Apple Silicon Mac, macOS 26 with Metal 4, Xcode 26 and Swift 6.2 or newer, and the package is arm64-only.

Current scope is text-only inference from the single pinned Gemma 4 checkpoint; the app and CLI do not execute tools themselves, and images, audio and video are not supported. Mikhaylov's stated next steps are iPhone and iPad ports and wider Mac benchmarking, including the base 16GB M4 Mac mini and other 8GB machines. The runtime's source and documentation are Apache 2.0 licensed; the model weights are downloaded separately from Hugging Face and remain governed by their own terms. The project is independent and, per its own README, is not affiliated with, sponsored by, or endorsed by Google.

Key facts

  • TurboFieldfare, a Swift and Metal runtime, runs Gemma 4 26B-A4B, a 26-billion-parameter mixture-of-experts model, in about 2GB of RAM by keeping a 1.35GB shared core and the FP16 KV cache resident and streaming needed experts from SSD per token.
  • The full model is 14.3GB on disk; the app downloads about 15GB from a pinned Hugging Face checkpoint and repacks it directly into TurboFieldfare's own .gturbo format without staging a second full copy.
  • The validated target machine is an 8GB M2 MacBook Air; the package requires macOS 26, Metal 4, Xcode 26, Swift 6.2 or newer, and is arm64-only.
  • It ships as a native Mac app, a CLI, and an experimental OpenAI-compatible loopback server on 127.0.0.1:8080 that supports function-tool declarations but requires the client to authorize and run every tool call.
  • The developer, Andrey Mikhaylov, documents 103 measured experiment results across kernels, caching, I/O, prefill, and decode; the project's source and docs are Apache 2.0 licensed and it states it is not affiliated with Google.

Why it matters

Running a 26-billion-parameter model in roughly 2GB of RAM challenges the assumption that large mixture-of-experts models need proportionally large memory. TurboFieldfare does it by never loading the full 14.3GB model, keeping only a small shared core and the KV cache resident while streaming the experts each token needs straight from SSD. That makes local, offline inference of a genuinely large model feasible on entry-level Apple Silicon hardware, including 8GB Macs, without cloud costs or a memory upgrade.

Who it affects

The project targets developers and hobbyists who want to run large models locally on Apple Silicon Macs, particularly the lower-memory 8GB models that are usually excluded from this kind of experiment. It is aimed at people comfortable building from source with Xcode and Swift, since there is no prebuilt binary distribution described. TurboFieldfare is an independent, solo project; it is explicitly not affiliated with, sponsored by, or endorsed by Google, which owns the Gemma model it runs.

How to use it

Clone the repository and run swift build -c release, then launch the Mac app, choose Download to fetch and repack the pinned Gemma 4 checkpoint (about 15GB, becoming a 14.3GB .gturbo install), choose Load Model, type a prompt, and choose Generate. The same .gturbo directory also works with a command-line interface and an experimental OpenAI-compatible loopback server on 127.0.0.1:8080 that supports streaming chat completions and function-tool declarations, though the client must authorize and execute every tool call itself since the server has no built-in authentication or TLS. TurboFieldfare's source and documentation are Apache 2.0 licensed; the Gemma model weights are downloaded separately from Hugging Face and remain governed by their own license terms.

How solid is it

The developer backs the design with 103 measured experiment results covering kernels, caching, I/O, prefill, and decode, published as part of the project's documentation, and the validated hardware target is an 8GB M2 MacBook Air. On Hacker News the project drew 707 points and 248 comments, a strong signal of community interest, though the performance figures are the author's own benchmarks rather than independently audited numbers. The author explicitly frames the measured throughput as a reference point rather than a performance ceiling, since prompt length, generated length, page-cache state, and hardware all affect results.

Risks and caveats

TurboFieldfare is text-only: it does not support images, audio, or video, and while its loopback server can pass through function-tool declarations, neither the app nor the CLI executes tools themselves. The author's own documentation warns the model can still repeat itself or produce incorrect answers, so outputs need checking. The loopback server has no authentication or TLS and is meant to stay on localhost. Because of the tight memory budget, only one TurboFieldfare process, whether the app, CLI, decode service, server, or test suite, should run at a time. The project is an independent, one-person effort, not an official Google release, and current support is limited to Apple Silicon Macs running macOS 26 and Metal 4.

“Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget.”

— Andrey Mikhaylov, creator of TurboFieldfare