FreeToken serves 753B-parameter GLM-5.2 on one workstation GPU
Researchers have presented FreeToken, an edge-native MoE serving system built on the idea that a personal machine should be treated not as a small GPU, but as a unified, elastic inference platform. The system co-designs the full serving stack around two realities of running AI locally: agent workloads keep changing their execution pattern as they run, and edge hardware exposes a mix of resources (CPU, GPU, memory) whose balance differs from one machine to the next. That co-design spans model layout and loading, which experts stay resident where, how CPU and GPU execution is split, reuse of agentic state, and runtime memory management. Rather than committing to one fixed offloading strategy, FreeToken continuously remaps computation and model state onto whatever resources are actually available at a given moment.
The team reports that FreeToken supports more than 20 MoE models and runs real coding and tool-using agents, across hardware ranging from an 8GB laptop GPU up to a single workstation GPU. The claimed effect is a jump in what these machines can practically serve: a 35B-parameter model on a laptop, a 284B-parameter model on a gaming desktop, and the 753B-parameter GLM-5.2 on a single workstation GPU.
The team frames FreeToken as turning open weights into deployable local software, so that machines people already own become a practical platform for frontier-scale models rather than requiring datacenter infrastructure. The system has been released at flashml.ai.
Key facts
- FreeToken is an edge-native MoE serving system that treats a personal machine as a unified, elastic inference platform instead of a small GPU.
- It supports more than 20 MoE models and runs real coding and tool-using agents.
- On hardware from an 8GB laptop GPU to a single workstation GPU, it claims to raise what a machine can serve: a 35B model on a laptop, a 284B model on a gaming desktop, and the 753B-parameter GLM-5.2 on a single workstation GPU.
- Instead of a fixed offloading strategy, FreeToken continuously maps computation and model state onto whatever memory and compute the machine has available at the time.
- The system is released at flashml.ai.
Why it matters
Frontier open-weight MoE models keep getting released, but serving them has largely assumed datacenter-grade infrastructure. FreeToken is pitched as a way to close that gap by co-designing the whole serving stack, model layout, expert residency, CPU-GPU execution, agentic state reuse, and memory management, around consumer hardware, rather than porting a datacenter serving approach downward.
Who it affects
Anyone trying to run large open-weight MoE models on their own laptop, gaming desktop, or single workstation GPU instead of renting cloud GPU capacity, including developers building or using coding and tool-using agents locally.
How to use it
The system is released at flashml.ai and is described as supporting more than 20 MoE models across hardware from an 8GB laptop GPU up to a single workstation GPU. The source text does not give pricing, licensing terms, or installation steps, so none of that can be stated here.
How solid is it
This is the team's own description of the system, published on Hugging Face's papers listing, with no benchmark numbers for latency, throughput, or accuracy given in the text, and no author names or institutional affiliations stated. The capability claims, including which model sizes run on which hardware tier, come from the authors rather than from an independent evaluation.
Risks and caveats
Without published benchmarks, it is not possible to judge how fast or how accurate inference is once a 753B model is spread across a single workstation GPU through offloading and dynamic remapping; running frontier-scale MoE models on constrained edge memory typically trades off speed or context length even when it is technically possible. The source text also does not explain the underlying mechanisms for expert residency or CPU-GPU execution in enough detail to assess them independently.