Glonce

"Persistent State Machines" framework claims sub-1mW LLM attention on FPGA

researchThe Glonce editorsHacker News🔥54Glonce rating54Impact62Novelty68Relevance87Surprise32Players27Credibility78

A Zenodo record, now at Version 8.0, describes "Persistent State Machines" (PSMs), a formal discrete framework that reworks the attention operator in large language models. Instead of the usual matrix arithmetic, computation is broadcast to stationary in-memory cells that each run local, deterministic state transitions. The record presents complete mathematical proofs for quantization error bounds, a multi-phase discrete Softmax construction under an explicit bounded-logits assumption, equivalence to a deterministic finite automaton with spatial factorization, and membership in the complexity class DSPACE(O(n)).

The authors back the theory with two FPGA evaluation flows. In a Low-Power Evaluation on a Zynq-7000 xc7z020 chip, a full 1024-cell array (with dimension d=128) was implemented as an out-of-context block design. By annotating the post-route netlist with a Switching Activity Interchange Format file taken from a post-route functional simulation, they show that self-activation gating at the hardware level keeps active switching confined to sparse cells; the estimated dynamic power of the core logic comes in below 1.0 mW, for a normalized dynamic energy of 3.81 x 10^-5 picojoules per operation. In a second, System-on-Chip evaluation on a larger UltraScale+ xcvu9p device, a 256-cell sub-array representing one integrated attention head was wired into a full SoC with AMBA AXI4 interconnects and an AMD Xilinx PCIe Gen3 x1 Bridge (XDMA v4.2), run under the AWS Cloud FPGA Developer environment. The design closed timing at a unified system clock of 62.5 MHz, with a Worst Negative Slack of +1.854 nanoseconds, while occupying only 0.67% of the device's logic slices and 0.00% of its DSP blocks.

Functional simulation with more than one thousand random vectors showed bit-exact agreement between the hardware design and a fixed-point software reference. The authors are explicit about the limits of what was measured: every energy and power figure is a simulation-based tool estimate for the synthesised logic, no physical FPGA board was measured, and system-level external memory energy is excluded entirely. The record also notes a related Japanese patent application, No. 2026-177318, filed and pending. The abstract does not name individual authors or an affiliated institution, and no peer-reviewed publication venue is given beyond the Zenodo listing itself.

Key facts

  • "Persistent State Machines" replace matrix-multiplication attention with stationary in-memory cells running local deterministic state transitions, with proofs covering quantization error bounds, a discrete Softmax construction, finite-automaton equivalence and DSPACE(O(n)) membership
  • On a Zynq-7000 xc7z020, a 1024-cell array (d=128) is estimated at under 1.0 mW dynamic power, or 3.81 x 10^-5 pJ per operation
  • On a larger UltraScale+ xcvu9p SoC, a 256-cell single-head sub-array with AXI4 and a PCIe Gen3 x1 bridge closed timing at 62.5 MHz (WNS +1.854 ns) using 0.67% of logic slices and 0.00% of DSP blocks
  • Over 1,000 random-vector simulations showed bit-exact agreement with a fixed-point software reference; all power and energy numbers are tool estimates, with no physical board measurement performed
  • A related Japanese patent application, No. 2026-177318, is pending; the record names no individual authors or institution

Why it matters

Attention is the most expensive part of running large language models, and almost every accelerator today attacks it with more or faster matrix multiplication. This record proposes a structurally different route: represent attention as a network of stationary memory cells that only switch state when their local inputs change, so idle cells draw effectively no dynamic power. If the approach holds up outside simulation, it points at a class of attention hardware built around sparsity and gating rather than raw throughput, which is a genuinely different design axis from the GPU and TPU lineage most inference hardware follows.

Who it affects

The material is aimed at hardware and ML-systems researchers working on custom accelerators for transformer inference, not at application developers or end users. Nothing here is a shipping product, an API or a chip anyone can buy; it is a formal architecture and synthesis result meant to be read, checked and potentially built on by people who design silicon or FPGA logic for AI workloads.

How to use it

There is no product, service or license to adopt. The output is a Zenodo record documenting the mathematics and two FPGA implementations, alongside a filed Japanese patent application (No. 2026-177318, patent pending). Anyone wanting to build on it would need to work from the published proofs and synthesis results themselves, and the patent filing signals the authors intend to control commercial use of the design.

How solid is it

The record includes real hardware synthesis on two distinct FPGA families with concrete, checkable numbers: post-route power estimates, timing closure with a stated Worst Negative Slack, resource utilisation percentages, and bit-exact validation against a fixed-point reference across more than a thousand random vectors. That is more rigor than a pure simulation claim. Set against that: the abstract names no individual authors or institution, there is no stated peer-review venue beyond a Zenodo posting now at "Version 8.0", and there is no comparison against existing attention-hardware baselines on GPUs, TPUs or other accelerators, so the claimed efficiency gain cannot yet be judged relative to the state of the art.

Risks and caveats

The authors state their own central caveat plainly: every power and energy figure is a simulation-based tool estimate for the synthesised logic, no physical FPGA board was ever measured, and energy spent on external memory access at the system level is excluded from the numbers entirely. Real silicon or board-level measurement frequently diverges from post-route simulation estimates, and excluding memory energy leaves out a cost that often dominates in practice for large models. Repeated version revisions (the record is at "Version 8.0") and the absence of named authors, institutional affiliation or a peer-reviewed venue also mean the work has not yet been independently vetted.

“All energy figures are simulation-based tool estimates for the synthesised logic; no physical FPGA board measurement was performed, and system-level external memory energy is strictly excluded.”

— Persistent State Machines record (Version 8.0)