Glonce

Reviser generates text via cursor actions, not left to right

researchThe Glonce editorsarXiv🔥69Glonce rating69Impact60Novelty75Relevance95Surprise45Players20Credibility80

A new paper proposes Reviser, a decoder-only Transformer that generates a response as a sequence of cursor-relative actions on a mutable canvas rather than producing tokens in a fixed left-to-right order. At each step the model predicts exactly one action token: INSERT(token), which writes a token at the cursor; MOVE(delta), which shifts the cursor by some offset; or STOP, which ends generation. Crucially, the model is autoregressive over this edit-history of actions, not over the final text order, so it can insert or revise earlier content as it goes. The authors say this gives genuinely non-monotonic generation, meaning the text does not have to be built strictly from start to end, while keeping the same simple next-token-style interface as ordinary autoregressive decoding.

The authors frame this against existing revision-capable methods, both non-autoregressive and edit-based, which they say typically get their flexibility by re-running computation over the whole sequence at each revision step. Reviser instead folds revision directly into the token-by-token action stream.

On a continuation benchmark, the authors ran arena-style evaluations and report that Reviser is strongly preferred to two diffusion-style text generation baselines, SEDD and MDLM, though no numeric win rate or margin is given in the source. Trajectory statistics from these runs show frequent backward cursor moves and mid-canvas insertions, which the authors use as evidence that the model is genuinely revising rather than just imitating ordinary end-append decoding through a more complicated interface.

Against size-matched ordinary autoregressive baselines, the authors report that Reviser is competitive at both the 100M and 300M parameter scales. Under what they call their shared FLOPs convention, a way of counting compute so different methods can be compared fairly, they also report that Reviser needs substantially less inference compute than representative multi-pass refinement and diffusion-style baselines, again without giving specific FLOPs figures or a percentage saving.

Key facts

  • Reviser is a decoder-only Transformer that generates text as cursor-relative actions, INSERT(token), MOVE(delta), or STOP, on a mutable canvas rather than as tokens in fixed left-to-right order.
  • It is autoregressive over the history of edit actions rather than over final text order, which the authors say enables genuinely non-monotonic generation.
  • In arena-style evaluations on a continuation benchmark, Reviser is reported as strongly preferred to the diffusion-style baselines SEDD and MDLM, with no numeric win rate given.
  • Trajectory statistics show frequent backward moves and mid-canvas insertions, which the authors take as evidence the model is really revising rather than mimicking end-append decoding.
  • Reviser is competitive with size-matched ordinary autoregressive baselines at both 100M and 300M parameter scales, and needs substantially less inference compute under the authors' shared FLOPs convention than multi-pass refinement and diffusion-style baselines, though no specific compute figures are given.

Why it matters

Most revision-capable text generators, whether non-autoregressive or edit-based, get their ability to insert or fix earlier content by re-running computation over the whole sequence at each revision step. Reviser instead builds revision into the generation interface itself: the model still predicts one token at a time, but that token is a cursor action rather than a piece of final text, so inserting, moving and stopping are native moves rather than an expensive extra pass bolted onto ordinary decoding.

Who it affects

The audience here is researchers working on language model architectures and generation methods, particularly anyone comparing autoregressive decoding against diffusion-style or edit-based alternatives. The source gives no author names, institutional affiliations, code, model weights or release plans, so there is nothing yet to point developers or end users toward.

How to use it

As described, using Reviser means running inference action by action: at each step the model outputs INSERT, MOVE or STOP, and the canvas is updated accordingly until STOP ends the sequence. This is the same shape as ordinary next-token decoding, just over actions instead of final words, which the authors present as the mechanism that lets a single model both write forward and revise what it already wrote. No pricing, license or availability details appear in the source.

How solid is it

The results come from the authors' own arena-style evaluations on a continuation benchmark against SEDD and MDLM, plus a separate comparison against size-matched ordinary autoregressive baselines at 100M and 300M parameters, and a compute comparison under their own shared FLOPs convention. Preference over SEDD and MDLM is reported as strong but with no numeric score or margin, and the compute saving is reported only as substantial, with no percentage or FLOPs figure given, so the size of both effects cannot be checked from the source.

Risks and caveats

This is an arXiv preprint with no stated peer review, and the source text names no authors, institutions or publication date. The continuation benchmark and the multi-pass refinement and diffusion-style baselines beyond SEDD and MDLM are not described beyond name or category, so how representative or how difficult these comparisons are cannot be judged from the text alone. All reported advantages rest on the authors' own evaluation setup and their own definition of shared FLOPs.