Glonce

GPT-6 Astra: Sebastian Raschka digs into the looped-transformer rumor

modelsThe Glonce editorsSebastian Raschka🔥80Glonce rating80Impact80Novelty70Relevance100Surprise65Players90Credibility60
GPT-6 Astra: Sebastian Raschka digs into the looped-transformer rumor

OpenAI released GPT-6 Astra last week. In Sebastian Raschka's assessment, it is likely the best model tried so far, ahead of the GPT-5.6 predecessor across writing, math and coding, and especially strong at 3D rendering and animation relative to other models. On the ARC-AGI-3 benchmark, which mixes logic puzzles with generalization tests, Astra scores 99.9%, against 7.8% for GPT-5.6 Sol. The broader agentic scores read as more mixed: on the Artificial Analysis Coding Agent Index v1.4 and the wider Intelligence Index v4.2, Astra sits at the frontier without pulling far ahead of rivals. Raschka's explanation: benchmarks such as Terminal-Bench v2.1 run on one shared harness (Terminus 2, in that case) across every model, which can understate a model tuned mainly around its own harness, even if that model performs better day to day.

Raschka points to computer use as Astra's most distinctive strength: operating real software through mouse and keyboard clicks inside the Codex or ChatGPT app, not just writing code or calling APIs. As a demo, Raschka's own test had Astra redraw a photo of the author in a browser version of MS Paint using only the mouse, run on the Medium and High effort tiers to avoid burning through tokens on the costlier Extra High and Max settings. Computer use is not brand new (Raschka used earlier GPT models for expense tasks in Excel earlier this year), but Raschka calls it still relatively immature, since LLMs are text models first and graphical-interface control is a newer, harness-dependent skill. The expectation is that both models and their harnesses keep improving at this over the coming months and years, eventually turning mundane computer tasks, filing a tax return, for instance, into something doable just by asking. In a smaller aside, Raschka relays advice from a colleague, who cites a recommendation attributed to “the Claude Code lead”: newer models understand a task well enough on their own that some existing AGENTS.md and SKILL.md instruction files may now be over-specifying rather than helping, and are worth pruning or rewriting rather than kept as permanent.

The Mac buildout fits that computer-use focus. Raschka points to recent reports that OpenAI bought tens of thousands of Mac Minis and Mac Studios, not to run training itself (GPUs handle that) but to give models macOS environments to practice on during reinforcement learning. In the training loop Raschka describes, a harness gives the model a task, shows a screenshot of the Mac's screen, lets the model predict a mouse or keyboard action, executes that action, then screenshots the result and repeats until the task succeeds or fails; pass or fail signals, plus other verifiers, then feed back into training, the same reinforcement learning with verifiable rewards (RLVR) approach used for text-based reasoning. The model itself still runs on GPUs and is fed to the Mac over an API. Nvidia's chief executive has said, in an aside Raschka relays rather than an OpenAI statement, that Astra was trained on about 100,000 Grace Blackwell GPUs. Despite the computer-use emphasis, Raschka is clear this is not a new training paradigm: Astra, like probably any LLM for the foreseeable future, is still fundamentally a reasoning model that produces an internal chain of thought before answering, trained the same way via RLVR.

The architecture question comes from elsewhere. About two days before Astra's release, The Information reported, citing inside sources it does not name, that Astra uses a technique called “recurrent depth,” or “looped transformers.” Raschka, who works on LLM architecture, made a video explaining the mechanism and addressing what it would mean for hiding a model's reasoning trace. A looped transformer reuses the same transformer blocks (units combining attention, a feedforward layer, normalization and shortcut connections, sometimes called layers) multiple times on one input instead of only once; unlike simply stacking more blocks, the weights stay identical across every pass. The idea is not new, tracing back to the 2018 Universal Transformers paper. As a concrete example, Raschka's piece walks through Nanbeige4.2-3B, an open-weight model released in July: its input passes through 22 transformer blocks once, and the resulting hidden states are then fed back through those same 22 blocks a second time, block 1 again, then block 2, and so on. Unrolled, that is 44 total block applications, but only 22 distinct sets of weights, half of what a conventional 44-block transformer would need.

Raschka's introduction promises his own verdict on whether looped transformers actually hide a model's chain of thought, plus a look at newer research on the topic, later in the piece. Sourced only to The Information's anonymous reporting, the hidden-reasoning claim remains an unconfirmed report rather than a settled fact.

Key facts

  • OpenAI released GPT-6 Astra last week; in Sebastian Raschka's assessment it is likely the best model tried so far, ahead of GPT-5.6 in writing, math and coding, and especially strong at 3D rendering and animation.
  • On the ARC-AGI-3 benchmark, Astra scores 99.9%, against 7.8% for GPT-5.6 Sol; on the Artificial Analysis Coding Agent Index v1.4 and Intelligence Index v4.2, it leads the frontier without a wide margin.
  • Astra can operate software through mouse and keyboard inside the Codex or ChatGPT app; Raschka's demo had it redraw a photo of the author in browser MS Paint, using the Medium and High effort tiers.
  • Recent reports say OpenAI bought tens of thousands of Mac Minis and Mac Studios to give models macOS to practice computer use on during reinforcement learning; Nvidia's chief executive has said Astra itself trained on about 100,000 Grace Blackwell GPUs.
  • The Information reported, two days before release and citing unnamed inside sources, that Astra uses “recurrent depth,” or “looped transformers”; Raschka's piece explains the technique using the open-weight Nanbeige4.2-3B, whose 22 transformer blocks run twice for 44 total applications but only 22 sets of weights.

Why it matters

GPT-6 Astra doubles as a case study in where frontier labs are pushing next: not only raw benchmark scores, where Astra leads without a landslide margin on agentic indices, but operating real software directly, the computer-use skill that has lagged text and code because most graphical interfaces were never built for a model to read. Training that skill by giving models actual macOS environments, rather than relying only on more GPU-bound text data, signals a shift in how the next round of capability gets built. The looped-transformer report adds a separate stake: if a frontier model reuses the same weights across multiple passes instead of stacking many distinct layers, and if that architecture also makes its chain of thought harder to see, that bears directly on how much anyone, safety researchers included, can trust a visible reasoning trace as a window into what a model is actually doing.

Who it affects

Anyone choosing a coding or creative-tool model gets a firsthand data point from someone who tested Astra directly rather than repeating a vendor's own benchmark chart. People building or evaluating agent harnesses should note the harness-dependency point: a benchmark score is partly a property of the harness, not only of the model being tested. AI safety and interpretability researchers who rely on chain-of-thought output as a monitoring signal are the direct audience for the looped-transformer question, since an architecture that obscures that trace changes what the signal is worth. LLM architecture researchers and open-weight model builders get a concrete worked example in Nanbeige4.2-3B. And the reported Mac Studio and Mac Mini purchases, plus the Grace Blackwell GPU figure, point to Apple's and Nvidia's hardware businesses as quiet beneficiaries of how OpenAI trains computer-use skills.

How to use it

Astra offers several effort tiers; Raschka's demo used Medium and High and skipped the costlier Extra High and Max to save tokens, so a practical first step is picking the lowest tier that still gets a task done rather than defaulting to the top setting. Its computer-use ability runs through the Codex or ChatGPT app, where it can be pointed at a graphical program directly instead of only a CLI or an API. Separately, anyone maintaining AGENTS.md or SKILL.md files for coding agents should treat Raschka's aside as a prompt to revisit them: content written to hand-hold an older, less capable model may now be constraining a newer one into worse answers, so pruning or rewriting stale instructions is worth doing rather than assuming more guidance always helps.

How solid is it

The hands-on impressions and the MS Paint demo are Raschka's own firsthand testing, and the benchmark comparisons lean on Artificial Analysis, which Raschka singles out as independent of the model makers and therefore more trustworthy than a vendor's self-reported numbers, though Raschka also flags that agentic scores can still be skewed by which harness a benchmark uses. The looped-transformer claim itself rests on one link back: The Information's report, citing inside sources it does not name, published about two days before Astra's launch; neither OpenAI nor this piece confirms it outright. The 100,000-GPU figure comes from Nvidia's own chief executive, not from an OpenAI disclosure, and the Mac-buying report is also secondhand. Raschka's own verdict on whether looped transformers actually hide a model's reasoning comes later in the piece, once he finishes walking through the mechanism itself.

Risks and caveats

The central architecture claim, that Astra uses looped transformers and that this might hide its chain of thought, comes from anonymous sourcing at The Information and is not verified in what is available here; treat it as a live rumor, not a confirmed fact. If it does turn out to be true and does obscure the reasoning trace, that weakens chain-of-thought output as a safety and interpretability signal, without any indication in this material of how much weaker. Benchmark comparisons across different harnesses can favor whichever model was tuned to that specific harness, so the reported frontier position on agentic indices may not reflect Astra's real capability relative to rivals. And two of the more attention-grabbing numbers, the 100,000-GPU training claim and the Mac-buying figures, are both other people's reporting relayed secondhand rather than something OpenAI has stated itself.

“it’s an exceptionally good model, likely the best I’ve used as of this writing.”

— Sebastian Raschka