PTP trains an inverse model to rebuild LLM prompts from responses

A paper submitted to arXiv on 31 July 2026, in the Computation and Language category, is titled "PTP: Previous-Token Prediction based LLM Inversion for Near-Exact Prompt Reconstruction". It tackles LLM inversion: working out which prompt produced an observed response.

The paper starts from a basic property of language models. They generate text by sampling the next token one step at a time, which creates a many-to-many mapping between prompts and responses. Many prompts can yield the same kind of output, and one prompt can yield many outputs, so inferring a prompt from an output is hard.

The authors describe prior work as treating prompt recovery as a semantic reconstruction task. Those methods fine-tune pretrained sequence-to-sequence models on large external datasets and need access to model weights or logits. Their goal is to generate prompts that are semantically plausible.

PTP takes a different route. The authors present what they call a functional approach to inverting a given LLM in a black-box setting, without auxiliary aids. They train an explicit inverse language model entirely from scratch, on data synthetically generated from the target LLM itself. The training objective mirrors the forward one: where the LLM learns next-token prediction, the inverse model is trained with previous-token prediction. The authors say this sets up a generative link between the forward and inverse processes, and that this link is what makes faithful prompt reconstruction possible.

Because the inverse model is generative, it can also sample. The authors say it naturally supports diverse prompt reconstructions, and that all of the sampled prompts induce similar responses when fed back through the forward, target LLM.

On results, the authors claim the approach generalises across datasets and transfers to reconstructing prompts from responses generated by different LLMs. They also say that across the set of token-based evaluation metrics for prompt and response reconstructions, it outperforms prior work. The abstract gives no metric values, so the size of the gain is not stated.

Key facts

  • PTP is a black-box LLM inversion method: it reconstructs a prompt from an observed response without needing model weights or logits.
  • An inverse language model is trained from scratch on data synthetically generated from the target LLM itself, using previous-token prediction as the mirror of next-token prediction.
  • Sampling from the inverse model yields diverse reconstructed prompts, and the authors say all of them induce similar responses under the target LLM.
  • The authors claim the approach generalises across datasets and transfers to responses generated by different LLMs.
  • The authors say it outperforms prior work across token-based metrics for prompt and response reconstruction; no numbers are given in the abstract.

Why it matters

Recovering a prompt from an output is hard because LLM sampling maps many prompts to many responses. The authors contrast PTP with earlier inversion work, which they say treats the task as semantic reconstruction. That earlier work fine-tunes pretrained sequence-to-sequence models on large external datasets and requires access to model weights or logits. PTP instead works in a black-box setting and uses no auxiliary aids: the inverse model is trained from scratch on text the target LLM generated itself. The title's phrase "near-exact" reconstruction signals the aim of recovering the actual prompt rather than a merely plausible one.

Who it affects

The paper is aimed at researchers working on LLM inversion and prompt reconstruction. The abstract describes a setting where only the target model's outputs are needed, so the method concerns anyone who can observe responses from a model but cannot inspect its internals.

How to use it

This is a research method, not a product. The abstract describes the recipe: generate synthetic data from the target LLM, train an inverse language model from scratch with previous-token prediction, then sample prompts from it given a response. No code release or availability is stated in the abstract text.

How solid is it

This is an arXiv preprint submitted on 31 July 2026, and the source here is the abstract only. All claims are the authors' own: generalisation across datasets, transfer across LLMs, and better results than prior work on token-based metrics. The abstract names no authors or institutions. It gives no numerical results, metric values or improvement margins, and it does not name the datasets, target LLMs or baseline prior-work methods. The abstract does not say the paper has been peer reviewed or accepted anywhere.

Risks and caveats

The headline claim of outperforming prior work cannot be checked from the abstract, since no figures, baselines or datasets are given. No model sizes, training cost or data volumes are given either, so the cost of training an inverse model per target LLM is unknown. No security, privacy or misuse implications are discussed in the abstract.

“we present a functional approach to inverting a given LLM in a black-box setting, without auxiliary aids”

— From the abstract of the PTP paper