Glonce

Compile by training turns text specs into neural functions, hits 83.6% accuracy

researchThe Glonce editorsHugging Face🔥66Glonce rating66Impact65Novelty70Relevance95Surprise55Players35Credibility85
Compile by training turns text specs into neural functions, hits 83.6% accuracy

A paper describes "compile by training," a way to turn a natural-language specification into a reusable neural function instead of calling a large remote model on every input. The motivation stated in the paper is that many recurring text tasks are easy to describe in words but hard to encode as rules, while routing each request through a large remote model brings repeated cost, latency, and dependency on that provider. The proposed fix works in two stages. At compile time, teacher models generate task-specific training examples, and those examples are used to train a small adapter for a compact interpreter. Once trained, the resulting function runs on its own, without the teacher models, and the paper says it can be stored, versioned, and composed the way ordinary software is. The authors test the approach on FuzzyBench-Hard, described as a subset on which a competing fast compiler called Program-as-Weights produced no exact matches at all. On that subset, compile by training reaches 83.6% semantic accuracy. The paper is explicit about the trade-off behind that number: compiling with this method takes roughly a minute, versus seconds for the fast compiler, so the accuracy gain is bought with slower compilation. The paper also reports a live deployment: the compiler runs in a public interactive service, and the authors demonstrate compiled functions in three settings, a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator. The abstract does not explain what "Claudish" refers to, and it names no authors, institutions, release date, model size, or training-data size, nor does it state the accuracy the Program-as-Weights fast compiler itself achieves on the same FuzzyBench-Hard subset.

Key facts

  • Compile by training turns a natural-language specification into a reusable neural function by having teacher models generate task-specific examples that train a small adapter for a compact interpreter.
  • On FuzzyBench-Hard, a subset where the Program-as-Weights fast compiler produced no exact matches, compile by training reaches 83.6% semantic accuracy.
  • Compiling takes roughly a minute, versus seconds for the fast compiler, trading compile speed for accuracy.
  • The compiled function runs without the teacher models afterward and can be stored, versioned, and composed like ordinary software.
  • The compiler is deployed in a public interactive service, demonstrated in a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator.

Why it matters

Teams that need a text function for a recurring task, say classifying a message or rewriting a sentence to a rule, usually either hand-code brittle rules or call a large remote model on every single input, paying repeated cost and latency and staying dependent on that provider. Compile by training offers a third path: describe the task in words once, and get back a small function that runs on its own afterward, with no further calls to the teacher models that helped train it.

Who it affects

The paper frames this for developers and teams building products around recurring, easy-to-describe text tasks who currently either write custom rules or lean on a remote large model per request. The three demonstrated uses, a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English-Claudish translator, point at practical assistant and interface features rather than research benchmarks alone.

How to use it

A specification in natural language is compiled into a neural function: teacher models first generate task-specific training examples, which then train a small adapter sitting on a compact interpreter. The compiled function can be stored, versioned, and composed like ordinary software, and does not need the teacher models to keep running. The paper reports the compiler already deployed in a public interactive service, with compiled functions shown working in the three demo settings above. Compiling this way takes roughly a minute, against seconds for the competing Program-as-Weights fast compiler, so it suits cases where accuracy matters more than compile-time speed.

How solid is it

The headline figure, 83.6% semantic accuracy, is measured specifically on FuzzyBench-Hard, a subset the paper says was chosen because the Program-as-Weights fast compiler got zero exact matches on it, a deliberately hard slice rather than the full benchmark. The paper does not give the fast compiler's own semantic-accuracy score on that same subset, so the comparison shown is exact matches for one method against semantic accuracy for the other, not a like-for-like accuracy gap. The abstract also does not state the subset's size or task domain, nor the size of the compiled model or its training data.

Risks and caveats

The abstract names no authors or institutions, gives no release date, and does not specify model size or training-data size, so provenance and reproducibility cannot be checked from the source alone. "Claudish," used to describe one of the three demo translators, is not defined anywhere in the text. The one quantified cost of the approach is compile time, about a minute versus seconds for the faster alternative; whether that cost holds at larger scale or across other task types is not addressed.

“The resulting function runs without the teachers and can be stored, versioned, and composed like ordinary software.”

— the paper