Glonce

JAZ agent framework matches MemGPT and ACE with a single primitive called invoke

researchThe Glonce editorsarXiv🔥77Glonce rating77Impact72Novelty78Relevance98Surprise63Players28Credibility87

Researchers built JAZ, a minimalist LLM agent framework that strips the usual agent loop down to a single LLM-based primitive called invoke, plus a set of built-in hooks for applying constraints and monitoring. The authors note that today's agent workflows often require extra engineering beyond the basic loop of alternating tool calls and observations, such as dedicated memory systems for long-horizon recall or self-improving systems that refine behavior over time. JAZ was built to test how far a minimal harness, little more than the agent loop itself, can go in handling tasks that those specialized systems were built for.

The invoke primitive is designed as the simplest loop satisfying two properties: first, the LLM can write arbitrary executable code, including recursive calls to invoke itself; second, everything the LLM can see, all inputs to invoke and its interaction history with the code environment, exists as variables inside that code environment. The authors frame invoke as a language primitive: a function whose implementation is supplied at runtime by an LLM each time it is called.

To validate this design, the team evaluated invoke using only prompting, with no manually designed tools, no custom harness, and no external systems such as memory modules or a file system, on tasks traditionally handled by specialized external harnesses. On long-horizon workflows that require recall beyond the model's context window, JAZ invoke outperformed Letta (MemGPT) by 8% while running at half the cost, on the recall-heavy portion of the StuLife benchmark. On continual self-improvement tasks, JAZ invoke outperformed ACE by 4% at a lower cost on the AppWorld benchmark.

Key facts

  • JAZ is a minimalist LLM agent framework built around one primitive, invoke, plus built-in hooks for constraints and monitoring.
  • invoke lets the LLM write arbitrary executable code, including recursive invoke calls, with all its visible inputs and interaction history stored as variables in the code environment.
  • Tested with only prompting and no custom tools, harness, memory system, or file system access.
  • On StuLife's recall-heavy tasks, JAZ invoke beat Letta (MemGPT) by 8% at half the cost.
  • On AppWorld's continual self-improvement tasks, JAZ invoke beat ACE by 4% at a lower cost.

Why it matters

The result challenges the assumption that long-horizon recall and continual self-improvement require purpose-built external systems like dedicated memory modules or self-improvement harnesses. JAZ shows that a single, minimal primitive with only prompting can match or beat specialized systems like Letta (MemGPT) and ACE on the tasks those systems were designed for, suggesting some of that specialized engineering may be unnecessary.

Who it affects

Researchers and engineers building LLM agent frameworks, particularly those working on memory-augmented agents or self-improving agent systems, since JAZ offers a simpler alternative design to benchmark against.

How to use it

JAZ is a research framework, not a packaged product: invoke is used purely through prompting, without manually designed tools, a custom harness, or external systems such as memory or file-system access, on workflows that would normally be implemented with specialized external harnesses.

How solid is it

The claims come from the paper's own benchmark comparisons on two tasks, StuLife's recall-heavy portion and AppWorld, against two named specialized systems, Letta (MemGPT) and ACE. The source text is an abstract-level extract that does not include author names, institutional affiliations, a submission date, or details on how StuLife's 'recall-heavy portion' or AppWorld's cost and scoring metrics are defined.

Risks and caveats

The AppWorld cost advantage over ACE is stated only qualitatively, as 'lower cost,' without a numeric figure, unlike the StuLife comparison against Letta which specifies half the cost. The mechanics behind the reported percentage gains and cost comparisons are not detailed in the available text.