Berkeley researchers say AI agents need redesigned data systems

Aditya G. Parameswaran, an Associate Professor of EECS and co-director of the EPIC Data Lab at UC Berkeley, together with collaborators, published a perspective on the BAIR (Berkeley Artificial Intelligence Research) blog on how falling AI inference costs change what data systems, meaning databases and the software that manages them, need to look like. The post states that GPT-4-class capability cost roughly $30 per million tokens in early 2023; the same capability now costs under $1, and some providers price it below $0.10. Across the benchmarks the authors tracked, inference prices fell between 9x and 900x per year, with a median decline near 50x. They argue that intelligence sufficient for most everyday knowledge work, short of what they call Nobel-Prize-winning genius-level reasoning, already exists and keeps getting cheaper, pointing toward an era of near-free intelligence.

From that premise the authors set out three intertwined challenges. Data systems for agents asks how databases should serve agents as users, since agents query very differently from humans or BI tools. Data systems of agents asks what substrate lets thousands of agents hold state, coordinate and survive failures over long-running tasks. Data systems by agents asks how agents that can already synthesize entire custom data systems on demand can be trusted to build ones that behave as intended.

On the first challenge, the authors describe agent database use as agentic speculation: a single user request, such as a root-cause question like why did coffee sales in Berkeley drop this year, can spawn a swarm of agents that between them issue thousands of individual SQL queries while exploring joins, aggregations and filters. On a text-to-SQL benchmark run with multiple agents per task, only 10 to 20 percent of the resulting sub-plans were distinct, meaning 80 to 90 percent of sub-queries duplicated work already done elsewhere, even though that redundancy also came with higher task success rates as more agents attempted a task. The authors propose fixes partly drawn from older database research: reusing results across overlapping sub-plans through multi-query optimization and shared scans, returning approximate answers that are good enough for an agent to keep moving, streaming partial results so an agent can judge whether it needs the rest, letting agents submit batches of queries with stated approximation tolerances instead of one query at a time, and offering higher-level primitives, such as DBT-style looping macros, instead of forcing agents to enumerate every SQL query by hand. They also suggest data systems could turn proactive: warning an agent about a query's likely latency before it runs, or handing over pre-built materialized and virtual views as context instead of making the agent construct them.

On the second challenge, the authors note that today's agent substrate, tools such as Claude Code and Codex plus assorted memory mechanisms, mostly relies on agents writing notes into unstructured markdown files that get searched later by grep or embedding-based retrieval. They argue this will not scale once agents handle most knowledge work: context windows cannot hold every relevant file fragment, retrieval latency matters even if windows keep growing, and serializing a large database or codebase into context is often not feasible at all. Knowledge graphs do not solve this either, in the authors' view, since they still lack a way to retrieve only the memory relevant to a specific task across multiple facets, such as pulling only the memories tagged to the right module, language, framework and failure mode when an agent is debugging a flaky test; they add that what gets retrieved should be corrective rather than a raw record of past mistakes, since replaying a mistake risks repeating it. Drawing on their own prior work, the authors describe a structured memory scheme in which each memory entry is tagged across attributes that are either wildcarded or matched against specific values. For a data agent, those attributes could be the columns and tables involved, the type of operation, and a natural-language corrective instruction, for example that date-time operations should use fiscal rather than calendar year conventions, or that a query on product name should prefer a column named product_cleaned over product. Defining such an application-specific schema, which the authors liken to a world model for memory, is called an open question, and they suggest agents themselves could help refine it over time; they add that structured memory could also help evolutionary frameworks manage large search spaces and let future agents learn from mined traces of past single- and multi-agent runs.

The piece also flags concurrency as unresolved: existing multiversioning and copy-on-write techniques may not hold up once thousands of agents try to edit shared memory or state at once, for instance when many candidate transactions are attempted and only one needs to survive while the rest are rolled back. The authors point to exactly-once semantics, CRDTs and operational transformation as relevant prior techniques, and suggest fuzzy memory updates might trade strict consistency for lower latency; they name a failure mode they call livelock, where compensating actions block any real progress. Beyond shared state, the authors list handling agent failures, choosing between direct and shared-state communication, and managing straggler agents as open problems, and note that durable-execution tools such as Temporal exist but have not been proven at swarms of thousands of agents. The available text of the post breaks off mid-sentence while introducing how several agents might negotiate a shared schema, so the third challenge, data systems synthesized by agents, is stated only as a framing question in the material we have and is not elaborated further.

Key facts

GPT-4-class inference cost roughly $30 per million tokens in early 2023 and now costs under $1, with some providers below $0.10; the authors' tracked benchmarks show per-year price drops of 9x to 900x, median near 50x.
On a text-to-SQL benchmark with multiple agents per task, only 10 to 20 percent of the resulting sub-plans were distinct, so 80 to 90 percent of sub-queries duplicated work, even as the redundancy tracked with higher task success rates.
The authors propose a structured memory scheme that tags entries by attributes such as table, column and operation type plus a natural-language corrective instruction, instead of relying on markdown files searched by grep or embeddings.
They warn that thousands of agents editing shared state at once risks a failure mode they call livelock, and argue existing multiversioning and copy-on-write techniques are unproven at that scale.
The perspective is led by Aditya G. Parameswaran, Associate Professor of EECS and co-director of the EPIC Data Lab at UC Berkeley, with collaborators, published on the BAIR blog.

Why it matters

The argument is that cheap inference does not just make individual AI answers cheaper, it turns agents into the primary users of data infrastructure. Once a routine request can spin up a swarm of agents each issuing hundreds of queries, the database and memory layers built for a handful of human analysts or BI dashboards become the bottleneck rather than the model itself. That reframes the interesting engineering problem in AI from making models smarter to making the systems around them handle agent-scale traffic, state and coordination.

Who it affects

The piece speaks directly to people building or operating data infrastructure, database engines, query layers, memory and retrieval systems, for products that already use or plan to use multiple cooperating agents. It also matters to teams building agent harnesses in the vein of Claude Code or Codex, since the memory and coordination gaps described sit exactly where those harnesses currently rely on markdown files and ad hoc scripting. Data scientists and platform teams running text-to-SQL or analytics agents are named directly through the sub-plan duplication example.

How to use it

The concrete near-term ideas are the ones a team could borrow before any of this becomes a shipped product: cache and reuse results across agents' overlapping sub-plans instead of recomputing them, offer approximate or partial answers so an agent can decide whether to keep digging, and expose batch or higher-level query interfaces rather than one raw SQL statement per agent turn. On the memory side, the suggested pattern is to tag stored corrections by concrete attributes, such as which table or operation they apply to, and match on those attributes plus wildcards, rather than depending only on free-text search over markdown notes.

How solid is it

This is a self-described perspective and landscape survey, not a benchmark paper or a product announcement, and the authors disclose that several of the directions discussed, including agentic speculation, structured memory and synthesizing data systems from scratch, are drawn from their own ongoing research. The one hard number offered, the 10 to 20 percent distinct sub-plan rate on a text-to-SQL benchmark, comes from an experiment the authors ran themselves and is not independently cited. The rest of the piece is framed explicitly as open questions and proposed directions rather than settled findings, and the source text available to us cuts off mid-sentence before the negotiation-between-agents example and the third challenge, data systems built by agents, are fully developed.

Risks and caveats

The authors flag their own open problems rather than glossing over them: livelock from competing compensating actions, concurrency techniques such as multiversioning and copy-on-write that may not hold at thousands of agents, and durable-execution tools like Temporal that are untested at that scale. No timeline, product or funding is attached to any of this, so readers should treat it as a research agenda rather than a description of infrastructure that exists today. The disclosed conflict, that the authors are describing challenges their own lab is actively working on, is worth keeping in mind when weighing how neutral the framing is.

“The current wisdom is that files are all you need.”

— Aditya G. Parameswaran and collaborators, BAIR blog