Simon Willison releases datasette-arrows 0.1a0, an Arrow export plugin for Datasette

Simon Willison releases datasette-arrows 0.1a0, an Arrow export plugin for Datasette

Simon Willison published datasette-arrows 0.1a0 on 18 November 2026. It is a plugin for Datasette that adds an Apache Arrow streaming export option.

The plugin builds on a recent change in the core project. Datasette 1.0a43 added the ability for plugins to implement streaming export formats. Datasette core already supports streaming CSV and TSV: you can export millions of lines of data from a table without loading it all into memory, because those two file formats support streaming data in chunks. The register_output_renderer() plugin hook can now be used to implement the same pattern for other formats.

Willison explains why the plugin exists: he has learned never to release a new plugin feature without accompanying it with a plugin that uses it. The new streaming hook therefore ships with a worked example.

He says the plugin came largely from an AI model. Claude Fable 5.2 suggested the Apache Arrow streaming format as a good option, and built the plugin in almost a single shot.

Using it takes one command: add the plugin to Datasette with datasette install datasette-arrows, and the export dialog gains a new Arrow option. The screenshot in the post comes from Datasette's main branch. The most recent release does not yet group streaming formats such as Arrow with TSV and CSV in the "All rows in this table" section, but Willison says that is coming in the next alpha release.

Key facts

  • datasette-arrows 0.1a0 is a Datasette plugin that adds an Apache Arrow streaming export option.
  • It relies on Datasette 1.0a43, which let plugins implement streaming export formats through the register_output_renderer() hook.
  • Core Datasette already streams CSV and TSV, so millions of lines can be exported from a table without loading them all into memory.
  • Willison says Claude Fable 5.2 suggested the Arrow streaming format and built the plugin in almost a single shot.
  • Install with datasette install datasette-arrows; grouping of Arrow with CSV and TSV under "All rows in this table" arrives in the next alpha release.

Why it matters

The release is a first real test of a new Datasette capability. Since 1.0a43, plugins can add their own streaming export formats, where before only CSV and TSV streamed. Willison follows a stated rule of shipping a plugin alongside any new plugin feature, and this one shows the hook in use for a format beyond those two. It is also a small example of an AI model choosing the format and writing nearly all of a working plugin.

Who it affects

Datasette users who want to export large tables in Apache Arrow format, and Datasette plugin authors who want to see how the streaming export pattern is used with register_output_renderer(). The impact is narrow and aimed at people already working with Datasette.

How to use it

Run datasette install datasette-arrows. After that, the export dialog gains a new Arrow option. In the current release the Arrow option is not yet grouped with TSV and CSV in the "All rows in this table" section; the post's screenshot is from the main branch, and Willison says the grouping is coming in the next alpha release.

How solid is it

This is a short first-person announcement from the plugin's author. The version is an early alpha (0.1a0). The claim that Claude Fable 5.2 suggested Arrow and built the plugin in almost a single shot is Willison's own account. The prompts, session details or the amount of human editing behind the Claude-built plugin are not described. No performance figures, file sizes or benchmarks for the Arrow export are given.

Risks and caveats

This is an alpha plugin that depends on an alpha feature of Datasette (1.0a43), so behaviour may change. The export dialog layout differs between the main branch and the latest release until the next alpha ships, and no date is given for that release. Without published benchmarks, how well the Arrow export performs on very large tables is untested in the source.

“I've learned never to release a new plugin feature without accompanying it with a plugin that uses it.”

— Simon Willison