Cloudflare cuts DNS cache memory by over 50%, frees 100TB

Cloudflare's Big Pineapple platform, the caching layer behind 1.1.1.1, Gateway DNS, DNS Firewall, AS112 and several other Cloudflare DNS services, holds more than 250 billion DNS cache entries at any given time. At that scale, wasting a single byte per entry costs Cloudflare more than 250 gigabytes of memory across its fleet. Five successive changes to how those entries are stored in memory cut the per-entry footprint by over 50%, freeing roughly 100 terabytes of memory fleet-wide, equal to the RAM in 130 of Cloudflare's Gen 13 servers. The cache got faster in the process rather than slower: insert throughput rose 43% and lookup latency dropped 19%, because the changes meant fewer memory allocations and better memory locality rather than a straight trade of speed for space.

To measure the impact of each change, Cloudflare's engineers benchmarked by filling the cache with synthetic entries built to match production traffic: 56% A records, 25% AAAA and 19% TXT, with one to four records per entry. TXT records stood in for every non-A/AAAA type, sized randomly between 64 and 224 bytes, close to the real average response size for variable-length record types. A custom allocator wrapping Rust's System allocator tracked the number and size of allocations per entry, alongside insert throughput and lookup latency, so a memory win could not hide a performance loss. Because a synthetic benchmark only approximates production rather than reproducing it, the team also tracked resident memory on live production instances as the changes rolled out.

The first change replaced Vec and String fields with Box<[T]> and Box. A cache entry, once written, is never modified again, so the capacity field that Vec and String carry to support future growth serves no purpose, yet still costs 8 bytes per field, and Vec often reserves extra heap space beyond what is actually stored. Boxed slices and strings cannot grow after creation, so they drop the capacity field entirely and never over-allocate. Each cache entry has 8 Vec or String fields, so the switch saves 8 bytes per field, 64 bytes per entry, plus whatever excess heap space Vec had reserved. Across the fleet's 250 billion-plus entries, this one change alone saved over 15 terabytes.

The second change merged three separate record lists, answer, authority and additional, into a single list addressed by offsets. Since the record count in each section fits in a 16-bit integer, the two boundaries between sections can be stored as two 2-byte offsets instead of two full lists, each of which previously cost an 8-byte pointer plus an 8-byte length. That swap saves 28 bytes per entry. Cloudflare's engineers note that these savings do not always map cleanly onto the bytes an individual field takes up: because Rust pads structs to satisfy alignment and rounds their size up to a multiple of that alignment, removing even a small field, or packing several boolean flags into one bitflag, as they also did, can shrink a struct by more than the size of the fields removed, by eliminating padding along with them.

The third change concerns the owner field on each DNS record, the domain the record belongs to. Querying example.com for an A record returns two records both owned by example.com, but if a CNAME is involved, as in the post's own example where example.com resolves via a CNAME to cdn.example.com, the resulting A records are owned by cdn.example.com instead. The wire format avoids repeating owner names via compression pointers under RFC 1035, but following those pointers on every cache lookup is expensive on the hot path, which is why Cloudflare had been storing the full owner name with each cached record instead, trading memory for speed. Since most cached records share their owner with the queried domain, the new Record struct makes the owner an Option<Box>: when the owner matches the query, the field is None and the response builder recovers the domain from the cache key at read time with no heap allocation, and only when the owner genuinely differs, as with records reached through a CNAME, is the full name stored on the heap. The source does not give a specific bytes-per-entry figure for this change, unlike three of the other four.

The fourth change reshaped how individual DNS records are represented. Rust enums are sized to their largest variant, and in Cloudflare's RecordData enum that variant is NAPTR, whose three text fields, domain name and two integers need 136 bytes, pushing the whole enum, tag and padding included, to 144 bytes regardless of which record type is actually stored. An A record needs only 4 bytes and an AAAA record 16, and A and AAAA together make up over 80% of traffic, so most cached records were wasting more than 120 bytes on padding. The fix boxes the larger, rarer variants, TXT, NAPTR and SVCB among them, onto a separate heap allocation reached through an 8-byte pointer, while keeping the small, common A and AAAA variants stored inline. That saves 120 bytes per A or AAAA record and shrinks the enum itself to 24 bytes; NAPTR, now paying for a heap pointer and allocation overhead on top of its data, actually costs slightly more, a tradeoff Cloudflare's engineers accept because NAPTR records are rare in practice. Boxing is not free, however. Cloudflare's jemalloc allocator groups allocations into fixed size classes: a 32-byte TXT request fits its 32-byte bin exactly, but a 40-byte MX request rounds up to a 48-byte bin and wastes 8 bytes. Boxed data also scatters across the heap instead of sitting next to the rest of its entry, so a lookup that follows a boxed pointer can force the CPU to fetch an extra cache line.

The fifth change stores each record's data as raw wire-format bytes rather than a parsed enum variant. Cloudflare's engineers considered caching the entire pre-built DNS response in wire format and patching only per-client fields like the message ID, but rejected it: DNSSEC records only appear when a client sets the DO flag, which would force either caching two versions of every response or filtering an already-built message, and parsing a full message on every lookup carries its own cost that the parsed-enum approach had avoided. As a middle ground, each record's data is now stored as a single Box<[u8]>, with every record encoded as a 2-byte length prefix followed by its raw bytes, eliminating both the per-variant enum overhead and the boxed heap allocations that the fourth change introduced, while packing the data contiguously for better CPU cache locality. The tradeoff is that records can no longer be randomly indexed and must be read sequentially, complicating features like round-robin rotation of A and AAAA records, a cost Cloudflare's engineers call negligible since each entry holds only a handful of records. The account of this final change, and any closing summary of how the five changes' savings add up, is cut off in the text available for this retelling.

Key facts

• Cloudflare's Big Pineapple platform, which caches more than 250 billion DNS entries for 1.1.1.1, Gateway DNS, DNS Firewall and AS112, made five changes to how cache entries are stored, cutting the per-entry memory footprint by over 50% and freeing roughly 100 terabytes fleet-wide, equal to the RAM in 130 Gen 13 servers.
• The cache also got faster: insert throughput rose 43% and lookup latency fell 19%, because the changes cut allocations and improved memory locality instead of trading speed for space.
• Replacing 8 Vec and String fields per entry with Box<[T]> and Box removed an unused capacity field and wasted heap space, saving 64 bytes per entry and over 15 terabytes fleet-wide from this change alone.
• Boxing the larger, rarer variants of the RecordData enum, such as TXT and NAPTR, while keeping common A and AAAA records stored inline saves 120 bytes per A or AAAA record, since those two types make up over 80% of traffic but previously forced every record into a 144-byte structure sized for the rare 136-byte NAPTR variant.
• At Cloudflare's scale, wasting a single byte per cache entry costs more than 250 gigabytes of memory across the fleet, which is the reason such small, low-level changes to struct layout translate into a 100-terabyte result.

Why it matters

Big Pineapple holds over 250 billion DNS cache entries at once, so at Cloudflare's scale a single wasted byte per entry costs more than 250 gigabytes of memory across the fleet. That is the case for treating cache entry layout as an engineering problem worth five separate rounds of optimization rather than a detail to ignore. The result, over 50% less memory per entry and about 100 terabytes freed fleet-wide, also came with a 43% faster insert path and 19% lower lookup latency, which matters because it shows memory and speed did not have to trade off here: fewer allocations and better memory locality delivered both at once.

Who it affects

Directly, this is Cloudflare's own infrastructure: the freed 100 terabytes, equal to 130 Gen 13 servers' worth of RAM, becomes capacity Cloudflare can use elsewhere across the fleet that runs 1.1.1.1, Gateway DNS, DNS Firewall and AS112. Indirectly, it touches everyone who queries those services, since lookup latency fell rather than rose. The more lasting audience is backend and systems engineers, particularly those writing Rust, who are building large in-memory caches of their own and can lift these same techniques.

How to use it

This is not a product with pricing or a signup; it is a set of reusable Rust techniques. For data that is written once and never mutated, replace Vec and String with Box<[T]> and Box to drop the unused capacity field and stop over-allocating heap space. Where several related lists share a natural order, store them as one contiguous list addressed by small integer offsets instead of separate pointer-and-length pairs. Where a field's value can be recovered from context, such as an owner name that usually equals the lookup key, make it optional and store it only in the exceptional case. For a sum type like an enum, box the large or rare variants and keep the small, common ones inline, but weigh that against the allocator's size-class rounding and the memory-locality cost of chasing extra pointers. And where nothing after insert needs to mutate the stored data, consider keeping it as raw encoded bytes instead of a parsed structure, accepting sequential rather than random access in exchange.

How solid is it

This comes from Cloudflare's own engineering blog, written in the first person by the team that built Big Pineapple, with Rust struct definitions and references to the jemalloc allocator throughout. Three of the five changes carry an explicit before-and-after byte count, and the combined result, over 50% less memory per entry, about 100 terabytes freed, 43% faster inserts, 19% faster lookups, was measured with a custom allocator that tracks per-entry allocations, then cross-checked against resident memory on live production instances during the rollout, since the team notes its synthetic benchmark only approximates real traffic. No individual author is named and no publication date appears in the material available here. The source text itself cuts off mid-sentence during the description of the fifth change, before any closing summary, so there is no statement of how the total savings split across the five individual changes.

Risks and caveats

As an account of Cloudflare's own system published on its own blog, none of these figures come from an outside party. The third change, dropping the redundant owner field, is a change with no specific byte savings given, unlike three of the other four. Boxing the RecordData enum's larger variants is described as a net win but not a free one: jemalloc's size-class rounding wastes a few bytes on some allocations, and scattering boxed data across the heap costs memory locality, which the team judges worth it mainly because A and AAAA dominate the traffic. Storing records as raw bytes removes random access entirely, adding complexity to features like round-robin rotation of A and AAAA records that the team calls negligible only because each entry holds few records. Nothing in the material says whether the 100 terabytes freed is a one-time reclaim or an ongoing steady-state saving.

“Wasting a single byte per entry costs more than 250 gigabytes of memory across our fleet.”

— Cloudflare engineering blog