TechMinute | Aug 30 | Cloudflare Freed 100TB of RAM

Watch the video: 100TB of RAM Freed — 9:16 HLS

Cloudflare just pulled off one of the quietest, most impressive engineering wins of the year: freed up 100 terabytes of RAM across its fleet — without adding a single server.

The story behind it is a masterclass in scale economics. Their 1.1.1.1 DNS resolver — the one answering a huge chunk of the internet's lookups — keeps roughly 250 billion records cached at any given moment. At that scale, wasting one byte per entry burns 250 gigabytes fleet-wide.

And waste there was:

• Vec overhead — Rust's standard vector carries 24 bytes of metadata and reserves spare capacity that just sits there unused
• Three separate memory lists per cache entry — answers, authority, additional records — each with its own pointer/length pairs and heap allocations, 48 bytes of metadata total
• Padded layouts — memory arranged for convenience, not density

The fix: swap the vector for Box<[T]> (16 bytes metadata, zero spare slots), merge the three lists into one contiguous allocation with 2-byte offsets (20 bytes metadata), and cut per-entry overhead by more than half. Result: ~100TB recovered — RAM that now serves more cache instead of sitting idle.

It's the kind of optimization invisible to users but enormous for the internet's plumbing. Free capacity, found in the code.

Source: Cloudflare Engineering Blog — "How we saved 100 terabytes of memory by optimizing 1.1.1.1's DNS cache"