Big Pineapple , the platform behind 1. 1. 1.
1 , Gateway DNS , DNS Firewall , AS112 , and several other Cloudflare DNS services, stores over 250 billion DNS cache entries at any given time. At that scale, wasting a single byte per entry costs more than 250 gigabytes of memory across our fleet. Five successive changes to how cache entries are stored in memory cut the per-entry footprint by over 50%.
Across our fleet, these changes freed up roughly 100 terabytes of memory, equivalent to the amount of RAM in 130 of our Gen 13 servers . The cache also got faster. Insert throughput rose 43% and lookup latency dropped 19%, as fewer allocations and better memory locality meant we did not trade speed for space.
What we cache On cold start, Big Pineapple starts out with an empty cache. As DNS queries arrive, the cache fills until it hits its maximum entry count, at which point we evict older or less popular items to make room. The exact cache size varies by data center.
When EDNS Client Subnet (ECS) is in use, authoritative servers return different answers depending on the client's network, so we cache multiple versions of the same query. This increases both the number of entries and the memory each one consumes, making the optimizations in this post especially impactful for ECS-heavy locations. Each item in the cache is a key-value pair.
The key identifies what was queried: The value stores the DNS response itself: the answer, authority, and additional record sections, along with metadata like the creation time, a hit counter, and the Time-to-Live (TTL). Both structs have room for improvement. Several fields use types that carry overhead we don't need once the entry is stored.
Benchmarking memory usage To measure the impact of each change, we benchmark by filling the cache with randomly generated entries that roughly match the traffic distribution we see in production: 56% A records, 25% AAAA , and 19% TXT . Each entry contains between one and four records. TXT records serve as a stand-in for all non- A / AAAA record types in the benchmark.
Their size is randomized between 64 and 224 bytes, close to the average response size we see for variable-length record types. We track memory usage using a custom allocator that wraps Rust’s System allocator and records the number and size of allocations per cache entry. Alongside memory, we measure insert throughput and lookup latency across the full cache flow to make sure memory savings don’t come at the cost of performance.
These inputs approximate production rather than reproduce it exactly. Process memory also depends on traffic mix, cache occupancy, allocator state, and memory used outside the cache. We therefore measured resident memory across production instances during the rollout.
The cost of capacity Vec stores three fields: a pointer to heap-allocated data, the current length, and the total capacity. When you push an item, Vec checks whether the length exceeds the capacity and reallocates if needed. If there’s room, it just appends the item and increments the length.
Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec . The over-allocated heap space is wasted as well, as a Vec with capacity for eight items but only five stored leaves three slots unused on the heap.
Using Box<[T]> solves both problems. It can’t grow after creation, so it doesn’t need a capacity field or reserve space for future elements. The same applies to String , which also carries a capacity field.
Box drops it. Each cache entry stores 8 Vec and String fields. Replacing them with Box<[T]> and Box saves 8 bytes per field, 64 bytes per entry.
It also eliminates the excess heap memory that Vec reserves for future growth. The combined savings add up to over 15 terabytes with over 250 billion cache entries. Fewer lists, fewer pointers Rather than storing the answer, authority, and additional sections in separate lists, we can store a single list with offsets to the start of each section.
Since DNS record counts per section fit in a u16 , we can use a u16 (2 bytes) for each offset, compared to the 8-byte pointer and 8-byte length that each separate Box<[T]> requires. This removes two lists, each with an 8-byte pointer and 8-byte length, and replaces them with two 2-byte offsets, saving 28 bytes per entry. These savings do not always map directly to the number of bytes removed from individual fields.
Rust inserts padding to satisfy alignment requirements and rounds a struct’s size up to a multiple of its alignment. Removing a small field can therefore eliminate additional padding. For example, we also packed several boolean fields into a single bitflag .
This reduced the surrounding padding, causing the struct to shrink by more than the size of the individual booleans. Dropping the owner Each DNS record has an owner, the domain the record belongs to. In many cases, this owner is identical to the domain being queried.
For example, a query for example. com A returns two records with the same owner: But when a CNAME is involved, for example, the record owner can differ from the queried domain: The DNS wire format handles repeated owners using name compression, as defined in RFC 1035 . Rather than encoding the same domain twice, subsequent occurrences store a 2-byte pointer to the first occurrence.
A domain like www. example. com can encode just www followed by a pointer to where example.
com already appeared in the message. This works well on the wire, but in our cache we store the full owner name alongside each record. Following compression pointers during cache lookups is expensive on the hot path, so we trade memory for speed.
Most records, however, have an owner identical to the queried domain. For those, we can drop the owner entirely and infer it at read time. When the owner differs, such as the A records behind a CNAME , we store the full name.
Originally published at blog.cloudflare.com
