Memory costs are increasing dramatically. Both RAM and hard disk drive prices have exploded over the past year. At Cloudflare, we run several massively distributed storage products (including our famous CDN) that rely on making efficient use of the memory we have deployed so we can continue to serve all of our customers.
With this in mind, we prototyped a way to expand effective cache capacity. By encoding eligible assets with Zstandard inside Pingora , the architecture trades a minor CPU increase for significant storage and cross-data center bandwidth savings. We have been prototyping a system called Cache Transcoding, which I built during my internship at Cloudflare as part of the 1.
- 1 Intern Program .
When an eligible response enters the cache, we encode it using Zstandard, or zstd, before writing it to disk. We keep that compressed form while the asset lives in the cache and moves between data centers via Tiered Cache , then decode it before serving the response to the client. In our initial testing, this encoding shrunk eligible assets to ⅓ of their original on-disk size on average.
The estimated extra CPU cost in our origin-facing proxy was small, but that is the trade. A small increase in CPU gives Cloudflare petabytes of effective cache capacity and reduces the data transferred between our data centers. The encoding cost is paid once when an asset enters the cache.
The storage and bandwidth savings continue every single time that asset is reused. What is Zstandard? Zstandard, or zstd, is a lossless compression algorithm developed by Yann Collet at Facebook and open sourced in 2016.
Lossless means that after compressed data is decoded, every byte is identical to the original. We can change how an asset is represented on disk without changing the asset itself. Zstd is designed to balance compression ratio with speed.
In our earlier browser compression testing , it compressed data 42% faster than Brotli while producing nearly the same file size, and produced files 11. 3% smaller than gzip at a comparable speed. That balance matters because Cache Transcoding would touch a large amount of traffic, so both encoding and decoding need to stay fast.
The prototype uses zstd level 3, giving us most of the compression benefit without turning cache fills into a CPU bottleneck. Cloudflare traditionally stores an asset using the content encoding supplied by its origin. If an origin sends an uncompressed response, we store those uncompressed bytes on disk and transfer them between data centers in the same form.
Cache Transcoding adds compression inside the cache itself. Not everything is worth compressing Transcoding does not mean compressing everything. Images, video, and fonts are usually compressed already.
In our traffic sample, this media slice represented 21. 4% of requests but 63. 3% of bytes.
Compressing it again would burn CPU for nothing. Compressible text is different. HTML, JSON, CSS, and JavaScript represented 67.
3% of requests and 22. 3% of bytes. Within that text slice, approximately 71% arrived uncompressed with Content-Encoding unset and it compresses well.
In our controlled test corpus, the eligible assets compressed by roughly 2. 8 times. Encoding is more expensive per byte, but assets are served far more often than they are filled.
By changing how assets are represented, existing hardware could store more customer content. Fewer bytes on disk mean each server can retain more objects. This increases cache density and reduces the likelihood that useful content is evicted because an uncompressed representation consumed more space than necessary.
The smaller representation also helps as an asset moves through Tiered Cache because it reduces the data transferred between Cloudflare data centers, making backbone usage more efficient. Paying the compression cost once Compression is never free. Encoding and decoding both use CPU, so the important question is whether the byte savings are worth the processing cost.
At zstd level 3 (often the default balance of speed and compression size output), our model kept the extra CPU cost to a few percent under the traffic and reuse assumptions we tested. We initially considered limiting transcoding to popular content, since hot assets are reused more, but it did not help. Decoding happens every time an asset is served, so limiting the feature to only the hottest content reduced the storage saving without cutting CPU by the same amount.
The simpler policy performed better. Transcoding all eligible compressible text at or above 4 kibibytes (KiB) captured nearly all of the measured storage benefit, while remaining within the CPU budget. How Cache Transcoding works On a cache miss, our Pingora-based proxy encodes the body using zstd before writing it to disk.
The cache metadata records that the stored representation is compressed and preserves the original content length. Before the response leaves the proxy, the body is decoded back to its original identity representation . On a cache hit, the stored zstd object is read from disk and decoded.
With Tiered Cache, the compressed representation is transferred from the upper tier to the lower tier in the compressed form. Decoding only happens on the client-facing hop. On a full cache miss, the upper tier fetches identity bytes from the origin.
Those bytes are encoded once, stored as zstd, and transferred to the lower tier in their compressed form. The lower tier also stores the zstd representation, then decodes it for the request path. If the lower tier misses but the upper tier already has the object, the origin is not involved.
The compressed object moves directly between the cache tiers. It remains compressed on the wire and on disk, then is decoded once at the lower tier. If the lower tier already has the object, no network transfer or encoding is needed.
The lower tier reads the zstd bytes from disk, decodes them, and passes the original asset onward.
Originally published at blog.cloudflare.com
