Cache Refresh Optimization Retrospective

The cache had plenty of CPU and memory headroom. Network throughput was the bottleneck.

Multiple ad servers periodically fetched campaign configuration from the cache. Campaign settings rarely changed. But the system pulled the entire dataset every cycle regardless of whether anything had been modified. As the number of servers grew, network throughput approached the instance’s baseline limit, and downscaling was off the table.

Data Separation

Looking at the cached campaign data, I found three different types bundled together.

Metadata. Campaign metadata and targeting conditions change infrequently. They only update when an advertiser modifies a campaign.

State data. Budget consumption updates with every ad impression. It must always reflect the latest state.

Shared data. Ad creatives are often referenced by multiple campaigns. Embedding them inside each campaign duplicates the same creative across campaigns, inflating both storage and transmission. I moved creatives into separate keys and had campaigns hold only the reference IDs.

I separated all three. Metadata and shared data switched to incremental refresh. State data continued refreshing every cycle.

Incremental Refresh

Switching from full refresh to incremental refresh requires knowing what has changed.

A batch job fetches the latest data from the database, then compares it against what is stored in the cache. Only items with different content are written. At the same time, the change index records a timestamp for that item. Storing timestamps as Sorted Set scores enables range queries for items changed after a specific point in time. The read side remembers when it last queried, fetches only the score range after that point, receives the changed item IDs, and pulls those items separately.

flowchart LR
    subgraph Write ["Write Path"]
        DB["DB"] --> BATCH["Batch"]
        BATCH --> CMP{"Compare
with cache"}
        CMP -->|"Changed"| WRITE["Update cache
+ record timestamp"]
        CMP -->|"Same"| SKIP["Skip"]
    end

    subgraph Read ["Read Path"]
        SVC["Service"] --> TS{"Changes since
last refresh?"}
        TS -->|"Yes"| FETCH["Fetch changes only"]
        TS -->|"No"| LOCAL["Keep local data"]
    end

Separating write and read paths was the key. The batch writes only changes. Services read only changes.

Different consumers needed different data scopes, and that had to be handled too. Some services only needed metadata. Others needed metadata plus creatives. I split the read interfaces by consumer so each service pulled only the scope it actually used.

Result

Network throughput dropped significantly. During cycles with no changes, almost no data was transmitted. The cache instance could be downscaled to a smaller type.

The structure became more complex in return. Change detection logic, timestamp management, partial local state updates, and a full-sync mechanism to recover from cache-source inconsistencies all came with the switch. If the cache had held small or frequently changing data, I would not have gone this far.

Looking back, the starting point of this work was accurately identifying the bottleneck. Confirming that the constraint was network, not CPU or memory, naturally led to data separation and incremental refresh as the direction.