Expiration Policy ​

Redis allows all data structures to have expiration times, automatically deleting keys when their time is up. You can imagine a "grim reaper" inside Redis, constantly monitoring keys set to expire and promptly deleting them when their time comes.

However, this raises questions about whether too many keys expiring at once could overwhelm the system. Since Redis is single-threaded, the time spent deleting expired keys might impact the processing of read and write commands, potentially causing delays.

Expired Key Collection ​

Redis maintains a separate dictionary for each key with an expiration time. It regularly scans this dictionary to remove expired keys. In addition to periodic scans, Redis employs a lazy deletion strategy, checking the expiration time of a key when it is accessed. If the key is expired, it is immediately deleted. Periodic deletions are centralized, while lazy deletions are scattered.

Periodic Scanning Strategy ​

By default, Redis performs ten expiration scans per second. During a scan, it randomly selects 20 keys from the expiration dictionary, deletes those that are expired, and if more than a quarter of them are expired, it repeats the process. To prevent excessive looping and potential thread stalls, the scanning duration is capped at 25 ms.

In a scenario where all keys in a large Redis instance expire simultaneously, Redis would repeatedly scan the expiration dictionary until expired keys become sparse. This could lead to noticeable delays in processing read and write requests. Additionally, frequent memory management could consume CPU resources.

When client requests arrive during a scanning operation, they may have to wait at least 25 ms for processing. If a client sets a very short timeout (e.g., 10 ms), many connections could time out, resulting in business exceptions. Furthermore, slow queries won’t be recorded in Redis's slowlog during this wait time, as slowlog only tracks logical processing delays.

Best Practices ​

To mitigate issues from mass expirations, developers should randomize expiration times. For example, add a random delay to the target expiration time:

redis.expire_at(key, random.randint(86400) + expire_ts)

In systems with events that can be discarded after completion, such as activity data, it's crucial to avoid setting all expiration times uniformly, as this can lead to a large number of keys expiring simultaneously.

During development, services like iReader encountered performance alerts due to mass expirations. Randomizing expiration times effectively resolved these issues.

Slave Expiration Policy ​

Slaves do not actively scan for expired keys; their handling is passive. When a key expires on the master, a DEL command is logged in the AOF file, which is then synchronized to all slaves. If the synchronization is delayed, there can be inconsistencies between master and slave data, resulting in scenarios where expired keys exist on the slave but not on the master.