Root Cause Analysis

A scheduled maintenance operation on the Redis infrastructure extended beyond the anticipated timeframe, resulting in temporary service disruptions affecting multiple applications reliant on Redis for caching.

‌

Timeline

08:04 UTC - Our automated monitoring systems detected anomalies within the Redis infrastructure, leading to temporary service disruptions across multiple applications reliant on Redis for caching and data storage.

08:08 UTC - We received confirmations that users are no longer receiving server errors.

‌

Resolution

Services were fully restored by 08:08 AM UTC. Comprehensive system checks confirmed the stability and performance of the Redis infrastructure post-maintenance.

We have initiated a comprehensive retrospective and are actively collaborating with our Redis service provider, to investigate the root causes of the incident. This investigation aims to thoroughly understand the factors that contributed to the unexpected downtime and to implement effective measures to enhance our system’s resilience.

‌