On 19th April 2023 between 11:12 and 11:45 UTC customers may have experienced an increase in 404 errors when attempting to retrieve existing consents. This was due to a change deployed in order to improve our platform’s telemetry. This change failed silently, and intermittently caused timeouts between two internal databases, while we simultaneously experienced a temporary spike in API traffic. As this issue was intermittent, it did not manifest during testing in our non-production environments. Approximately 30 minutes following the deployment into production the rate of API requests received declined and the platform recovered without any active intervention. We subsequently identified the breaking change and rolled this back, to resolve the underlying root cause and ensure our platform remained resilient during increased traffic loads. We held a internal post mortem within three days of this incident closing, and we have identified a number of improvements that we will be implementing as soon as possible to prevent an incident of this type reoccurring. These include improving our testing in non-production environments, enhanced alerting for the impacted service, and a review of our deployment strategy. If you have any questions regarding this incident, or post mortem, please reach out to support@yapily.com