Elevated latencies and timeouts

Incident Report for Fauna

Resolved

This incident has been resolved.

Posted Dec 09, 2020 - 09:06 PST

Monitoring

Changes that result in reduced peak memory usage have been deployed. We are monitoring the system.

Posted Dec 09, 2020 - 08:14 PST

Update

Further investigation has identified this issue also caused a similar event of elevated latencies and timeouts lasting from 1:41-2:24.

Posted Dec 09, 2020 - 02:56 PST

Identified

The issue has been identified as an excessive memory usage as part of processing user queries. We have a fix that we are deploying.

Posted Dec 09, 2020 - 02:13 PST

Investigating

We have experienced timeouts and elevated latencies in cloud, starting from 5:10AM-6:10 UTC and with smaller magnitude from 6:30-9:10.

Posted Dec 09, 2020 - 02:12 PST

This incident affected: Classic Region Group (FQL API).