In an abundance of caution we have left this incident in a monitoring state for an extended period of time, but have not seen a reoccurrence of the original issue. This incident is now resolved. We have implemented additional monitoring and improved automated validation tasks as a result.
Posted Dec 03, 2019 - 14:47 PST
Update
The background validation process has been paused to ensure it does not have an impact on normal operations after a spike in the system error rate earlier today. Issues for known impacted customers have been resolved. Once the behavior of the validation system has been validated, the process will be resumed.
Posted Dec 02, 2019 - 14:52 PST
Monitoring
Automated validation of the cluster continues. Issues for known impacted customers have been resolved. If you believe you are impacted please contact support at support@fauna.com.
Posted Nov 27, 2019 - 09:35 PST
Update
The build deployed on 22 Nov has resolved the source of the replication issue. Repair work is ongoing for the subset of users impacted.
Posted Nov 25, 2019 - 07:49 PST
Update
We’ve deployed a new build that appears to address the symptom and are actively exploring repairing affected data for the subset of users this impacts.
Posted Nov 22, 2019 - 10:07 PST
Investigating
We are seeing limited reports of index updates not being fully replicated to all replicas in the cluster. All data is present, but the indexes do not always display the data. Engineering is investigating the cause.
Posted Nov 15, 2019 - 23:25 PST
This incident affected: Global Region Group (FQL API).