We are investigating increased error rates in SecureAuth Identity Platform.

Incident Report for SecureAuth Service

Postmortem

On Dec 7th around 7:42 am PST, SecureAuth detected that several services were encountering issues and customers were impacted. Upon further investigation, the root cause of the service interruption was due to a wide-spread outage in AWS US-East-1 region. At 8:39 am PST, SecureAuth switched the MFA Cloud Services to the backup cluster hosted in AWS’s US-West-2 region. However, IdPs that run from the AWS US-East-1 region could not be moved and certain customers were continuing to be impacted. AWS started recovery around 2 PM PST and SecureAuth’s services started to recover as well. The AWS US-East-1 region eventually became fully functional around Dec 8th 1:32 PM PST. SecureAuth will engage with AWS to discuss the root cause of the outage and devise plans for future reliability improvements.

Posted Dec 08, 2021 - 23:12 UTC

Resolved

AWS service has been fully restored and all SecureAuth services returned back to normal.

Posted Dec 08, 2021 - 17:44 UTC

Update

We are continuing to monitor for any further issues.

Posted Dec 07, 2021 - 19:39 UTC

Monitoring

AWS has restored some of the impacted services but has no ETA on full resolution. We will continue to monitor.

Posted Dec 07, 2021 - 19:36 UTC

Update

We are continuing to investigate this issue.

Posted Dec 07, 2021 - 18:22 UTC

Update

AWS is encountering networking issues in US-East-1 region and impacted several services. There is currently no ETA from AWS on the resolution timeline.

Posted Dec 07, 2021 - 17:19 UTC

Update

We are continuing to investigate this issue.

Posted Dec 07, 2021 - 16:54 UTC

Investigating

We are investigating increased error rates in SecureAuth Identity Platform and selected Cloud Services.

Posted Dec 07, 2021 - 16:37 UTC

This incident affected: SecureAuth Titan Services (Dashboard Service, Push-to-Accept Service, SMS Service).