Investigating intermittent issues with Mobile Service affecting Push and TOTP

Incident Report for SecureAuth Service

Postmortem

RCA - Intermittent Mobile Service issues - 12.05.2025

Issue Description: On December 5, 2025, a subset of customers experienced intermittent disruptions affecting the Push-2-Accept and TOTP multi-factor authentication (MFA) methods for end users.

Cause: At approximately 5:00PM PST on December 5, 2025, an update was applied to the nginx ingress routing configuration for a specific tenant. Due to the complexity of the tenant’s ingress rules, the change exceeded nginx’s processing capacity, resulting in performance degradation and intermittent routing failures. Consequently, a small portion of network traffic could not be serviced, causing intermittent MFA issues for a small percentage of customers.

Recovery: The ingress configuration changes were rolled back, restoring nginx to a stable state and resolving the affected MFA functionality.

Corrective Actions:

The tenant with highly complex ingress routing requirements will be migrated to a dedicated custom deployment to prevent similar issues in the future.

Posted Dec 08, 2025 - 12:07 PST

Resolved

We have monitored the results of the fix for an extended period and have not observed any issues since the fix was implemented. The issue has been fully resolved.

An RCA will be be posted once the post mortem investigation is complete.

Posted Dec 06, 2025 - 08:38 PST

Monitoring

We've identified the issue and have implemented a fix. We are continuing to monitor results and will provide a root cause once post mortem investigation is complete.

Posted Dec 06, 2025 - 00:05 PST

Investigating

We are investigating reports of intermittent issues with the Mobile Service potentially affecting Push and TOTP MFA methods. We will post updates as we gain understanding to the issue.

SMS OTP and Telephony OTP services are operational and can be used as an alternative MFA option.

If you have questions or require additional assistance, please log a ticket with support at https://support.secureauth.com.

Posted Dec 05, 2025 - 22:50 PST

This incident affected: Workforce (Push, Link-to-Accept Service).