Two-Factor Authentication SMS tokens Delay
Incident Report for CommCare HQ
Postmortem

Overview

CommCare HQ experienced an issue with Two-Factor Authentication in which SMS tokens delivery was delayed. Users received SMS tokens after several minutes and could not login as the token had expired.

Summary of the incident

On November 27,15:56 UTC, Users started to experience a delay in receiving their Two-Factor Authentication SMS tokens. Investigations led to the discovery that the issue was related to an issue with our SMS carrier as per their incident report here: https://status.twilio.com/incidents/9stk7dtx9z16
The issue was resolved at 20:15 UTC.

Our Next Steps

We will continue to encourage our users to create backup tokens and save them, as they can be used when there is a delay with SMS tokens delivery.
We sincerely apologize for any service disruption you may have experienced. Thank you for your patience and support. Please reach out to support@dimagi.com if you have further questions about the incident.

Posted Nov 28, 2020 - 08:52 UTC

Resolved
Dear users, the issue related to delayed Two-Factor Authentication SMS tokens has been resolved. Once more, we apologize for the inconvenience this has caused.
Posted Nov 27, 2020 - 20:22 UTC
Update
Dear users, we are continuing to monitor the issue with Two-Factor Authentication. Users experiencing the issue may continue using their back-up tokens. We will send another update in the next 30 minutes. We apologize for the inconvenience caused.
Posted Nov 27, 2020 - 20:03 UTC
Identified
Dear users, we are currently experiencing an issue with our Two-Factor authentication. SMS tokens for some users are delayed, this is caused by an issue with our SMS carrier. We apologize for the inconvenience that this has caused. Users experiencing the issue may try using their back-up tokens to login. We will send another update in the next hour. Please reach out to support@dimagi.com if you have more questions or need more information on the issue.
Posted Nov 27, 2020 - 18:58 UTC
This incident affected: www.commcarehq.org (SMS / Messaging).