Scheduled maintenance
Twilio Services
Issues related to Twilio’s Software
External Connectivity
Issues found outside of Twilio’s Network
Zipwhip Gateway
Issues related to Zipwhip's Services
System Metrics
Reporting Delays For Super SIM Connection Events and Usage Records
Incident
Report for Twilio
Resolved
We are no longer experiencing reporting delay issues for Super SIM Connection Events and Usage Records in the DE1 and SG1 regions this issue has been resolved.
However, while attempting to replay Super SIM Connection Events and Usage Records that were not published during the incident, we have identified an issue that prevents us from completing the process.
Unfortunately, this means that the events that were not published are lost and cannot be recovered. We estimated that the incident affected around 100% of devices that were connected to the SG1 internet breakout region (using APN sg1.super) between March 1st 8 AM UTC and March 2nd 10:30 AM UTC, and about 11% of devices connected to the DE1 internet breakout region (using APN de1.super) between March 2nd 8AM UTC and March 2nd 10:30AM UTC.
However, while attempting to replay Super SIM Connection Events and Usage Records that were not published during the incident, we have identified an issue that prevents us from completing the process.
Unfortunately, this means that the events that were not published are lost and cannot be recovered. We estimated that the incident affected around 100% of devices that were connected to the SG1 internet breakout region (using APN sg1.super) between March 1st 8 AM UTC and March 2nd 10:30 AM UTC, and about 11% of devices connected to the DE1 internet breakout region (using APN de1.super) between March 2nd 8AM UTC and March 2nd 10:30AM UTC.
Update
We have successfully mitigated the reporting delay issue for Super SIM Connection Events and Usage Records in the DE1 and SG1 regions. New Connection Events and Usage Records will be published again in near real time for devices connected in those regions. We are continuing to work on the process to replay events that were not published during the incident and will provide an update once the process is complete. Please note that this incident did not impact connectivity. We will provide another update in 2 hours or as soon as more information becomes available.
Monitoring
We have successfully mitigated the reporting delay issue for Super SIM Connection Events and Usage Records in the DE1 and SG1 regions. New Connection Events and Usage Records will be published again in near real time for devices connected in those regions. We are currently starting the process to replay those events that were not published during the incident and will provide an update once the process is complete. Please note that this incident did not impact connectivity. We will provide another update in 2 hours or as soon as more information becomes available.
Identified
We are experiencing reporting delays for Super SIM Connection Events and Usage Records in the DE1 and SG1 region. The issue started on Wednesday 1st of March at 00:00 PT for devices connected in SG1, and on Thursday 2nd of March at 00:00 PT for devices connected in DE1. The issue has been identified and we are actively working on restoring the service. The issue has no impact on connectivity. We are working to identify the source of the delay. We will provide another update in 1 hour or sooner once more information becomes available.