Resolved -
The emergency maintenance window conducted this morning has successfully concluded.
We are pleased to confirm that all data backlogs—including transactional data and reporting for Insights, Speed of Service (SOS), BinUsage, and View metric reporting—have been fully processed and restored.
All systems are now operating at 100% capacity, and full system functionality has been confirmed.
Thank you for your patience and cooperation as our QSR teams worked diligently to complete this final recovery phase.
This incident is now considered closed.
Oct 23, 21:51 EDT
Update -
To ensure the complete and final recovery of all remaining transactional data from Monday's outage, we are implementing a focused maintenance window.
This emergency maintenance window is necessary to finalize the recovery process and restore full transactional data integrity across all impacted services.
Emergency Maintenance Window Details:
Date: Thursday, October 23rd
Time: 6:00 AM ET
Affected Services: This maintenance will specifically impact users polling Speed of Service (SOS) data, whether accessed via the Insights Portal or directly through the API.
Expected Downtime: There is potential for up to 15 minutes of downtime during this window.
We will provide an update following the completion of the 6:00 AM ET maintenance window to confirm full system recovery and restoration of all data backlogs.
We appreciate your cooperation as we take this necessary step to resolve the incident.
Oct 22, 15:23 EDT
Update -
Our teams continue to focus intensely on clearing the internal processing backlog affecting Insights, Speed of Service, BinUsage, and View metric reporting. While core AWS stability is restored, these residual service degradations persist.
Reporting Metrics Restoration (End of Day Today): We estimate that the backlog for Speed of Service, BinUsage, and View Metric reporting data should be fully processed and up-to-date by the end of the day today, October 21st.
Full Data Integrity (Within 24 Hours): The remaining transactional data backlogs are expected to be fully completed within approximately 24 hours.
We will provide a final resolution update tomorrow once the full transactional data backlog is cleared.
We appreciate your continued understanding as we work diligently to complete this final recovery phase.
Oct 21, 16:38 EDT
Update -
AWS has confirmed that their service outage has been resolved, and our core systems have stabilized.
However, we are tracking residual degradation within our platform impacting multiple services, specifically Insights, Speed of Service, BinUsage, and View metric reporting. As a result, some reporting data for these services may be temporarily unavailable or incomplete as we work to process the backlog.
We will provide a follow-up update as soon as we have successfully restored full data integrity and performance across all impacted services.
We appreciate your continued patience as we work to finalize the recovery.
Oct 21, 10:15 EDT
Update -
AWS has confirmed that their upstream service outage has been resolved, and our core systems are stable.
However, we are tracking residual degradation within our platform impacting two specific areas: Speed of Service and the Insights Portal reporting. We are continuing to monitor services and will provide an update once full system functionality is restored.
Oct 20, 20:15 EDT
Update -
AWS services are reporting that their systems have mostly recovered, and have reduced throttling. QSR teams have successfully implemented our mitigation plan.
We are monitoring closely, but please note that some service latency or brief intermittent errors may persist until AWS confirms full resolution.
We will provide an update when we confirm full system stability, including an estimated timeline for working through our remaining operational backlog.
Oct 20, 18:08 EDT
Update -
We are still experiencing some issues across multiple Enterprise services, as well as our Support phone service, due to an ongoing AWS outage. Our team is actively working to mitigate the impact, but you may encounter degraded performance and elevated error rates until AWS fully resolves the issue.
Oct 20, 13:57 EDT
Update -
We are currently experiencing disruptions across multiple systems, including Support Phone and Email services, due to ongoing AWS performance issues. This may result in delayed email responses and intermittent call connectivity. Access issues with QSR services or the Insights Portal are also related. We are actively monitoring the situation and will provide updates as they become available.
Oct 20, 12:25 EDT
Update -
AWS has provided an update reporting degraded performance and elevated error rates again, which is impacting multiple systems on our end. We are actively monitoring the situation and will share updates as they become available.
https://health.aws.amazon.com/health/status
Toast Integration Notice: QSR systems remain fully operational; however, we are unable to connect to the Toast APIs to retrieve orders due to the ongoing AWS issues affecting Toast services as well.
https://status.toasttab.com/
Oct 20, 11:14 EDT
Monitoring -
Systems are returning to normal following the resolution of the AWS outage. The Kitchen Status Webhook remains affected. We are actively monitoring and will provide further updates shortly. If you continue to experience issues, please contact QSR Support: via email at Support@QSRAutomations.com or Phone 855.980.7328 option 6.
Oct 20, 08:51 EDT
Identified -
We are currently experiencing issues across multiple Enterprise services, as well as our Support phone service, due to an ongoing AWS outage. Our team is actively working to mitigate the impact, but you may encounter degraded performance and elevated error rates until AWS fully resolves the issue.
Oct 20, 05:59 EDT