Status Tracker v0.23
Dec 3, 04:39 UTC
Resolved – This incident has been resolved.
Dec 3, 04:38 UTC
Update – We saw a recurrence of the large hosted runner incident (https://www.githubstatus.com/incidents/qq1m7mqcl6zk) from 12/1/2024. We’ve applied the same mitigation and see improvements. We will continue to work on a long term solution.
Dec 3, 04:16 UTC
Update – We are investigating reports of degraded performance for Hosted Runners
Dec 3, 04:11 UTC
Investigating – We are currently investigating this issue.
Dec 3, 03:40 UTC
Resolved – This incident was resolved by hotfix and rolled out to all environments by 2024-12-02 07:06 UTC.
We apologize for any inconveniences this may have caused you, your team, and our mutual customers.
Dec 2, 04:26 UTC
Monitoring – A hotfix is rolling out to all environments and expected to complete within the next 2 hours. We will continue to monitor the issue during this time and provide an update to confirm the impact is mitigated.
Dec 2, 02:11 UTC
Identified – We have identified the root cause in a code path that processes the response from loading static macro content. A fix has been produced and is going through QA to ensure the issue is resolved before rolling out to all sites. We expect to resolve the issue across all Confluence sites by the end of the day.
Dec 2, 01:33 UTC
Investigating – We have identified a problem with Connect where Confluence macro rendering is degraded. We are investigating the root cause and will provide another update by 02:00 UTC.
Incident began at 2024-12-02 19:00 and ended at 2024-12-02 20:50 (times are in Coordinated Universal Time (UTC)).
We recommend that you retry any failed API calls to the Play Developer Subscriptions and In-App Purchases API that occurred during this disruption window.
Affected products: Play Developer Subscriptions and In-App Purchases API
Dec 3, 00:55 UTC
Resolved – This incident has been resolved.
Dec 3, 00:22 UTC
Monitoring – A fix has been implemented and we are monitoring the results.
Dec 3, 00:09 UTC
Investigating – We are currently investigating this issue.
Dec 2, 19:34 EST
Resolved – We confirmed that this was an overly-sensitive alert
Dec 2, 13:28 EST
Monitoring – We believe this is a false alarm on an overly sensitive alert and are working on a fix for the alert.
Dec 2, 13:22 EST
Investigating – We are investigating reports of issues with our systems.
Dec 3, 00:34 UTC
Resolved – Between 20:48 UTC and 22:34 UTC, some customers in the US region may have experienced issues creating or editing alert conditions through the New Relic UI or API. This issue has been resolved and impacted services have returned to normal operation. Data ingest and alert notifications were not impacted.
Dec 2, 23:00 UTC
Investigating – Some customers in the US region may experience issues creating or editing alert conditions through the New Relic UI or API. Data ingest and alert notifications are not impacted at this time. We are investigating and will continue to provide updates here as more information is available.
Dec 2, 23:32 UTC
Resolved – This incident has been resolved.
Dec 2, 23:20 UTC
Monitoring – The team have identified the issue and deployed a fix. We are continuing to monitor the situation.
Dec 2, 22:47 UTC
Investigating – We are currently investigating reports of Webflow.com not loading for some customers. Customers can access their Dashboard and sites directly at webflow.com/dashboard in the meantime.
Dec 2, 21:42 UTC
Resolved – This incident has been resolved.
Dec 2, 20:30 UTC
Monitoring – A fix has been implemented and we are monitoring the results. It may take a small amount of time for DNS changes to propagate
Dec 2, 20:11 UTC
Identified – The issue has been identified and a fix is being implemented.
Dec 2, 19:56 UTC
Investigating – We have identified the cause of the issue, and is currently working on resolution
Dec 2, 11:25 PST
Resolved – The issue has now been resolved.
Starting at 10:40AM PT, some customers experienced an elevated error rate in the Assistants API. A fix was introduced, and performance was stabilized at 10:52 AM PT.
Dec 2, 11:13 PST
Monitoring – A fix has been implemented and performance has normalized. We will continue to monitor.
Dec 2, 11:00 PST
Investigating – We are currently experiencing degraded performance for the Assistants API. We are currently investigating and will post an update as soon as possible.
Dec 2, 12:39 EST
Resolved – Roll-up properties are no longer experiencing delays. We’ve identified and addressed the cause of the delays that began at 8:29 AM EST (UTC -05:00) on Dec 2, 2024 as a problem with an underlying system failure. The issue is now fully resolved.
Dec 2, 11:15 EST
Identified – We’ve identified the cause of the Roll-up Properties being delayed up to 6 hours for EU customers since 8:29 AM EST (UTC -05:00) on Dec 2, 2024. These delays may impact tools such as reports, emails, etc using roll-up properties in HubSpot. Our team is working on reducing the delay and will update this page when more information is available.