Day: December 3, 2024

Make – Scenario Queue Not Being Processed

by

Dec 4, 06:58 CET
Resolved – This incident has been resolved.

Dec 3, 19:57 CET
Monitoring – We have identified an issue in our code and have rolled out a fix. The changes have been applied, and we will be closely monitoring the situation.

Dec 3, 18:46 CET
Investigating – We are currently experiencing an issue with scenario queue execution. It appears to be similar to the problem we encountered earlier today. This issue is being observed on eu1.make.com and us1.make.com.
Our team is actively investigating the problem and will provide an update within the next 2 hours or sooner if possible.
Thank you for your patience.

Canva – Some users might experience difficulty opening or creating designs

by

Dec 4, 15:12 AEDT
Resolved – This incident has been resolved.

Dec 4, 15:12 AEDT
Update – We are continuing to monitor for any further issues.

Dec 4, 14:33 AEDT
Monitoring – A fix has been implemented and we are monitoring the results.

Dec 4, 14:23 AEDT
Investigating – We are currently investigating this issue.

OpenAI – ChatGPT & API Unable to Generate Images

by

Dec 3, 17:27 PST
Resolved – The issue has been resolved.

Starting at 4:20 PM PT, some customers experienced errors when attempting to generate images from ChatGPT and the API. A fix was implemented, and the issue was fully resolved at 5:15 PM PT.

Dec 3, 17:05 PST
Monitoring – We have implemented a fix and the issue appears to be resolved. We will continue to monitor performance, and will update once we have confirmed resolution.

Dec 3, 16:55 PST
Investigating – We are currently experiencing an issue where requests to ChatGPT & API are unable to create images. We are currently investigating and will post an update as soon as able.

Anthropic – Claude.ai and Console experiencing increased latency and intermittent unavailability

by

Dec 3, 16:06 PST
Resolved – This incident has been resolved.

Dec 3, 12:46 PST
Update – We are continuing to monitor for any further issues.

Dec 3, 12:46 PST
Monitoring – A fix has been implemented and we are monitoring the results.

Dec 3, 11:14 PST
Update – We are continuing to investigate issues with claude.ai.

Dec 3, 11:08 PST
Investigating – We are currently investigating issues with logging in and signing up to claude.ai

GitHub – [Retroactive] Incident with Pull Requests

by

Dec 3, 23:30 UTC
Resolved – On December 3rd, between 23:29 and 23:43 UTC, Pull Requests experienced a brief outage and teams have confirmed the issue to be resolved. Due to brevity of incident it was not publicly statused at the time however an RCA will be conducted and shared in due course.

GitLab – Project mirror disabled due to excessive notifications

by
December 3, 2024 14:04 UTC
Investigating – We are currently investigating emails being sent for older project mirrors and imports. More details about this incident can be found in gitlab.com/gitlab-com/gl-infra/production/-/issues/18929

December 3, 2024 14:47 UTC
Identified – We’ve turned off the functionality that sends out email updates temporarily for project mirrors. We are continuing to investigate this incident.

December 3, 2024 15:51 UTC
Identified – We’ve turned off the functionality relating to schedules for project mirroring. Project mirroring will be reenabled once we resolve this issue. Mirrored projects will not be updated during this time.

December 3, 2024 18:49 UTC
Monitoring – Project mirroring has been re-enabled on GitLab.com and we are monitoring to make sure no further issues arise. See https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18929 for further details.

December 3, 2024 20:26 UTC
Resolved – After seeing no further issues arise during our monitoring period, we are considering this incident resolved. Please review https://gitlab.com/gitlab-com/gl-infra/production/-/issues/18929 for more details.

GitHub – Incident with Pull Requests and API Requests

by

Dec 3, 20:05 UTC
Resolved – This incident has been resolved.

Dec 3, 20:05 UTC
Update – Pull Requests is operating normally.

Dec 3, 20:04 UTC
Update – Actions is operating normally.

Dec 3, 20:02 UTC
Update – API Requests is operating normally.

Dec 3, 19:59 UTC
Update – We have taken mitigating actions and are starting to see recovery but are continuing to monitor and ensure full recovery. Some users may still see errors.

Dec 3, 19:54 UTC
Update – Some users will experience problems with certain features of pull requests, actions, issues and other areas. We are aware of the issue, know the cause, and are working on a mitigation.

Dec 3, 19:48 UTC
Investigating – We are investigating reports of degraded performance for API Requests, Actions and Pull Requests

Figma – Service disruption

by

Dec 3, 19:22 UTC
Resolved – This incident has been resolved.

Dec 3, 18:19 UTC
Monitoring – A fix has been implemented and Figma should now be operational. We are continuing to monitor.

Dec 3, 17:49 UTC
Investigating – We are currently investigating an issue with loading Figma

Bubble – Issues with Main Bubble Cluster

by

Dec 3, 13:01 EST
Resolved – Our systems are functional and we are closing out this incident.

Dec 3, 12:47 EST
Investigating – We are investigating reports of issues with our systems.

Supabase – Realtime Postgres Changes degraded

by

Dec 3, 17:04 UTC
Resolved – This incident has been resolved.

Dec 3, 14:15 UTC
Monitoring – A fix has been implemented and we are monitoring the results.

Dec 3, 14:15 UTC
Identified – We have determined Postgres Changes was using an incorrect number of connections to check RLS policies.
This caused the system to reuse the same replication slot, leading to failed connections against Postgres Changes.

The issue has been identified and we’re deploying corrective measures.

Dec 3, 14:03 UTC
Investigating – We are currently investigating this issue.