Day: December 11, 2024

Rovo – OpenAI’s API is returning 5xx for GPT models

by

Dec 12, 07:26 UTC
Resolved – Between 3:16pm PST and 7:38pm PST, we experienced an outage for Rovo and all AI features across Atlassian Products. The issue has been resolved and the service is operating normally.

Dec 12, 03:50 UTC
Update – We are witnessing significant recovery of the services. We will continue to monitor the situation closely.

Dec 12, 02:35 UTC
Update – We are actively communicating with the affected AI Service provider and closely monitoring the ongoing outage situation. While we have observed some signs of recovery, the majority of API calls are still resulting in errors.

We have successfully restored the Smart Answers in Search feature by rerouting requests to an alternative model.

All other Atlassian Intelligence capabilities and Rovo remain unavailable at this time due to the ongoing AI Service provider outage.

Dec 12, 00:47 UTC
Investigating – We have reached out to our AI Service Provider and they are actively working on restoring the API.

Additionally, we are working on routing the traffic for Smart answers in Search capability to another model.

We will provide more details within the next hour.

OpenAI – API, ChatGPT & Sora Facing Issues

by

Dec 11, 22:23 PST
Resolved – Between 3:16pm PST and 7:38pm PST on December 11, OpenAI’s services were unavailable. We began to see recovery for API traffic starting at approximately 5:40pm PST, and for ChatGPT and Sora around 6:50pm PST. We mitigated the issues at 7:38 pm PST, and all services are now fully operational.

OpenAI will run a full root-cause analysis of this outage and will share details on this page when complete.

Dec 11, 19:53 PST
Monitoring – API, ChatGPT, and Sora traffic has largely recovered. We are monitoring the situation to ensure full resolution.

Dec 11, 18:54 PST
Update – We are continuing to work towards remediation. API traffic recovery is ongoing and we are bringing ChatGPT traffic back up by region. Sora is partially beginning to recover.

Dec 11, 17:50 PST
Update – We are continuing to work towards remediation. API and ChatGPT traffic is partially recovered. Sora remains down.

Dec 11, 17:03 PST
Update – We are continuing to work on a fix for this issue.

Dec 11, 16:59 PST
Update – We are continuing to work on a fix for this issue.

Dec 11, 16:55 PST
Update – We have identified a pathway to recovery and we are starting to see some traffic successfully return. Continuing to work to return service to normal.

Dec 11, 16:24 PST
Update – ChatGPT, Sora, and the API remain down. We have identified the issue and are rolling out a remediation. We are working as fast as we can to return service to normal and apologize for the downtime.

Dec 11, 15:53 PST
Identified – We have reports of API calls returning errors, and difficulties logging in to platform.openai.com and ChatGPT. We have identified the issue and are working to roll out a fix.

Dec 11, 15:45 PST
Update – We are continuing to investigate this issue.

Dec 11, 15:42 PST
Update – We are continuing to investigate this issue.

Dec 11, 15:17 PST
Investigating – We are currently investigating the issue and will provide more updates shortly.

Notion – Issue with Notion AI functionality

by

Dec 11, 21:13 PST
Resolved – Notion AI functionality has been fully restored and is now operating at full capacity across all features and services.

Dec 11, 17:07 PST
Identified – Users may experience issues with Notion AI functionality due to a vendor outage. Our team is monitoring the situation and will provide updates when new information becomes available.

Compass – Issues accessing Compass

by

Dec 12, 03:55 UTC
Resolved – This incident has been resolved.

Dec 12, 03:18 UTC
Identified – We’re seeing recovery across all of our systems. The team is still working on ensuring that impact is fully mitigated.

Dec 12, 02:38 UTC
Investigating – We’re seeing issues accessing the Compass application. The team is working on identifying the underlying cause.