Supabase Outage History

Recovery efforts are completed and this incident is now resolved. Thank you for your patience throughout this incident.

Our infrastructure provider has resolved the underlying incident, and impacted services have recovered. We have completed recovery efforts for affected projects, re-enabled project creation, pausing, and compute resizes in the us-east-1 region, and are now monitoring the platform to ensure continued stability.

Our infrastructure provider has resolved the underlying incident, and the majority of impacted projects have now recovered. Our team is currently working through the remaining projects that did not automatically self-heal during recovery and is actively restoring them manually where needed. We have also re-enabled project creation, pausing, and compute resizes in the us-east-1 region. We are continuing to monitor platform stability closely and will provide additional updates as recovery progresses.

Our team continues to make progress restoring impacted projects safely and as quickly as possible. To support recovery efforts, project pausing and new project creation in the us-east-1-a4 region continue to be disabled. While recovery efforts are moving forward, full resolution is still expected to take several more hours based on the latest guidance from our infrastructure provider. We understand the impact this is having and appreciate your continued patience. We’ll continue to share updates as more information becomes available.

Our team continues to make progress restoring impacted projects safely and as quickly as possible. To support recovery efforts, project pausing and new project creation in the us-east-1-a4 region continue to be disabled. While recovery efforts are moving forward, full resolution is still expected to take several more hours based on the latest guidance from our infrastructure provider. We understand the impact this is having and appreciate your continued patience. We’ll continue to share updates as more information becomes available.

Our team is actively working to bring impacted projects back online as quickly and safely as possible. While we are seeing progress, full recovery may still take several hours based on the latest guidance from our infrastructure provider. We appreciate your patience and will continue to share updates as more information becomes available.

Affected projects are continuing to recover. We are monitoring the recovery of the AZ and will provide regular updates until full recovery.

Affected projects are continuing to recover. We are monitoring the recovery of the AZ and will provide regular updates until full recovery.

Affected projects are continuing to recover. We are monitoring the recovery of the AZ and will provide regular updates until full recovery.

We're starting to see a number of affected projects starting to recover. We're still working with our upstream providers on a full recovery of all affected projects.

We are continuing to work on a fix for this issue.

We are continuing to work on a fix for this issue.

We've disabled pausing projects in us-east-1. We recommend not changing project configurations at this time. Users cannot access their databases or run any authentication-dependent services. We are continuing to monitor this situation with our upstream provider.

We have disabled project creation in us-east-1. We will provide another update in an hour, if not sooner.

We are waiting on a resolution from our upstream provider. We will provide an update in an hour.

We have identified the root cause of the connection issues as an outage with an upstream provider affecting a single availability zone (us-east-1a). All affected projects are located in this zone.

Increased connection times for Supavisor since about 00:10 UTC is affecting a limited number of projects in us-east-1

This is now resolved. Thank you for your patience while we worked to resolve this issue.

The exceptions fix has been rolled out, and we are monitoring for continued stability.

We are continuing to work on a fix. We’ll share further updates as progress is made.

We are continuing to work on a fix for this issue.

The issue has been identified and we are working on a fix.

We are investigating 403 errors for PostgREST requests across multiple regions.

All projects are patched and this is now resolved. Thank you for your patience while we ensured all projects were updated.

The fix has been rolled out, and we are monitoring for continued stability. Any remaining impacted projects are being manually patched.

The permanent fix has now been rolled out across the fleet. We are continuing to manually patch any remaining impacted projects to restore service.

We’ve identified additional impacted projects and are currently working to manually patch and restore them. We are also continuing to roll out a fix across the fleet to prevent further impact.

All impacted projects with open support tickets have now been manually patched and restored. We are currently rolling out a fix across the fleet to prevent further impact. We will continue to monitor progress and provide updates.

Our team is actively working on a fix and is manually patching impacted projects to restore service as quickly as possible. If you are running a version of Postgres older than 15.1.1.57, please avoid restarting your project until this issue is resolved.

We’ve identified that a recent change to the Postgres systemd service has a dependency on a prestart script. This script is not present on some older projects, and when those projects restart, Postgres may fail to start successfully. This can result in projects remaining offline. We are working on a fix to restore service. If you are running a version of Postgres older than 15.1.1.57, please avoid restarting your project until this issue is resolved.

We’ve identified that a recent change to the Postgres systemd service has a dependency on a prestart script. This script is not present on some older projects, and when those projects restart, Postgres may fail to start successfully. This can result in projects remaining offline. We are working on a fix to restore service.

This incident has been resolved.

We believe all users affected by this particular issue have been resolved. The team is going to keep an eye on these error rates to ensure we catch any that didn't originally appear, or that no new issues arise. We appreciate your patience as we worked through this issue.

The team is continuing their mitigation efforts. We've fixed and restarted most of the affected projects. We're continuously looking for any others that are affected so we can be sure to get them all fixed.

The team is still working to bring these back online, and have a fix under way. Many users can also resolve this on their own via a project restart. This can be performed from the dashboard for your own projects at any time. But for those who are still seeing issues after a restart, we will be pushing a fix soon.

We have identified this issue across multiple regions, not just eu-west-3 as originally suspected. We are expanding the scope of efforts to bring affected projects back online. Users can also resolve this, in most cases, on their own via a project restart. This can be performed from the dashboard for your own projects at any time.

The team is continuing to work through affected projects; however, a project restart is also effective. This can be performed from the dashboard for your own projects at any time.

We are seeing an increase in projects unavailable in eu-west-3 following an upstream issue with EC2 instances in the region. The team is working on restoring access to these projects

This issue has now been resolved and projects have returned to normal.

Projects have recovered and capacity has been restored. All previously affected regions have been re-enabled, and we will continue to monitor to ensure stability.

Capacity has been freed, and the regions have been re-enabled. The team is currently working on resolving any failed project starts, resizes, restarts, or other configuration changes during this event.

US-East-2 and AP-Northeast-1 have been disabled for project creation and configuration change actions, and the team is currently working to free additional capacity in these regions.

We are seeing new project creation, resize requests, and project restart failures due to capacity issues in us-east-1. We are disabling project creation, project resize actions, and project restarts in these regions. We have already reached out to our provider for additional capacity, and will update here as we have additional information.

The issue has been resolved and DNS resolution is now operating normally.

A fix has been implemented, and we are seeing improvements in DNS resolution. We’re closely monitoring to ensure the issue is fully resolved and services remain stable.

We are currently experiencing an issue propagating new DNS records for our zone. We are working closely with our upstream network providers technical team to implement a fix and will share updates as we learn more.

We are currently investigating reports that newly created projects may be unreachable. Initial findings indicate DNS resolution failures. We will provide an update as soon as more information is available.

All metrics have been nominal and stable for several hours now, and our upstream provider has resolved their incident.

We've received confirmation from our networking provider that there is recovery across all regions. We are able to corroborate this with our own metrics collections. We will continue to monitor for a bit, but we believe this to be resolved at this time. We apologize for the trouble, and we appreciate your patience.

We've seen a marked improvement in error rates across all regions in the last few minutes. We are still waiting on confirmation from our provider on status, but we believe things to be improved at the moment. We'll keep working on this until we have for sure reached a resolution.

We recognize that some of our users, especially in parts of North and South America, are continuing to experience DNS and 5XX errors when trying to connect to their projects. We have escalated this issue with our network provider and will work with them continuously until the problems are resolved.

Some users continue to experience errors. We are raising the issue with our provider.

Our upstream provider has reported their incident resolved. We will continue to monitor for any errors our customers may be experiencing for another 30 minutes.

The fix that our upstream provider implemented has taken effect. Service has significantly improved across all previously impacted regions. There are still residual elevated error rates in a handful of regions and we will continue to work with our provider to eliminate them.

Our upstream provider has rolled the fix out to most regions. While we are seeing improvement, we are still seeing increased error rates in Mexico and Brazil and are continuing to work with them until this is fully resolved.

Upstream provider mitigations are still under way, and we are seeing improvements incrementally as routing regions have the mitigations applied. We are still seeing particular impact in South America, but some users in North America may still be affected as well. We anticipate continued incremental improvements and will update as we get more information.

Our upstream networking provider has identified a fix and is currently implementing a fix. We are seeing preliminary improvements to error rates for users in North America, but are still following this issue closely with our provider.

Our upstream networking partner is continuing their investigation. We will continue to post regular updates as we have more information.

Our upstream networking provider has discovered an issue and has declared an incident on their side. They are currently working toward resolution. We are actively working with them and will provide updates on progress as more information is available.

We continue to have active lines of investigation going with our upstream network provider partners, and are still working toward a resolution. This issue affects the network-level access to projects. The projects themselves and the data in them are safe and unaffected. Users are seeing two separate symptoms of these networking issues: 1. DNS Lookup failures for supabase project URLs 2. 530 responses to HTTP requests to supabase project API Endpoints

We continue to work with partners and investigate.

We have further clarified the scope and impact of the issues currently affecting users. A subset of users across North and South America are experiencing DNS resolution failures and HTTP 530 errors. These symptoms indicate a networking-related issue impacting the availability of Supabase projects. We are actively working with both internal teams and external network providers to isolate the root cause and determine the most effective path to resolution. We will continue to provide updates as more information becomes available.

We are actively investigating this issue and working with relevant partner organizations. At this time, we do not believe this issue is specific to any particular network provider. A subset of users continue to be impacted in both South and North America.

We are currently investigating reports of users experiencing login issues when using Supabase Auth. At this time, the impact appears to be limited to a subset of users, primarily in North and South America. The exact cause and scope are still being determined. Our team is actively working to identify the root of the issue and will provide updates as more information becomes available. We appreciate your patience while we investigate.

We are currently investigating reports of users experiencing login issues when using Supabase Auth. At this time, the impact appears to be limited to a subset of users, primarily in South America. The exact cause and scope are still being determined. Our team is actively working to identify the root of the issue and will provide updates as more information becomes available. We appreciate your patience while we investigate.

Capacity has been fully restored in the impacted APAC regions, and project creation is operating normally.

We are recovering capacity in the impacted APAC regions and are seeing improvements in project creation success rates. We will continue to monitor to ensure stability.

We are currently investigating an issue affecting project creation in some APAC regions. Users may experience failures when attempting to create new projects in impacted regions. Existing projects are not impacted. Our team is actively working to mitigate the issue and restore normal availability. We will share further updates as more information becomes available.

The issue causing increased error rates during plan upgrades has been resolved. All systems are operating normally.

A fix has been applied, and we are monitoring to ensure stability.

We have identified the cause of increased error rates affecting plan upgrades and are working on a fix. We will provide further updates as progress is made.

We are currently investigating increased error rates affecting plan upgrades. Our team is actively working to identify the root cause and restore normal functionality. We will provide updates as more information becomes available.

Log ingestion has recovered and the incident is considered resolved. Some log data was lost during the incident between around 11pm to 11:45pm ETC on March 31.

We have identified the issue and a fix is in place. We will continue to monitor to ensure stability.

We are seeing degradation of logs and are currently investigating the cause of this issue.

The issue has been resolved and Realtime services have returned to normal.

The fix has been implemented and we’re monitoring to ensure stability.

We appreciate your patience as we continue to work on a fix. We will provide updates as they become available.

We are working on a fix and we’ll continue to provide updates as progress is made.

We've identified the issue and are currently working on a fix, we will provide an update on our progress soon.

We're still investigating network connectivity issues with Realtime.

We are investigating network connectivity issues with Realtime.”

Project creation in all regions are now successful.

A fix has been implemented and we are monitoring the results.

We have identified the issue and a fix is underway.

We are currently investigating project creation failures across multiple regions.

Project creation recovery has been confirmed across all regions. Regional dashboards show error rates returned to normal, and previously failing project creation operations are succeeding again.

A fix has been implemented and we are seeing project creation recovery across regions. We're moving the incident to monitoring while we confirm full stability.

We have identified the root cause and are currently working on a fix.

We are currently investigating reports of projects failing to be created.

Error rates have returned to baseline and all regions are operating as expected.

We've seen error rates return to normal levels, and all regions are now holding steady. We will continue to monitor to make sure things remain stable.

We are continuing to see some periodic spikes in error rates; however, total rate of errors are decreasing across most regions. We're continuing efforts to get all systems back to green. When error rates spike, management API functions, which would mean dashboard actions, project creation, configuration changes, and other functions remain impacted. Existing projects are unaffected and connectivity via the Data APIs, Postgres connections, and Supavisor are unaffected.

The team is continuing to deploy mitigations, and we are seeing error rates improve, but are not back to baseline yet. All management API functions, which would mean dashboard actions, project creation, configuration changes, and other functions remain impacted. Existing projects are unaffected and connectivity via the Data APIs, Postgres connections, and Supavisor are unaffected.

We have identified the source of the increased error rates, and are working on a fix now. All management API functions, which would mean dashboard actions, project creation, configuration changes, and other functions remain impacted. Existing projects are unaffected and connectivity via the Data APIs, Postgres connections, and Supavisor are unaffected.

We've expanded this to all management API functions, which would mean dashboard actions, project creation, configuration changes, and other functions would also be impacted. Error rates are currently improving, but the team is investigating the root cause to ensure continued stability. Existing projects are unaffected and connectivity via the Data APIs, Postgres connections, and Supavisor are unaffected.

We are currently investigating the cause, but users may see issues creating projects across all regions. Existing projects are not affected and remain up and stable.

Both regions are now working as usual.

Response times in both the ap-northeast-1 (Tokyo) and ap-northeast-2 (Seoul) regions have now stabilised. We will continue to monitor the systems closely to ensure stability.

Response times in the ap-northeast-1 (Tokyo) region have improved following mitigation efforts. We are currently rolling out the fix to the ap-northeast-2 (Seoul) region. Further updates will be provided as the rollout continues.

We are continuing to investigate reports of Edge Functions experiencing high response times in Asia Pacific regions Our team is still working to determine the underlying cause. We understand the disruption this may be causing and are treating this with high priority.

We are continuing to investigate reports of Edge Functions experiencing high response times in Asia Pacific regions Our team is still working to determine the underlying cause. We understand the disruption this may be causing and are treating this with high priority.

We are currently investigating the cause for increased response times.

The issue affecting Edge Functions has now been fully resolved, and users have reported Edge Functions operating as expected. The root cause was a denial-of-service event that led to elevated error rates and 504 responses across impacted regions. Mitigations have been implemented to restore stability. We will be publishing a full post-mortem with additional details, once our internal review is complete. Thank you for your patience while we worked to resolve this issue.

The fix has been implemented and early indicators suggest it has resolved the 504 errors affecting Edge Functions. We are continuing to monitor system performance closely to ensure stability and confirm full recovery across all affected regions. We will provide a final update once we are confident the issue is fully resolved.

We believe we have identified the underlying cause of the 504 errors affecting Edge Functions across multiple regions. The team are currently implementing corrective measures to address the issue and restore service stability.

We are continuing to investigate reports of Edge Functions returning 504 errors across multiple regions. Our team is still working to determine the underlying cause. We understand the disruption this may be causing and are treating this with high priority.

We are currently investigating reports of Edge Functions returning 504 errors across multiple regions. The team has been alerted and is actively investigating the root cause. We are working to identify the scope of affected regions. We will provide an update as soon as we have more information.

Systems have remained stable, and we are confident the issue has been resolved.

Connection and query times are back to normal levels. We will continue monitoring to ensure stability.

The team is investigating connectivity issues via Supavisor in EU-Central-1. Data API, direct connections, and connections via the dedicated pooler are unaffected. You can read more on the dedicated pooler here: https://supabase.com/docs/guides/database/connecting-to-postgres#dedicated-pooler

Metrics collection continues to operate normally in US-West-2, and full metrics collection has been fully restored with no blips in stability.

The team has deployed additional capacity and has restored metrics collection operations. All metrics collection in region is fully operational and the team is actively monitoring the fixes deployed.

The team has deployed a fix and is actively monitoring recovery of partially degraded metrics collection. Some US-West-2 users may see metrics periodically available, but others may still see missing metrics. This does not impact project availability or functionality, which remain healthy.

The team is continuing to work on a fix for metrics collection. Some US-West-2 users may see metrics periodically available, but others may still see missing metrics. We'll continue to update here as we have more information. This does not impact project availability or functionality, which remain healthy.

We have identified the components resulting in the delayed metrics collection, and the team is working on a fix. This does not impact project availability or functionality, which remain healthy.

We have discovered an issue with metrics collection for projects in US-West-2. Some users may not be able to see metrics for these projects in the dashboard, and functions such as automatic disk resizes may be delayed as well. This does not impact project availability or functionality, which remain healthy. The team is currently investigating.