PlanetScale Outage History

The incident has been resolved.

Mitigations are in place and we're seeing signs of recovery. We continue to monitor the health of the system.

Insights metrics are underreporting for Postgres and Vitess databases. Database traffic and connectivity are unaffected. We are actively investigating the issue and are deploying mitigations.

Connection issues have been resolved. If you experience any further issues, please contact our support team.

We remediated the issue, and clients can now connect to their database branches in us-east-1 normally. We will continue monitoring the situation.

We are observing connectivity issues on Postgres in the us-east-1 region. We are remediating.

API and dashboard service is restored.

We have increased the memory available to the API and are monitoring the performance and stability of the API and dashboard to confirm service is restored.

We have identified the cause of the errors - a pattern of requests causing the application to run out of memory and crash - and are working to better handle this traffic pattern now.

We are aware that both our API and dashboard are returning errors. We are investigating the cause and will restore service as soon as possible.

The changes we applied are effectively preventing further unexpected restarts to the high availability component shared by Postgres clusters. We have not seen further restarts to this component nor unplanned failovers in Postgres clusters since applying these changes.

We applied changes to prevent unexpected restarts of the high availability component shared by Postgres clusters. We have not seen further restarts of this component nor unplanned Postgres failovers since applying these changes. We are actively monitoring the situation.

We have identified the likely cause of unexpected restarts to the high availability component used by Postgres clusters, and are applying changes to prevent recurrences.

A high availability component that Postgres clusters depend on is restarting unexpectedly, triggering unplanned failovers in Postgres clusters.

This incident has been resolved.

We're continuing to monitor. We've updated the status history to reflect the time periods where customers experienced connection issues.

At **12:51 PM UTC** and **1:38 PM UTC** some Vitess databases in us-east-1 were impacted by connection errors for approximately 5 minutes due to an issue in our edge network. We resolved the issue and are no longer seeing errors. We will be taking action to protect customers from this issue happening again in the future and apologize for the error.

We're investigating an issue with connection errors to Vitess databases in us-east-1.

Vitess branch creation is available once again.

We're rolling out a mitigation now that will allow Vitess branch creation to succeed while we continue to investigate. No database operations or anything about Postgres database creation are impacted. Next update by 12:00 UTC.

New Vitess databases are being accepted by the PlanetScale API but we are unable to provision the infrastructure to serve them. We've confirmed this impact and are investigating the cause. We will provide further updates by 11:00 UTC.

Problem has been resolved. We saw three nodes fail across three availability zones simultaneously. Impacting databases hosted on those nodes. All databases have been moved to healthy nodes and we're investigating how we can further reduce impact from similar events in the future.

Impacted databases are recovering, we're continuing to monitor.

We're working on a fix for the impacted databases.

We are currently investigating impact.

The issue has been resolved.

A fix has been applied, and we are monitoring recovery.

We are investigating internal errors on some API endpoints. Database connectivity is unaffected.

This incident is resolved.

We've deployed a fix and are seeing error rates resolve.

We're continuing to work on a fix and will share an update once it has been deployed.

We're aware of an issue impacting new branch creation, deploy requests and cluster size changes for Vitess databases deployed to GCP. We are working on a fix.

We've made a fix and the errors have resolved.

The dashboard and API are currently returning 500 errors. We are rolling out a fix.

No further disruption is expected, please contact support with any further issues.

All databases have recovered, but we are continuing to watch the situation closely.

We are continuing to investigate

PlanetScale experienced the failure of VMs across multiple availability zones, causing a small number of branches to need rescheduling before recovery was possible. We expect all branches in this state have recovered, and we are continuing to monitor for further failures.

We are continuing to investigate and recover.

A small number of databases in GCP us-central1 are stuck with "primary is not serving, there may be a reparent operation in progress".

We've resolved the issue. SSO/SAML is working again.

We've identified a potential fix, testing, and the preparing to roll it out.

We are continuing to investigate. We've confirmed an issue with our SSO integration, but are still searching for a root cause.

PlanetScale is aware of issues logging in via SSO/SAML. We are currently investigating.

All services operational.

All clusters are online and serving traffic. We are monitoring out of caution before returning to green status.

Most affected clusters are now online. The last few are coming up now.

Clusters are returning to an online state.

We've identified the issue with an isolated set of Postgres databases. The clusters are currently in the process of coming back online.

We are investigating the unavailability of a small about of databases in us-east-1