Fly.io Outage History
Past incidents and downtime events
Complete history of Fly.io outages, incidents, and service disruptions. Showing 50 most recent incidents.
March 2026(15 incidents)
Machines failing to start in DFW
3 updates
In addition to freeing up existing capacity, the team has provisioned new capacity in DFW and we are monitoring the results.
We freed up some capacity on our workers to allow for successful Machine starts.
The Machines start failure rate is elevated in DFW.
Metrics currently experiencing issues
3 updates
This incident has been resolved. We're unable to recover the lost metrics from that one hour.
We have implemented a fix. There has been approximately 1h of lost metrics from 06:07UTC. We're monitoring the cluster for further issues
We are currently investigating an issue with our metrics cluster.
Machines failing to start in DFW
4 updates
This incident has been resolved. Machine creates in DFW continue to work normally.
A fix has been implemented and we are monitoring the results.
The team is currently rolling out additional capacity in DFW which should help ease Machine start failures across the region.
We are investigating reports of machines failing to start in the DFW (Dallas) region with "insufficient memory" errors. This may cause deployment failures for applications running in DFW. Our team is actively working to restore full capacity in the region. If you are affected, deploying to an alternate region may serve as a temporary workaround. We will provide updates as the situation progresses.
IPv6 networking issues in SJC region
3 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating intermittent network issues in SJC region impacting outbound public IPv6 access from Machines. Connecting to IPv6 internet resources from apps hosted in SJC region may be slow or fail at this time. IPv4 access, as well as 6PN private networking, are unaffected.
Connection Issues in SJC
2 updates
This incident has been resolved.
Between 13:55 and 14:03 UTC machines and MPG clusters hosted in the SJC region saw elevated connection errors. Users may have seen errors connecting to or from most machines in the region, as well as with deployments or updates to machines in the region. Networking has returned to normal in the region, and we are continuing to monitor closely to ensure stable recovery.
Fly ssh console command failing
3 updates
This incident has been resolved.
A fix has been implemented and we are seeing `ssh console` commands succeed as normal.
We have identified an issue causing new `fly ssh console` connections to fail with 500 errors. A fix is in progress.
Sprites Operations: 401 errors for certain organizations
2 updates
This incident has been resolved.
Organizations with names prefixed with numerical digits may experience 401 errors. Affected operations include actions such as Sprite creation, listing, etc... A fix has been implemented since 2026-03-14 12:30 UTC and we are monitoring the results!
Setting secrets and creating apps is degraded
4 updates
This incident has been resolved.
While the secret storage service was in a read-only state, app creation requests queued up, due to the retry logic and insufficient request concurrency limits in our GraphQL API. This prevented our GraphQL API from serving any other requests. We have scaled up the GraphQL API and are continuing to monitor the situation.
A fix has been implemented and we are monitoring the results.
An ongoing data migration in our secret storage service is causing degraded Machines API functionality.
Private networking issues in SYD region
3 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating a private networking failure between SYD and other regions. Apps continue to run, and private networking within SYD is unaffected.
Routing issues in NA regions
3 updates
This incident has been resolved. Due to a BGP issue, we saw some North American traffic routed to edges in Singapore (sin). Users in North America would have seen additional request latency during this period.
A fix has been implemented and we are monitoring the results.
We're aware of routing issues affecting some customers in North America regions, and we're actively investigating.
Elevated GraphQL API errors
3 updates
This incident was caused by a failed Redis node that powers our GraphQL API. We were able to recreate the Redis node and restore service. We are still investigating the root cause of the failure. In the mean time, all API endpoints now appear to be stable and errors have dropped to baseline level.
A fix has been implemented and we are monitoring the results.
We're investigating elevated GraphQL errors that affect some API endpoints.
Cost Explorer fails to load
2 updates
This incident has been resolved.
We are currently investigating this issue. The page currently displays: "We’re having trouble loading the cost breakdown."
Certificates issues affecting API and proxy
1 update
Between 19:54 and 20:06 UTC, our Vault cluster serving app certificates was unavailable. This caused various API requests to fail, mainly operations on certificates but also app creates and IP assignments. As the failure mode was Vault requests hanging rather than failing immediately, TLS requests through fly-proxy for domains where the certificate was not cached on the local node remained open for a long time while proxy attempted to fetch the certificate; this caused some connections to fail as too many connection slots were taken up by requests waiting on Vault. The root cause of this incident was a partially completed update to the Vault cluster. We will be implementing safeguards in the proxy for this failure mode, as well as improving certificate storage longer-term.
Machines failing to boot in EWR
4 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Issues with the Machines API
4 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're currently investigating issues with the Machines API. Customer deployments and the Fly dashboard may be affected.
February 2026(28 incidents)
Slow API requests
9 updates
This incident has been resolved. All platform and API operations are working normally.
API and platform operations have normalized. We are continuing to monitor to ensure full and stable recovery. Background jobs are almost fully caught up. Users may still see slightly slower requests creating new apps / orgs, but they should complete successfully. Sprite and MPG cluster creations are processing as normal.
A second fix has been deployed and database load has returned to normal, resulting in API response times beginning to normalize. Most Machines API requests should succeed as normal, and deploys to existing apps should also work. We are working through a backlog of background jobs. New app / organization creations and other other operations that use these will continue to see increased latency or failures while we work thorough these. New MPG cluster and new Sprite creation continues to be impacted.
An initial fix has been deployed and we are seeing improvements in load and API performance. Some operations that rely on the Graphql API, such as new app creations and some deployments, will continue to fail at this time. We are continuing to work on restoring full availability.
We are currently seeing full API failures for requests to our Graphql API and elevated failures for the machines API. Direct calls to these apis may fail, along with many flyctl commands. We have identified the cause of the issue and are continuing to work on a fix. Existing running machines and apps should continue to be reachable, but creates, deploys, or other features relying on platform API calls will fail at this time.
New Sprite creations are also timing out or failing at this time. We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
We have identified the cause of the increased latency and are working on a fix. The most common errors we are seeing is timeouts when users attempt to perform an action against a newly created app / machine resource. Those may timeout or fail with an `app|machine not found` error
We are investigating increased in API request latency and timeouts with the main platform API. This is impacting multiple operations, including creating, querying or performing actions against machines, as well as platform level operations like adding payment methods.
Capacity issues in iad and dfw
3 updates
This incident has been resolved.
We have provisioned additional capacity in dfw and iad and are monitoring to ensure machine and builder starts are succeeding consistently.
These regions (Dallas, TX dfw and Ashburn, VA iad) are currently low on capacity. New machine creates in these regions might fail temporarily, and Depot builders may be unavailable, causing deploys to hang in "Waiting for Depot builder". If you are having issues with Depot builders, consider moving them to a different non-iad, non-dfw region in your fly.io dashboard's "Settings" page under "App builders", or try `--depot=false`.
Capacity isssues in iad and dfw
6 updates
This incident has been resolved.
We're continuing to monitor after having added more capacity to our DFW and IAD regions. Deploys or machine starts using existing volumes in these regions may still hit a capacity issue. Users should use `fly volume fork --vm-memory ` to fork the volume to a host with more capacity, then retry the deploy or start command using the new volume.
We have added additional capacity in DFW and IAD regions and are monitoring the impact. New machine creates and deploys without volumes are seeing improved success rates. Deploys using depot builders in those regions are also improving, with much quicker builder start times. Deploys or machine starts using existing volumes in these regions may still hit a capacity issue. Users should use `fly volume fork --vm-memory ` to fork the volume to a host with more capacity, then retry the deploy or start command using the new volume.
We've identified some newly created Managed Postgres clusters are failing to come up healthy in these regions.
New machine creates in these regions might fail temporarily, and Depot builders may be unavailable. If you are having issues with Depot builders, consider moving them to a different region, or try `--depot=false`.
We have identified the problem and are working on a fix.
Sprites API degradation
3 updates
This incident has been resolved.
A slow deploy is causing Sprites API degradation. We are implementing a fix.
A slow deploy is causing Sprites API degradation. We are implementing a fix.
Metrics are degraded
5 updates
Metrics processing has caught up, and we don't see any data loss.
Delayed metrics are still being processed.
Metrics are coming back online, but it will take a little time to process what's backed up in the queues.
We're continuing to work with VictoriaMetrics support on a fix for this issue.
In some cases data is missing or lagging. We've identified the problem and are working on a fix.
Sprite creations failing
3 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating issues creating new Sprites.
Degraded Managed Postgres Control Plane
2 updates
This incident has been resolved as of 20:30 UTC.
We are currently investigating issues with the MPG control plane. Users may experience delays or hanging when creating or deleting databases via the dashboard or CLI.
Deploys hanging at waiting for Depot Builder
5 updates
This incident has been resolved.
The fix has been rolled out and we are seeing deploys using depot builder succeeding normally. We continue to monitor to ensure full recovery. Depot builders have been reenabled as the default option for new deploys
A fix is being rolled out. Fly builders continue to be the default while this is deployed
We are again seeing elevated latency provisioning depot builders on new deploys. Users may see deploys using Depot builders hang or timeout at the "Waiting for Depot Builder" step. We are working on a fix. We are switching all deploys to use the default Fly builders in the meantime. If desired users can manually switch back to depot builders using `fly deploy --depot=true` but may continue to see latency issues at this time.
We have seen elevated latency provisioning Depot builders during deployments over the past hour. This caused some deploys to hang or timeout at the "Waiting for Depot Builder" step in this period. Latency has improved and builder provision times are back to normal. We're continuing to monitor to ensure latency remains normal.
Networking issues for users connecting through lhr
3 updates
Network traffic in LHR has been stable for some time now, we are not seeing any further issues.
A fix has been implemented and we are monitoring the results.
We’re currently investigating this issue.
Investigating registry issues affecting deploys
5 updates
This incident has been resolved.
While we have seen some improvement from the previous fix, we are still seeing elevated rates of Registry connection issues. Users may continue to see slower machine creates and deploys due to slow image pulls. Deploys may succeed on a retry. We are continuing to work on restoring normal registry performance
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Control plane state delayed on some hosts possibly causing network or deployment disruption
4 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
flyctl deploy timeouts
3 updates
Earlier today, an issue caused elevated rate limiting and some deployment timeouts. A fix is in place and deployments are back to normal.
A fix has been implemented and we are monitoring the results.
We’re investigating elevated 429 errors from flaps causing deployment timeouts. Affected deploys are failing with: ✖ Failed: error waiting for release_command machine XX to finish running: timeout reached waiting for machine's state to change Your machine never reached the state "destroyed".
Degraded Managed Postgres Control Plane in ORD
5 updates
This incident has been resolved.
A fix has been implemented and we are seeing full recovery of the control plane in ORD. With that recovery we are seeing impacted replicas catching up and clusters returning to normal health. We're continuing to monitor for full recovery.
We are continuing to work on a fix for this issue.
The issue has been identified and we are working on a fix. The majority of MPG clusters in ORD continue to run normally, though some users may still see degraded replicas at this time. Some clusters in the region will have experienced a primary -> replica failover.
We are currently investigating issues with the MPG control plane in ORD. A small number of clusters in the region may be seeing replication lag or PGBouncers connectivity issues at this time.
Issues with deploying apps using Depot builders for new accounts
4 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Some new Fly.io users may encounter an "upgrade your organization" error message when attempting to deploy apps for the first time. We're currently working with Depot to figure out what's causing the issue. In the meantime, you should be able to work around the issue by using Fly builders with `fly deploy --depot=false`.
Creating new sprites is degraded
6 updates
This incident has been resolved.
Sprite creation appears to be back to normal operation now.
We've identified the cause of the delay following creates and we're deploying a fix.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
Sprite creation generates an error that the sprite "is not assigned to compute." Eventually the sprite transitions from an unknown state to warm, so there is a delay before the sprite is usable.
Degraded MPG clusters in IAD
5 updates
This incident has been resolved.
We've rolled out a fix for the remaining impacted clusters, and we're now monitoring the results.
We've rolled out a fix for some additional impacted clusters, and we're continuing to work on the remaining clusters.
We've identified the issue - some MPG clusters in IAD should be seeing improvements, and we're working on rolling out a fix for the remaining impacted clusters.
We're currently looking into an issue with MPG clusters in the IAD region.
Issue creating new Sprites in IAD
4 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're currently looking into an issue that's preventing new Sprites from being created in IAD. Sprite creation from other regions are unaffected.
Degraded network in AMS
6 updates
This incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are still working on restoring the MPG clusters. Most of them should be operational already.
Affected hosts are starting to come back online. We are working on restoring affected MPG clusters.
One of our upstream providers is experiencing a major power issue in their AMS datacenter. Managed Postgres instances in AMS are experiencing an outage as our control plane for Managed Postgres is taken down by the incident.
One of our upstream providers is performing an emergency DC maintenance. You may see degraded connectivity on some of your apps in AMS. Most apps in AMS are not affected.
Machines API issues
4 updates
This incident has been resolved.
A fix has been implemented and we are seeing Machines API connectivity improve in APAC regions. We continue monitoring for full recovery.
The issue has been identified and we are seeing Machines API performance improve in most regions since ~16:20 UTC. Machines API calls in the SYD, NRT, SIN region may continue to see 5xx errors or higher latency at this time. We are continuing to work on restoring full API performance in all regions
We are investigating widespread Machines API issues since 16:00 UTC. You may experience 5xx errors or higher latency at this time.
Private Networking and Certificate Resolution Issues in SYD
5 updates
This incident has been resolved.
A fix has been implemented and we are seeing private networking / certificates in SYD improving. We are continuing to monitor for full recovery.
Private Networking (6PN) is degraded in SYD region. Communication between Machines in SYD region and Machines in other regions may fail at this time. Newly created Machines in SYD may fail to sync to other regions (may not show up in Machines API List endpoint, or state may be incorrect). We are working with our upstream providers to resolve this issue.
We are continuing to investigate this issue.
We are investigating communication issues between some SYD (Sydney, Australia) region hosts and our Certificate vault. Requests hitting SYD region edges may see issues resolving certificates at this time, especially for newly issued or not recently used certificates.
Network issues on newly-created machines
3 updates
This issue is now resolved.
We have successfully run a fix to re-sync our global and regional state stores in order to bring machines back to a healthy state, and we're monitoring the situation to confirm that there are no more issues.
Machines created after the delayed machine registration incident (https://status.flyio.net/incidents/3npj6935byt4) may have incomplete networking configurations and could be unable to receive traffic. We've identified the issue and are deploying code fixes as well as updating created machines. This affects a small number of machines on all our regions.
MPG Degraded clusters in AMS, IAD and SIN regions
8 updates
All MPG clusters are back to full, normal operations.
All MPG clusters are reachable.
We are still continuing cleanup on some clusters.
All cluster primary and pgBouncer machines are now healthy and operating normally. We are still continuing cleanup on some clusters with lagging or degraded replicas, but this should not impact writes or reads to clusters.
We are continuing to work on restoring all clusters to full health.
With the underlying incident stabilizing (https://status.flyio.net/incidents/3npj6935byt4) we are seeing improvements amongst impacted clusters. We continue to work on restoring all clusters to full health.
A number of clusters in IAD, AMS, and SIN regions continue to see degraded replicas and PGBouncers at this time. A smaller number of clusters in these regions are also seeing disruption to their primaries. We continue to work on restoring full cluster health in all regions.
A small number of MPG clusters in the AMS and IAD region are currently in degraded states due to downstream impact from this Machines API issue: https://status.flyio.net/incidents/3npj6935byt4 Most of the impacted clusters may see a degraded replica or PG Bouncer in their statuspage. A very small number may be unable to connect to their MPG primary node, the team is working to restore connectivity as the top priority. Users may also see delays registering new clusters in these regions at this time.
Delayed Machine Registration + Token Errors
7 updates
This incident has been resolved.
A fix has been deployed across all impacted hosts. We are seeing a sharp reduction in Token errors since 20:00 UTC and other metrics are recovering as well. We are continuing to monitor closely
We saw some improvement from the previous fix, however errors remained elevated on some hosts. We have identified the root cause of the remaining errors as a communication issue between the hosts and our Token database. We are preparing a fix that should resolve these.
We have rolled out an initial fix for the token issues and are monitoring for improvements.
While Machine registration error rates have improved, we are now seeing elevated error rates verifying user tokens during some actions. Users may see errors like "failed to launch VM: permission_denied: bolt token: failed to verify service token: no verified tokens" when deploying or creating machines. We are investigating
A fix has been rolled out and most hosts are registering machines as normal. A few hosts remain with elevated error rates, we are continuing to fix these. Users who experience an error creating or deploying a new machine should re-try the operation.
We have identified elevated error rates registering new machines with our global state tracking service on some hosts. We have identified the issue and are deploying a fix. Users may have seen elevated machine create, start, or deployment failures over the past ~20 minutes.
Network maintenance in YYZ
3 updates
Network maintenance has concluded.
Managed Postgres clusters in YYZ should be operating normally.
An upstream network provider is performing an emergency network maintenance in the YYZ region. Machines in YYZ may see some packet loss. Managed Postgres clusters in YYZ are experiencing management plane issues. Clusters may see delayed fail-overs and changes in cluster size may not be possible during the maintenance period.
IPv6 Issues in YYZ
3 updates
This incident has been resolved.
A fix has been implemented and we're seeing IPv6 networking return to normal in YYZ. We'll continue to monitor to ensure full recovery.
We are currently investigating degraded IPv6 networking in the YYZ (Toronto) region. Users with machines in this region may see issues connecting to their machines over IPv6. Users with static egress IPs may see issues connecting outbound over IPv6 from this region at this time. IPv4 is not impacted and continues to work normally.
Elevated latency and packetloss in North American regions
3 updates
This incident has been resolved.
Network performance issues between North American regions have resolved and we're continuing to monitor.
We are currently investigating intermittent spikes of increased latency and packet loss between North American regions over the past hour. Users may see degraded network performance on traffic in and out of the IAD and SJC regions at this time. We are working with our upstream networking providers to investigate and mitigate these issues.
Congestion in CDG and FRA
2 updates
This incident has been resolved.
We are experiencing elevated weekend congestion in CDG (France) and FRA (Germany).
Sprites are returning not found or unauthorized when they shouldn't be.
6 updates
This incident has been resolved.
We've been able to restore missing sprites and tokens. We're monitoring for any additional issues.
We're working on a fix to restore missing sprites and tokens.
We identified the source of the problem as an upstream DNS issue Tigris experienced, now resolved. We're currently assessing the impact on Sprites.
We are continuing to investigate this issue.
We're currently investigating this issue.
January 2026(7 incidents)
Grafana Log Search Display Issue
3 updates
This has been resolved. If you are still experiencing any issues, you may need to log out and then back in.
No logs are displayed in Grafana Log Search when using the default `*` query. You can try the following workarounds: 1. Replace the default `*`query with `NOT ""` 2. Viewing logs from the “fly app” tab or the “explore” tab, Thank you for your kind understanding as we work through resolving this!
No logs are displayed in Grafana Log Search when using the default `*` query. As a temporary workaround, please replace `*` with `NOT ""` query. Thank you for your kind understanding as we work through resolving this!
Delayed metric reporting in NRT and SIN regions
5 updates
This incident has been resolved. All hosts in SIN and NRT are reporting up to date metrics.
Currently one host in SIN is still finishing working through it's metrics backlog and is reporting delayed metrics. Other hosts in NRT and SIN are reporting metrics correctly. If needed, users with impacted machines on the remaining host can use `fly machine clone` to create new machines in the region, which should land on a different host.
Most hosts in NRT and SIN have completed backfilling their metrics and are up to date in fly-metrics.net. Four hosts are still working through the backlog; machines on those hosts are still reporting delayed metrics at this time.
We are continuing to process the metrics backlog in NRT and SIN. Progress is being made, but due to the volume of metrics this may still take some time to fully complete. At this time users with machines on impacted hosts will see metrics beginning to backfill into fly-metrics.net. However many will not be fully caught up yet. This impacts metrics only, the underlying machines continue to work normally.
A small number of hosts in the NRT (Tokyo) and SIN (Singapore) are reporting delayed metrics to the hosted Grafana charts at fly-metrics.net. Users with machines on impacted hosts will see delayed or spotty metrics in their Grafana charts. Only metrics for these machines are impacted. The underlying machines continue to receive and serve traffic as usual, and all machine actions(stopping, starting, deploys etc.) continue to work normally. We are processing the backlog of metrics on these hosts, but metrics will be delayed until this is complete.
Congestion in CDG and FRA
1 update
We are experiencing elevated weekend congestion in CDG (France) and FRA (Germany).
Delays issuing certificates
3 updates
This incident has been resolved.
We have identified the congestion and released a fix, we'll continue to monitor while the jobs catch up.
We are currently investigating possible delays issuing ACME certificates for new hostnames.
Errors creating new Sprites
3 updates
This incident has been resolved.
We are continuing to investigate this issue.
We're currently investigating an issue that's preventing new Sprites from being created.
MPG network instability in LAX
3 updates
This incident has been resolved.
Connections are back to normal. We'll keep monitoring the region.
We identified network partitions in the LAX region. We are investigating the problem.
Machines errors in JNB region
2 updates
This incident has been resolved.
A bad deploy of an internal service in JNB region may cause Machines API requests for JNB region machines to fail. At this time, it may not be possible to create or update machines in JNB region, but apps continue to run. The deploy is being reverted.