Scheduled snapshots not running on some deployments
Incident Report for Elastic Cloud
Resolved
The fix has been rolled out to all regions and verified. Thanks for your patience!
Posted Apr 27, 2020 - 20:40 UTC
Identified
We have confirmed that our fix works and are working on rolling it out to all production environments. Once that's complete and we've verified the fix, we'll post another update. We expect this to take approximately two hours.
Posted Apr 27, 2020 - 18:07 UTC
Update
We're still working on testing the fix in our staging environment and canary regions. We will have another update in one hour.
Posted Apr 27, 2020 - 17:21 UTC
Investigating
Regularly scheduled snapshots are not happening on approximately 2% of cloud deployments across all regions and cloud providers. Larger deployments are impacted. We have a fix in progress and will be rolling it to production after testing.

We will post an update in one hour.
Posted Apr 27, 2020 - 16:04 UTC
This incident affected: GCP Singapore (asia-southeast1) (Deployment snapshots: GCP asia-southeast1), Azure Netherlands (azure-westeurope) (Deployment snapshots: Azure azure-westeurope), GCP Mumbai (asia-south1) (Deployment snapshots: GCP asia-south1), GCP Oregon (us-west1) (Deployment snapshots: GCP us-west1), Azure Singapore (azure-southeastasia) (Deployment snapshots: Azure azure-southeastasia), AWS N. Virginia (us-east-1) (Deployment snapshots: AWS us-east-1), AWS Sydney (ap-southeast-2) (Deployment snapshots: AWS ap-southeast-2), Azure Washington (azure-westus2) (Deployment snapshots: Azure azure-westus2), GCP Belgium (europe-west1) (Deployment snapshots: GCP europe-west1), AWS N. California (us-west-1) (Deployment snapshots: AWS us-west-1), GCP Sao Paulo (southamerica-east1) (Deployment snapshots: GCP southamerica-east1), AWS Singapore (ap-southeast-1) (Deployment snapshots: AWS ap-southeast-1), GCP London (europe-west2) (Deployment snapshots: GCP europe-west2), AWS São Paulo (sa-east-1) (Deployment snapshots: AWS sa-east-1), AWS Tokyo (ap-northeast-1) (Deployment snapshots: AWS ap-northeast-1), GCP Montreal (northamerica-northeast1) (Deployment snapshots: GCP northamerica-northeast1), Azure Tokyo (azure-japaneast) (Deployment snapshots: Azure azure-japaneast), GCP Frankfurt (europe-west3) (Deployment snapshots: GCP europe-west3), GCP Iowa (us-central1) (Deployment snapshots: GCP us-central1), GCP Tokyo (asia-northeast1) (Deployment snapshots: GCP asia-northeast1), Azure London (azure-uksouth) (Deployment snapshots: Azure azure-uksouth), GCP N. Virginia (us-east4) (Deployment snapshots: GCP us-east4), Azure Virginia (azure-eastus2) (Deployment snapshots: Azure azure-eastus2), AWS Oregon (us-west-2) (Deployment snapshots: AWS us-west-2), GCP South Carolina (us-east1) (Deployment snapshots: GCP us-east1), GCP Sydney (australia-southeast1) (Deployment snapshots: GCP australia-southeast1), AWS Frankfurt (eu-central-1) (Deployment snapshots: AWS eu-central-1), AWS London (eu-west-2) (Deployment snapshots: AWS eu-west-2), and AWS Ireland (eu-west-1) (Deployment snapshots: AWS eu-west-1).