Azure Instability

Incident Report for ESS (Public)

Resolved

The Azure DNS instability incident has been resolved.
Posted Apr 02, 2021 - 00:34 UTC

Update

Upon further review of how provider DNS instability effected our platform, we identified that the level of impact for users will be determined by the behavior of the client. For example, clients that do not often attempt to re-resolve may not have observed any issues. However, clients that needed to re-establish connectivity to deployments may have seen a greater level of impact. We will continue to monitor until the provider status page resolves.
Posted Apr 01, 2021 - 22:58 UTC

Update

The platform has fully recovered, we'll continue to monitor until the provider status page clears.
Posted Apr 01, 2021 - 22:39 UTC

Monitoring

We're seeing healthy DNS resolutions and the platform is showing signs of recovery, we will continue monitoring to ensure a full recovery.
Posted Apr 01, 2021 - 22:26 UTC

Identified

Azure has identified issues with their DNS service https://status.azure.com. We're working to find out more from the upstream provider and are evaluating options for mitigation.
Posted Apr 01, 2021 - 22:11 UTC

Investigating

We've been alerted of connectivity issues within Azure regions, we're continuing to investigate and will engage the upstream provider for further details. Another update will be provided in 30 minutes or as we discover more details.
Posted Apr 01, 2021 - 21:47 UTC
This incident affected: Global services (Cloud console), Azure Singapore (azure-southeastasia) (Elasticsearch connectivity: Azure azure-southeastasia, Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-southeastasia, Deployment snapshots: Azure azure-southeastasia, Deployment hosts: Azure azure-southeastasia, APM connectivity: Azure azure-southeastasia, Kibana connectivity: Azure azure-southeastasia), Azure Netherlands (azure-westeurope) (Azure Infrastructure health: azure-westeurope, Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-westeurope, APM connectivity: Azure azure-westeurope, Elasticsearch connectivity: Azure azure-westeurope, Kibana connectivity: Azure azure-westeurope), Azure Washington (azure-westus2) (Elasticsearch connectivity: Azure azure-westus2, Kibana connectivity: Azure azure-westus2, Azure Infrastructure health: azure-westus2, APM connectivity: Azure azure-westus2, Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-westus2), Azure Iowa (azure-centralus) (Azure Infrastructure health: azure-centralus, Kibana connectivity: Azure azure-centralus, Elasticsearch connectivity: Azure azure-centralus, APM connectivity: Azure azure-centralus, Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-centralus), Azure New South Wales (azure-australiaeast) (Elasticsearch connectivity: Azure azure-australiaeast, APM connectivity: Azure azure-australiaeast, Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-australiaeast, Kibana connectivity: Azure azure-australiaeast, Azure Infrastructure health: azure-australiaeast), Azure Virginia (azure-eastus2) (Elasticsearch connectivity: Azure azure-eastus2, Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-eastus2, APM connectivity: Azure azure-eastus2, Azure Infrastructure health: azure-eastus2, Kibana connectivity: Azure azure-eastus2), Azure Tokyo (azure-japaneast) (Elasticsearch connectivity: Azure azure-japaneast, Kibana connectivity: Azure azure-japaneast, Azure Infrastructure health: azure-japaneast, APM connectivity: Azure azure-japaneast, Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-japaneast), and Azure London (azure-uksouth) (Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-uksouth, APM connectivity: Azure azure-uksouth, Kibana connectivity: Azure azure-uksouth, Azure Infrastructure health: azure-uksouth, Elasticsearch connectivity: Azure azure-uksouth).