Potential orchestration disruption in Azure regions

Incident Report for ESS (Public)

Resolved

Mitigation has been completed and all affected Azure regions are operational.
Posted Nov 24, 2021 - 17:58 UTC

Monitoring

A fix has been applied to the affected hosts and the network configuration has been confirmed as functional.
We are monitoring the situation.

Full orchestration of existing deployments, and the creation of new ones, should now be available in affected regions.
Posted Nov 24, 2021 - 13:43 UTC

Identified

We have identified a network configuration error that has impacted a number of our hosts. Work is underway to rectify this error, and determine the cause.

Further updates will follow within the next 30mins
Posted Nov 24, 2021 - 13:02 UTC

Investigating

We are investigating some potential issues with the orchestration and deployment of hosted clusters in Azure regions. This may impact the ability of customers to deploy certain versions, or upgrade their existing deployment.

Access to existing hosted deployments is not affected.

Updates will be given in the next 30mins
Posted Nov 24, 2021 - 12:28 UTC
This incident affected: Azure London (azure-uksouth) (Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-uksouth), Azure Virginia (azure-eastus2) (Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-eastus2), and Azure Netherlands (azure-westeurope) (Deployment orchestration (Create/Edit/Restart/Delete): Azure azure-westeurope).