Resolved
This incident has been resolved.
Posted Aug 10, 2021 - 06:22 UTC
Update
We’ve tested out our remediation process and have begun running it against the full fleet, which will take some time to complete successfully. We will update this incident once we have completed the remediation across all regions.
In the meanwhile, if you have a stateless application (Kibana, APM, Fleets, and Enterprise Search), force restarting any unhealthy application will work around this issue.
Posted Aug 09, 2021 - 15:53 UTC
Update
We're testing our remediation process and are running against canary instances; once validated, we'll run against the full fleet. We'll update again when remediation is complete.
Posted Aug 09, 2021 - 13:58 UTC
Identified
We are currently experiencing network connectivity issues in a small percentage of customers' stack components. This does not impact Elasticsearch (including ML) connectivity but may impact Kibana, APM, Fleets, and Enterprise Search. We've determined the issue and are working on a remediation process. We'll update again in an hour.
Posted Aug 09, 2021 - 13:22 UTC
This incident affected: GCP Sydney (australia-southeast1) (Kibana connectivity: GCP australia-southeast1, APM connectivity: GCP australia-southeast1), GCP São Paulo (southamerica-east1) (APM connectivity: GCP southamerica-east1, Kibana connectivity: GCP southamerica-east1), GCP South Carolina (us-east1) (APM connectivity: GCP us-east1, Kibana connectivity: GCP us-east1), Azure Ireland (azure-northeurope) (APM connectivity: Azure azure-northeurope, Kibana connectivity: Azure azure-northeurope), GCP Tokyo (asia-northeast1) (Kibana connectivity: GCP asia-northeast1, APM connectivity: GCP asia-northeast1), GCP N. Virginia (us-east4) (APM connectivity: GCP us-east4, Kibana connectivity: GCP us-east4), Azure Washington (azure-westus2) (Kibana connectivity: Azure azure-westus2, APM connectivity: Azure azure-westus2), Azure New South Wales (azure-australiaeast) (APM connectivity: Azure azure-australiaeast, Kibana connectivity: Azure azure-australiaeast), GCP Seoul (asia-northeast3) (Kibana connectivity: GCP asia-northeast3, APM connectivity: GCP asia-northeast3), AWS Frankfurt (eu-central-1) (APM connectivity: AWS eu-central-1, Kibana connectivity: AWS eu-central-1), GCP London (europe-west2) (APM connectivity: GCP europe-west2, Kibana connectivity: GCP europe-west2), GCP Montreal (northamerica-northeast1) (APM connectivity: GCP northamerica-northeast1, Kibana connectivity: GCP northamerica-northeast1), Azure Tokyo (azure-japaneast) (Kibana connectivity: Azure azure-japaneast, APM connectivity: Azure azure-japaneast), GCP Singapore (asia-southeast1) (Kibana connectivity: GCP asia-southeast1, APM connectivity: GCP asia-southeast1), GCP Taiwan (asia-east1) (APM connectivity: GCP asia-east1, Kibana connectivity: GCP asia-east1), GCP Finland (europe-north1) (Kibana connectivity: GCP europe-north1, APM connectivity: GCP europe-north1), GCP Netherlands (europe-west4) (APM connectivity: GCP europe-west4, Kibana connectivity: GCP europe-west4), Azure Iowa (azure-centralus) (Kibana connectivity: Azure azure-centralus, APM connectivity: Azure azure-centralus), GCP Mumbai (asia-south1) (Kibana connectivity: GCP asia-south1, APM connectivity: GCP asia-south1), Azure London (azure-uksouth) (APM connectivity: Azure azure-uksouth, Kibana connectivity: Azure azure-uksouth), AWS Seoul (ap-northeast-2) (APM connectivity: AWS ap-northeast-2, Kibana connectivity: AWS ap-northeast-2), AWS Paris (eu-west-3) (Kibana connectivity: AWS eu-west-3, APM connectivity: AWS eu-west-3), AWS Mumbai (ap-south-1) (APM connectivity: AWS ap-south-1, Kibana connectivity: AWS ap-south-1), Azure Virginia (azure-eastus) (Kibana connectivity: Azure azure-eastus, APM connectivity: Azure azure-eastus), Azure Texas (azure-southcentralus) (Kibana connectivity: Azure azure-southcentralus, APM connectivity: Azure azure-southcentralus), Azure Virginia (azure-eastus2) (APM connectivity: Azure azure-eastus2, Kibana connectivity: Azure azure-eastus2), Azure Netherlands (azure-westeurope) (APM connectivity: Azure azure-westeurope, Kibana connectivity: Azure azure-westeurope), AWS Ohio (us-east-2) (APM connectivity: AWS us-east-2, Kibana connectivity: AWS us-east-2), Azure Singapore (azure-southeastasia) (APM connectivity: Azure azure-southeastasia, Kibana connectivity: Azure azure-southeastasia), AWS Hong Kong (ap-east-1) (APM connectivity: AWS ap-east-1, Kibana connectivity: AWS ap-east-1), Azure France (azure-francecentral) (Kibana connectivity: Azure azure-francecentral, APM connectivity: Azure azure-francecentral), GCP Belgium (europe-west1) (Kibana connectivity: GCP europe-west1, APM connectivity: GCP europe-west1), GCP Frankfurt (europe-west3) (Kibana connectivity: GCP europe-west3, APM connectivity: GCP europe-west3), AWS Tokyo (ap-northeast-1) (Kibana connectivity: AWS ap-northeast-1, APM connectivity: AWS ap-northeast-1), AWS Central (ca-central-1) (Kibana connectivity: AWS ca-central-1, APM connectivity: AWS ca-central-1), GCP Iowa (us-central1) (Kibana connectivity: GCP us-central1, APM connectivity: GCP us-central1), GCP Oregon (us-west1) (Kibana connectivity: GCP us-west1, APM connectivity: GCP us-west1), AWS São Paulo (sa-east-1) (Kibana connectivity: AWS sa-east-1, APM connectivity: AWS sa-east-1), AWS N. California (us-west-1) (Kibana connectivity: AWS us-west-1, APM connectivity: AWS us-west-1), AWS Oregon (us-west-2) (Kibana connectivity: AWS us-west-2, APM connectivity: AWS us-west-2), AWS N. Virginia (us-east-1) (Kibana connectivity: AWS us-east-1, APM connectivity: AWS us-east-1), AWS Ireland (eu-west-1) (Kibana connectivity: AWS eu-west-1, APM connectivity: AWS eu-west-1), AWS Singapore (ap-southeast-1) (Kibana connectivity: AWS ap-southeast-1, APM connectivity: AWS ap-southeast-1), AWS London (eu-west-2) (Kibana connectivity: AWS eu-west-2, APM connectivity: AWS eu-west-2), and AWS Sydney (ap-southeast-2) (APM connectivity: AWS ap-southeast-2, Kibana connectivity: AWS ap-southeast-2).