Resolved
This incident has been resolved.
Posted Oct 08, 2020 - 18:12 UTC
Monitoring
The fix has been rolled out to all regions. We are monitoring the situation and if we do not see any further issues we will close the incident in 30 minutes.
Posted Oct 08, 2020 - 17:37 UTC
Update
As we’ve been making changes to disable TLS < 1.2 we found that there was a missing set of supported cipher suites:
ECDHE-ECDSA-AES128-SHA
ECDHE-RSA-AES128-SHA
ECDHE-ECDSA-AES256-SHA
ECDHE-RSA-AES256-SHA
ECDHE-RSA-DES-CBC3-SHA
AES128-GCM-SHA256
AES256-GCM-SHA384
AES128-SHA256
AES128-SHA
AES256-SHA
DES-CBC3-SHA
We were able to determine that this change affected less than 1% of overall traffic and the majority of clients are not affected.
Engineers are actively pushing a fix to correct the issue. We will post a follow up shortly as we make progress.
If you’re having issues around TLS negotiation or affected by the problem, please reach out to support@elastic.co.
Posted Oct 08, 2020 - 17:09 UTC
Investigating
As we’ve been making changes to disable TLS < 1.2 we have had reports of issues with TLS connections to clusters. We found that there was a change to the cipher suites that was inadvertently included in the changes we were rolling out and we are working on deploying a fix to correct the problem. As we work on the issue, we will post updates as we make progress.
If you’re having issues please reach out to support@elastic.co.
Posted Oct 08, 2020 - 16:01 UTC
This incident affected: AWS Tokyo (ap-northeast-1) (Elasticsearch connectivity: AWS ap-northeast-1, Kibana connectivity: AWS ap-northeast-1, APM connectivity: AWS ap-northeast-1), AWS Seoul (ap-northeast-2) (Kibana connectivity: AWS ap-northeast-2, Deployment hosts: AWS ap-northeast-2, Deployment metrics: AWS ap-northeast-2), AWS São Paulo (sa-east-1) (Elasticsearch connectivity: AWS sa-east-1, Kibana connectivity: AWS sa-east-1, APM connectivity: AWS sa-east-1), AWS N. California (us-west-1) (Elasticsearch connectivity: AWS us-west-1, Kibana connectivity: AWS us-west-1, APM connectivity: AWS us-west-1), AWS Oregon (us-west-2) (Elasticsearch connectivity: AWS us-west-2, Kibana connectivity: AWS us-west-2, APM connectivity: AWS us-west-2), AWS Ireland (eu-west-1) (Elasticsearch connectivity: AWS eu-west-1, Kibana connectivity: AWS eu-west-1, APM connectivity: AWS eu-west-1), AWS N. Virginia (us-east-1) (Elasticsearch connectivity: AWS us-east-1, Kibana connectivity: AWS us-east-1, APM connectivity: AWS us-east-1), AWS Singapore (ap-southeast-1) (Elasticsearch connectivity: AWS ap-southeast-1, Kibana connectivity: AWS ap-southeast-1, APM connectivity: AWS ap-southeast-1), and AWS Sydney (ap-southeast-2) (Elasticsearch connectivity: AWS ap-southeast-2, APM connectivity: AWS ap-southeast-2, Kibana connectivity: AWS ap-southeast-2).