We identified that between Feb 12th at 21:30 UTC and Feb 18 at 10:00 UTC some network related settings were not being properly applied on newly created deployment nodes in AWS. We determined that this impacted 2-3% of AWS deployments, specifically the following situations:
1. Newly created deployments.
2. Deployments that had topology changes resulting in node creation during the window.
The impact would have been observed by nodes occasionally dropping and returning to their respective clusters (node left events). The root cause was found to be a failure in settings application and has been remediated, we are also working on improving how we monitor these configuration settings to prevent such a situation in the future.
We would like to apologize for the inconvenience and if you still see issues please reach out to through https://support.elastic.co
and we will be happy to assist.