Will AKS ever get better at moving volumes between nodes to speed up PVC mounts? #5654
Alexander Overvoorde (OvervCW)
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Whenever a pod shuts down that mounts a PVC and a new pod is scheduled to run on a different AKS node, the pod, without fail, will take up to 5 minutes to actually start running there due to issues like this:
Multi-Attach error for volume "pvc-XYZ" Volume is already exclusively attached to one node and can't be attached to anotherAttachVolume.Attach failed for volume "pvc-XYZ" : rpc error: code = NotFound desc = failed to get azure instance id for node "aks-XYZ" (instance not found)This has been an issue for as long as I've used AKS, over 6 years now, and the situation hasn't seemed to improve since then.
I have several questions about this:
For reference, we are running a Kubernetes cluster using version 1.33.x and have also created new clusters over the years, and they all demonstrate these issues, so it does not seem to be a weird state that one specific cluster is in.
This delay in remounting can have serious effects on the availability of services, even when they are replicated, which is why I think starting a discussion about this is warranted.
All reactions