We find that if the xtradb-operator crashes and restarts in the middle of the deleteStatefulSet method, it will never be able to delete the PVCs and secrets.

More concrete, inside deleteStatefulSet, the operator is supposed to do two things:

1. delete the statefulset if the statefulset is found

2. delete all the PVCs and secrets used by the statefulset

If the operator crashes between 1 and 2, an intermediate state where the statefulset is deleted but the PVCs still exist will appear. When the operator restarts and enters deleteStatefulSet again, it cannot find the statefulset and will not be able to proceed to delete the PVC as shown in the code:

{{}}

{{}}

Fix
There are multiple potential solutions. We can move PVCs/secrets deletion before statefulset deletion, or we can choose to still issue PVCs/secrets deletion even if the statefulset is not found.

We are willing to help fix the bug.