Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

backup delete is failing if the cluster was deleted before

Description

I had a group replication cluster with some backups (I did not use the cert manager) and then deleted the cluster.
Then I tried to delete the backup (on AWS S3 which works in general) but creation of the new container for backup deletion is failing because SSL secret is missing (it was deleted when I deleted the cluster).

Notice that we have these two related tickets to deletion of secrets:
https://jira.percona.com/browse/K8SPS-125 (Operator should not remove cluster secret in case of CR deletion)
https://jira.percona.com/browse/K8SPS-158 (cert manager certificate renew is not working after delete+apply)
So we should think about the how it should work in general here.

NAME READY STATUS RESTARTS AGE percona-server-mysql-operator-7c984f7c9-msl6p 1/1 Running 0 138m xb-backup1-s3-us-west-fz8rc 0/1 Completed 0 116m xb-backup2-s3-gcp-z8xgn 0/1 Completed 0 88m xb-backup3-s3-azure-d7s9z 0/1 Completed 0 80m xb-delete-backup1-zckt7 0/1 ContainerCreating 0 7m48s xb-restore-restore1-m6v8z 0/1 Completed 0 65m xb-restore-restore2-jxrb6 0/1 Completed 0 54m xb-restore-restore3-4vwmx 0/1 Completed 0 41m # issue in pod xb-delete-backup1-zckt7 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 8m4s default-scheduler Successfully assigned test/xb-delete-backup1-zckt7 to gke-plavi-1222-default-pool-3fbfca63-qt7h Warning FailedMount 6m1s kubelet Unable to attach or mount volumes: unmounted volumes=[tls], unattached volumes=[bin datadir tls kube-api-access-c4q62]: timed out waiting for the condition Warning FailedMount 3m46s kubelet Unable to attach or mount volumes: unmounted volumes=[tls], unattached volumes=[tls kube-api-access-c4q62 bin datadir]: timed out waiting for the condition Warning FailedMount 111s (x11 over 8m3s) kubelet MountVolume.SetUp failed for volume "tls" : secret "cluster1-ssl" not found Warning FailedMount 88s kubelet Unable to attach or mount volumes: unmounted volumes=[tls], unattached volumes=[datadir tls kube-api-access-c4q62 bin]: timed out waiting for the condition

Environment

None

Smart Checklist

Activity

Show:

Slava Sarzhan December 12, 2022 at 3:09 PM

The issue was fixed.

Slava Sarzhan December 5, 2022 at 3:22 PM

the behavior should be the same as we have e.g. in PXC  operator.

Done

Details

Assignee

Reporter

Labels

Needs QA

Yes

Fix versions

Affects versions

Priority

Smart Checklist

Created September 27, 2022 at 9:46 AM
Updated February 29, 2024 at 8:09 PM
Resolved December 28, 2022 at 12:38 PM

Flag notifications