Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

[BUG] xtradb operator does not delete PVC after scaling-down leading to resource leak

Description

Description
If the pxc.size in the spec gets changed, the xtradb operator updates the stateful set size. The operator decreases the size of the statefulset and delete the pod if it sees that the pxc.size decreases. However, the operator does not delete the PVC used by the deleted pod, which leads to storage resource leak.

Fix
During each reconcile, we can make the operator list all the PVC used by xtradb and check whether its owner pod is still running. If not, the operator can delete the PVC as long as delete-pxc-pvc finalizer is present.

We are willing to send a PR to deal with the resource leak problem.

Environment

None

Smart Checklist

Activity

Lalit Choudhary December 2, 2021 at 2:51 PM

Hi

Thank you for the details.  I'm marking this case open and please let us know once you have patch.

sieveteam December 1, 2021 at 5:59 AM

Hello Sergey, thanks for the reply.

The resource leak can become significant if the MongoDB cluster experiences a steep scale down.

I agree that a potential solution is to add a configurable finalizer to recycle the orphan PVC.

The users can decide whether to recycle more space or minimize the time for sync.

I can try to send a PR for this one.

Sergey Pronin November 29, 2021 at 8:11 AM

hello  - we track the same here https://jira.percona.com/browse/K8SPXC-603

 

We don't track it as a bug, as in scale up event the Pod will reuse existing PVC and minimize the time for sync. 

We think it should be done through some finalizer, which the user can enable.

What are your thoughts?

Details

Assignee

Reporter

Affects versions

Priority

Smart Checklist

Created November 29, 2021 at 2:26 AM
Updated March 5, 2024 at 5:42 PM

Flag notifications