Restore is failing - PXC Cluster and Xtrabackup versions are not in sync

Description

Hi all, 

I haver created a PXC cluster trough PMM DBaas and the PXC version is `8.0.25-15.1`.  Backups were taken by the operator and when I tried to restore one of them I got the following error:

 

 After some investigation and debugging it looks like the issue is operator is using Xtrabackup `8.0.23` version and that can not restore `8.0.25`. I have tried manually and I have copied the files out of the backup like:

If I used Xtrabackup version 8.0.25+ the restore worked when I used 8.0.23 the restore failed.

Reproducing

Create a cluster with PXC 8.0.25 and try to restore it with Xtrabackup 8.0.23.

Recommendation

Operator should handle this case when the versions are not matching a few options:

  • always keep xtrabackup version up to date

  • if version are not the same install the lates xtrabackup version

  • Give a proper error to the client if versions are mismatching 

  • update our documentation how can you keep Xtrabackup version up to date or how can you install a specific version in you operator

 

Environment

None

Smart Checklist

Activity

Slava Sarzhan June 14, 2022 at 10:44 AM

do not worry. I will contact PMM team to discuss this issue.

Tibor Korocz (Percona) June 14, 2022 at 10:40 AM

This is only an issue with DBaaS because it allows you to install a PXC version without checking if the xtrabackup version in the operator can handle that.. Of course sometimes we can get lucky when the PXC version and xtrabackup versions are in match but without actually checking that we can not be sure.. 

I do not have this test environment running anymore. but if you need me to do more testing I could install one.. 

Slava Sarzhan June 13, 2022 at 6:35 PM

I can't reproduce it without PMM DBaas.

As you can see for the backup and restore the Xtrabackup version 8.0.25 is used. If this issue is still actual I need to see CR.

Slava Sarzhan June 13, 2022 at 6:10 PM

It is an interesting issue. In case of 1.10.0 the Xtrabackup version 8.0.25 should be used. Does PMM DBaas use version service? All this stuff should be handled by VS. I will try to reproduce it without DBaas.

Details

Assignee

Reporter

Affects versions

Priority

Smart Checklist

Created April 7, 2022 at 9:29 AM
Updated March 5, 2024 at 5:39 PM