xtrabackup prepare fails if checkpoint lsn is more than the last lsn

Description

https://forums.percona.com/t/xtrabackup-error-during-prepare/9788 also talks about this issue.

 

The error I get is 

 

Created 128 and tracked 128 new rollback segment(s) in the temporary tablespace.

128 are now active. 8.0.23 started; log sequence number 3585483693664

Allocated tablespace ID 1 for sys/sys_config, old maximum was 0

xtrabackup: Last MySQL binlog file position 198671947, file name mysql-bin.002091 xtrabackup: error: The transaction log file is corrupted.

xtrabackup: error: The log was not applied to the intended LSN!

xtrabackup: Log applied to lsn 3585483693664

xtrabackup: The intended lsn is 3585483693674

 

However, I don't think backup was bad. It is working fine with previous version 8.0.22-15-1.bionic.

We also see this issue with at least 3 backups and they were not all taken from same host. All those backups can't be bad. 

 

It looks like new xtrabackup version is unable to restore some backups taken from those older servers where we set sql_log_bin to 0. Those are servers where we do only bulk load operations, and so turned off writing to bin logs. We bulk load same files to all replicas rather than using MySQL replication.

Environment

None

Attachments

1

Smart Checklist

Activity

Show:

Rahul Malik June 3, 2021 at 12:23 PM

Problem:
Backups taken before fix of pxb-2274 can have checkpoint lsn more than
the last LSN from backup.
This could happen because checkpoint lsn were copied from the redo file
instead of querying from log_status.

Fix:
set last_lsn to checkpoint_lsn if it is more than last_lsn

Rahul Malik April 12, 2021 at 8:11 AM
Edited

Backups taken before fix of pxb-2274 can have checkpoint lsn more than
the last LSN from backup.
This could happen because checkpoint lsn were copied from the redo file
instead of querying from log_status.

Fix:
set last_lsn to checkpoint_lsn if it is more than last_lsn

Jira Bot April 1, 2021 at 9:56 AM

To:
CC:

Hi, I'm jira-bot, Percona's Jira automation tool. I've detected that someone from
Percona has made an edit to the Summary field of an issue that you reported.

I'm not sentient (yet) so I'm not sure whether the person fixed a typo, changed
a few words, or completely rewrote the text. In any case, it is Percona Engineering's
intention to make the Summary and Description of an issue as accurate as possible
so that we're fixing the actual problem you're encountering, and to avoid
misunderstandings about symptoms and causes.

If the current Summary does not accurately reflect the problem you are reporting,
or if you feel the change was otherwise inappropriate in some way, please add a
new comment explaining things and we'll address it as soon as we can.

This message will be added only once per issue, regardless of how many times
the Summary is edited.

message-code:summary-edited

Rahul Malik April 1, 2021 at 9:41 AM
Edited

thanks , We are able to reproduce the bug.

Nagavamsi Ponnekanti March 30, 2021 at 4:34 PM

I am attaching full output from xtrabackup. It seems to print some cnf options that it uses.

Done

Details

Assignee

Reporter

Time tracking

3d 5h logged

Priority

Smart Checklist

Created March 27, 2021 at 6:18 PM
Updated March 6, 2024 at 6:43 PM
Resolved April 16, 2021 at 12:39 PM