SST fails with gtid=on: Move file mysqld-bin.index to /otherfs/mysqld-bin.log.index failed: Destination file exists

Description

1. Setup PXC cluster with two nodes and gtid enabled:
e.g.

./anydbver deploy virtual-machine pxc node1 virtual-machine pxc galera-master:default ./anydbver ssh default bash /vagrant/tools/tools/fix_wsrep_cluster_address.sh /etc/my.cnf

in my.cnf:

enforce_gtid_consistency=ON gtid_mode=ON

2. Create a partition, move binary logs:

dd if=/dev/zero of=/disk1 bs=512 count=1M; yum install -y e2fsprogs; mkfs.ext4 -F /disk1; mkdir -p /archdest/mysql/logfile;mount -o loop /disk1 /archdest/mysql/logfile ; chown mysql:mysql /archdest/mysql/logfile/. systemctl stop mysqld

add to my.cnf:

log_bin = /archdest/mysql/logfile/mysqld-bin.log log_bin_index = /archdest/mysql/logfile/mysqld-bin.log.index relay-log= /archdest/mysql/logfile/relay-bin.log relay-log-index = /archdest/mysql/logfile/mysql-relay-bin.index max_binlog_size=524288

systemctl start mysqld

3. Create next binlog file:

create database test use test INSTALL PLUGIN keyring_file SONAME 'keyring_file.so' create table t(id int auto_increment primary key, c varchar(10000)) ENCRYPTION='Y'; insert into t (c) select REPEAT('0123456789',1000); -- repeat 10x insert into t (c) select REPEAT('0123456789',1000) FROM t t1, t t2, t t3;

4. repeat 2 on second node

5. force SST on second node
cd /var/lib/mysql
rm -f bin xtrabackup_* *.log galera.cache grastate.dat

SST fails with:
210226 03:59:54 [01] Moving mysqld-bin.log.index to /archdest/mysql/logfile//mysqld-bin.log.index
210226 03:59:54 [01] Copying mysqld-bin.log.index to /archdest/mysql/logfile/mysqld-bin.log.index
210226 03:59:54 [01] ...done
210226 03:59:54 [01] Removing mysqld-bin.log.index
Error: Move file mysqld-bin.index to /archdest/mysql/logfile//mysqld-bin.log.index failed: Destination file exists

2021-02-26T03:59:54.629109Z 0 [ERROR] [MY-000000] [WSREP-SST] ------------ innobackup.move.log (END) ------------

put binary log index file in the datadir is not working as a workaround due to: https://jira.percona.com/browse/PXC-3601

log_bin_index = /var/lib/mysql/mysqld-bin.log.index

Environment

None

AFFECTED CS IDs

CS0016072

Smart Checklist

Activity

Show:

Marcelo Altmann March 18, 2022 at 6:17 PM

 Can you please add a note on the limitations page of PS/PXC:

It's highly advised to not use dots (.) on binlog log_bin and log_bin_index variables as The value MySQL export on status variables and the actual value it uses will divert. This causes tools that rely on this information, such as backup tools to have unpredicted behavior.

Marcelo Altmann March 18, 2022 at 6:13 PM

As explained above, this is rather a server issue. MySQL will remove the last dot on log_bin when creating it to disk file, while still reporting the file with dot in status variables. Fixing this is not trivial and would have to take into consideration updates from previous versions to not break the server on upgrades.

Having in mind the easy workaround (do not use dots on log_bin and log_bin_index) we have decided to adjust this as documentation on the limitations page.

KennT April 22, 2021 at 3:50 AM

Also, upstream report for a single unnumbered binlog file is created (not multiple numbered files), due to multiple '.' in the log_bin name

http://bugs.mysql.com/103404

 

 

George Lorch April 21, 2021 at 5:50 PM

upstream report for '.' in log_bin_index https://bugs.mysql.com/bug.php?id=103400

Sveta Smirnova March 22, 2021 at 2:16 PM

May we update documentation at least?

Done

Details

Assignee

Reporter

Time tracking

4d 1h logged

Components

Affects versions

Priority

Smart Checklist

Created February 26, 2021 at 4:23 AM
Updated March 6, 2024 at 9:14 PM
Resolved July 19, 2022 at 3:14 PM

Flag notifications