Oh holy smokes

The script I used to build the replica has a code path to setup a replica if the mysql is encrypted using
xtrabackup … --keyring-file-data=/srv/mysql2022/keyring/component_keyring_file …

this file was present but empty, the prepare was done with

/usr/bin/xtrabackup --component-keyring-config=/usr/lib/mysql/plugin/component_keyring_file.cnf --prepare

which resulted in a usable database able to act as a replica even without any encryption parameter set.

it showed an Error I overlooked:
2025-02-07T14:59:58.637157+01:00 0 [ERROR] [MY-011825] [Xtrabackup] Unable to read /usr/lib/mysql/plugin/component_keyring_file.cnf passed as --component-keyring-config parameter
Now, if the Backup is in the datadir, you should be able to start mysql
after that you can configure the replication:

but as said the mysql just worked as an replica, but xtrabackup somehow got confused with this.

After using the correct (aka not using component-keyring ) code path the replica also runs and xtrabackup now works also on the replica.

I don’t know why using an empty keyring file with xtrabackup results in working database or why a xtrabackup fails on it. Sure my fault, but perhaps xtrabackup should check if keyring files are empty and fail?

Thanks for all your work.

Hello

I tried something else:

Setup 2 VMs Ubuntu 24.04, PS8 8.4.3-3 / xtrabackup version 8.4.0-2 as before, same config except server_id and VM-name dependent Path/file names

no replica at this point

/usr/bin/xtrabackup --backup --use-memory=1G --target-dir=./ working on both VMs

lets say vm1 is primary, vm2 will get replica

• on vm2 :

  • systemctl stop mysql and delete /var/lib/mysql

  • ssh vm2 “/usr/bin/xtrabackup --parallel=2 --backup --stream=xbstream" | xbstream -x -C /var/lib/mysql“

  • /usr/bin/xtrabackup --prepare --target-dir=/var/lib/mysql

  • chown -R mysql:mysql /var/lib/mysql

  • systemctl start mysql but no change replication source …

  • /usr/bin/xtrabackup --backup --use-memory=1G --target-dir=./ stops working

  • xtrabackup created additional files on restore (backup-my.cnf and some xtrabackup_*) which I removed , but xtrabackup still fails

Next try

• on vm1:

  • systemctl stop mysql

  • tar cf /somenfs/t.tar /var/lib/mysql

  • systemctl start mysql

• on vm2:

  • systemctl stop mysql and delete /var/lib/mysql

  • tar xf /somenfs/t.tar (into /var/lib/mysql)

  • systemctl start mysql but no change replication source

  • /usr/bin/xtrabackup --backup --use-memory=1G --target-dir=./ works!

in this case no restore with xtrabackup was done, as I copied the files offline with tar.

Next try:

• on vm1:

  • xtrabackup --backup --target-dir=somewhere

  • xtrabackup --prepare --target-dir=somewhere

• on vm2:

  • systemctl stop mysql and delete /var/lib/mysql

  • rsync -Pav somewhere /var/lib/mysql

  • systemctl start mysql

  • /usr/bin/xtrabackup --backup --use-memory=1G --target-dir=./ works!

So using

• ssh vm1 /usr/bin/xtrabackup --parallel=2 --backup --stream=xbstream" | xbstream -x -C somewhere

• xtrabackup --prepare --target-dir=somewhere

produces something that the next xtrabackup run don’t like. the db starts fine and making a replica of it works also.

xbstream -V
xbstream Ver 8.4.0-2 for Linux (x86_64) (revision id: d4373834) on vm1&vm2

We used this way creating replicas countless times and did not have this issue with ps8.0

Hi

Thank you for the information; we will try the described steps locally and keep you posted.

Hello

What I did:

• Setup 2 Ubuntu 24-04 VMs

• install Percona Serer for MySQL : Server version: 8.4.3-3 Percona Server (GPL), Release '3', Revision '505340da'

• install xtrabackup version 8.4.0-2 based on MySQL server 8.4.0 Linux (x86_64) (revision id: d4373834)

• both system have identical setup regarding path/config/… except for server_id as we will

• on both systems /usr/bin/xtrabackup --backup --use-memory=1G --target-dir=./ work

• setup 1 system as replica, using xtrabackup using --backup --stream=xbstream" | xbstream -x -C ./
  this also shows xtrabackup is working on a primary and activate the reeplica:

• issue /usr/bin/xtrabackup --backup --use-memory=1G --safe-slave-backup --target-dir=./ on the replica
  which results in:
  2025-02-03T09:20:21.316815+01:00 0 [ERROR] [MY-011825] [InnoDB] Cannot restore from xtrabackup, InnoDB running in read-only mode!

• /usr/bin/xtrabackup --backup --use-memory=1G --target-dir=./ on primary still works

What I do not understand is why it says “Cannot restore from xtrabackup, InnoDB running in read-only mode” as we try to create a backup.

Tried to stop replica; reset replica all, [super]read_only=off, restarted mysql does not help, after being a replica that server can not be backuped
anymore.
Any information I could provide?

Hi

Please provide all the steps to repeat the behaviour locally.