Duplicate
Details
Assignee
Marcelo AltmannMarcelo Altmann(Deactivated)Reporter
Itamar CroitoruItamar CroitoruAffects versions
Priority
Medium
Details
Details
Assignee
Marcelo Altmann
Marcelo Altmann(Deactivated)Reporter
Itamar Croitoru
Itamar CroitoruAffects versions
Priority
Smart Checklist
Smart Checklist
Smart Checklist
Created May 2, 2022 at 4:52 PM
Updated March 6, 2024 at 6:26 PM
Resolved May 2, 2022 at 5:52 PM
I have a UrBackup Server 2.2.11 on FreeNAS.
I am taking a backup of a VM on ESXi of Ubuntu Server 18.04 LTS.
The VM has mySQL Server 5.7 installed.
Backups are nightly and have been working fine using Xtra Backup 2.4.
Yesterday, I ran a monthly update cycle that upgraded Xtra Backup 2.4.24-1.bionic to 2.4.25-1.bionic.
Since that upgrade, backups have continually failed. After incremental backups started to fail, I tried full backups but they fail with the exact same details.
I ran a force install with apt to download back to the previous version and backups are working again.
Here is the forced downgrade command:
sudo apt install percona-xtrabackup-24=2.4.24-1.bionic -V
DB server uName:
Linux <hostname> 4.15.0-176-generic #185-Ubuntu SMP Tue Mar 29 17:40:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
A dpkg search for mysql resulted in this:
rc bacula-director-mysql 5.2.6+dfsg-9.1ubuntu3 amd64 network backup service - MySQL storage for Director
ii libdbd-mysql-perl 1:4.050-3.bionic amd64 Perl5 database interface to the MariaDB/MySQL database
rc libmysqlclient18:amd64 5.5.50-0ubuntu0.14.04.1 amd64 MySQL database client library
ii libmysqlclient20:amd64 5.7.37-0ubuntu0.18.04.1 amd64 MySQL database client library
rc libmysqlcppconn7 1.1.3-5 amd64 MySQL Connector for C++ (library)
ii mysql-client-5.7 5.7.37-0ubuntu0.18.04.1 amd64 MySQL database client binaries
ii mysql-client-core-5.7 5.7.37-0ubuntu0.18.04.1 amd64 MySQL database core client binaries
ii mysql-common 5.8+1.0.4 all MySQL database common files, e.g. /etc/mysql/my.cnf
ii mysql-server 5.7.37-0ubuntu0.18.04.1 all MySQL database server (metapackage depending on the latest version)
rc mysql-server-5.5 5.5.50-0ubuntu0.14.04.1 amd64 MySQL database server binaries and system database setup
ii mysql-server-5.7 5.7.37-0ubuntu0.18.04.1 amd64 MySQL database server binaries and system database setup
ii mysql-server-core-5.7 5.7.37-0ubuntu0.18.04.1 amd64 MySQL database server binaries
rc mysql-workbench 6.0.8+dfsg-2 amd64 MySQL Workbench - a visual database modeling, administration and queuing tool
ii mysqltuner 1.7.2-1 all high-performance MySQL tuning script
UrBackup provided this log:
2022-05-02 09:25:18(info): Starting unscheduled full file backup...
2022-05-02 09:26:18(info): Backing up "root" without snapshot.
2022-05-02 09:26:18(info): Indexing of "root" done. 37592 filesystem lookups 0 db lookups and 0 db updates
2022-05-02 09:26:18(info): <hostname>: Loading file list...
2022-05-02 09:26:19(info): <hostname>: Started loading files...
2022-05-02 09:26:20(info): Referencing snapshot on "<hostname>" for path "root" failed: FAILED
2022-05-02 09:32:35(warning): Error getting complete file "ozrOUZmYtjHCWHTMoZNG|root/usr/local/var/urbackup/data/filelist_new_0.ub" from <hostname>. Errorcode: CANNOT_OPEN_FILE (3)
2022-05-02 09:43:19(info): Waiting for file transfers...
2022-05-02 09:43:21(error): mariadbxtrabackup: Error reading tar header (1). Unexpected length 0
2022-05-02 09:43:20(error): mariadbxtrabackup: Starting binary backup of MariaDB at Mon May 2 09:43:19 EDT 2022...
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: recognized server arguments: --datadir=/var/lib/mysql --tmpdir=/tmp
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: recognized client arguments: --backup=1 --stream=tar --extra-lsndir=/var/lib/mariadbbackup/last.new --user=<redacted> --password=*
2022-05-02 09:43:21(error): mariadbxtrabackup: 220502 09:43:20 version_check Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup' as '<redacted>' (using password: YES).
2022-05-02 09:43:21(error): mariadbxtrabackup: 220502 09:43:20 version_check Connected to MySQL server
2022-05-02 09:43:21(error): mariadbxtrabackup: 220502 09:43:20 version_check Executing a version check against the server...
2022-05-02 09:43:21(error): mariadbxtrabackup: 220502 09:43:20 version_check Done.
2022-05-02 09:43:21(error): mariadbxtrabackup: 220502 09:43:20 Connecting to MySQL server host: localhost, user: <redacted>, password: set, port: not set, socket: not set
2022-05-02 09:43:21(error): mariadbxtrabackup: Using server version 5.7.37-0ubuntu0.18.04.1
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup version 2.4.25 based on MySQL server 5.7.35 Linux (x86_64) (revision id: 90fe9d0)
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: uses posix_fadvise().
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: cd to /var/lib/mysql
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: open files limit requested 0, set to 1024
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: using the following InnoDB configuration:
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: innodb_data_home_dir = .
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: innodb_log_group_home_dir = ./
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: innodb_log_files_in_group = 2
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup: innodb_log_file_size = 50331648
2022-05-02 09:43:21(error): mariadbxtrabackup: InnoDB: Number of pools: 1
2022-05-02 09:43:21(error): mariadbxtrabackup: 13:43:20 UTC - xtrabackup got signal 11 ;
2022-05-02 09:43:21(error): mariadbxtrabackup: This could be because you hit a bug or data is corrupted.
2022-05-02 09:43:21(error): mariadbxtrabackup: This error can also be caused by malfunctioning hardware.
2022-05-02 09:43:21(error): mariadbxtrabackup: Attempting to collect some information that could help diagnose the problem.
2022-05-02 09:43:21(error): mariadbxtrabackup: As this is a crash and something is definitely wrong, the information
2022-05-02 09:43:21(error): mariadbxtrabackup: collection process might fail.
2022-05-02 09:43:21(error): mariadbxtrabackup:
2022-05-02 09:43:21(error): mariadbxtrabackup: Thread pointer: 0x0
2022-05-02 09:43:21(error): mariadbxtrabackup: Attempting backtrace. You can use the following information to find out
2022-05-02 09:43:21(error): mariadbxtrabackup: where mysqld died. If you see no messages after this, something went
2022-05-02 09:43:21(error): mariadbxtrabackup: terribly wrong...
2022-05-02 09:43:21(error): mariadbxtrabackup: stack_bottom = 0 thread_stack 0x10000
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup(my_print_stacktrace+0x3b)[0x55e37367722b]
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup(handle_fatal_signal+0x285)[0x55e37332e735]
2022-05-02 09:43:21(error): mariadbxtrabackup: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7f54aafac980]
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup(my_tmpdir+0xa)[0x55e37366cb5a]
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup(+0x5a66a5)[0x55e3730216a5]
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup(ds_open+0x12)[0x55e37301e0d2]
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup(_Z22xtrabackup_backup_funcv+0xf35)[0x55e373010545]
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup(main+0x1185)[0x55e372ff3ab5]
2022-05-02 09:43:21(error): mariadbxtrabackup: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f54a8db4c87]
2022-05-02 09:43:21(error): mariadbxtrabackup: xtrabackup(_start+0x2a)[0x55e3730074ea]
2022-05-02 09:43:21(error): mariadbxtrabackup:
2022-05-02 09:43:21(error): mariadbxtrabackup: Please report a bug at https://jira.percona.com/projects/PXB
2022-05-02 09:43:21(error): Script "mariadbxtrabackup" returned a nun-null value 2. Failing backup.
2022-05-02 09:43:21(error): Client <hostname> went offline.
2022-05-02 09:43:22(info): Waiting for file hashing and copying threads...
2022-05-02 09:43:24(info): Waiting for metadata download stream to finish
2022-05-02 09:43:24(error): Error saving metadata. Could not open output file at "/mnt/iS3-R10-UrBackup-Storage/<hostname>/220502-0925/.hashes/urbackup_backup_scripts/mariadbxtrabackup/.dir_metadata"
2022-05-02 09:43:24(info): Writing new file list...
2022-05-02 09:43:28(info): All metadata was present
2022-05-02 09:43:29(info): Transferred 5.57562 GB - Average speed: 46.8493 MBit/s
2022-05-02 09:43:29(info): Time taken for backing up client <hostname>: 18m 11s
2022-05-02 09:43:29(error): Backup failed