LP #1294431: cluster SST fail - /usr/bin/du: Argument list too long

**Reported in Launchpad by pawouk last update 09-02-2015 23:01:26

Hi Guys,

we have quite a lot of tables and databases (1000DB, each with 60tables).

When I try to add a new node to the cluster I receive an error:

WSREP_SST: [INFO] Evaluating xbstream -c ${INFO_FILE} | socat -u stdio TCP:10.100.2.3:4444; RC=( ${PIPESTATUS[@]} ) (20140319 02:22:23.431)

/usr//bin/wsrep_sst_xtrabackup-v2: line 234: /usr//bin/du: Argument list too long

WSREP_SST: [INFO] Sleeping before data transfer for SST (20140319 02:22:25.456)
WSREP_SST: [INFO] Streaming the backup to joiner at 10.100.2.3 4444 (20140319 02:22:35.463)
WSREP_SST: [INFO] Evaluating innobackupex --defaults-file=/etc/mysql/my.cnf $INNOEXTRA --galera-info --stream=$sfmt ${TMPDIR} 2>${DATA}/innobackup.backup.log | pv -f -i 10 -N donor -s 2>0 | socat -u stdio TCP:10.100.2.3:4444; RC=( ${PIPESTATUS[@]} ) (20140319 02:22:35.468)
WSREP_SST: [ERROR] innobackupex finished with error: 1. Check /var/lib/mysql/data//innobackup.backup.log (20140319 02:22:35.697)
WSREP_SST: [ERROR] Cleanup after exit with status:22 (20140319 02:22:35.702)
140319 2:22:35 [ERROR] WSREP: Failed to read from: wsrep_sst_xtrabackup-v2 --role 'donor' --address '10.100.2.3:4444/xtrabackup_sst' --auth 'xtrabackuper:h@wagaV8$uyE' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/data/' --defaults-file '/etc/mysql/my.cnf' --gtid '7002b138-9d04-11e2-0800-0dbb30030abe:1402150009'
140319 2:22:35 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'donor' --address '10.100.2.3:4444/xtrabackup_sst' --auth 'xtrabackuper:h@wagaV8$uyE' --socket '/var/run/mysqld/mysqld.sock' --datadir '/var/lib/mysql/data/' --defaults-file '/etc/mysql/my.cnf' --gtid '7002b138-9d04-11e2-0800-0dbb30030abe:1402150009': 22 (Invalid argument)
140319 2:22:35 [Warning] WSREP: 0 (sql1002): State transfer to 1 (sql1003) failed: -1 (Operation not permitted)
140319 2:22:35 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 1402150009)

The problematic code line is:

payload=$(du --block-size=1 -c */.ibd */.MYI */.MYI ibdata1 | awk 'END { print $1 }')

When I try to run it manualy at mysql data directory I receive the same error.

root@sql1002:/var/lib/mysql/data# du --block-size=1 -c */.ibd */.MYI */.MYI ibdata1
-bash: /usr/bin/du: Argument list too long
root@sql1002:/var/lib/mysql/data#

I believe it can be fixed by replacing the line with this command:

payload=$(find . -name ".ibd" -name ".MYI" -name "ibdata1" -type f -print0 | xargs -0 du --block-size=1 -c | awk 'END { print $1 }')