(xbcloud) Correct chunk size for 1 of 2 files, the other is tiny (swift problem ?)
Description
Environment
Attachments
Smart Checklist
Activity

Marcelo Altmann February 16, 2023 at 1:08 PM
Hi .
The parallel on xbcloud was to attempt a fix on your slave lag.
This ticket is about xbcrypt creating the extra chunk.
Baptiste Mille-Mathias February 16, 2023 at 7:44 AM
Hello,
Thanks Marcelo for the discussion over this issue.
I think the title should updated to reflected the root cause of the problem.
For the record I backup using the command
xtrabackup --defaults-extra-file=/root/.xtrabackup.ini --read-buffer-size=200MB --encrypt-chunk-size=200MB --backup --compress | xbcloud --defaults-file=/root/.xtrabackup.ini put full --s3-bucket={s3_bucket
}
root@sql6:~# cat ~/.xtrabackup.ini
[xtrabackup]
check_privileges=1
compress-threads=4
encrypt=AES256
encrypt-key=dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
encrypt-threads=4
parallel=8
stream=xbstream[xbcloud]
s3-access-key=dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
s3-endpoint=s3.de.cloud.ovh.net
s3-secret-key=dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
storage=s3
parallel=16
Changing value of key `parallel` as you asked to me on chat did not help fixing this issue.
When I added `--read-buffer-size=200MB --encrypt-chunk-size=200MB
` last week the size of the "small" file changed from 64kb to 122bytes.
Best

Marcelo Altmann February 15, 2023 at 4:42 PM
That is due to encryption:
Only --compress
Same table --compress & --encrypt
Baptiste Mille-Mathias September 14, 2021 at 12:35 PM
Hello,
I see the same behaviour with 2.4.23.

Lalit Choudhary September 14, 2021 at 11:22 AM
Hi
Thank you for the report.
what version of xtrabackup are you using? try the latest xtrabackup version and if you still see the issue please add reproducible steps with a command that you are using.
Details
Assignee
UnassignedUnassignedReporter
Baptiste Mille-MathiasBaptiste Mille-MathiasPriority
Medium
Details
Details
Assignee
Reporter
Priority
Smart Checklist
Open Smart Checklist
Smart Checklist
Open Smart Checklist
Smart Checklist

I've seen on the object storage (swift from OBH) the chuck files are strange, The even ones have a correct size of 10MB the odd ones have a tiny size, often around 64KB.
I attach a screenshot
I'm not sure this is actually an issue but seemed strange to me because the doc says chunk are 10MB