(xbcloud) Correct chunk size for 1 of 2 files, the other is tiny (swift problem ?)

Description

I've seen on the object storage (swift from OBH) the chuck files are strange, The even ones have a correct size of 10MB the odd ones have a tiny size, often around 64KB.

I attach a screenshot

I'm not sure this is actually an issue but seemed strange to me because the doc says chunk are 10MB

Environment

None

Attachments

2

Smart Checklist

Activity

Show:

Marcelo Altmann February 16, 2023 at 1:08 PM

Hi .

 

The parallel on xbcloud was to attempt a fix on your slave lag.

This ticket is about xbcrypt creating the extra chunk.

Baptiste Mille-Mathias February 16, 2023 at 7:44 AM

Hello,

Thanks Marcelo for the discussion over this issue.

I think the title should updated to reflected the root cause of the problem.

 

For the record I backup using the command

xtrabackup --defaults-extra-file=/root/.xtrabackup.ini --read-buffer-size=200MB --encrypt-chunk-size=200MB --backup --compress | xbcloud --defaults-file=/root/.xtrabackup.ini put full --s3-bucket={s3_bucket}
 

root@sql6:~# cat ~/.xtrabackup.ini 
[xtrabackup]
check_privileges=1
compress-threads=4
encrypt=AES256
encrypt-key=dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
encrypt-threads=4
parallel=8
stream=xbstream[xbcloud]
s3-access-key=dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
s3-endpoint=s3.de.cloud.ovh.net
s3-secret-key=dxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
storage=s3
parallel=16

Changing value of  key `parallel` as you asked to me on chat did not help fixing this issue.

When I added `--read-buffer-size=200MB --encrypt-chunk-size=200MB` last week the size of the "small" file changed from 64kb to 122bytes.

Best

Marcelo Altmann February 15, 2023 at 4:42 PM

That is due to encryption:

Only --compress

 

Same table --compress & --encrypt

Baptiste Mille-Mathias September 14, 2021 at 12:35 PM

Hello,

I see the same behaviour with 2.4.23.

Lalit Choudhary September 14, 2021 at 11:22 AM

Hi 

Thank you for the report.

what version of xtrabackup are you using?  try the latest xtrabackup version and if you still see the issue please add reproducible steps with a command that you are using.

 

Details

Assignee

Reporter

Priority

Smart Checklist

Created April 19, 2021 at 1:20 PM
Updated March 6, 2024 at 6:42 PM