LP #1742751: mysqldump takes 5x as long with 5.7.20-19

Description

**Reported in Launchpad by William Taylor last update 17-01-2018 03:54:06

Originally posted this on the Percona forums with no response yet. This is probably a better place to post though.

After we upgraded to 5.7.20-19 from 5.7.19, mysql backups for this table jumped from 42 mins to 5hrs.
The backup time for the innodb table doubled but the tokudb table took more than 5 times longer.
I saw there was a memory leak fixed in mysqldump but unless there were other changes there I
don't see that being the issue.

Any ideas?

  1. Time: 2018-01-04T12:22:36.800014Z

  2. User@Host: root[root] @ localhost [] Id: 148138

  3. Schema: cdr Last_errno: 0 Killed: 0

  4. Query_time: 2568.367009 Lock_time: 0.000000 Rows_sent: 523136203 Rows_examined: 523136203 Rows_affected: 0

  5. Bytes_sent: 188647575431
    SET timestamp=1515068556;
    SELECT /*!40001 SQL_NO_CACHE */ * FROM `cdr_main`;

  1. Time: 2018-01-04T12:22:56.091752Z

  2. User@Host: root[root] @ localhost [] Id: 148138

  3. Schema: cdr Last_errno: 0 Killed: 0

  4. Query_time: 19.015636 Lock_time: 0.000000 Rows_sent: 11859695 Rows_examined: 11859695 Rows_affected: 0

  5. Bytes_sent: 239650071
    SET timestamp=1515068576;
    SELECT /*!40001 SQL_NO_CACHE */ * FROM `line_item_ratings`;

After upgrade to 5.7.20-19

  1. Time: 2018-01-10T20:18:38.510011Z

  2. User@Host: root[root] @ localhost [] Id: 29239

  3. Schema: cdr Last_errno: 1681 Killed: 0

  4. Query_time: 18269.061056 Lock_time: 0.000000 Rows_sent: 525785773 Rows_examined: 525785773 Rows_affected: 0

  5. Bytes_sent: 189596705933
    SET timestamp=1515615518;
    SELECT /*!40001 SQL_NO_CACHE */ * FROM `cdr_main`;

  1. Time: 2018-01-10T20:19:24.966083Z

  2. User@Host: root[root] @ localhost [] Id: 29239

  3. Schema: cdr Last_errno: 1681 Killed: 0

  4. Query_time: 44.798296 Lock_time: 0.000000 Rows_sent: 11859695 Rows_examined: 11859695 Rows_affected: 0

  5. Bytes_sent: 239650071
    SET timestamp=1515615564;
    SELECT /*!40001 SQL_NO_CACHE */ * FROM `line_item_ratings`;

SHOW TABLE STATUS:

Name: cdr_main
Engine: TokuDB
Version: 10
Row_format: tokudb_zlib
Rows: 526232788
Avg_row_length: 323
Data_length: 169998403381
Max_data_length: 9223372036854775807
Index_length: 35156877283
Data_free: 18446743920195218452
Auto_increment: 829412264
Create_time: 2016-07-11 11:16:33
Update_time: 2018-01-10 17:08:38
Check_time: NULL
Collation: latin1_swedish_ci
Checksum: NULL
Create_options:
Comment:

Name: line_item_ratings
Engine: InnoDB
Version: 10
Row_format: Dynamic
Rows: 6108827
Avg_row_length: 35
Data_length: 214663168
Max_data_length: 0
Index_length: 261586944
Data_free: 5242880
Auto_increment: NULL
Create_time: 2016-08-24 11:34:06
Update_time: NULL
Check_time: NULL
Collation: latin1_swedish_ci
Checksum: NULL
Create_options:
Comment:

Environment

None

Smart Checklist

Activity

Show:

lpjirasync January 24, 2018 at 11:52 AM

**Comment from Launchpad by: William Taylor on: 16-01-2018 16:46:49

It looks like its not the new mysql version.
It appears to be this kernel "kernel-3.10.0-693.11.6.el7.x86_64"
rolling back to "kernel-3.10.0-693.11.1.el7.x86_64" seems to have resolved the problems.
This is on CentOS Linux release 7.4.1708 (Core)

lpjirasync January 24, 2018 at 11:52 AM

**Comment from Launchpad by: Roel Van de Paar on: 15-01-2018 21:15:29

Tests do not show the issue. If we receive further OS/settings info, we can test again.

lpjirasync January 24, 2018 at 11:52 AM

**Comment from Launchpad by: Roel Van de Paar on: 15-01-2018 21:14:50

Discussion in the thread.

lpjirasync January 24, 2018 at 11:52 AM

**Comment from Launchpad by: Roel Van de Paar on: 13-01-2018 00:32:27

@Shako, can you please test this as discussed? Thank you

Done

Details

Assignee

Reporter

Priority

Smart Checklist

Created January 24, 2018 at 11:51 AM
Updated January 24, 2018 at 11:52 AM
Resolved January 24, 2018 at 11:52 AM