Severe degradation of Innodb buffer pool load times at startup after migration from 5.5. to 5.6

Description

We have 80GB Innodb buffer pool which loaded in about 5 minutes in Percona MySQL Server 5.5 (with disk read rates peaking at 1000MB/s) but it takes more than 30 minutes with current Percona 5.6.39 (disk rate steady at 30MB/s).

I did an investigation and saw that the original (5.5) implementation of Innodb buffer pool load at startup issued asynchronous io requests for tablespace pages (see: https://github.com/percona/percona-server/blob/1d6035b9f6bd98446ea1600d9535b951dad96bb2/storage/innobase/buf/buf0lru.c#L2875).

But as of MySQL 5.6 you adopted implentation from the upstream which is buggy. Oracle implementors moved async page request into buf_read_page_async function and (I think that due to a copy-paste error) replaced second parameter of buf_read_page_low from FALSE to TRUE which effectively forces read IO to be synchronous.

There is an unstream bug report for the issue https://bugs.mysql.com/bug.php?id=73583, but it ignored by Oracle.

Environment

CentOS 7
x86_64
Percona-Server-server-56-5.6.39-rel83.1.el7.x86_64

Smart Checklist

Activity

Show:

Jira Bot March 29, 2020 at 4:15 PM

Hello ,
It's been 507 days since this issue went into Incomplete and we haven't heard
from you on this.

At this point, our policy is to Close this issue, to keep things from getting
too cluttered. If you have more information about this issue and wish to
reopen it, please reply with a comment containing "jira-bot=reopen".

Jira Bot February 28, 2020 at 8:55 PM

Hello ,
It's jira-bot again. Your bug report is important to us, but we haven't heard
from you since the previous notification. If we don't hear from you on
this in 7 days, the issue will be automatically closed.

Jira Bot January 25, 2020 at 10:20 PM

Hello ,
I'm jira-bot, Percona's automated helper script. Your bug report is important
to us but we've been unable to reproduce it, and asked you for more
information. If we haven't heard from you on this in 3 more weeks, the issue
will be automatically closed.

Lalit Choudhary November 8, 2018 at 1:20 PM

Hi

could you please disable performnce_schema and try.

Add following entry in my.cnf
performance_schema=OFF
 This issue could be duplicate of  https://jira.percona.com/browse/PS-1621

Test case at https://jira.percona.com/browse/PS-1538

 

Sergei Turchanov August 17, 2018 at 6:50 AM

Is there any progress on this issue?

Incomplete

Details

Assignee

Reporter

Priority

Smart Checklist

Created April 1, 2018 at 6:02 AM
Updated March 6, 2024 at 1:33 PM
Resolved March 29, 2020 at 4:15 PM