Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Issues

Select view

Select search mode

 
7 of 7

LP #1108035: PXC innodb_buffer_pool_restore_at_startup and wsrep_recovery

Done

Description

**Reported in Launchpad by Ilias Bertsimas last update 22-11-2013 06:59:14

We have noticed an issue on PXC when we have 30GB of buffer pool and the option innodb_buffer_pool_restore_at_startup enabled so the buffer pool is warmed up quick during startup.
When the node joins (state JOINER ) the cluster before the buffer pool restore is done reading all the pages it had loaded last time cluster flow control will be paused and the whole cluster will stall as the newly joined node is not able to keep up due to lack of io which is being consumed by the buffer pool restore operation. In our case with our workload it may stay stalled for 4-5 mins.

It may not be a bug but it would be best if it was possible to have the buffer pool restore operation complete first and then the wsrep continue joining the cluster or the other way around. Now percona server's innodb_buffer_pool_restore_at_startup and wsrep are oblivious to each other causing the issue described above.

Environment

None

Smart Checklist

Details

Assignee

Reporter

Labels

Priority

Smart Checklist

Created January 12, 2018 at 1:35 PM
Updated December 25, 2018 at 5:25 PM
Resolved December 15, 2023 at 2:26 PM

Activity

lpjirasyncJanuary 12, 2018 at 1:36 PM

**Comment from Launchpad by: Valerii Kravchuk on: 22-06-2013 08:49:56

Yes, we had the problem with innodb_buffer_pool_populate=ON at every node restart after successful full state transfer. As soon as we set it to OFF, we had not got a problem.

lpjirasyncJanuary 12, 2018 at 1:36 PM

**Comment from Launchpad by: Alex Yurchenko on: 07-06-2013 14:02:03

Valerii, Raghu, this does not even look like a bug. At first I'd make sure that there really was no state divergence.

lpjirasyncJanuary 12, 2018 at 1:36 PM

**Comment from Launchpad by: Raghavendra D Prabhu on: 07-06-2013 13:33:32

@Valerii, that doesn't seem to be related to this. Have you been able to repeat this with innodb_buffer_pool_populate ON and not repeat it with it being OFF? That looks like an entirely different issue. Yes, you can report a separate bug for this.

lpjirasyncJanuary 12, 2018 at 1:36 PM

**Comment from Launchpad by: Valerii Kravchuk on: 07-06-2013 11:42:05

Can mysqld startup with 144G of buffer pool, innodb_buffer_pool_populate set BUT wsrep_recover NOT set be affected by this bug (or fixed by the patch) when IST ends up with:

...
130607 1:09:43 [ERROR] WSREP: Local state seqno (2249918697) is greater than group seqno (2249916946): states diverged. Aborting to avoid potential data loss. Remove '/data/mysql//grastate.dat' file and restart if you wish to continue. (FATAL)
at galera/src/replicator_str.cpp:state_transfer_required():34

and node can join the cluster only after full state transfer? Should I report separate bug for this?

Should the manual mention this problem for PXC when describing innodb_buffer_pool_populate variable?

lpjirasyncJanuary 12, 2018 at 1:36 PM

**Comment from Launchpad by: Raghavendra D Prabhu on: 07-06-2013 10:08:37

Note, due to a bug, mysqld --verbose --help won't show innodb_buffer_pool_populate but it is there.