Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

TokuDB and MyRocks native partitioning never releases memory

Description

Running sysbench with 6 threads and 20 tables structured like

with table_open_cache=10 and table_open_cache_instances=8 on 8 core machine eventually consumes all memory.  On 8.0 one can see a significant amount og memory being utilized by TABLE_SHARE::mem_root:

In this ^^ example, we see 11Gb used here where the process VSZ is around 17Gb and about to trip OOM killer in this environment.

 

In discussion w/ , we believe that the NP handlers are allocating memory off the wrong mem root, they should be using THD for transient open/close memory instead of the global TABLE_SHARE which is never released until the server is shut down.

Environment

None

AFFECTED CS IDs

249553

Smart Checklist

Activity

Show:

Satya Bodapati April 1, 2019 at 6:46 AM

Satya Bodapati April 1, 2019 at 3:51 AM

the patch for fixes this issue too

 

Satya Bodapati March 29, 2019 at 6:49 AM
Edited

There is more to this. Not straightforward. Just using thd->mem_root is also wrong.

Partition_share is one per partitioned table across handlers --> will use new() and freed when there are no references to part_share.

Partition_base::create: should use table->mem_root instead of table_share->mem_root.

DROP/RENAME etc : should use thd->mem_root (TABLE* is not available). This is similar to non-native part.

Satya Bodapati March 27, 2019 at 7:08 AM

Native partitioning is always using table_share->mem_root. table_share is not freed until shutdown (unless evicted I think)

See

Done

Details

Assignee

Reporter

Labels

Time tracking

2d logged

Components

Fix versions

Affects versions

Priority

Smart Checklist

Created March 26, 2019 at 5:28 PM
Updated March 6, 2024 at 12:15 PM
Resolved April 1, 2019 at 11:23 AM