There are probably already existing reports on this, but I wanted to check what happens on new version and with protections in place when we operate with huge tables in MyRocks.
I set
rocksdb_block_cache_size=6G
and limit total memory available for MySQL to 10G
I have multiple sbtest tables, each 50000000 rows.
Now I execute the following sequence:
set session rocksdb_bulk_load = 1;
set session rocksdb_max_row_locks=40000000;
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
insert into sbtest2 (k,c,pad) select k,c,pad from sbtest3;
insert into sbtest4 (k,c,pad) select k,c,pad from sbtest2;
insert into sbtest5 (k,c,pad) select k,c,pad from sbtest4; ERROR 2013 (HY000): Lost connection to MySQL server during query
At this point MySQL was killed by OOM, as resident memory exceeded 10G
I understand this is probably too strict memory limits, but this is exactly to show the problem.
The expected behavior is for the transaction to fail, not server being killed by OOM.
The perfect solution would be for MyRocks to handle transaction without using much memory over specified limites.
Medium
Server version is 8.0.21-12
There are probably already existing reports on this, but I wanted to check what happens on new version and with protections in place when we operate with huge tables in MyRocks.
I set
rocksdb_block_cache_size=6G
and limit total memory available for MySQL to 10G
I have multiple sbtest tables, each 50000000 rows.
Now I execute the following sequence:
set session rocksdb_bulk_load = 1;
set session rocksdb_max_row_locks=40000000;
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
insert into sbtest2 (k,c,pad) select k,c,pad from sbtest3;
insert into sbtest4 (k,c,pad) select k,c,pad from sbtest2;
insert into sbtest5 (k,c,pad) select k,c,pad from sbtest4;
ERROR 2013 (HY000): Lost connection to MySQL server during query
At this point MySQL was killed by OOM, as resident memory exceeded 10G
I understand this is probably too strict memory limits, but this is exactly to show the problem.
The expected behavior is for the transaction to fail, not server being killed by OOM.
The perfect solution would be for MyRocks to handle transaction without using much memory over specified limites.