Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Killing ALTER partition table query causes table->get_ref_count() == 0 assertion failure

Description

Description

When killing an alter partition table query in the stage committing alter table to storage engine, the InnoDB will roll back the operation with dict_partitioned_table_remove_from_cache.

size_t name_len = strlen(name); .... if ((strncmp(name, prev_table->name.m_name, name_len) == 0) && dict_table_is_partition(prev_table)) { btr_drop_ahi_for_table(prev_table); dict_table_remove_from_cache(prev_table); }

InnoDB only compares the first name_len bytes of string name and prev_table->name.m_name in strncmp. Imaging the first name_len bytes of prev_table->name.m_name equals the name, InnoDB will remove the table from the Dictionary Cache, which removes the wrong table. InnoDB will hit the Assertion failure in dict_table_remove_from_cache_low.

(gdb) p name $2 = 0x7f26ecdf4070 "test/a" (gdb) p prev_table->name.m_name $3 = 0x7f26cba1e300 "test/a_1#p#p0" (gdb) p name_len $4 = 6

Because the first 6 bytes of test/a_1#p#p0 equal test/a in my test case, the InnoDB will remove the test.a_1 instead of test.a during the query

(gdb) p thd->m_query_string $5 = {str = 0x7f26c8085030 "ALTER TABLE test.a DROP PARTITION pmax", length = 38}

The assertion fails.

ut_a(table->get_ref_count() == 0);
(gdb) p table->n_ref_count._M_i $6 = 1

The error log is

2024-06-17T04:55:36.556026Z 303 [ERROR] [MY-013183] [InnoDB] Assertion failure: dict0dict.cc:1894:table->get_ref_count() == 0 thread 139805159565056 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 04:55:36 UTC - mysqld got signal 6 ; Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware. Build ID: 6a40cea985cb9f0bc84a984b229e27b22b31f04e Server Version: 8.0.28-19-debug Percona Server (GPL), Release 19, Revision 31e88966cd3-debug Thread pointer: 0x7f26c8020000 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7f26ecdf9ba8 thread_stack 0x100000 /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x55) [0x5565462bf3a9] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(print_fatal_signal(int)+0x2b8) [0x5565450bbc2e] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(my_server_abort()+0x74) [0x5565450bbf6d] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(my_abort()+0xd) [0x5565462b5771] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(ut_dbg_assertion_failed(char const*, char const*, unsigned long)+0x1e3) [0x55654679091c] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(+0x52b9228) [0x55654690d228] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(dict_table_remove_from_cache(dict_table_t*)+0x1d) [0x55654690d93a] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(dict_partitioned_table_remove_from_cache(char const*)+0x184) [0x55654690dac1] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(+0x4dabace) [0x5565463fface] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(+0x3928f40) [0x556544f7cf40] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(mysql_alter_table(THD*, char const*, char const*, HA_CREATE_INFO*, TABLE_LIST*, Alter_info*)+0x3ea4) [0x556544f8954f] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(Sql_cmd_alter_table::execute(THD*)+0x5e1) [0x556544d8ddc3] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(mysql_execute_command(THD*, bool)+0x5c1a) [0x556544e93db2] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(dispatch_sql_command(THD*, Parser_state*, bool)+0x7ce) [0x556544e9609a] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(dispatch_command(THD*, COM_DATA const*, enum_server_command)+0x16fe) [0x556544e8b388] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(do_command(THD*)+0x5c9) [0x556544e89317] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(+0x3a4fdb2) [0x5565450a3db2] /home/jinyou.ma/opt/mysql/8.0.28/bin/mysqld(+0x551cf2d) [0x556546b70f2d] /lib64/libpthread.so.0(+0x81cf) [0x7f270fb7d1cf] /lib64/libc.so.6(clone+0x43) [0x7f270dbd0d83] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7f26c8085030): ALTER TABLE test.a DROP PARTITION pmax Connection ID (thread ID): 303 Status: KILL_CONNECTION

The backtrace is

(gdb) bt #0 0x00007f270dbe5a4f in raise () from /lib64/libc.so.6 #1 0x00007f270dbb8db5 in abort () from /lib64/libc.so.6 #2 0x00005565450bc073 in my_server_abort () at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/signal_handler.cc:284 #3 0x00005565462b5771 in my_abort () at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/mysys/my_init.cc:264 #4 0x000055654679091c in ut_dbg_assertion_failed (expr=0x5565487ca423 "table->get_ref_count() == 0", file=0x5565487c9d08 "/mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/storage/innobase/dict/dict0dict.cc", line=1894) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/storage/innobase/ut/ut0dbg.cc:99 #5 0x000055654690d228 in dict_table_remove_from_cache_low (table=0x7f26cb97a8b8, lru_evict=0) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/storage/innobase/dict/dict0dict.cc:1894 #6 0x000055654690d93a in dict_table_remove_from_cache (table=0x7f26cb97a8b8) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/storage/innobase/dict/dict0dict.cc:1977 #7 0x000055654690dac1 in dict_partitioned_table_remove_from_cache (name=0x7f26ecdf4070 "test/a") at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/storage/innobase/dict/dict0dict.cc:2007 #8 0x00005565463fface in innobase_dict_cache_reset (schema_name=0x7f26c8085ba8 "test", table_name=0x7f26c8085bc0 "a") at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/storage/innobase/handler/ha_innodb.cc:4295 #9 0x0000556544f7cf40 in mysql_inplace_alter_table (thd=0x7f26c8020000, schema=..., new_schema=..., table_def=0x7f26cb9783d8, altered_table_def=0x7f26c80977d8, table_list=0x7f26c8085fc0, table=0x0, altered_table=0x7f26c80ee820, ha_alter_info=0x7f26ecdf4e30, inplace_supported=HA_ALTER_INPLACE_NO_LOCK_AFTER_PREPARE, alter_ctx=0x7f26ecdf5980, columns=std::set with 0 elements, fk_key_info=0x7f26c812ca50, fk_key_count=0, fk_invalidator=0x7f26ecdf4d70) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/sql_table.cc:13726 #10 0x0000556544f8954f in mysql_alter_table (thd=0x7f26c8020000, new_db=0x7f26c8085ba8 "test", new_name=0x0, create_info=0x7f26ecdf6e60, table_list=0x7f26c8085fc0, alter_info=0x7f26ecdf6cc0) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/sql_table.cc:17635 #11 0x0000556544d8ddc3 in Sql_cmd_alter_table::execute (this=0x7f26c8086648, thd=0x7f26c8020000) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/sql_alter.cc:369 #12 0x0000556544e93db2 in mysql_execute_command (thd=0x7f26c8020000, first_level=true) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/sql_parse.cc:4828 #13 0x0000556544e9609a in dispatch_sql_command (thd=0x7f26c8020000, parser_state=0x7f26ecdf8a60, update_userstat=false) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/sql_parse.cc:5466 #14 0x0000556544e8b388 in dispatch_command (thd=0x7f26c8020000, com_data=0x7f26ecdf9b30, command=COM_QUERY) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/sql_parse.cc:2031 #15 0x0000556544e89317 in do_command (thd=0x7f26c8020000) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/sql_parse.cc:1414 #16 0x00005565450a3db2 in handle_connection (arg=0x7f26d254c200) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/sql/conn_handler/connection_handler_per_thread.cc:308 #17 0x0000556546b70f2d in pfs_spawn_thread (arg=0x7f270d05bfa0) at /mnt/jenkins/workspace/ps8.0-autobuild-RELEASE/test/percona-server-8.0.28-19/storage/perfschema/pfs.cc:2948 #18 0x00007f270fb7d1cf in start_thread () from /lib64/libpthread.so.0 #19 0x00007f270dbd0d83 in clone () from /lib64/libc.so.6

Reproduce

  • create two tables in mysql

create database test; create table test.a ( x int) PARTITION BY RANGE (x) ( PARTITION p0 VALUES LESS THAN (10000), PARTITION pmax VALUES LESS THAN MAXVALUE ); create table test.a_1 like test.a; select count(*) from test.a_1;

  • kill alter query in shell

while true; do { mysql -BNe 'select concat("kill ",id ,";") from information_schema.processlist where state = "committing alter table to storage engine";' | mysql -vvv ; } ; done

  • execute alter query in second shell

while true; do { mysql -BNe "ALTER TABLE test.a ADD PARTITION (PARTITION pmax VALUES LESS THAN MAXVALUE);" ; mysql -BNe " ALTER TABLE test.a DROP PARTITION pmax;" ; }; done

Environment

None

AFFECTED CS IDs

CS0046848, CS0049899

Activity

jinyou.ma 
October 11, 2024 at 5:28 AM

The following patch will fix the issue

diff --git a/storage/innobase/dict/dict0dict.cc b/storage/innobase/dict/dict0dict.cc index 5c1e6896638..f99114d055e 100644 --- a/storage/innobase/dict/dict0dict.cc +++ b/storage/innobase/dict/dict0dict.cc @@ -2006,7 +2006,8 @@ void dict_partitioned_table_remove_from_cache(const char *name) { } if ((strncmp(name, prev_table->name.m_name, name_len) == 0) && - dict_table_is_partition(prev_table)) { + dict_table_is_partition(prev_table) && + (strncmp(dict_name::PART_SEPARATOR, prev_table->name.m_name + name_len, dict_name::PART_SEPARATOR_LEN) == 0)) { btr_drop_ahi_for_table(prev_table); dict_table_remove_from_cache(prev_table); }

Details

Assignee

Reporter

Upstream Bug URL

Needs QA

Affects versions

Priority

Created June 17, 2024 at 5:21 AM
Updated October 15, 2024 at 5:57 PM

Flag notifications