Issues

Select view

Select search mode

 
50 of 200

Rollback causes MySQL to crash and restart

Incomplete

Description

The database deployed in a private environment crashes and restarts MySQL due to rollback. The error message is as follows:

2023-08-23T01:29:40.351223Z 142838 [ERROR] InnoDB: trying to read page [page id: space=977910041, page number=655360] in nonexisting or being-dropped tablespace
2023-08-23T01:29:40.351226Z 142838 [ERROR] InnoDB: Trying to do I/O to a tablespace which does not exist. I/O type: read, page: [page id: space=977910041, page number=655360], I/O length: 16384 bytes
2023-08-23T01:29:40.351230Z 142838 [ERROR] InnoDB: trying to read page [page id: space=977910041, page number=655360] in nonexisting or being-dropped tablespace
2023-08-23T01:29:40.351233Z 142838 [ERROR] InnoDB: Trying to do I/O to a tablespace which does not exist. I/O type: read, page: [page id: space=977910041, page number=655360], I/O length: 16384 bytes
2023-08-23T01:29:40.351236Z 142838 [ERROR] InnoDB: trying to read page [page id: space=977910041, page number=655360] in nonexisting or being-dropped tablespace
2023-08-23T01:29:40.368657Z 142838 [ERROR] [FATAL] InnoDB: Unable to read page [page id: space=977910041, page number=655360] into the buffer pool after 100 attempts. The most probable cause of this error may be that the table has been corrupted. Or, the table was compressed with with an algorithm that is not supported by this instance. If it is not a decompress failure, you can try to fix this problem by using innodb_force_recovery. Please see http://dev.mysql.com/doc/refman/5.7/en/ for more details. Aborting...
2023-08-23 01:29:40 0x7f2959ce3700  InnoDB: Assertion failure in thread 139815577073408 in file ut0ut.cc line 924
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.7/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
01:29:40 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
Please help us make Percona Server better by reporting any
bugs at https://bugs.percona.com/
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=679
max_threads=4097
thread_count=524
connection_count=523
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1639374 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Build ID: Not Available 
Server Version: 5.7.43-47-log Percona Server (GPL), Release 47, Revision ff1a4d42212
Thread pointer: 0x7f2518093500
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f2959ce2e10 thread_stack 0x40000
mysqld(my_print_stacktrace+0x3f)[0xf3a08f]
mysqld(handle_fatal_signal+0x5a1)[0xd69441]
/lib64/libpthread.so.0(+0x12cf0)[0x7f42ed3bbcf0]
/lib64/libc.so.6(gsignal+0x10f)[0x7f42eb541acf]
/lib64/libc.so.6(abort+0x127)[0x7f42eb514ea5]
mysqld[0x786443]
mysqld[0x7867e1]
mysqld(_Z16buf_page_get_genRK9page_id_tRK11page_size_tmP11buf_block_tmPKcmP5mtr_tbP7dberr_t+0x598)[0x11d90a8]
mysqld(_Z32btr_free_externally_stored_fieldP12dict_index_tPhPKhPKmP14page_zip_des_tmbP5mtr_t+0x3d9)[0x11b8829]
mysqld(_Z26btr_cur_pessimistic_updatemP9btr_cur_tPPmPP16mem_block_info_tS4_PP9big_rec_tP5upd_tmP9que_thr_tmP5mtr_t+0x2dc)[0x11b937c]
mysqld[0x12cfe31]
mysqld[0x12d0a85]
mysqld(_Z12row_undo_modP11undo_node_tP9que_thr_t+0xbd5)[0x12d3855]
mysqld(_Z13row_undo_stepP9que_thr_t+0x7a)[0x1115cca]
mysqld(_Z15que_run_threadsP9que_thr_t+0xc80)[0x10a1d50]
mysqld[0x116be0f]
mysqld[0x116c750]
mysqld(_Z22trx_rollback_for_mysqlP5trx_t+0x338)[0x116cc88]
mysqld[0xfec30b]
mysqld(_Z15ha_rollback_lowP3THDb+0xef)[0x7c19ff]
mysqld(_ZN13MYSQL_BIN_LOG8rollbackEP3THDb+0xbd)[0xed68ed]
mysqld(_Z17ha_rollback_transP3THDb+0x82)[0x7c1b72]
mysqld(_Z14trans_rollbackP3THD+0x2e)[0xd37d3e]
mysqld(_Z21mysql_execute_commandP3THDb+0x35ce)[0xc8860e]
mysqld(_Z11mysql_parseP3THDP12Parser_stateb+0x505)[0xc8cd75]
mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0xf96)[0xc8ddc6]
mysqld(_Z10do_commandP3THD+0x1ff)[0xc8f4cf]
mysqld(handle_connection+0x2e0)[0xd50110]
mysqld(pfs_spawn_thread+0x15b)[0xf588fb]
/lib64/libpthread.so.0(+0x81ca)[0x7f42ed3b11ca]
/lib64/libc.so.6(clone+0x43)[0x7f42eb52ce73]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7f2518012480): rollback
Connection ID (thread ID): 142838
Status: NOT_KILLED

Environment

Master slave architecture, one master and one slave;

Attachments

2
  • 23 Aug 2023, 10:23 AM
  • 23 Aug 2023, 10:19 AM

Details

Assignee

Reporter

Labels

Found by Automation

Yes

Planned Version/s

Needs QA

Yes

Affects versions

Priority

Smart Checklist

Created August 23, 2023 at 10:20 AM
Updated March 6, 2024 at 9:35 AM
Resolved December 10, 2023 at 6:37 AM

Activity

Show:

Aaditya Dubey December 10, 2023 at 6:37 AM

Hi ,

We still haven't heard any news from you. So I assume issue does not persist anymore and will close the ticket. If you disagree just reply and create a follow-up new Jira Ticket.

Aaditya Dubey October 10, 2023 at 9:14 AM

Hi ,

Thank you for the report.
Looks like it is happening due to corruption, However will be great if you have test case which repeats this issue, Please share if you have one.

zongyang August 23, 2023 at 10:22 AM

Important parameter configuration:

  1. buffer pool
    innodb_buffer_pool_size = 96g
    innodb_buffer_pool_instances = 8
    innodb_log_file_size = 1g
    innodb_sort_buffer_size = 4194304
    innodb_log_buffer_size = 134217728
     

  2. temp table
    max_heap_table_size = 67108864
    tmp_table_size = 67108864
     

  3. undo log
    innodb_undo_tablespaces=4
    innodb_undo_logs=128
    innodb_max_undo_log_size=5G
    innodb_undo_log_truncate = 1
    innodb_purge_rseg_truncate_frequency=128