Duplicate
Details
Details
Assignee
Unassigned
UnassignedReporter
张晨鹏
张晨鹏Time tracking
10m logged
Fix versions
Priority

Smart Checklist
Smart Checklist
Created October 25, 2019 at 4:19 AM
Updated March 6, 2024 at 10:03 PM
Resolved October 28, 2019 at 10:39 AM
pxc version
Percona XtraDB Cluster 5.7.26,WSREP version 31.37
kernal version
3.10.0-862.el7.x86_64
linux version
CentOS Linux release 7.5.1804
=============================
my.cnf:
key_buffer_size = 64M
myisam_sort_buffer_size = 8M
read_rnd_buffer_size = 262144
back_log = 50
innodb_strict_mode = 1
max_allowed_packet = 128M
max_connect_errors = 1000000
interactive_timeout = 900
wait_timeout = 900
skip_grant_tables = 0
skip_name_resolve = 1
host_cache_size = 0
sql_mode = 'STRICT_ALL_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'
sysdate_is_now = 1
transaction_isolation = REPEATABLE-READ
explicit_defaults_for_timestamp = 1
secure_file_priv = NULL
binlog_cache_size = 1M
binlog_format = ROW
max_binlog_size = 512M
max_binlog_cache_size = 4G
sync_binlog = 0
master_info_repository = TABLE
relay_log_info_repository = TABLE
slave_parallel_type = LOGICAL_CLOCK
slave_parallel_workers = 4
relay_log_recovery = ON
tmp-table-size = 32M
max-heap-table-size = 32M
query_cache_type = 0
query_cache_size = 0
max_connections = 300
thread_cache_size = 1000
open-files-limit = 65535
table-definition-cache = 4096
table-open-cache = 4096
join_buffer_size = 1M
read_buffer_size = 1M
sort_buffer_size = 1M
innodb_autoextend_increment = 8
innodb_autoinc_lock_mode = 2
innodb_buffer_pool_instances = 8
innodb_buffer_pool_size = 8G
innodb_buffer_pool_load_at_startup = ON
innodb_buffer_pool_dump_at_shutdown = ON
innodb_data_file_path = ibdata1:512M:autoextend
innodb_fast_shutdown = 1
innodb_file_per_table = 1
innodb_force_recovery = 0
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
innodb_lock_wait_timeout = 50
innodb_log_buffer_size = 8M
innodb_log_file_size = 512M
innodb_log_files_in_group = 3
innodb_open_files = 65535
innodb_purge_threads = 4
innodb_read_io_threads = 8
innodb_write_io_threads = 8
innodb_show_verbose_locks = 1
innodb_sync_spin_loops = 30
innodb_thread_concurrency = 24
innodb_print_all_deadlocks = 1
innodb_monitor_enable = all
==================================
before mysql crash
OS status:
Thu Oct 24 22:23:04 CST 2019
22:23:04 up 29 days, 2:18, 1 user, load average: 1.57, 1.06, 1.11
total used free shared buff/cache available
Mem: 32011 13221 4335 0 14453 15568
====================================
after mysql crash
mysql-error.log:
14:23:04 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
Please help us make Percona XtraDB Cluster better by reporting any
bugs at https://jira.percona.com/projects/PXC/issues
key_buffer_size=67108864
read_buffer_size=1048576
thread_count=38
connection_count=33
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 684363 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x7fde08012100
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fde8e539d30 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x3b)[0xf416fb]
/usr/sbin/mysqld(handle_fatal_signal+0x471)[0x7b5c51]
/lib64/libpthread.so.0(+0xf5d0)[0x7fe1e864c5d0]
/lib64/libc.so.6(+0x155ab8)[0x7fe1e686bab8]
/usr/sbin/mysqld(_Z13log_write_lowPKhm+0x5d)[0x103759d]
/usr/sbin/mysqld(_ZN5mtr_t7Command12finish_writeEm+0x55)[0x1062eb5]
/usr/sbin/mysqld(_ZN5mtr_t7Command7executeEv+0x247)[0x1064a07]
/usr/sbin/mysqld(_ZN5mtr_t6commitEv+0x89)[0x1064d69]
/usr/sbin/mysqld[0x12e6ac2]
/usr/sbin/mysqld(_Z12row_undo_insP11undo_node_tP9que_thr_t+0x36c)[0x12e72bc]
/usr/sbin/mysqld(_Z13row_undo_stepP9que_thr_t+0x425)[0x11136d5]
/usr/sbin/mysqld(_Z15que_run_threadsP9que_thr_t+0x897)[0x109a817]
/usr/sbin/mysqld[0x1168ffb]
/usr/sbin/mysqld[0x11699d0]
/usr/sbin/mysqld(_Z22trx_rollback_for_mysqlP5trx_t+0x152)[0x1169da2]
/usr/sbin/mysqld[0xfd8aa8]
/usr/sbin/mysqld(_Z15ha_rollback_lowP3THDb+0x12f)[0x830e9f]
/usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG8rollbackEP3THDb+0xe0)[0xedc580]
/usr/sbin/mysqld(_Z17ha_rollback_transP3THDb+0x66)[0x831116]
/usr/sbin/mysqld(_Z14trans_rollbackP3THD+0x3e)[0xdb319e]
/usr/sbin/mysqld(_ZN3THD7cleanupEv+0x52)[0xcb3272]
/usr/sbin/mysqld(_ZN3THD17release_resourcesEv+0x408)[0xcb6b68]
/usr/sbin/mysqld(handle_connection+0x1d6)[0xdd33f6]
/usr/sbin/mysqld(pfs_spawn_thread+0x1b4)[0xf59ff4]
/lib64/libpthread.so.0(+0x7dd5)[0x7fe1e8644dd5]
/lib64/libc.so.6(clone+0x6d)[0x7fe1e6813ead]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0): is an invalid pointer
Connection ID (thread ID): 711281
Status: KILL_CONNECTION
You may download the Percona XtraDB Cluster operations manual by visiting
http://www.percona.com/software/percona-xtradb-cluster/. You may find information
in the manual which will help you identify the cause of the crash.
/var/log/message:
Oct 24 22:23:05 systemd: mysql.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 24 22:23:05 mysql-systemd: /usr/bin/mysql-systemd: line 295: kill: (24570) - No such process
Oct 24 22:23:05 mysql-systemd: ERROR! Failed to stop the server with PID: 24570 with 1 status
Oct 24 22:23:05 systemd: mysql.service: control process exited, code=exited status=1
Oct 24 22:23:05 mysql-systemd: /usr/bin/mysql-systemd: line 232: kill: (24570) - No such process
Oct 24 22:23:05 systemd: mysql.service: control process exited, code=exited status=3
Oct 24 22:23:05 mysql-systemd: WARNING: mysql already dead
Oct 24 22:23:05 mysql-systemd: ERROR! Stale PID file: /xxx/xxx/mysql.pid
Oct 24 22:23:05 systemd: Unit mysql.service entered failed state.
Oct 24 22:23:05 systemd: mysql.service failed.