mysqld got signal 11

Description

Debian-9.5, Percona Server 5.6.41-84.1 (Percona Server (GPL), Release 84.1, Revision b308619) running on HP DL180G6 (2x Xeon E5645 @ 2.40GHz) 48 Gb RAM, 4x SAS HDD HP 600GB 15K in RAID1 (for mysql-data)

Sudden crush occurs with next error:

 

23:23:38 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,.
something is definitely wrong and this may fail.
Please help us make Percona Server better by reporting any
bugs at http://bugs.percona.com/

key_buffer_size=104857600
read_buffer_size=33554432
max_used_connections=367
max_threads=402
thread_count=5
connection_count=3
It is possible that mysqld could use up to.
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 26453457 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x7fe6c4004520
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7fe6e0d33daf thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x2c)[0x562374e33f4c]
/usr/sbin/mysqld(handle_fatal_signal+0x481)[0x562374bbc801]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x110c0)[0x7fedde8810c0]
/usr/sbin/mysqld(+0x6e6440)[0x562374e6e440]
/usr/sbin/mysqld(_ZN17Query_cache_query16unlock_n_destroyEv+0x27)[0x562374bfa867]
/usr/sbin/mysqld(ZN11Query_cache12move_by_typeEPPhPP17Query_cache_blockPmS3+0x3f1)[0x562374bfdf91]
/usr/sbin/mysqld(_ZN11Query_cache10pack_cacheEv+0x71)[0x562374bfe4c1]
/usr/sbin/mysqld(_ZN11Query_cache4packEmj+0x44)[0x562374bfe7d4]
/usr/sbin/mysqld(_Z20reload_acl_and_cacheP3THDmP10TABLE_LISTPi+0x3e0)[0x562374c57810]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x4750)[0x562374c3f020]
/usr/sbin/mysqld(_ZN13sp_instr_stmt9exec_coreEP3THDPj+0x53)[0x562374d69eb3]
/usr/sbin/mysqld(_ZN12sp_lex_instr23reset_lex_and_exec_coreEP3THDPjb+0x29c)[0x562374d6c1fc]
/usr/sbin/mysqld(_ZN12sp_lex_instr29validate_lex_and_execute_coreEP3THDPjb+0xaa)[0x562374d6cb3a]

/usr/sbin/mysqld(_ZN13sp_instr_stmt7executeEP3THDPj+0x2f8)[0x562374d6cfd8]
/usr/sbin/mysqld(_ZN7sp_head7executeEP3THDb+0x463)[0x562374bc24a3]
/usr/sbin/mysqld(_ZN7sp_head17execute_procedureEP3THDP4ListI4ItemE+0x656)[0x562374bc56a6]
/usr/sbin/mysqld(_ZN14Event_job_data7executeEP3THDb+0x8c4)[0x562374cdd674]
/usr/sbin/mysqld(_ZN19Event_worker_thread3runEP3THDP28Event_queue_element_for_exec+0xe5)[0x562374db2665]
/usr/sbin/mysqld(event_worker_thread+0x4d)[0x562374db271d]
/usr/sbin/mysqld(pfs_spawn_thread+0x146)[0x562374e6b4e6]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7494)[0x7fedde877494]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7feddc7a8acf]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7fe524071dd0): is an invalid pointer
Connection ID (thread ID): 1116891
Status: NOT_KILLED

You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.

 

The time, when crash happens, there's no load on server (scripts, user access, backups etc.)

 

my.cnf:

===

[mysqld]

port = 3306
connect_timeout=100
net_read_timeout=90
net_write_timeout=120

max_connections = 400
max-connect-errors = 1000
skip-name-resolve
open_files_limit = 100000

datadir = /mysql-data
tmpdir = /ssd

join_buffer_size = 32M
sort_buffer_size = 32M
read_buffer_size = 32M
read_rnd_buffer_size = 32M
key_buffer_size = 100M

max_allowed_packet = 512M

query_cache_size = 64M
query_cache_type = ON

socket = /var/run/mysqld/mysqld.sock

table_open_cache = 10000

thread_cache_size = 32

  1.  

    1.  

      1. tmp_table_size = max_heap_table_size
        max_heap_table_size = 80M
        tmp_table_size = 80M

 

  1.  

    1.  

      1.  

        1.  

          1. Replication #####
            server-id = 1
            binlog-format = MIXED
            log_bin= /var/log/mysql/mysql-bin.log
            expire_logs_days= 5
            max_binlog_size= 1024M

          2. End replication #####

  1.  

    1.  

      1.  

        1.  

          1. Logs #####
            expire_logs_days=7
            sync_binlog = 0
            #log_slow_queries = /var/log/mysql/bide-slow.log
            #long_query_time = 10

  1.  

    1.  

      1.  

        1.  

          1. End Logs #####

  1.  

    1.  

      1.  

        1.  

          1. InnoDB #####
            innodb_file_per_table
            innodb_flush_method = O_DIRECT

  1. 50-80% of memory
    innodb_buffer_pool_size = 25G
    innodb_log_buffer_size = 16M

  1. Set .._log_file_size to 25 % of buffer pool size
    innodb_log_file_size = 900M

innodb_flush_log_at_trx_commit = 2

innodb_buffer_pool_instances = 30

interactive_timeout = 21600
wait_timeout = 7200
innodb_lock_wait_timeout = 200
innodb_read_io_threads = 8
innodb_write_io_threads = 8

  1. (numCPU) x 2 + numHDD
    innodb_thread_concurrency = 16

innodb_adaptive_hash_index = OFF

innodb_io_capacity = 1500
innodb_io_capacity_max = 2000

  1.  

    1.  

      1.  

        1.  

          1. End InnoDB #####

event_scheduler=ON

  1.  

    1.  

      1.  

        1.  

          1. Audit plugin #####

  2. load plugin
    plugin-load=server_audit=server_audit.so

  1. only audit connections and DDL queries
    server_audit_events=QUERY

  1. enable logging
    server_audit_logging=ON

server_audit_incl_users='D14_Bakushin'

  1. flat file
    server_audit_output_type=FILE
    server_audit_file_path=/var/log/mysql/bide-audit.log
    server_audit_file_rotate_size=100000000
    server_audit_file_rotations=9
    ########################

sql-mode=""

ignore-db-dir=lost+found

===

 Previously MySQL-5.6 was used, then it was removed and Percona Server was installed up over existing database files.

 

 

 

 

{{}}

Environment

None

Smart Checklist

Activity

Dmitry November 28, 2018 at 7:56 AM

Thank you,  

 

— Here from what version you are moved from MySQL to Percona server is an important thing. 

It was mysql-5.6.32 as I remember. Hardware and OS are the same.

 

— It would be good if you run mysql_upgrade which fix table structure differences and corrupt table problems.

Yes, I'll do that when it can be possible and monitor situation.

 

— Percona has it's own Audit_Log Plugin, tested with Percona server, use of it will be good to avoid problems.

Yes, we use MariaDB audit plugin. It worked without any problems with "vanilla" mysql-5.6.  I know about percona audit and will test it ASAP.

 

Lalit Choudhary November 12, 2018 at 1:11 PM

Hi Dmitry,

Thank you for information.

I did test this issue on Percona Server 5.6.41 with https://perconadev.atlassian.net/browse/PS-1085#icft=PS-1085 test case which has similar backtrace crash. I cannot reproduce described behavior. 

 

> Did you run mysql_upgrade after replacing the binaries? No. Simply removed MySQL and installed Percona Server. Possible tables "mismatches and errors"? Same way (removing mysql -> installing percona) I migrated from mysql to percona on slave server - it runs OK.

Here from what version you are moved from MySQL to Percona server is an important thing. 

It would be good if you run mysql_upgrade which fix table structure differences and corrupt table problems.

Also, In the provided log  I see following,

181108 2:23:49 server_audit: MariaDB Audit Plugin version 1.4.0 STARTED.

Percona has it's own Audit_Log Plugin, tested with Percona server, use of it will be good to avoid problems.

https://www.percona.com/doc/percona-server/5.7/management/audit_log_plugin.html

After fixing this, if you get crash then update this bug with crash error info. 

 

Dmitry November 8, 2018 at 3:07 PM

Thank you for answer.

 

> Did you run mysql_upgrade after replacing the binaries?

No. Simply removed MySQL and installed Percona Server. Possible tables "mismatches and errors"?

Same way (removing mysql -> installing percona) I migrated from mysql to percona on slave server - it runs OK.

 

> Also, Provide full mysql error log for crash instance

This was full error log. Also, at 01:00 AM some complicate srcipt started and crashed mysql after 1,5 hours of working with next errors:

2018-11-08 02:23:42 29128 [Note] Plugin 'FEDERATED' is disabled.
2018-11-08 02:23:43 29128 [ERROR] Can't open shared library '/usr/lib/mysql/plugin/libaudit_plugin.so' (errno: 0 /usr/lib/mysql/plugin/libaudit_plugin.so
2018-11-08 02:23:43 29128 [Warning] Couldn't load plugin named 'MCAFEE' with soname 'libaudit_plugin.so'.
2018-11-08 02:23:43 29128 [ERROR] Function 'server_audit' already exists
2018-11-08 02:23:43 29128 [Warning] Couldn't load plugin named 'server_audit' with soname 'server_audit.so'.
2018-11-08 02:23:43 29128 [Note] InnoDB: Using atomics to ref count buffer pool pages
2018-11-08 02:23:43 29128 [Note] InnoDB: The InnoDB memory heap is disabled
2018-11-08 02:23:43 29128 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-11-08 02:23:43 29128 [Note] InnoDB: Memory barrier is not used
2018-11-08 02:23:43 29128 [Note] InnoDB: Compressed tables use zlib 1.2.8
2018-11-08 02:23:43 29128 [Note] InnoDB: Using Linux native AIO
2018-11-08 02:23:43 29128 [Note] InnoDB: Using CPU crc32 instructions
2018-11-08 02:23:43 29128 [Note] InnoDB: Initializing buffer pool, size = 25.0G
2018-11-08 02:23:44 29128 [Note] InnoDB: Completed initialization of buffer pool
2018-11-08 02:23:44 29128 [Note] InnoDB: Highest supported file format is Barracuda.
2018-11-08 02:23:44 29128 [Note] InnoDB: The log sequence numbers 24284754405431 and 24284754405431 in ibdata files do not match the log sequence number
2018-11-08 02:23:44 29128 [Note] InnoDB: Database was not shutdown normally!
2018-11-08 02:23:44 29128 [Note] InnoDB: Starting crash recovery.
2018-11-08 02:23:44 29128 [Note] InnoDB: Reading tablespace information from the .ibd files...
2018-11-08 02:23:47 29128 [Note] InnoDB: Restoring possible half-written data pages.
2018-11-08 02:23:47 29128 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Last MySQL binlog file position 0 28703, file name mysql-bin.000151
2018-11-08 02:23:49 29128 [Note] InnoDB: 128 rollback segment(s) are active.
2018-11-08 02:23:49 29128 [Note] InnoDB: Waiting for purge to start
2018-11-08 02:23:49 29128 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.41-84.1 started; log sequence number 24457337590681
181108 2:23:49 server_audit: server_audit_incl_users set to 'D14_Bakushin'.
181108 2:23:49 server_audit: MariaDB Audit Plugin version 1.4.0 STARTED.
181108 2:23:49 server_audit: logging started to the file /var/log/mysql/bide-audit.log.
2018-11-08 02:23:49 29128 [Note] Recovering after a crash using /var/log/mysql/mysql-bin
2018-11-08 02:23:49 29128 [Note] Starting crash recovery...

2018-11-08 02:23:49 29128 [Note] Crash recovery finished.
2018-11-08 02:23:50 29128 [Note] RSA private key file not found: /mysql-data//private_key.pem. Some authentication plugins will not work.
2018-11-08 02:23:50 29128 [Note] RSA public key file not found: /mysql-data//public_key.pem. Some authentication plugins will not work.
2018-11-08 02:23:50 29128 [Note] Server hostname (bind-address): '*'; port: 3306
2018-11-08 02:23:50 29128 [Note] IPv6 is available.
2018-11-08 02:23:50 29128 [Note] - '::' resolves to '::';
2018-11-08 02:23:50 29128 [Note] Server socket created on IP: '::'.
2018-11-08 02:23:50 29128 [Warning] 'user' entry 'root@bide' ignored in --skip-name-resolve mode.
2018-11-08 02:23:50 29128 [Warning] 'proxies_priv' entry '@ root@bide' ignored in --skip-name-resolve mode.
2018-11-08 02:23:50 29128 [Note] Event Scheduler: Loaded 1 event
2018-11-08 02:23:50 29128 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.41-84.1-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Percona Server (GPL), Release 84.1, Revision b308619
2018-11-08 02:23:50 29128 [Note] Event Scheduler: scheduler thread started with id 1

 

> Please share event definition and definition of all object included in it like store procedure and tables it referring to.

mysql> select name from mysql.proc;

mysql> select name from mysql.proc; +-------------------------------------+ | name | +-------------------------------------+ | AllInsp | | lk_esvg | | lk_esvg_1 | | lk_patent_fizlic | | lk_patent_fizlic_1 | | lk_prazdn | | lk_prazdn_minus | | RUB | | create_synonym_db | | extract_schema_from_file_name | | extract_table_from_file_name | | format_bytes | | format_path | | format_statement | | format_time | | ps_is_account_enabled | | ps_is_consumer_enabled | | ps_is_instrument_default_enabled | | ps_is_instrument_default_timed | | ps_is_thread_instrumented | | ps_setup_disable_background_threads | | ps_setup_disable_consumer | | ps_setup_disable_instrument | | ps_setup_disable_thread | | ps_setup_enable_background_threads | | ps_setup_enable_consumer | | ps_setup_enable_instrument | | ps_setup_enable_thread | | ps_setup_reload_saved | | ps_setup_reset_to_default | | ps_setup_save | | ps_setup_show_disabled | | ps_setup_show_disabled_consumers | | ps_setup_show_disabled_instruments | | ps_setup_show_enabled | | ps_setup_show_enabled_consumers | | ps_setup_show_enabled_instruments | | ps_statement_avg_latency_histogram | | ps_thread_id | | ps_thread_stack | | ps_trace_statement_digest | | ps_trace_thread | | ps_truncate_all_tables | | sys_get_config | +-------------------------------------+

Our main database is 'analiz_0'. So, if filter according to main database we get only 6 rows:

 

mysql> show procedure status where Db='analiz_0'\G

*************************** 1. row *************************** Db: analiz_0 Name: AllInsp Type: PROCEDURE Definer: webuser@localhost Modified: 2017-10-06 11:38:12 Created: 2017-10-06 11:38:12 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8_general_ci *************************** 2. row *************************** Db: analiz_0 Name: lk_esvg Type: PROCEDURE Definer: webuser@127.0.0.1 Modified: 2017-11-15 12:52:28 Created: 2017-11-15 12:52:28 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8_general_ci *************************** 3. row *************************** Db: analiz_0 Name: lk_esvg_1 Type: PROCEDURE Definer: webuser@127.0.0.1 Modified: 2017-11-14 10:46:42 Created: 2017-11-14 10:46:42 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8_general_ci *************************** 4. row *************************** Db: analiz_0 Name: lk_patent_fizlic Type: PROCEDURE Definer: webuser@127.0.0.1 Modified: 2017-11-15 12:51:43 Created: 2017-11-15 12:51:43 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8_general_ci *************************** 5. row *************************** Db: analiz_0 Name: lk_patent_fizlic_1 Type: PROCEDURE Definer: webuser@127.0.0.1 Modified: 2017-11-14 10:43:32 Created: 2017-11-14 10:43:32 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8_general_ci *************************** 6. row *************************** Db: analiz_0 Name: RUB Type: PROCEDURE Definer: root@localhost Modified: 2016-09-23 13:01:49 Created: 2016-09-23 13:01:49 Security_type: DEFINER Comment: character_set_client: utf8 collation_connection: utf8_general_ci Database Collation: utf8_general_ci

 

There're complicate SQLs with hundreds or even thousands of queries. Pasting them here will be like a spamming.

 

> Provide SQL command and SHOW CREATE TABLE output for tables included in Store procedure

Same thing: too many information will be. Too many...

Maybe there's something else, more concrete to see and paste here?

 

 

Dmitry November 8, 2018 at 1:38 PM

Thank you for answer.

 

> Did you run mysql_upgrade after replacing the binaries?

No. Simply removed MySQL and installed Percona Server. Possible tables "mismatches and errors"?

Same way (removing mysql -> installing percona) I migrated from mysql to percona on slave server - it runs OK.

 

> Also, Provide full mysql error log for crash instance

This was full error log. Also, at 01:00 AM some complicate srcipt started and crashed mysql after 1,5 hours of working with next errors:

2018-11-08 02:23:42 29128 [Note] Plugin 'FEDERATED' is disabled.
2018-11-08 02:23:43 29128 [ERROR] Can't open shared library '/usr/lib/mysql/plugin/libaudit_plugin.so' (errno: 0 /usr/lib/mysql/plugin/libaudit_plugin.so
2018-11-08 02:23:43 29128 [Warning] Couldn't load plugin named 'MCAFEE' with soname 'libaudit_plugin.so'.
2018-11-08 02:23:43 29128 [ERROR] Function 'server_audit' already exists
2018-11-08 02:23:43 29128 [Warning] Couldn't load plugin named 'server_audit' with soname 'server_audit.so'.
2018-11-08 02:23:43 29128 [Note] InnoDB: Using atomics to ref count buffer pool pages
2018-11-08 02:23:43 29128 [Note] InnoDB: The InnoDB memory heap is disabled
2018-11-08 02:23:43 29128 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2018-11-08 02:23:43 29128 [Note] InnoDB: Memory barrier is not used
2018-11-08 02:23:43 29128 [Note] InnoDB: Compressed tables use zlib 1.2.8
2018-11-08 02:23:43 29128 [Note] InnoDB: Using Linux native AIO
2018-11-08 02:23:43 29128 [Note] InnoDB: Using CPU crc32 instructions
2018-11-08 02:23:43 29128 [Note] InnoDB: Initializing buffer pool, size = 25.0G
2018-11-08 02:23:44 29128 [Note] InnoDB: Completed initialization of buffer pool
2018-11-08 02:23:44 29128 [Note] InnoDB: Highest supported file format is Barracuda.
2018-11-08 02:23:44 29128 [Note] InnoDB: The log sequence numbers 24284754405431 and 24284754405431 in ibdata files do not match the log sequence number
2018-11-08 02:23:44 29128 [Note] InnoDB: Database was not shutdown normally!
2018-11-08 02:23:44 29128 [Note] InnoDB: Starting crash recovery.
2018-11-08 02:23:44 29128 [Note] InnoDB: Reading tablespace information from the .ibd files...
2018-11-08 02:23:47 29128 [Note] InnoDB: Restoring possible half-written data pages.
2018-11-08 02:23:47 29128 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Last MySQL binlog file position 0 28703, file name mysql-bin.000151
2018-11-08 02:23:49 29128 [Note] InnoDB: 128 rollback segment(s) are active.
2018-11-08 02:23:49 29128 [Note] InnoDB: Waiting for purge to start
2018-11-08 02:23:49 29128 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.41-84.1 started; log sequence number 24457337590681
181108 2:23:49 server_audit: server_audit_incl_users set to 'D14_Bakushin'.
181108 2:23:49 server_audit: MariaDB Audit Plugin version 1.4.0 STARTED.
181108 2:23:49 server_audit: logging started to the file /var/log/mysql/bide-audit.log.
2018-11-08 02:23:49 29128 [Note] Recovering after a crash using /var/log/mysql/mysql-bin
2018-11-08 02:23:49 29128 [Note] Starting crash recovery...

2018-11-08 02:23:49 29128 [Note] Crash recovery finished.
2018-11-08 02:23:50 29128 [Note] RSA private key file not found: /mysql-data//private_key.pem. Some authentication plugins will not work.
2018-11-08 02:23:50 29128 [Note] RSA public key file not found: /mysql-data//public_key.pem. Some authentication plugins will not work.
2018-11-08 02:23:50 29128 [Note] Server hostname (bind-address): '*'; port: 3306
2018-11-08 02:23:50 29128 [Note] IPv6 is available.
2018-11-08 02:23:50 29128 [Note] - '::' resolves to '::';
2018-11-08 02:23:50 29128 [Note] Server socket created on IP: '::'.
2018-11-08 02:23:50 29128 [Warning] 'user' entry 'root@bide' ignored in --skip-name-resolve mode.
2018-11-08 02:23:50 29128 [Warning] 'proxies_priv' entry '@ root@bide' ignored in --skip-name-resolve mode.
2018-11-08 02:23:50 29128 [Note] Event Scheduler: Loaded 1 event
2018-11-08 02:23:50 29128 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.41-84.1-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Percona Server (GPL), Release 84.1, Revision b308619
2018-11-08 02:23:50 29128 [Note] Event Scheduler: scheduler thread started with id 1

 

> Please share event definition and definition of all object included in it like store procedure and tables it referring to.

mysql> select name from mysql.proc;

gives 44 rows.

select name from mysql.proc\G
*************************** 1. row ***************************
name: AllInsp
*************************** 2. row ***************************
name: lk_esvg
*************************** 3. row ***************************
name: lk_esvg_1
*************************** 4. row ***************************
name: lk_patent_fizlic
*************************** 5. row ***************************
name: lk_patent_fizlic_1
*************************** 6. row ***************************
name: lk_prazdn
*************************** 7. row ***************************
name: lk_prazdn_minus
*************************** 8. row ***************************
name: RUB
*************************** 9. row ***************************
name: create_synonym_db
*************************** 10. row ***************************
name: extract_schema_from_file_name
*************************** 11. row ***************************
name: extract_table_from_file_name
*************************** 12. row ***************************
name: format_bytes
*************************** 13. row ***************************
name: format_path
*************************** 14. row ***************************
name: format_statement
*************************** 15. row ***************************
name: format_time
*************************** 16. row ***************************
name: ps_is_account_enabled
*************************** 17. row ***************************
name: ps_is_consumer_enabled
*************************** 18. row ***************************
name: ps_is_instrument_default_enabled
*************************** 19. row ***************************
name: ps_is_instrument_default_timed
*************************** 20. row ***************************
name: ps_is_thread_instrumented
*************************** 21. row ***************************
name: ps_setup_disable_background_threads
*************************** 22. row ***************************
name: ps_setup_disable_consumer
*************************** 23. row ***************************
name: ps_setup_disable_instrument
*************************** 24. row ***************************
name: ps_setup_disable_thread
*************************** 25. row ***************************
name: ps_setup_enable_background_threads
*************************** 26. row ***************************
name: ps_setup_enable_consumer
*************************** 27. row ***************************
name: ps_setup_enable_instrument
*************************** 28. row ***************************
name: ps_setup_enable_thread
*************************** 29. row ***************************
name: ps_setup_reload_saved
*************************** 30. row ***************************
name: ps_setup_reset_to_default
*************************** 31. row ***************************
name: ps_setup_save
*************************** 32. row ***************************
name: ps_setup_show_disabled
*************************** 33. row ***************************
name: ps_setup_show_disabled_consumers
*************************** 34. row ***************************
name: ps_setup_show_disabled_instruments
*************************** 35. row ***************************
name: ps_setup_show_enabled
*************************** 36. row ***************************
name: ps_setup_show_enabled_consumers
*************************** 37. row ***************************
name: ps_setup_show_enabled_instruments
*************************** 38. row ***************************
name: ps_statement_avg_latency_histogram
*************************** 39. row ***************************
name: ps_thread_id
*************************** 40. row ***************************
name: ps_thread_stack
*************************** 41. row ***************************
name: ps_trace_statement_digest
*************************** 42. row ***************************
name: ps_trace_thread
*************************** 43. row ***************************
name: ps_truncate_all_tables
*************************** 44. row ***************************
name: sys_get_config

Our main database is 'analiz_0'. So, if filter according to main database we get only 6 rows:

mysql> show procedure status where Db='analiz_0'\G

*************************** 1. row ***************************
Db: analiz_0
Name: AllInsp
Type: PROCEDURE
Definer: webuser@localhost
Modified: 2017-10-06 11:38:12
Created: 2017-10-06 11:38:12
Security_type: DEFINER
Comment:
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: utf8_general_ci
*************************** 2. row ***************************
Db: analiz_0
Name: lk_esvg
Type: PROCEDURE
Definer: webuser@127.0.0.1
Modified: 2017-11-15 12:52:28
Created: 2017-11-15 12:52:28
Security_type: DEFINER
Comment:
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: utf8_general_ci
*************************** 3. row ***************************
Db: analiz_0
Name: lk_esvg_1
Type: PROCEDURE
Definer: webuser@127.0.0.1
Modified: 2017-11-14 10:46:42
Created: 2017-11-14 10:46:42
Security_type: DEFINER
Comment:
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: utf8_general_ci
*************************** 4. row ***************************
Db: analiz_0
Name: lk_patent_fizlic
Type: PROCEDURE
Definer: webuser@127.0.0.1
Modified: 2017-11-15 12:51:43
Created: 2017-11-15 12:51:43
Security_type: DEFINER
Comment:
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: utf8_general_ci
*************************** 5. row ***************************
Db: analiz_0
Name: lk_patent_fizlic_1
Type: PROCEDURE
Definer: webuser@127.0.0.1
Modified: 2017-11-14 10:43:32
Created: 2017-11-14 10:43:32
Security_type: DEFINER
Comment:
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: utf8_general_ci
*************************** 6. row ***************************
Db: analiz_0
Name: RUB
Type: PROCEDURE
Definer: root@localhost
Modified: 2016-09-23 13:01:49
Created: 2016-09-23 13:01:49
Security_type: DEFINER
Comment:
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: utf8_general_ci

There're complicate SQLs with hundreds or even thousands of queries. Pasting them here will be like a spamming.

 

> Provide SQL command and SHOW CREATE TABLE output for tables included in Store procedure

Same thing: too many information will be. Too many... 

Maybe there's something else, more concrete to see and paste here?

 

 

 

Lalit Choudhary November 8, 2018 at 11:59 AM

Hi Dmitry,

Thank you for report and information.

The crash log has an event execution and procedure call. As you mentioned there's no load on the server (scripts, user access, backups etc.) at the time of crash then this crash could be triggered by an event call. 

Please share event definition and definition of all object included in it like store procedure and tables it referring to.

Provide SQL command and SHOW CREATE TABLE output for tables included in Store procedure.

Also, Provide full mysql error log for crash instance.

backtrace looks similar to https://jira.percona.com/browse/PS-1085

>Previously MySQL-5.6 was used, then it was removed and Percona Server was installed up over existing database files.

Did you run mysql_upgrade after replacing the binaries?

Cannot Reproduce

Details

Assignee

Reporter

Affects versions

Priority

Smart Checklist

Created November 8, 2018 at 9:10 AM
Updated March 6, 2024 at 12:42 PM
Resolved December 28, 2018 at 2:42 PM

Flag notifications