Disabling the table encryption while innodb-force-recovery=6 or read-only=1 leads to a server crash

General

Escalation

General

Escalation

Description

gdb:

(gdb) bt
+bt
#0  0x00007f027065fa01 in __pthread_kill (threadid=<optimized out>, signo=11) at ../nptl/sysdeps/unix/sysv/linux/pthread_kill.c:61
#1  0x000000000189e77f in my_write_core (sig=11) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/mysys/stacktrace.c:249
#2  0x0000000001618601 in handle_fatal_signal (sig=11) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/signal_handler.cc:223
#3  <signal handler called>
#4  0x00000000019f4c6d in OSMutex::enter (this=0x8) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/storage/innobase/include/sync0types.h:484
#5  0x0000000001ac1884 in os_event::set (this=0x0) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/storage/innobase/include/os0event.h:58
#6  0x0000000001ac16ff in os_event_set (event=0x0) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/storage/innobase/os/os0event.cc:277
#7  0x0000000001d396c3 in fil_crypt_set_encrypt_tables (val=0) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/storage/innobase/fil/fil0crypt.cc:2863
#8  0x00000000019f16b4 in innodb_encrypt_tables_update (thd=0x7f0243c19000, var=0x2cb4dc0 <mysql_sysvar_encrypt_tables>, var_ptr=0x2ded750 <srv_encrypt_tables>, save=0x7f0243c27cf8) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/storage/innobase/handler/ha_innodb.cc:21321
#9  0x00000000014f9d79 in sys_var_pluginvar::global_update (this=0x7f02627e27f8, thd=0x7f0243c19000, var=0x7f0243c27cd8) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/sql_plugin.cc:3843
#10 0x00000000013fad84 in sys_var::update (this=0x7f02627e27f8, thd=0x7f0243c19000, var=0x7f0243c27cd8) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/set_var.cc:183
#11 0x00000000013fc4f2 in set_var::update (this=0x7f0243c27cd8, thd=0x7f0243c19000) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/set_var.cc:836
#12 0x00000000013fbd61 in sql_set_variables (thd=0x7f0243c19000, var_list=0x7f0243c1ba10, free_joins=true) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/set_var.cc:672
#13 0x00000000014c83be in mysql_execute_command (thd=0x7f0243c19000, first_level=true) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/sql_parse.cc:3879
#14 0x00000000014cdaad in mysql_parse (thd=0x7f0243c19000, parser_state=0x7f0270c5c470) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/sql_parse.cc:5873
#15 0x00000000014c2921 in dispatch_command (thd=0x7f0243c19000, com_data=0x7f0270c5cc50, command=COM_QUERY) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/sql_parse.cc:1516
#16 0x00000000014c17a9 in do_command (thd=0x7f0243c19000) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/sql_parse.cc:1047
#17 0x00000000015f5468 in handle_connection (arg=0x7f025430e190) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/sql/conn_handler/connection_handler_per_thread.cc:312
#18 0x00000000018c2220 in pfs_spawn_thread (arg=0x7f026c818b20) at /home/hrvoje/worktable/PS-5.7.23-24_dbg/storage/perfschema/pfs.cc:2190
#19 0x00007f027065ae25 in start_thread (arg=0x7f0270c5d700) at pthread_create.c:308
#20 0x00007f026e82bbad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

sql:

# mysqld options required for replay:  --innodb-force-recovery=6
SET GLOBAL innodb_encrypt_tables=OFF;

Reproduced on debug build of: RM-390 (57a9574def950d8dfc8ae5ba6f1367a7bad39622)

I'm able to reproduce this on a release build as well:

mysql> show variables like 'innodb_force_recovery';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| innodb_force_recovery | 6     |
+-----------------------+-------+
1 row in set (0.00 sec)

mysql> SET GLOBAL innodb_encrypt_tables=OFF;
ERROR 2013 (HY000): Lost connection to MySQL server during query

Environment

None

Linked work items

is caused by

PS-1525

Implement the innodb_encrypt_tables system variable

Activity

Julia Vural
March 4, 2025 at 9:02 PM

It appears that this issue is no longer being worked on, so we are closing it for housekeeping purposes. If you believe the issue still exists, please open a new ticket after confirming it's present in the latest release.

Tomislav Plavcic
January 28, 2020 at 9:34 PM

I have another case where --read-only=1 so this is probably related to the fact that in both cases the server is in read-only mode. From what I see this affects 5.7.29 (debug/release), but not 8.0.18 (since the variable innodb_force_recovery is not available there).

Another test case:

# mysqld options required for replay:  --innodb-read-only=1
set global innodb_encrypt_tables='OFF';

GDB:

Core was generated by `/home/tomislav.plavcic/build/PS220120-percona-server-5.7.29-32-linux-x86_64-deb'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __pthread_kill (threadid=<optimized out>, signo=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
57      ../sysdeps/unix/sysv/linux/pthread_kill.c: No such file or directory.
[Current thread is 1 (Thread 0x7f08bab7f700 (LWP 12778))]
(gdb) bt
#0  __pthread_kill (threadid=<optimized out>, signo=11) at ../sysdeps/unix/sysv/linux/pthread_kill.c:57
#1  0x000055fd393bcf2b in my_write_core (sig=11) at /home/tomislav.plavcic/build/percona-server_dbg/mysys/stacktrace.c:261
#2  0x000055fd3910bf67 in handle_fatal_signal (sig=11) at /home/tomislav.plavcic/build/percona-server_dbg/sql/signal_handler.cc:230
#3  <signal handler called>
#4  0x000055fd394b3f89 in OSMutex::enter (this=0x8) at /home/tomislav.plavcic/build/percona-server_dbg/storage/innobase/include/sync0types.h:494
#5  0x000055fd3958ce3a in os_event::set (this=0x0) at /home/tomislav.plavcic/build/percona-server_dbg/storage/innobase/include/os0event.h:69
#6  0x000055fd3958cabe in os_event_set (event=0x0) at /home/tomislav.plavcic/build/percona-server_dbg/storage/innobase/os/os0event.cc:334
#7  0x000055fd398267f0 in fil_crypt_set_encrypt_tables (val=0) at /home/tomislav.plavcic/build/percona-server_dbg/storage/innobase/fil/fil0crypt.cc:2892
#8  0x000055fd394aff35 in innodb_encrypt_tables_update (thd=0x7f085f419000, var=0x55fd3a84a580 <mysql_sysvar_encrypt_tables>, var_ptr=0x55fd3a984e90 <srv_encrypt_tables>, save=0x7f085f427be0)
    at /home/tomislav.plavcic/build/percona-server_dbg/storage/innobase/handler/ha_innodb.cc:21424
#9  0x000055fd38fddb6c in sys_var_pluginvar::global_update (this=0x7f08adbe2030, thd=0x7f085f419000, var=0x7f085f427bc0) at /home/tomislav.plavcic/build/percona-server_dbg/sql/sql_plugin.cc:3851
#10 0x000055fd38ecfd2b in sys_var::update (this=0x7f08adbe2030, thd=0x7f085f419000, var=0x7f085f427bc0) at /home/tomislav.plavcic/build/percona-server_dbg/sql/set_var.cc:190
#11 0x000055fd38ed15f4 in set_var::update (this=0x7f085f427bc0, thd=0x7f085f419000) at /home/tomislav.plavcic/build/percona-server_dbg/sql/set_var.cc:843
#12 0x000055fd38ed0e52 in sql_set_variables (thd=0x7f085f419000, var_list=0x7f085f41ba88, free_joins=true) at /home/tomislav.plavcic/build/percona-server_dbg/sql/set_var.cc:679
#13 0x000055fd38fa7f94 in mysql_execute_command (thd=0x7f085f419000, first_level=true) at /home/tomislav.plavcic/build/percona-server_dbg/sql/sql_parse.cc:3944
#14 0x000055fd38fadd94 in mysql_parse (thd=0x7f085f419000, parser_state=0x7f08bab7e460, update_userstat=false) at /home/tomislav.plavcic/build/percona-server_dbg/sql/sql_parse.cc:5927
#15 0x000055fd38fa1db7 in dispatch_command (thd=0x7f085f419000, com_data=0x7f08bab7ed80, command=COM_QUERY) at /home/tomislav.plavcic/build/percona-server_dbg/sql/sql_parse.cc:1539
#16 0x000055fd38fa0b42 in do_command (thd=0x7f085f419000) at /home/tomislav.plavcic/build/percona-server_dbg/sql/sql_parse.cc:1060
#17 0x000055fd390e6760 in handle_connection (arg=0x7f0865589a60) at /home/tomislav.plavcic/build/percona-server_dbg/sql/conn_handler/connection_handler_per_thread.cc:325
#18 0x000055fd393e2e18 in pfs_spawn_thread (arg=0x7f08adff7020) at /home/tomislav.plavcic/build/percona-server_dbg/storage/perfschema/pfs.cc:2198
#19 0x00007f08ba58a6db in start_thread (arg=0x7f08bab7f700) at pthread_create.c:463
#20 0x00007f08b85ae88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Hrvoje Matijakovic
November 1, 2018 at 5:04 AM

Setting the innodb_force_recovery to values between 0-5 works ok:

mysql> show variables like 'innodb_force_recovery';
+-----------------------+-------+
| Variable_name         | Value |
+-----------------------+-------+
| innodb_force_recovery | 0     |
+-----------------------+-------+
1 row in set (0.01 sec)

mysql> SET GLOBAL innodb_encrypt_tables=OFF;
Query OK, 0 rows affected (0.00 sec)

Resize issue view side panel

Won't Do

Details

Assignee

Unassigned

Reporter

Hrvoje Matijakovic

Labels

rotated-key-encryption

Time tracking

15m logged

Affects versions

5.7.29-32

Priority

Low

Created November 1, 2018 at 4:55 AM

Updated March 4, 2025 at 9:02 PM

Resolved March 4, 2025 at 9:02 PM

Disabling the table encryption while innodb-force-recovery=6 or read-only=1 leads to a server crash

Description

Environment

Linked work items

is caused by

Activity

Julia Vural
March 4, 2025 at 9:02 PM

Tomislav Plavcic
January 28, 2020 at 9:34 PM

Hrvoje Matijakovic
November 1, 2018 at 5:04 AM

Details

Assignee

Reporter

Labels

Time tracking

Affects versions

Priority

Flag notifications

Something's gone wrong

Disabling the table encryption while innodb-force-recovery=6 or read-only=1 leads to a server crash

Description

Environment

Linked work items

is caused by

Activity

Julia Vural March 4, 2025 at 9:02 PM

Tomislav Plavcic January 28, 2020 at 9:34 PM

Hrvoje Matijakovic November 1, 2018 at 5:04 AM

Details

Assignee

Reporter

Labels

Time tracking

Affects versions

Priority

Flag notifications

Something's gone wrong

Julia Vural
March 4, 2025 at 9:02 PM

Tomislav Plavcic
January 28, 2020 at 9:34 PM

Hrvoje Matijakovic
November 1, 2018 at 5:04 AM