offline-mode is persisted when doing dba.rebootClusterFromCompleteOutage

Description

Hello! When restoring the cluster via dba.rebootClusterFromCompleteOutage(), the variable offline-mode gets persisted, overriding the value we had manually set on the my.cnf.  This action is invisible to the human operator, and can lead to the mistake of starting a node assuming offline_mode will be ON, when in fact the persisted variable will disabled and allow traffic to the node.

I set some options and make the cluster multi-primary:

 

Then I make sure I have offline-mode=ON in *my.cnf* (it's the same on every node in the cluster)

 

 

I then (cleanly) stop every instance, and start them again, and they all appear as individual OFFLINE nodes:

 

 

Then we move to bootstrap; We verify that before bootstrapping, offline_mode is correctly ON:

 

 

And we verify the variable is NOT among the persisted variables:

 

 

We reboot the cluster (this command is the one triggering the issue)

 

 

And check persisted variables again:

When we restart the next time we will have *offline_mode=OFF* even when we would have expected it to be ON, because we have it set that way in my.cnf, and we were not aware that the variable was persisted....which doesn't make much sense to persist, or at least not without some seriously visible warning.

See also https://dev.mysql.com/doc/mysql-shell/8.0/en/configuring-automatic-rejoin-of-instances.html

 

Hope it makes sense!

Environment

None

AFFECTED CS IDs

CS0031017

Activity

Show:
Done

Details

Assignee

Reporter

Needs QA

Yes

Components

Priority

Smart Checklist

Created November 10, 2022 at 3:48 AM
Updated March 6, 2024 at 9:56 AM
Resolved May 11, 2023 at 10:29 PM