Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

pods not restarted if custom config is updated inside secret or configmap

Description

If we store custom config inside configmap or secret and then later update the secret/config map with new config the pods won't be restarted so they will use the old config.

Initially used variable:

mysql> show variables like '%wsrep_debug%'; +---------------+--------+ | Variable_name | Value | +---------------+--------+ | wsrep_debug | SERVER | +---------------+--------+ 1 row in set (0.01 sec)

Secret after updating:

$ k get secrets cluster1-pxc -oyaml apiVersion: v1 data: init.cnf: W215c3FsZF0Kd3NyZXBfZGVidWc9Q0xJRU5UCg== kind: Secret metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","data":{"init.cnf":"W215c3FsZF0Kd3NyZXBfZGVidWc9Q0xJRU5UCg=="},"kind":"Secret","metadata":{"annotations":{},"name":"cluster1-pxc","namespace":"pxc-test"}} creationTimestamp: "2021-07-07T16:04:03Z" name: cluster1-pxc namespace: pxc-test resourceVersion: "27617" uid: a6a26b0f-034d-42fc-8201-e994a7164378 type: Opaque $ echo -n "W215c3FsZF0Kd3NyZXBfZGVidWc9Q0xJRU5UCg==" | base64 --decode [mysqld] wsrep_debug=CLIENT

File in the container:

$ k exec -ti cluster1-pxc-0 -c pxc -- cat /etc/percona-xtradb-cluster.conf.d/init.cnf [mysqld] wsrep_debug=CLIENT

Variable value in the server is still the old one:

mysql> show variables like '%wsrep_debug%'; +---------------+--------+ | Variable_name | Value | +---------------+--------+ | wsrep_debug | SERVER | +---------------+--------+ 1 row in set (0.00 sec)

As you can see the mongod is still using the old value for the variable.

It is the same if we just use a custom created config map instead of a secret.

Environment

None

Smart Checklist

Activity

Show:

Tomislav Plavcic July 13, 2021 at 6:25 AM
Edited

It seems this might not work for proxysql when it will be fixed for pxc/haproxy because of this:

# IMPORTANT INFORMATION REGARDING THIS CONFIGURATION FILE: ######################################################################################## # On startup, ProxySQL reads its config file (if present) to determine its datadir. # What happens next depends on if the database file (disk) is present in the defined # datadir (i.e. "/var/lib/proxysql/proxysql.db"). # # If the database file is found, ProxySQL initializes its in-memory configuration from # the persisted on-disk database. So, disk configuration gets loaded into memory and # then propagated towards the runtime configuration. # # If the database file is not found and a config file exists, the config file is parsed # and its content is loaded into the in-memory database, to then be both saved on-disk # database and loaded at runtime. # # IMPORTANT: If a database file is found, the config file is NOT parsed. In this case # ProxySQL initializes its in-memory configuration from the persisted on-disk # database ONLY. In other words, the configuration found in the proxysql.cnf # file is only used to initial the on-disk database read on the first startup. # # In order to FORCE a re-initialise of the on-disk database from the configuration file # the ProxySQL service should be started with "systemctl start proxysql-initial". # ########################################################################################

This part particularly:

# IMPORTANT: If a database file is found, the config file is NOT parsed. In this case # ProxySQL initializes its in-memory configuration from the persisted on-disk # database ONLY.

So even if we change the config file content in the secret/config map and it is visible in the pod and restart the pods for proxysql it will only read the config from it's own internal database and not the config.

I think most probably we will need to document this unless there is some proxysql startup option to always read the config file and even if so that we actually want this.

Done

Details

Assignee

Reporter

Labels

Needs QA

Yes

Fix versions

Affects versions

Priority

Smart Checklist

Created July 7, 2021 at 5:09 PM
Updated March 5, 2024 at 5:48 PM
Resolved October 8, 2021 at 3:21 PM

Flag notifications