Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Node cannot join cluster unless encryption is explicitly disabled

Description

When attempting to add a node to a newly bootstrapped cluster, SST fails with the following error:

 

-------- 2020-03-26T01:03:06.850222Z 0 [ERROR] [MY-000000] [Galera] handshake with remote endpoint ssl://192.168.1.101:4567 failed: asio.ssl:67567754: 'invalid padding' ( 67567754: 'error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding') This error is often caused by SSL issues. For more information, please see: https://per.co.na/pxc/encrypt_cluster_traffic -------- 2020-03-26T01:03:08.350426Z 0 [ERROR] [MY-000000] [Galera] handshake with remote endpoint ssl://192.168.1.101:4567 failed: asio.ssl:67567754: 'invalid padding' ( 67567754: 'error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding') This error is often caused by SSL issues. For more information, please see: https://per.co.na/pxc/encrypt_cluster_traffic -------- 2020-03-26T01:03:09.850442Z 0 [ERROR] [MY-000000] [Galera] handshake with remote endpoint ssl://192.168.1.101:4567 failed: asio.ssl:67567754: 'invalid padding' ( 67567754: 'error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding') This error is often caused by SSL issues. For more information, please see: https://per.co.na/pxc/encrypt_cluster_traffic -------- 2020-03-26T01:03:11.350509Z 0 [ERROR] [MY-000000] [Galera] handshake with remote endpoint ssl://192.168.1.101:4567 failed: asio.ssl:67567754: 'invalid padding' ( 67567754: 'error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding') This error is often caused by SSL issues. For more information, please see: https://per.co.na/pxc/encrypt_cluster_traffic

The workaround is to explicitly set pxc-encrypt-cluster-traffic=OFF in the cnf file

 

OS: Centos 7

 

full cnf: 

 

[client] socket=/var/lib/mysql/mysql.sock [mysqld] serverid=1 datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock log-error=/var/log/mysqld.log pid-file=/var/run/mysqld/mysqld.pidlog_error_verbosity=3 binlog_expire_logs_seconds=604800 wsrep_provider=/usr/lib64/galera4/libgalera_smm.so wsrep_cluster_address=gcomm://192.168.1.101,192.168.1.32,192.168.1.117 binlog_format=ROW wsrep_slave_threads=8 wsrep_log_conflicts innodb_autoinc_lock_mode=2 wsrep_node_address=192.168.1.32wsrep_cluster_name=dani-pxc-cluster wsrep_node_name=pxc2 pxc_strict_mode=ENFORCING wsrep_sst_method=xtrabackup-v2

 

Environment

None

Smart Checklist

Activity

Lalit Choudhary March 26, 2020 at 10:41 AM

 

Working Test:

node1 and node2

[mysqld] ssl-ca=/etc/mysql/certs/ca.pem ssl-cert=/etc/mysql/certs/server-cert.pem ssl-key=/etc/mysql/certs/server-key.pem [client] ssl-ca=/etc/mysql/certs/ca.pem ssl-cert=/etc/mysql/certs/client-cert.pem ssl-key=/etc/mysql/certs/client-key.pem

 

 

mysql> show variables like '%ssl%'; +--------------------+----------------------------------+ | Variable_name | Value | +--------------------+----------------------------------+ | have_openssl | YES | | have_ssl | YES | | mysqlx_ssl_ca | | | mysqlx_ssl_capath | | | mysqlx_ssl_cert | | | mysqlx_ssl_cipher | | | mysqlx_ssl_crl | | | mysqlx_ssl_crlpath | | | mysqlx_ssl_key | | | ssl_ca | /etc/mysql/certs/ca.pem | | ssl_capath | | | ssl_cert | /etc/mysql/certs/server-cert.pem | | ssl_cipher | | | ssl_crl | | | ssl_crlpath | | | ssl_fips_mode | OFF | | ssl_key | /etc/mysql/certs/server-key.pem | +--------------------+----------------------------------+ 17 rows in set (0.02 sec) mysql> show variables like 'pxc%encrypt%'; +-----------------------------+-------+ | Variable_name | Value | +-----------------------------+-------+ | pxc_encrypt_cluster_traffic | ON | +-----------------------------+-------+ 1 row in set (0.01 sec)

 

Copy the same SSL files from node1, SST successful and node2 started as part of a cluster.

[root@node2 mysql]# mkdir -p /etc/mysql/certs/ [root@node2 mysql]# cd /etc/ [root@node2 etc]# chown -R mysql:mysql mysql/ [root@node2 etc]# service mysqld start Redirecting to /bin/systemctl start mysqld.service [root@node2 etc]# mysql> show status like 'wsrep_cluster_size'; +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | wsrep_cluster_size | 2 | +--------------------+-------+ 1 row in set (0.01 sec)

 

Daniel Guzman Burgos March 26, 2020 at 10:26 AM

KennT March 26, 2020 at 3:08 AM

The SSL certs for each node have to be the same. So you will have to copy the SSL certs over for each node.

https://www.percona.com/doc/percona-xtradb-cluster/LATEST/security/encrypt-traffic.html

Not a Bug

Details

Assignee

Reporter

Affects versions

Priority

Smart Checklist

Created March 26, 2020 at 1:31 AM
Updated March 30, 2020 at 2:27 PM
Resolved March 26, 2020 at 10:41 AM

Flag notifications