proxysql-admin crashing kubernetes nodes

Description

After updating to 1.6 version of operator a lot of proxysql-admin zombie  processes created on nodes causing stucks and failures of other pods.

 

  1. ps aux | grep proxysql-admin | grep defunct | wc -l
    97517
     

I have built docker image from this repo https://github.com/percona/percona-docker/tree/main/proxysql (Dockerfile.k8s) but with ubi7 instead of ubi8 and problem disappeared.

 

 

Environment

None

Smart Checklist

Activity

Slava Sarzhan February 4, 2021 at 4:30 PM

We can not use old version of proxysql due to the following bug https://jira.percona.com/browse/PSQLADM-256. So, we need to wait when a new version of proxysql will be available to fix zombie issues. 

Slava Sarzhan January 14, 2021 at 5:20 PM

I have built proxysql-1.7 image with old proxysql. The fix will be available in the next version of proxysql-admin. So, PXCO 1.8.0 will use the latest version of proxysql.

Sergey Pronin November 19, 2020 at 8:41 AM

Discussed in details w Mykola:

  • we use haproxy by default

  • bug is not critical

we will change (again) release notes for 1.6 and recommend using previous proxysql image. If by PXC 1.7 proxysql-admin issue is not fixed - we are going to build proxysql-1.7 image with old proxysql-admin.

 

cc

Sergey Pronin November 19, 2020 at 7:59 AM

 we need to rebuild 1.6.0 proxysql image with proxysql-admin v2.0.12. 

 

Slava Sarzhan November 18, 2020 at 1:59 PM

Hi ,

I can confirm this issue but it is not connected with OS version. I can reproduce it on centos7/centos8 and ubi7/ubi8. This issue is connected with the version of proxysql-admin (ver is 2.0.14 for PXCO 1.6.0) which we use for proxysql docker image. In PXCO 1.5.0 we use proxysql-admin ver 2.0.12 and this issue does not exist. I have created bug report for ProxySQL/Admin team regarding this issue . More detailed information regarding this issue can be found using the following link: https://jira.percona.com/browse/PSQLADM-282

As a workaround you can use proxysql image from PXCO 1.5.0 it should work.

Done

Details

Assignee

Reporter

Time tracking

2d 5m logged

Fix versions

Priority

Smart Checklist

Created November 17, 2020 at 9:52 AM
Updated March 5, 2024 at 6:02 PM
Resolved February 2, 2021 at 3:49 PM