Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Too many connections brings down cluster

Description

When it fails one node just doesn't finish it's transactions anymore. The log is full of "Too many connections" messages and that's it. The cluster hangs.

Log messages:

// 2019-03-31T00:40:07.882609Z 688242683 [Note] Aborted connection 688242683 to db: 'db_name' user: ‘user’ host: ‘***’ (Got an error reading communication packets) 2019-03-31T00:40:07.882724Z 688245257 [Note] Aborted connection 688245257 to db: 'db_name' user: ‘user’ host: ‘***’ (Got an error reading communication packets) 2019-03-31T00:40:07.882847Z 688243397 [Note] Aborted connection 688243397 to db: 'db_name' user: ’user’ host: ‘****’ (Got an error reading communication packets) 2019-03-31T00:40:07.882964Z 688242021 [Note] Aborted connection 688242021 to db: 'db_name' user: ‘user’ host: ‘***’ (Got an error reading communication packets) 2019-03-31T00:40:08.471147Z 0 [Warning] Too many connections 2019-03-31T00:40:08.474331Z 0 [Warning] Too many connections 2019-03-31T00:40:08.479334Z 0 [Warning] Too many connections 2019-03-31T00:40:08.484204Z 0 [Warning] Too many connections 2019-03-31T00:40:08.485755Z 0 [Warning] Too many connections 2019-03-31T00:40:08.487274Z 0 [Warning] Too many connections 2019-03-31T00:40:08.501518Z 0 [Warning] Too many connections 2019-03-31T00:40:08.517456Z 0 [Warning] Too many connections

This happens regardless of the load, sometimes at night when the load is minimal.

Environment

mysql --version
mysql Ver 14.14 Distrib 5.7.23-23, for debian-linux-gnu (x86_64) using 6.3

Smart Checklist

Activity

Show:

Aaditya Dubey December 10, 2023 at 8:42 AM

Hi ,

Closing the report, no activity for a long!

Egor Lyutov April 8, 2019 at 8:30 AM

 Thanks for the answer.  I ran pt-pmp.  When another error occurs, I load the output.

Jericho Rivera April 4, 2019 at 5:53 AM

This might be similar to:

https://jira.percona.com/browse/PXC-2415

Which is affected by:

https://bugs.mysql.com/bug.php?id=92108

https://bugs.mysql.com/bug.php?id=91941

 

Can you run pt-pmp on the affected node when this happens again and then upload to this ticket?

Incomplete

Details

Assignee

Reporter

Labels

Priority

Smart Checklist

Created April 1, 2019 at 2:42 AM
Updated March 6, 2024 at 10:12 PM
Resolved December 10, 2023 at 8:42 AM

Flag notifications