missing online nodes in reader hostgroup while upgrade in progress

Description

Opening this as it was discussed already in slack.
When the upgrade is done PXC nodes are upgraded 1 by one (writer last), but when one node is terminated and becomes online it is not added immediately to proxysql config so you can get into situation without online hosts in reader group.

Here's how it looks:

node-2 was upgraded and ready, but it was not yet added to proxysql config and node-1 was then terminated for upgrade (SHUNNED status here), so no nodes left for reads

Environment

None

Smart Checklist

Activity

Done

Details

Assignee

Reporter

Time tracking

1d 5h 50m logged

Fix versions

Affects versions

Priority

Smart Checklist

Created June 26, 2020 at 7:52 AM
Updated March 5, 2024 at 6:14 PM
Resolved July 6, 2020 at 2:35 PM