Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Everest and db operator pods restart many times

Description

Tested on gke with Everest v0.5.0-30-gabd86b6 version

Install postgres operator

Create postgres db of 2 nodes

Install mysql operator

Create mysql db of 3 nodes

Install mongodb operator

Create mongodb db of 3 nodes

Create manual and scheduled backups for all 3 db.

After a while, everest and other operators restart.

Pod status

DB status

Frontend displayed the error

Everest backend displayed the error in logs, it was not able to get the token from kubernetes

Everest backend, operator, pxc operator, psmdb operator, everest deployment logs and pod description yaml attached as pod_restart_backend_secrets_not_available.tar.gz

Note: The pod restart issue has been seen many times, but the backend issue with secrets came only once.

Environment

None

Attachments

2

Activity

Manish Chawla June 5, 2024 at 12:01 PM

This issue has not been observed lately, so closing this issue.

Michal December 28, 2023 at 4:43 PM

I could not replicate this. I can see the operators restarting but the root cause is reconciliation of the GKE cluster and the restarts are due to networking not being available during this time.

Is the cluster available during the time you see the restarts?

Diogo Recharte December 21, 2023 at 12:28 PM

Thank you Manish, I don’t have anything to add, that was what I observed as well. I already reached out to Andrew explaining this issue and he’ll take a look.

Manish Chawla December 21, 2023 at 12:22 PM

, Please add to this issue if you have more observations. Thanks.

Done

Details

Assignee

Reporter

Sprint

Affects versions

Priority

Smart Checklist

Created December 21, 2023 at 12:19 PM
Updated June 5, 2024 at 12:01 PM
Resolved June 5, 2024 at 12:01 PM