Client fails to send non-UTF-8 query analytics content to server

Description

Hello All,

We are currently testing PMM2, it is working quiet well so far, however we experiencing some problems with one of our Nodes. We currently monitor a Galera Cluster (Percona XtraDB Cluster 5.7.28-31.41.1.el8 running on CentOS Linux release 8.1.1911 (Core)) of 5 Nodes, 4 are already in PMM2, 3 of this 4 nodes are working well, system metric, mysql metrics and slow query logs are collected/sent to the server well.

One node only sends its metrics but not the slow query log, after enabling debug mode, I've seen the following messages:

Based on the messages, it seems, the rpc agent used internally is not able to marshall the message containing the slow query log, because of a non UTF-8 Character, however I analysed the slow query log of all servers and all slow query logs contain non UTF-8 characters, so why is only this one node not working?

May a bug in the pmm-agent? I am not sure, please help.

Additional Info:
We now turned off slow query log on that server, cleared the logfile entirely, then we enabled the slow query log again and restarted pmm-agent, afterward the slow queries were reported to the server for a few minutes until the same error appeared again. Seems like a bug in pmm-agent while marshalling slow query logs containing non UTF-8 chars and this is causing a cacellation of the rpc connection to the server which ends up in a whole restart of the agent it seems.

If you need any further Information, feel free to contact me.

How to test

None

How to document

None

AFFECTED CS IDs

274829

Attachments

3

Smart Checklist

hide

Activity

Show:

Jira Bot June 24, 2020 at 10:08 AM

To:
CC:

Hi, I'm jira-bot, Percona's Jira automation tool. I've detected that someone from
Percona has made an edit to the Summary field of an issue that you reported.

I'm not sentient (yet) so I'm not sure whether the person fixed a typo, changed
a few words, or completely rewrote the text. In any case, it is Percona Engineering's
intention to make the Summary and Description of an issue as accurate as possible
so that we're fixing the actual problem you're encountering, and to avoid
misunderstandings about symptoms and causes.

If the current Summary does not accurately reflect the problem you are reporting,
or if you feel the change was otherwise inappropriate in some way, please add a
new comment explaining things and we'll address it as soon as we can.

This message will be added only once per issue, regardless of how many times
the Summary is edited.

message-code:summary-edited

Nailya Kutlubaeva June 17, 2020 at 10:12 AM

tested and works ok.

Alexey Palazhchenko June 16, 2020 at 9:11 AM

There were a lot of changes – please re-test the latest dev-latest build

Former user June 5, 2020 at 1:50 PM

Verified with perconalab/pmm-server-fb:PR-878-a87ac46.

Done

Details

Assignee

Reporter

Priority

Components

Needs QA

Yes

Needs Doc

No

Fix versions

Story Points

Sprint

Affects versions

Smart Checklist Progress

Environment

OS: CentOS Linux release 8.1.1911 (Core) (minimal installations with epel repo activated and nagios plugins for mysql)

MySQL Galera: Percona XtraDB Cluster 5.7.28-31.41.1.el8

Langpacks:

glibc-all-langpacks.x86_64 2.28-72.el8 @BaseOS
glibc-langpack-de.x86_64 2.28-72.el8 @BaseOS
glibc-langpack-en.x86_64 2.28-72.el8 @BaseOS
langpacks-de.noarch 1.0-12.el8 @AppStream
langpacks-en.noarch 1.0-12.el8 @AppStream
slang.x86_64 2.3.2-3.el8 @anaconda

 

Most recent updates are installed.

Smart Checklist

Created January 24, 2020 at 3:22 PM
Updated November 13, 2024 at 8:44 AM
Resolved June 17, 2020 at 10:12 AM