Cannot add MySQL to node "Failed to get PMM Server parameters from local pmm-agent: pmm-agent is not connected to PMM Server." when adding MySQL

Description

New client install on a DB server.
Installed latest version of pmm2_client using yum.

"pmm-admin config" appears to work : no errors reported.

"pmm-admin add mysql" fails with the error:
"Failed to get PMM Server parameters from local pmm-agent: pmm-agent is not connected to PMM Server."

Running the 'config' and 'add' with debug gives the following:

[root@[hostname] ~]# pmm-admin config --debug --force --server-insecure-tls --server-url=https://admin:'[password]'@[pmmServerIP]:443 [agentServerIP] generic [nodeName]
DEBUG 2021-08-18 13:04:34.17419395Z: Running: pmm-agent --server-address=[pmmServerIP]:443 --server-username=admin --server-password=[password] --server-insecure-tls --debug setup --force --metrics-mode=auto [agentServerIP] generic [nodeName]
DEBUG 2021-08-18 13:04:36.127131469Z: Result: &commands.configResult{Warning:"", Output:"Checking local pmm-agent status...\npmm-agent is running.\nRegistering pmm-agent on PMM Server...\nRegistered.\nConfiguration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated.\nReloading pmm-agent configuration...\nConfiguration reloaded.\nChecking local pmm-agent status...\npmm-agent is running."}
DEBUG 2021-08-18 13:04:36.127176577Z: Error: <nil>
Checking local pmm-agent status...
pmm-agent is running.
Registering pmm-agent on PMM Server...
Registered.
Configuration file /usr/local/percona/pmm2/config/pmm-agent.yaml updated.
Reloading pmm-agent configuration...
Configuration reloaded.
Checking local pmm-agent status...
pmm-agent is running.
[root@[hostname] ~]# pmm-admin add mysql --debug --username=[pmm2_user] --password=[password] --query-source=perfschema --service-name=[nodeName] --host=127.0.0.1 --port=3306 --environment=PROD
DEBUG 2021-08-18 13:04:56.745545862Z: POST /local/Status HTTP/1.1
Host: 127.0.0.1:7777
User-Agent: Go-http-client/1.1
Content-Length: 3
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

{}

DEBUG 2021-08-18 13:04:56.747133428Z: HTTP/1.1 200 OK
Content-Length: 412
Content-Type: application/json
Date: Wed, 18 Aug 2021 13:04:56 GMT
Grpc-Metadata-Content-Type: application/grpc

{
"agent_id": "/agent_id/dcded051-d223-4068-b821-1421d9809f0e",
"runs_on_node_id": "",
"server_info": {
"url": "https://admin:[password]@[pmmServerIP]:443/",
"insecure_tls": true,
"connected": false,
"version": "",
"latency": null,
"clock_drift": null
},
"agents_info": [
],
"config_filepath": "/usr/local/percona/pmm2/config/pmm-agent.yaml",
"agent_version": "2.20.0"
}
Failed to get PMM Server parameters from local pmm-agent: pmm-agent is not connected to PMM Server.
[root@[hostname] ~]# pmm-agent --debug
INFO[2021-08-18T15:05:05.716+02:00] Using /usr/local/percona/pmm2/exporters/node_exporter component=main
INFO[2021-08-18T15:05:05.716+02:00] Using /usr/local/percona/pmm2/exporters/mysqld_exporter component=main
INFO[2021-08-18T15:05:05.716+02:00] Using /usr/local/percona/pmm2/exporters/mongodb_exporter component=main
INFO[2021-08-18T15:05:05.716+02:00] Using /usr/local/percona/pmm2/exporters/postgres_exporter component=main
INFO[2021-08-18T15:05:05.716+02:00] Using /usr/local/percona/pmm2/exporters/proxysql_exporter component=main
INFO[2021-08-18T15:05:05.716+02:00] Using /usr/local/percona/pmm2/exporters/rds_exporter component=main
INFO[2021-08-18T15:05:05.716+02:00] Using /usr/local/percona/pmm2/exporters/azure_exporter component=main
INFO[2021-08-18T15:05:05.716+02:00] Using /usr/local/percona/pmm2/exporters/vmagent component=main
DEBU[2021-08-18T15:05:05.716+02:00] Loaded configuration: &{ID: ListenAddress:127.0.0.1 ListenPort:7777 Server:{Address: Username: Password: InsecureTLS:false WithoutTLS:false} Paths:{ExportersBase:/usr/local/percona/pmm2/exporters NodeExporter:/usr/local/percona/pmm2/exporters/node_exporter MySQLdExporter:/usr/local/percona/pmm2/exporters/mysqld_exporter MongoDBExporter:/usr/local/percona/pmm2/exporters/mongodb_exporter PostgresExporter:/usr/local/percona/pmm2/exporters/postgres_exporter ProxySQLExporter:/usr/local/percona/pmm2/exporters/proxysql_exporter RDSExporter:/usr/local/percona/pmm2/exporters/rds_exporter AzureExporter:/usr/local/percona/pmm2/exporters/azure_exporter VMAgent:/usr/local/percona/pmm2/exporters/vmagent TempDir:/tmp PTSummary:/usr/local/percona/pmm2/tools/pt-summary PTPgSummary:/usr/local/percona/pmm2/tools/pt-pg-summary PTMySqlSummary:/usr/local/percona/pmm2/tools/pt-mysql-summary PTMongoDBSummary:/usr/local/percona/pmm2/tools/pt-mongodb-summary SlowLogFilePrefix:} Ports:{Min:42000 Max:51999} Debug:true Trace:false Setup:{NodeType: NodeName: MachineID: Distro: ContainerID: ContainerName: NodeModel: Region: Az: Address: MetricsMode: DisableCollectors: CustomLabels: Force:false SkipRegistration:false}} component=main
INFO[2021-08-18T15:05:05.716+02:00] Starting... component=client
DEBU[2021-08-18T15:05:05.717+02:00] Starting gRPC server on http://127.0.0.1:45317/ ... component=local-server/gRPC
DEBU[2021-08-18T15:05:05.717+02:00] Reflection and channelz are enabled. component=local-server/gRPC
ERRO[2021-08-18T15:05:05.717+02:00] PMM Server address is not provided, halting. component=client
INFO[2021-08-18T15:05:05.717+02:00] Starting local API server on http://127.0.0.1:7777/ ... component=local-server/JSON
DEBU[2021-08-18T15:05:05.717+02:00] Debug handlers:
http://127.0.0.1:7777/debug/metrics
http://127.0.0.1:7777/debug/vars
http://127.0.0.1:7777/debug/requests
http://127.0.0.1:7777/debug/events
http://127.0.0.1:7777/debug/pprof component=local-server/JSON
INFO[2021-08-18T15:05:05.719+02:00] Started. component=local-server/JSON
PANI[2021-08-18T15:05:05.719+02:00] listen tcp 127.0.0.1:7777: bind: address already in use component=local-server/JSON
panic: (*logrus.Entry) 0xc0003a3e30

goroutine 133 [running]:
github.com/sirupsen/logrus.Entry.log(0xc0001a2cb0, 0xc000130d80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/tmp/go/pkg/mod/github.com/sirupsen/logrus@v1.4.2/entry.go:239 +0x2ea
github.com/sirupsen/logrus.(*Entry).Log(0xc000282cb0, 0x0, 0xc000341f90, 0x1, 0x1)
/tmp/go/pkg/mod/github.com/sirupsen/logrus@v1.4.2/entry.go:268 +0xf0
github.com/sirupsen/logrus.(*Entry).Panic(0xc000282cb0, 0xc000341f90, 0x1, 0x1)
/tmp/go/pkg/mod/github.com/sirupsen/logrus@v1.4.2/entry.go:306 +0x55
github.com/percona/pmm-agent/agentlocal.(*Server).runJSONServer.func2(0xc000282cb0, 0xc0000260e0)
/tmp/go/src/github.com/percona/pmm-agent/agentlocal/agent_local.go:297 +0xf2
created by github.com/percona/pmm-agent/agentlocal.(*Server).runJSONServer
/tmp/go/src/github.com/percona/pmm-agent/agentlocal/agent_local.go:294 +0xcf3

Noting the error :  listen tcp 127.0.0.1:7777: bind: address already in use component=local-server/JSON 

I checked the port usage:

[root@[hostname] ~]# netstat -nltp | grep 7777
tcp 0 0 127.0.0.1:7777 0.0.0.0:* LISTEN 29361/pmm-agent

 

Environment

Linux 3.10.0-862.14.4.el7.x86_64 x86_64

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Smart Checklist

Activity

Show:

Lalit Choudhary October 14, 2021 at 11:52 AM

I just realised this is in the wrong queue. 
I'll add it in the correct one. Sorry.

Thank you. I'm closing this please report bug case in for product PMM

Andrew Poodle August 18, 2021 at 1:25 PM

I just realised this is in the wrong queue. 
I'll add it in the correct one. Sorry.

Not a Bug

Details

Assignee

Reporter

Components

Priority

Smart Checklist

Created August 18, 2021 at 1:20 PM
Updated March 5, 2024 at 10:29 AM
Resolved October 14, 2021 at 11:53 AM

Flag notifications