grpc: received message larger than max
General
Escalation
General
Escalation
Description
How to test
1. Having a long-running PMM-Instance
2. Upgrade PMM to this FB
3. Now try to Upgrade via UI to another latest version when your step 2 already succeeded
How to document
None
Attachments
2
Smart Checklist
Activity
Show:
Yoni Sade April 4, 2023 at 11:13 AM
After upgrade to v2.36.0 getting the same error:
INFO[2023-04-04T11:10:01.205+00:00] RPC /server.Server/Readiness done in 28.364183ms. request=3e354549-d2d9-11ed-8dd0-0242ac110002
INFO[2023-04-04T11:10:04.438+00:00] Starting RPC /server.Server/Readiness ... request=402700bd-d2d9-11ed-8dd0-0242ac110002
INFO[2023-04-04T11:10:04.442+00:00] RPC /server.Server/Readiness done in 3.799264ms. request=402700bd-d2d9-11ed-8dd0-0242ac110002
WARN[2023-04-04T11:10:06.888+00:00] Collect for 49642 buckets took 2.234694129s. component=qan
ERRO[2023-04-04T11:10:06.888+00:00] rpc error: code = ResourceExhausted desc = grpc: received message larger than max (22650814 vs. 20971520)
failed to sent CollectRequest to QAN
github.com/percona/pmm/managed/services/qan.(*Client).Collect
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/managed/services/qan/client.go:291
github.com/percona/pmm/managed/services/agents.(*Handler).Run.func3
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/managed/services/agents/handler.go:134
runtime/pprof.Do
/usr/local/go/src/runtime/pprof/runtime.go:40
github.com/percona/pmm/managed/services/agents.(*Handler).Run
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/managed/services/agents/handler.go:133
github.com/percona/pmm/managed/services/agents/grpc.(*agentServer).Connect
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/managed/services/agents/grpc/agent_server.go:40
github.com/percona/pmm/api/agentpb._Agent_Connect_Handler
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/api/agentpb/agent_grpc.pb.go:102
github.com/grpc-ecosystem/go-grpc-middleware/validator.StreamServerInterceptor.func1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/validator/validator.go:72
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/percona/pmm/managed/utils/interceptors.StreamServiceEnabledInterceptor.func1
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/managed/utils/interceptors/service.go:49
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).StreamServerInterceptor.func1
/home/builder/go/pkg/mod/github.com/!percona-!lab/go-grpc-prometheus@v0.0.0-20230105215234-10537622c253/server_metrics.go:128
github.com/percona/pmm/managed/utils/interceptors.Stream.func1.1
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:139
github.com/percona/pmm/managed/utils/interceptors.logRequest
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:78
github.com/percona/pmm/managed/utils/interceptors.Stream.func1
/home/builder/rpm/BUILD/pmm-8269774a7083e5eb4222c6736c64e321ec7f84c8/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:136
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:58
google.golang.org/grpc.(*Server).processStreamingRPC
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.53.0/server.go:1627
google.golang.org/grpc.(*Server).handleStream
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.53.0/server.go:1708
google.golang.org/grpc.(*Server).serveStreams.func1.2
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.53.0/server.go:965
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1594 agent_id=pmm-server request=d537c1e4-d2d6-11ed-8dd0-0242ac110002
INFO[2023-04-04T11:10:07.628+00:00] Starting RPC /server.Server/Readiness ... request=420dba39-d2d9-11ed-8dd0-0242ac110002
INFO[2023-04-04T11:10:07.632+00:00] RPC /server.Server/Readiness done in 3.691483ms. request=420dba39-d2d9-11ed-8dd0-0242ac110002
INFO[2023-04-04T11:10:10.737+00:00] Starting RPC /server.Server/Readiness ... request=43e80b0b-d2d9-11ed-8dd0-0242ac110002
INFO[2023-04-04T11:10:10.740+00:00] RPC /server.Server/Readiness done in 3.018471ms. request=43e80b0b-d2d9-11ed-8dd0-0242ac110002
INFO[2023-04-04T11:10:13.849+00:00] Starting RPC /server.Server/Readiness ... request=45c301b3-d2d9-11ed-8dd0-0242ac110002Puneet Kala September 5, 2022 at 2:58 PM
Thanks a lot for testing and reporting this, I have reopened the ticket for our developers to look at, and we will try to work on this again, we have also seen this again on our internal environments and setups, hence reopening without the need for our support team to re-verify this.
Yoni Sade September 5, 2022 at 1:54 PM
After upgrade to v2.30.0 still seeing these errors:
ERRO[2022-09-05T13:52:03.518+00:00] rpc error: code = ResourceExhausted desc = grpc: received message larger than max (26986974 vs. 20971520)
failed to sent CollectRequest to QAN
github.com/percona/pmm/managed/services/qan.(*Client).Collect
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/managed/services/qan/client.go:273
github.com/percona/pmm/managed/services/agents.(*Handler).Run.func3
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/managed/services/agents/handler.go:134
runtime/pprof.Do
/usr/local/go/src/runtime/pprof/runtime.go:40
github.com/percona/pmm/managed/services/agents.(*Handler).Run
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/managed/services/agents/handler.go:133
github.com/percona/pmm/managed/services/agents/grpc.(*agentServer).Connect
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/managed/services/agents/grpc/agent_server.go:39
github.com/percona/pmm/api/agentpb._Agent_Connect_Handler
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/api/agentpb/agent_grpc.pb.go:98
github.com/grpc-ecosystem/go-grpc-middleware/validator.StreamServerInterceptor.func1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/validator/validator.go:72
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/percona/pmm/managed/utils/interceptors.StreamServiceEnabledInterceptor.func1
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/managed/utils/interceptors/service.go:49
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).StreamServerInterceptor.func1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:121
github.com/percona/pmm/managed/utils/interceptors.Stream.func1
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:124
github.com/percona/pmm/managed/utils/interceptors.logRequest
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:77
github.com/percona/pmm/managed/utils/interceptors.Stream
/home/builder/rpm/BUILD/pmm-570df2e9e13e2c139c8b3cb6b8dfdf8685a24e45/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:121
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:58
google.golang.org/grpc.(*Server).processStreamingRPC
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.48.0/server.go:1565
google.golang.org/grpc.(*Server).handleStream
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.48.0/server.go:1640
google.golang.org/grpc.(*Server).serveStreams.func1.2
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.48.0/server.go:932
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1571 agent_id=pmm-server request=5bc37a1d-2cee-11ed-ba2e-0242ac110002Yoni Sade August 10, 2022 at 8:25 AM
After upgrade to v2.29.1 still seeing these errors:
ERRO[2022-08-10T08:22:04.845+00:00] rpc error: code = ResourceExhausted desc = grpc: received message larger than max (36222461 vs. 20971520)
failed to sent CollectRequest to QAN
github.com/percona/pmm/managed/services/qan.(*Client).Collect
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/managed/services/qan/client.go:274
github.com/percona/pmm/managed/services/agents.(*Handler).Run.func3
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/managed/services/agents/handler.go:135
runtime/pprof.Do
/usr/local/go/src/runtime/pprof/runtime.go:40
github.com/percona/pmm/managed/services/agents.(*Handler).Run
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/managed/services/agents/handler.go:134
github.com/percona/pmm/managed/services/agents/grpc.(*agentServer).Connect
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/managed/services/agents/grpc/agent_server.go:40
github.com/percona/pmm/api/agentpb._Agent_Connect_Handler
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/api/agentpb/agent_grpc.pb.go:98
github.com/grpc-ecosystem/go-grpc-middleware/validator.StreamServerInterceptor.func1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/validator/validator.go:72
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/percona/pmm/managed/utils/interceptors.StreamServiceEnabledInterceptor.func1
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/managed/utils/interceptors/service.go:50
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).StreamServerInterceptor.func1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:121
github.com/percona/pmm/managed/utils/interceptors.Stream.func1
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:125
github.com/percona/pmm/managed/utils/interceptors.logRequest
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:78
github.com/percona/pmm/managed/utils/interceptors.Stream
/home/builder/rpm/BUILD/pmm-2c740d96c9399a5140f5a16b2063024b895753c4/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:122
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49
github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1
/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:58
google.golang.org/grpc.(*Server).processStreamingRPC
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.47.0/server.go:1549
google.golang.org/grpc.(*Server).handleStream
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.47.0/server.go:1624
google.golang.org/grpc.(*Server).serveStreams.func1.2
/home/builder/go/pkg/mod/google.golang.org/grpc@v1.47.0/server.go:922
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1571 agent_id=pmm-server request=094b1566-1883-11ed-ad61-0242ac110002Aaditya Dubey June 27, 2022 at 12:19 PM
Hi ,
Thank you for the updates.
This issue is getting tracked via and which is expected to release in 2.29.0, please keep following the same to get updates.
Done
Smart Checklist
Open Smart Checklist
Smart Checklist
Open Smart Checklist
Created May 16, 2022 at 11:29 AM
Updated March 6, 2024 at 1:12 AM
Resolved November 28, 2022 at 2:46 PM
Seeing this error every now and then in pmm-managed.log:
ERRO[2022-05-16T11:09:04.928+00:00] rpc error: code = ResourceExhausted desc = grpc: received message larger than max (33565605 vs. 20971520) failed to sent CollectRequest to QAN github.com/percona/pmm-managed/services/qan.(*Client).Collect /home/builder/rpm/BUILD/pmm-managed-ed249a02ab49a3ac0249c60911af1f4d280d5219/src/github.com/percona/pmm-managed/services/qan/client.go:252 github.com/percona/pmm-managed/services/agents.(*Handler).Run.func3 /home/builder/rpm/BUILD/pmm-managed-ed249a02ab49a3ac0249c60911af1f4d280d5219/src/github.com/percona/pmm-managed/services/agents/handler.go:135 runtime/pprof.Do /usr/local/go/src/runtime/pprof/runtime.go:40 github.com/percona/pmm-managed/services/agents.(*Handler).Run /home/builder/rpm/BUILD/pmm-managed-ed249a02ab49a3ac0249c60911af1f4d280d5219/src/github.com/percona/pmm-managed/services/agents/handler.go:134 github.com/percona/pmm-managed/services/agents/grpc.(*agentServer).Connect /home/builder/rpm/BUILD/pmm-managed-ed249a02ab49a3ac0249c60911af1f4d280d5219/src/github.com/percona/pmm-managed/services/agents/grpc/agent_server.go:41 github.com/percona/pmm/api/agentpb._Agent_Connect_Handler /home/builder/go/pkg/mod/github.com/percona/pmm@v0.0.0-20220404174627-9de9a91ebd60/api/agentpb/agent_grpc.pb.go:98 github.com/grpc-ecosystem/go-grpc-middleware/validator.StreamServerInterceptor.func1 /home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/validator/validator.go:72 github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1 /home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49 github.com/percona/pmm-managed/utils/interceptors.StreamServiceEnabledInterceptor.func1 /home/builder/rpm/BUILD/pmm-managed-ed249a02ab49a3ac0249c60911af1f4d280d5219/src/github.com/percona/pmm-managed/utils/interceptors/service.go:50 github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1 /home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49 github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).StreamServerInterceptor.func1 /home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:121 github.com/percona/pmm-managed/utils/interceptors.Stream.func1 /home/builder/rpm/BUILD/pmm-managed-ed249a02ab49a3ac0249c60911af1f4d280d5219/src/github.com/percona/pmm-managed/utils/interceptors/interceptors.go:125 github.com/percona/pmm-managed/utils/interceptors.logRequest /home/builder/rpm/BUILD/pmm-managed-ed249a02ab49a3ac0249c60911af1f4d280d5219/src/github.com/percona/pmm-managed/utils/interceptors/interceptors.go:78 github.com/percona/pmm-managed/utils/interceptors.Stream /home/builder/rpm/BUILD/pmm-managed-ed249a02ab49a3ac0249c60911af1f4d280d5219/src/github.com/percona/pmm-managed/utils/interceptors/interceptors.go:122 github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1.1.1 /home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:49 github.com/grpc-ecosystem/go-grpc-middleware.ChainStreamServer.func1 /home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:58 google.golang.org/grpc.(*Server).processStreamingRPC /home/builder/go/pkg/mod/google.golang.org/grpc@v1.45.0/server.go:1548 google.golang.org/grpc.(*Server).handleStream /home/builder/go/pkg/mod/google.golang.org/grpc@v1.45.0/server.go:1623 google.golang.org/grpc.(*Server).serveStreams.func1.2 /home/builder/go/pkg/mod/google.golang.org/grpc@v1.45.0/server.go:921 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:1581 agent_id=pmm-server request=6348dc32-d146-11ec-8397-0242ac110002I also saw this error many times in the web UI when trying to Upgrade to v2.28.0 (browser console showed: "Failed to load resource: the server responded with a status of 429 (Too Many Requests"):
grpc: received message larger than max (15165483 vs. 5242880)Upgrade log:
ProjectName: pmm-update Version: 2.27.0 PMMVersion: 2.27.0 Timestamp: 2022-04-12 11:11:07 (UTC) FullCommit: 3112681ff16267dad7b883601869a8ea7a6381be Starting "yum --verbose info installed pmm-update" ... Loading "changelog" plugin Loading "fastestmirror" plugin Loading "ovl" plugin Config time: 0.014 rpmdb time: 0.000 ovl: Copying up (0) files from OverlayFS lower layer Yum version: 3.4.3 Installed Packages Name : pmm-update Arch : noarch Version : 2.27.0 Release : 64.2204121111.3112681.el7 Size : 2.2 M Repo : installed From repo : pmm2-server Committer : Nikita Beletskii <nikita.beletskii@percona.com> Committime : Tue Oct 19 12:00:00 2021 Buildtime : Tue Apr 12 11:11:09 2022 Install time: Mon Apr 25 09:52:53 2022 Installed by: System <unset> Changed by : System <unset> Summary : Tool for updating packages and OS configuration for PMM Server URL : https://github.com/percona/pmm-update License : AGPLv3 Description : Tool for updating packages and OS configuration for PMM ServerStarting "yum update --assumeyes pmm-update" ... Loaded plugins: changelog, fastestmirror, ovl Loading mirror speeds from cached hostfile * base: download.cf.centos.org * epel: d2lzkl7pfhq30w.cloudfront.net * extras: download.cf.centos.org * updates: download.cf.centos.org Resolving Dependencies --> Running transaction check ---> Package pmm-update.noarch 0:2.27.0-64.2204121111.3112681.el7 will be updated ---> Package pmm-update.noarch 0:2.28.0-64.2204291517.0e0bfc6.el7 will be an update --> Finished Dependency ResolutionDependencies Resolved================================================================================ Package Arch Version Repository Size ================================================================================ Updating: pmm-update noarch 2.28.0-64.2204291517.0e0bfc6.el7 pmm2-server 823 kTransaction Summary ================================================================================ Upgrade 1 PackageTotal download size: 823 k Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : pmm-update-2.28.0-64.2204291517.0e0bfc6.el7.noarch 1/2 Cleanup : pmm-update-2.27.0-64.2204121111.3112681.el7.noarch 2/2 Verifying : pmm-update-2.28.0-64.2204291517.0e0bfc6.el7.noarch 1/2 Verifying : pmm-update-2.27.0-64.2204121111.3112681.el7.noarch 2/2 Updated: pmm-update.noarch 0:2.28.0-64.2204291517.0e0bfc6.el7 Complete! Starting "yum --verbose info installed pmm-update" ... Loading "changelog" plugin Loading "fastestmirror" plugin Loading "ovl" plugin Config time: 0.016 rpmdb time: 0.000 ovl: Copying up (0) files from OverlayFS lower layer Yum version: 3.4.3 Installed Packages Name : pmm-update Arch : noarch Version : 2.28.0 Release : 64.2204291517.0e0bfc6.el7 Size : 2.2 M Repo : installed From repo : pmm2-server Committer : Nikita Beletskii <nikita.beletskii@percona.com> Committime : Tue Oct 19 12:00:00 2021 Buildtime : Fri Apr 29 15:17:04 2022 Install time: Mon May 16 11:20:35 2022 Installed by: System <unset> Changed by : System <unset> Summary : Tool for updating packages and OS configuration for PMM Server URL : https://github.com/percona/pmm-update License : AGPLv3 Description : Tool for updating packages and OS configuration for PMM Servertime="2022-05-16T11:20:36Z" level=info msg="pmm-update:\nbefore update = {Version:2.27.0 FullVersion:2.27.0-64.2204121111.3112681.el7 BuildTime:2022-04-12 11:11:09 +0000 UTC Repo:pmm2-server}\n after update = {Version:2.28.0 FullVersion:2.28.0-64.2204291517.0e0bfc6.el7 BuildTime:2022-04-29 15:17:04 +0000 UTC Repo:pmm2-server}" time="2022-05-16T11:20:36Z" level=info msg="Version changed, exiting." ProjectName: pmm-update Version: 2.28.0 PMMVersion: 2.28.0 Timestamp: 2022-04-29 15:17:01 (UTC) FullCommit: 0e0bfc6194bc83292ad4fc6abba869c72b517305 Starting "yum --verbose info installed pmm-update" ... Loading "changelog" plugin Loading "fastestmirror" plugin Loading "ovl" plugin Config time: 0.015 rpmdb time: 0.000 ovl: Copying up (0) files from OverlayFS lower layer Yum version: 3.4.3 Installed Packages Name : pmm-update Arch : noarch Version : 2.28.0 Release : 64.2204291517.0e0bfc6.el7 Size : 2.2 M Repo : installed From repo : pmm2-server Committer : Nikita Beletskii <nikita.beletskii@percona.com> Committime : Tue Oct 19 12:00:00 2021 Buildtime : Fri Apr 29 15:17:04 2022 Install time: Mon May 16 11:20:35 2022 Installed by: System <unset> Changed by : System <unset> Summary : Tool for updating packages and OS configuration for PMM Server URL : https://github.com/percona/pmm-update License : AGPLv3 Description : Tool for updating packages and OS configuration for PMM ServerStarting "yum update --assumeyes pmm-update" ... Loaded plugins: changelog, fastestmirror, ovl Loading mirror speeds from cached hostfile * base: download.cf.centos.org * epel: d2lzkl7pfhq30w.cloudfront.net * extras: download.cf.centos.org * updates: download.cf.centos.org No packages marked for update Starting "yum --verbose info installed pmm-update" ... Loading "changelog" plugin Loading "fastestmirror" plugin Loading "ovl" plugin Config time: 0.022 rpmdb time: 0.000 ovl: Copying up (0) files from OverlayFS lower layer Yum version: 3.4.3 Installed Packages Name : pmm-update Arch : noarch Version : 2.28.0 Release : 64.2204291517.0e0bfc6.el7 Size : 2.2 M Repo : installed From repo : pmm2-server Committer : Nikita Beletskii <nikita.beletskii@percona.com> Committime : Tue Oct 19 12:00:00 2021 Buildtime : Fri Apr 29 15:17:04 2022 Install time: Mon May 16 11:20:35 2022 Installed by: System <unset> Changed by : System <unset> Summary : Tool for updating packages and OS configuration for PMM Server URL : https://github.com/percona/pmm-update License : AGPLv3 Description : Tool for updating packages and OS configuration for PMM Servertime="2022-05-16T11:20:39Z" level=info msg="pmm-update:\nbefore update = {Version:2.28.0 FullVersion:2.28.0-64.2204291517.0e0bfc6.el7 BuildTime:2022-04-29 15:17:04 +0000 UTC Repo:pmm2-server}\n after update = {Version:2.28.0 FullVersion:2.28.0-64.2204291517.0e0bfc6.el7 BuildTime:2022-04-29 15:17:04 +0000 UTC Repo:pmm2-server}" time="2022-05-16T11:20:39Z" level=info msg="Version did not change." Starting "ansible-playbook --flush-cache /usr/share/pmm-update/ansible/playbook/tasks/update.yml" ... [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'PLAY [localhost] ***************************************************************TASK [Gathering Facts] ********************************************************* ok: [localhost]TASK [detect /srv/pmm-distribution] ******************************************** ok: [localhost]TASK [detect containers] ******************************************************* ok: [localhost]TASK [force container] ********************************************************* skipping: [localhost]TASK [Remove percona-dashboard without architecture] *************************** ok: [localhost]TASK [Update percona-dashboards package] ***************************************changed: [localhost]TASK [Create grafana config] ***************************************************TASK [Create grafana directory] ************************************************ ok: [localhost] => (item=/srv/grafana) changed: [localhost] => (item=/srv/grafana/plugins)TASK [grafana : Set Grafana folder for plugins on /srv partition for all users] *** ok: [localhost]TASK [Copy new version of grafana.ini] ***************************************** changed: [localhost]TASK [grafana : Create provisioning directory] ********************************* ok: [localhost] => (item=datasources) ok: [localhost] => (item=plugins) ok: [localhost] => (item=dashboards)TASK [Copy grafana provisioning files] ***************************************** ok: [localhost] => (item=datasources) ok: [localhost] => (item=plugins) ok: [localhost] => (item=dashboards)TASK [Upgrade grafana database (Get the latest schema)] ************************ changed: [localhost]TASK [Supervisord start | Start supervisord service for AMI/OVF] ******** skipping: [localhost]TASK [Check that supervisor socket exists] ************************************* ok: [localhost]TASK [Supervisord start | Start supervisord for docker] **************** skipping: [localhost]TASK [Run initialization playbook] *********************************************TASK [initialization : Get current version] ************************************ ok: [localhost]TASK [initialization : Get image version] ************************************** ok: [localhost]TASK [initialization : Set current version if VERSION fail doen't exist] ******* skipping: [localhost]TASK [initialization : Setting current and image version] ********************** ok: [localhost]TASK [initialization : Setting current and image version] ********************** ok: [localhost]TASK [initialization : Print current version] ********************************** ok: [localhost] => { "msg": "Current version: 2.27.0 Image Version: 2.28.0" }TASK [Checking if we need an update or not] ************************************TASK [dashboards_upgrade : Check that old plugin dir exists] ******************* ok: [localhost]TASK [dashboards_upgrade : Find custom plugin in old plugin dir] *************** skipping: [localhost]TASK [dashboards_upgrade : Synchronization plugins (old version before 2.22.0)] *** skipping: [localhost]TASK [dashboards_upgrade : Find custom …