Timeout while adding mongos node
General
Escalation
General
Escalation
Description
How to test
None
How to document
None
Activity
Show:
Aaditya Dubey August 13, 2024 at 9:41 AM
Hi @Amrita Khemani
We still haven't heard any news from you. So I assume the fix/workaround works fine and will close the ticket. If you disagree, just reply and create a follow-up.
Nurlan Moldomurov July 23, 2024 at 1:31 AM
@Amrita Khemani seems like we can close this task, right?
Nurlan Moldomurov July 15, 2024 at 10:25 AM
oh, so you need to use https://percona-pmm.readme.io/reference/changemongodbexporter v1/inventory/Agents/ChangeMongoDBExporter instead of v1/inventory/Agents/ChangePostgresExporter
Amrita Khemani July 15, 2024 at 10:01 AM
@Nurlan Moldomurov These are the logs
time="2024-07-15T09:56:26.953+00:00" level=error msg="RPC /inventory.Agents/ChangePostgresExporter done in 3.118044ms with panic: interface conversion: inventorypb.Agent is *inventorypb.MongoDBExporter, not *inventorypb.PostgresExporter\nStack: goroutine 393115484 [running]:\nruntime/debug.Stack()\n\t/usr/local/go/src/runtime/debug/stack.go:24 +0x5e\ngithub.com/percona/pmm/managed/utils/interceptors.logRequest.func1()\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:49 +0x451\npanic({0x2ad4fe0?, 0xc000c1faa0?})\n\t/usr/local/go/src/runtime/panic.go:770 +0x132\ngithub.com/percona/pmm/managed/services/inventory.(*AgentsService).ChangePostgresExporter(0xc00054e620, {0x3a050c8, 0xc0028b0bd0}, 0x2d7dc80?)\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/managed/services/inventory/agents.go:575 +0xa5\ngithub.com/percona/pmm/managed/services/inventory/grpc.(*agentsServer).ChangePostgresExporter(0x5496fd0?, {0x3a050c8?, 0xc0028b0bd0?}, 0x40e55a?)\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/managed/services/inventory/grpc/agents_server.go:343 +0x25\ngithub.com/percona/pmm/api/inventorypb._Agents_ChangePostgresExporter_Handler.func1({0x3a050c8?, 0xc0028b0bd0?}, {0x2d7dc80?, 0xc004688a80?})\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/api/inventorypb/agents_grpc.pb.go:830 +0xce\nmain.runGRPCServer.UnaryServerInterceptor.func8({0x3a050c8, 0xc0028b0bd0}, {0x2d7dc80, 0xc004688a80}, 0x10?, 0xc01d158c30)\n\t/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/validator/validator.go:47 +0x65\nmain.runGRPCServer.ChainUnaryServer.func10.1({0x3a050c8?, 0xc0028b0bd0?}, {0x2d7dc80?, 0xc004688a80?})\n\t/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/chain.go:48 +0x45\nmain.runGRPCServer.UnaryServiceEnabledInterceptor.func7({0x3a050c8, 0xc0028b0bd0}, {0x2d7dc80, 0xc004688a80}, 0xc0011fdcc0, 0xc004688bc0)\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/managed/utils/interceptors/service.go:39 +0x8a\nmain.runGRPCServer.ChainUnaryServer.func10.1({0x3a050c8?, 0xc0028b0bd0?}, {0x2d7dc80?, 0xc004688a80?})\n\t/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/chain.go:48 +0x45\nmain.runGRPCServer.(*ServerMetrics).UnaryServerInterceptor.func5({0x3a050c8, 0xc0028b0bd0}, {0x2d7dc80, 0xc004688a80}, 0x4?, 0xc004688c00)\n\t/home/builder/go/pkg/mod/github.com/!percona-!lab/go-grpc-prometheus@v0.0.0-20230116133345-3487748d4592/server_metrics.go:114 +0x87\nmain.runGRPCServer.Unary.func6.1()\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:107 +0x58\ngithub.com/percona/pmm/managed/utils/interceptors.logRequest(0xc0193f1730, {0xc016d0f2c0, 0x2c}, 0xc00457b860)\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:78 +0x16b\nmain.runGRPCServer.Unary.func6({0x3a050c8, 0xc0028b0a80}, {0x2d7dc80, 0xc004688a80}, 0xc0011fdcc0, 0xc004688c00)\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/managed/utils/interceptors/interceptors.go:105 +0x345\nmain.runGRPCServer.ChainUnaryServer.func10({0x3a050c8, 0xc0028b0a80}, {0x2d7dc80, 0xc004688a80}, 0xc0011fdcc0, 0x80?)\n\t/home/builder/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.4.0/chain.go:53 +0x123\ngithub.com/percona/pmm/api/inventorypb._Agents_ChangePostgresExporter_Handler({0x2f0c400, 0xc00007d0b0}, {0x3a050c8, 0xc0028b0a80}, 0xc01c5c2a00, 0xc000d17200)\n\t/home/builder/rpm/BUILD/pmm-74e57527735bd062c4bd37adbd89c31bb14ebc15/src/github.com/percona/pmm/api/inventorypb/agents_grpc.pb.go:832 +0x143\ngoogle.golang.org/grpc.(*Server).processUnaryRPC(0xc0002c7200, {0x3a050c8, 0xc0028b09f0}, {0x3a192e0, 0xc016663c80}, 0xc01612cb40, 0xc000d177a0, 0x54d96a8, 0x0)\n\t/home/builder/go/pkg/mod/google.golang.org/grpc@v1.63.0/server.go:1369 +0xdf8\ngoogle.golang.org/grpc.(*Server).handleStream(0xc0002c7200, {0x3a192e0, 0xc016663c80}, 0xc01612cb40)\n\t/home/builder/go/pkg/mod/google.golang.org/grpc@v1.63.0/server.go:1780 +0xe8b\ngoogle.golang.org/grpc.(*Server).serveStreams.func2.1()\n\t/home/builder/go/pkg/mod/google.golang.org/grpc@v1.63.0/server.go:1019 +0x8b\ncreated by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 393096218\n\t/home/builder/go/pkg/mod/google.golang.org/grpc@v1.63.0/server.go:1030 +0x125\n" request=8072b263-4290-11ef-8453-0242ac110002
time="2024-07-15T09:56:27.100+00:00" level=info msg="Configuration reloaded." component=victoriametrics
Nurlan Moldomurov July 15, 2024 at 8:33 AM
Could you check logs from pmm-managed and share with us?
Done
Details
Details
Assignee
Unassigned
UnassignedReporter
Amrita Khemani
Amrita KhemaniPriority
MediumComponents
Labels
Needs QA
Yes
Needs Doc
Yes
Smart Checklist
Open Smart Checklist
Smart Checklist
Open Smart Checklist
Created July 12, 2024 at 12:20 PM
Updated August 13, 2024 at 9:44 AM
Resolved August 13, 2024 at 9:42 AM
I am getting these errors while adding mongos node for monitoring, as database size is big
level=error msg="time=\"2024-07-12T17:47:22+05:30\" level=error msg=\"Failed to get database names: canceled while checking out a connection from connection pool: context canceled; maxPoolSize: 100, connections in use by cursors: 0, connections in use by transactions: 0, connections in use by other operations: 1.\"" agentID=/agent_id/eb30d802-7f5e-41d7-b045-d2e15fdeb2e3 component=agent-process type=mongodb_exporter
Is there any option to add timeout in mongodb exporter?