General
Escalation
General
Escalation
Description
How to test
None
How to document
None
AFFECTED CS IDs
CS0040437
Activity
Show:
Nurlan Moldomurov October 22, 2024 at 6:06 AM
Nurlan Moldomurov July 24, 2024 at 9:09 AM
@Jaime Sicam we are planning to close this task as done, could you provide your feedback, please?
Nurlan Moldomurov January 29, 2024 at 11:22 PM
@Jaime Sicam could you try this https://www.notion.so/percona/Using-ENV-variables-in-PMM-AMI-518b8f435d954e49ba186874cbc3d685?pvs=4?
Nurlan Moldomurov January 10, 2024 at 9:51 AM
@Aaditya Dubey ability to provide env variable to PMM Server on AMI will be provided in PMM 3
Nurlan Moldomurov November 17, 2023 at 6:07 AM
@Jaime Sicam, could you provide exact command used to run PMM Server docker image? And how did you verify that it doesn't increased the scrape size?
Done
Pinned fields
Click on the next to a field label to start pinning.
Details
Assignee
Reporter
Priority
Medium
Components
Labels
Needs QA
Yes
Needs Doc
Yes
Affects versions
Lead time
196
Smart Checklist
Open Smart Checklist
Open Smart Checklist
Created October 31, 2023 at 7:20 AM
Updated October 22, 2024 at 6:06 AM
Resolved July 31, 2024 at 9:20 AM
Adding PMM_PROMSCRAPE_MAX_SCRAPE_SIZE in /usr/lib/systemd/system/pmm-agent.service to customize scrape size is not respected by vmagent binary:
[Unit] Description=pmm-agent After=time-sync.target network.target [Service] Environment="PMM_PROMSCRAPE_MAX_SCRAPE_SIZE=12M" Type=simple ExecStart=/usr/sbin/pmm-agent --config-file=/usr/local/percona/pmm2/config/pmm-agent.yaml Restart=always RestartSec=2s [Install] WantedBy=multi-user.target
Then apply configuration and restart pmm-agent:
# systemctl daemon-reload # systemctl restart pmm-agent
You can see that pmm-agent uses the environment variable but vmagent does not:
# strings /proc/`pidof pmm-agent`/environ LANG=en_US.UTF-8 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin INVOCATION_ID=73768ed0b65f41f6a7b41c49346519c0 JOURNAL_STREAM=8:37549 SYSTEMD_EXEC_PID=5017 PMM_PROMSCRAPE_MAX_SCRAPE_SIZE=12M # strings /proc/`pidof vmagent`/environ VMAGENT_remoteWrite_basicAuth_password=eyJrIjoieUdCczV0NjFiNW1wWVBzWkZoOFQyTGFoUWhuU1pFSUkiLCJuIjoicG1tLWFnZW50LXBzNTctZWw5LTEudGVzdC5jb20tNDg5MjI3NDE3MDI4OTgyMzUxOSIsImlkIjoxfQ== VMAGENT_remoteWrite_basicAuth_username=api_key # ps aux|grep vmagent root 5050 0.2 2.3 723720 46756 ? Sl 07:10 0:00 /usr/local/percona/pmm2/exporters/vmagent -envflag.enable=true -envflag.prefix=VMAGENT_ -httpListenAddr=127.0.0.1:42002 -loggerLevel=INFO -promscrape.config=/usr/local/percona/pmm2/tmp/vm_agent/agent_id/99b94ecf-a85d-40b2-834b-bfb5a235af70/vmagentscrapecfg -promscrape.maxScrapeSize=64MiB -remoteWrite.maxDiskUsagePerURL=1073741824 -remoteWrite.tlsInsecureSkipVerify=true -remoteWrite.tmpDataPath=/usr/local/percona/pmm2/tmp/vmagent-temp-dir -remoteWrite.url=https://10.0.0.5:443/victoriametrics/api/v1/write root 5065 0.0 0.1 6408 2208 pts/0 S+ 07:11 0:00 grep --color=auto vmagent
As you can see above that the promscrape.maxScrapeSize is still 64MB and not 12MB.
I suspect it's because of this code:
https://github.com/percona/pmm/blob/main/managed/services/agents/vmagent.go
"-envflag.prefix=VMAGENT_", } sort.Strings(args) var envs []string if !params.ExternalVM() { envs = []string{ "VMAGENT_remoteWrite_basicAuth_username={{.server_username}}", "VMAGENT_remoteWrite_basicAuth_password={{.server_password}}", } } for _, env := range os.Environ() { if strings.HasPrefix(env, envvars.ENVvmAgentPrefix) { envs = append(envs, env) } } sort.Strings(envs)
It looks like the new environment variable will only incorporate existing environment variables prefixed with "VMAGENT_" so "PMM_PROMSCRAPE_MAX_SCRAPE_SIZE" will be excluded.