Description

Adding PMM_PROMSCRAPE_MAX_SCRAPE_SIZE in /usr/lib/systemd/system/pmm-agent.service to customize scrape size is not respected by vmagent binary:

 

[Unit] Description=pmm-agent After=time-sync.target network.target [Service] Environment="PMM_PROMSCRAPE_MAX_SCRAPE_SIZE=12M" Type=simple ExecStart=/usr/sbin/pmm-agent --config-file=/usr/local/percona/pmm2/config/pmm-agent.yaml Restart=always RestartSec=2s [Install] WantedBy=multi-user.target

Then apply configuration and restart pmm-agent:

 

# systemctl daemon-reload # systemctl restart pmm-agent

You can see that pmm-agent uses the environment variable but vmagent does not:

 

# strings /proc/`pidof pmm-agent`/environ LANG=en_US.UTF-8 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin INVOCATION_ID=73768ed0b65f41f6a7b41c49346519c0 JOURNAL_STREAM=8:37549 SYSTEMD_EXEC_PID=5017 PMM_PROMSCRAPE_MAX_SCRAPE_SIZE=12M # strings /proc/`pidof vmagent`/environ VMAGENT_remoteWrite_basicAuth_password=eyJrIjoieUdCczV0NjFiNW1wWVBzWkZoOFQyTGFoUWhuU1pFSUkiLCJuIjoicG1tLWFnZW50LXBzNTctZWw5LTEudGVzdC5jb20tNDg5MjI3NDE3MDI4OTgyMzUxOSIsImlkIjoxfQ== VMAGENT_remoteWrite_basicAuth_username=api_key # ps aux|grep vmagent root        5050  0.2  2.3 723720 46756 ?        Sl   07:10   0:00 /usr/local/percona/pmm2/exporters/vmagent -envflag.enable=true -envflag.prefix=VMAGENT_ -httpListenAddr=127.0.0.1:42002 -loggerLevel=INFO -promscrape.config=/usr/local/percona/pmm2/tmp/vm_agent/agent_id/99b94ecf-a85d-40b2-834b-bfb5a235af70/vmagentscrapecfg -promscrape.maxScrapeSize=64MiB -remoteWrite.maxDiskUsagePerURL=1073741824 -remoteWrite.tlsInsecureSkipVerify=true -remoteWrite.tmpDataPath=/usr/local/percona/pmm2/tmp/vmagent-temp-dir -remoteWrite.url=https://10.0.0.5:443/victoriametrics/api/v1/write root        5065  0.0  0.1   6408  2208 pts/0    S+   07:11   0:00 grep --color=auto vmagent

As  you can see above that the promscrape.maxScrapeSize is still 64MB and not 12MB.

 

I suspect it's because of this code:

https://github.com/percona/pmm/blob/main/managed/services/agents/vmagent.go

"-envflag.prefix=VMAGENT_",     }    sort.Strings(args)    var envs []string     if !params.ExternalVM() {         envs = []string{             "VMAGENT_remoteWrite_basicAuth_username={{.server_username}}",             "VMAGENT_remoteWrite_basicAuth_password={{.server_password}}",         }     }     for _, env := range os.Environ() {         if strings.HasPrefix(env, envvars.ENVvmAgentPrefix) {             envs = append(envs, env)         }     }     sort.Strings(envs)

It looks like the new environment variable will only incorporate existing environment variables prefixed with "VMAGENT_" so "PMM_PROMSCRAPE_MAX_SCRAPE_SIZE" will be excluded.

 

 

How to test

None

How to document

None

AFFECTED CS IDs

CS0040437

Activity

Show:

Nurlan Moldomurov July 24, 2024 at 9:09 AM

we are planning to close this task as done, could you provide your feedback, please?

Nurlan Moldomurov January 29, 2024 at 11:22 PM

Nurlan Moldomurov January 10, 2024 at 9:51 AM

ability to provide env variable to PMM Server on AMI will be provided in PMM 3

Nurlan Moldomurov November 17, 2023 at 6:07 AM

, could you provide exact command used to run PMM Server docker image? And how did you verify that it doesn't increased the scrape size?

Done
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Unassigned

Reporter

Jaime Sicam

Priority

Medium

Components

Labels

Needs QA

Yes

Needs Doc

Yes

Affects versions

Lead time

196

Smart Checklist

Created October 31, 2023 at 7:20 AM
Updated October 22, 2024 at 6:06 AM
Resolved July 31, 2024 at 9:20 AM

Flag notifications