Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Prometheus alerting rule changes needs docker restart to activate

Description

STR:

  1. Install PMM2 2.3.0

  2. Configure Alertmanager (run was-staging-start job with --setup-alertmanager option)

  3. Open PMM settings page

  4. Add alert manager url and rule

  5. Apply changes

  6. Check PMM_Server/prometheus/rule page. Added rule should be displayed

  7. Open Pmm Settings page

  8. Change Alertmanager rule and submit the form

  9. Check again PMM_Server/prometheus/rule page
    Expected result: the new rule is displayed

Given result: The old rule is displayed

Expected results: Prometheus alerting rules changes shouldn't need docker restart
 
 

 

See Smart Checklist.

Original report:
Hello
following yout blog about alertmanager I tried it.

Starting with the mentioned Postgres down rule I got it working, but any changes to the alertmanager rules in the GUI did not fire on the testet values.

In the comments of the blog article I found "try to look http:///prometheus" 
This worked with the docker container on https://<pmmserver>:8443/prometheus and there only the Postgres Down Rule was mentioned. After docker stop/start the new rules also are mentioned.

Please make prometheus reloading the alertmanager rules after the "Aplly Alertmanager Settings" button in pmm-settings is pressed and the rules are valid.

Thanks again for the PMM itself. Very helpful tool.

Hubertus

How to test

None

How to document

None

Smart Checklist

hide

Activity

Show:

Alexey Palazhchenko May 20, 2020 at 10:23 AM

We have to add an API test for that. We already use Alertmanager API in our tests, and we should use it there.

Alexey Palazhchenko May 13, 2020 at 11:31 AM

there is no feature build

C W March 24, 2020 at 1:26 PM

pkill -HUP -x prometheus should be all that you need instead of a full restart:

Matthias Crauwels March 24, 2020 at 12:51 PM

less intrusive than restarting the entire container might be to go inside the container (docker exec...) and run supervisorctl restart prometheus

Alexey Palazhchenko March 19, 2020 at 7:31 AM

Moved to 2.5.0 because implementation is blocked by .

Done

Details

Assignee

Reporter

Priority

Components

Needs QA

Needs Doc

Fix versions

Story Points

Sprint

Affects versions

Smart Checklist Progress

Environment

docker container with 2.3.0 pmm-server:2
alertmanager in another docker container

Smart Checklist

Created March 3, 2020 at 12:15 PM
Updated April 22, 2025 at 7:12 AM
Resolved May 20, 2020 at 12:08 PM