Alertmanager - priority routing and appending additional metadata

I am wondering if there is any interest in the idea of expanding alertmanager somewhat. Currently alertmanager sends out alerts to each configured receiver for an alert without any information regarding what receivers the alert was sent to or what the responses were.

I can see uses cases where it would be useful to send an alert to one receiver and then append the response from the receiver onto the alert and then send to another receiver.

Because alertmanager is a bit of a one-way street (apart from being able to silence via the API) it means that there is no way of linking or feeding back into the alert resources. Arguably alertmanager is not the place for that anyway, because the alerts are consumed in other channels and this is typically where the feedback should be posted.

An example of a workflow that this would suit would be:

  • alertmanager send alert to slack (let us say route priority 1)
  • slack response contains a slack thread id
  • alertmanager then appends slack response details to the alert and sends to a webhook (say route priority 2)

This would allow the webhook application to do something/analyse something and continue to interact and update the slack alert thread with additional information.

I guess there is an argument for having an alertmanager API method that allows for the updating/commenting on an alert and alertmanager could then update the alert channels itself. However that would be a lot of out of scope overhead for alertmanager itself to take on in terms of integration endpoint methods, etc.

It just seems to me that alertmanager could provide this useful information, with a simple priority and append methodology, making the information available to upstream receivers so that they could further interact with the specific alert resources created. I use slack as an example, but the same is true for many other integrations I am certain, pagerduty, etc.

Anyway just some thoughts. I realise that “simple” is a word used to often :slight_smile:

Does the idea have any merit?

Posted to - Priority routing and appending additional metadata · Discussion #2937 · prometheus/alertmanager · GitHub
Sorry I did not see Discussions there.