Hello! First of all: I’d like to thank everyone in the prometheus development and maintenance team! Prometheus has been a gamechanger in our company, and everyday more and more colleagues are amazed with the Prometheus capabilities.
I’ve been searching for quite a while, but my google-fu skills aren’t that good enough (that’s why I’m still a junior dev, hahahaha)
In our use case, we have some alerts that come from prometheus to alertmanager at non-business hours. Every instance our prometheus monitors have a different business-hours, so a general silence wouldn’t be effective.
What I would like to do is: Create a silence for alerts coming from an specific instance every night, and its expiration date will be right after its’ business hours starts.
So, what happens if an alert is fired when the silencer is still there, and then the silence is expired? Does alertmanager resends the alert to the receiver? Or its gonna be completely ignored untill prometheus resends that alert?