I have a histogram that measures duration of “steps” inside of a job. the labels are roughly as follows:
step_duration_seconds{ stepName, job}
I’d like to display a panel that shows the top 3 steps that are taking the most time (averaged).
The base query I’m starting with is:
sum by (stepName) (rate(step_duration_seconds_sum[$__rate_interval]))
/ sum by (jobStepName) (rate(step_duration_seconds_count[$__rate_interval]))
This should get me the average duration of each step. However, I’m at a loss for how to display this. I want to pick the top 3 and put into a table. Any hints on how this would be done?