Hi,
There is a kube_pod_status_ready metric in k8s which you can use to check whether some pod is ready. We’ve been using it like this:
count(kube_pod_status_ready{condition="true"} == 1) by (cluster, environment)
What I want to do is to check which cluster/environment is scaling up and down too often. So, perfectly I’d like to have a function that would represent how many times the pod number metric changed direction in last interval. Af far as I understand the Prometheus documentation, there is no such metric that can do that out of the box.
The closest that I’ve come is delta function but it’s not working as I would like it to since it will only calculate the difference between now() and now() - interval. However, if the number of pods is changing multiple times(i.e. 4 times) from 2 to 3 and back, this function will return metric that’s either 0 or 1 depending on the time when the change happened.
Perfect function would return 8, 4 for scaling up and 4 for scale down. Also, if the pods are scaling up by 3 pods and later on scale down by 3 pods, perfect function would return 2 (1 for scale up, and one for scale down).
Is there such or similar function?
Thanks in advance