Prometheus negative cpu utilization with rate

PromQL
1

Prometheus reporting negative value of cpu utilization as taken with rate function.

Not able to fix this issue even after putting so much hours. This is understood that this is causing due to missed scrap. But not sure how rate function actually normalizing values which end up having negative values for cpu utilization.

Here is link of snapshots of error condition.
100 - (avg by (instance) (rate(node_cpu_seconds_total{mode=“idle”,instance=~"$host"}[1m])) * 100)
Response: attached snapshot link.

http://skras.ddns.net/TI/Promcpu

Troubleshooting Query: node_cpu_seconds_total{mode=“idle”,instance=~"$host"}
Troubleshooting Query Response:

Object
request:Object
url:“api/datasources/proxy/1/api/v1/query_range?query=node_cpu_seconds_total%7Bmode%3D%22idle%22%2Cinstance%3D~%22sermon%22%7D&start=1619717745&end=1619717880&step=15”
method:“GET”
hideFromInspector:false
response:Object
status:“success”
data:Object
resultType:“matrix”
result:Array[4]
0:Object
metric:Object
values:Array[10]
0:Array[1619717745,322555.9]
1:Array[1619717760,322570.32]
2:Array[1619717775,322580]
3:Array[1619717790,322580]
4:Array[1619717805,322580]
5:Array[1619717820,322627.9]
6:Array[1619717835,322642.11]
7:Array[1619717850,322656.44]
8:Array[1619717865,322670.74]
9:Array[1619717880,322685.1]
1:Object
2:Object
3:Object

2

It looks like you have repeated values over a long period of time:

2:Array[1619717775,322580]
3:Array[1619717790,322580]
4:Array[1619717805,322580]

This is causing the extrapolation math to exaggerate the idle value, which is causing the negative value.

Are you using any kind of caching or pushgateway in your setup? If yes, this is likely the cause of the failure. Without more info on your setup, it’s hard to say what the root cause is.