Hey everyone! Was writing because we have a pod label cardinality issue with prometheus and, besides a shard bases approach I was wondering if there are any other solutions.
Ideally, I would love to scrape our pods, rollup the metrics without the pod label, then drop the original metric immediately after to save memory. I understand that dropping the pod label immediately leads to bad metrics, but I would love to keep it around only as long as its needed to rollup.
Is this possible or are we going to have to shard?