Metrics aggregation best practices

I currently have a Prometheus deployment on my Kubernetes cluster, deployed with Prometheus Operator.
What would be the recommended way to aggregate metrics for all replicas of a deployment ?
The simple solution of using avg/sum function in Grafana over pods doesn’t seem to be really efficient, if I have a lot of changes in my scaling metrics won’t be stable at all.