I’d like to find a definitive method for calculating how much storage space is required for our prometheus data. We’ll be monitoring approximately 3K RHEL nodes, via node_exporter. We want to keep 24 months of data. We’ll also be using the Infiniband Exporter (runs on one node in the IB fabric) and the exporter for Slurm on approximately 10 HPC schedulers.

On the prometheus storage page there’s a formula, tho not explained entirely:

needed_disk_space = retention_time_seconds * ingested_samples_per_second * bytes_per_sample

How does one determine the “ingested_samples_per_second” ?

At present, I have only a small portion of our total 3K nodes in the prometheus.yml.

Will this formula accurately calculate our storage requirement?

I found this stack exchange article, but I do not know if it is authoritative:

How to calculate disk space required by Prometheus v2.2? - DevOps Stack Exchange.

It references this prometheus.io link, but that’s now 404:

https://prometheus.io/docs/prometheus/2.2/storage/

This formula suggested at the end of the stackexchange article to calculate bytes of storage required is:

retention_time_seconds *

rate(prometheus_tsdb_head_samples_appended_total[2h]) *

(rate(prometheus_tsdb_compaction_chunk_size_bytes_sum[2h]) / rate(prometheus_tsdb_compaction_chunk_samples_sum[2h]))

At present, I have only a small portion of our total 3K nodes in my prometheus.yml.

Will this formula accurately calculate our storage requirement?

Thanks for any help.