# Single Prometheus restarted after it ups for 2 weeks

**URL:** <https://discuss.prometheus.io/t/single-prometheus-restarted-after-it-ups-for-2-weeks/1648>\
**Category:** General Help/Support\
**Created:** [May 16, 2023, 7:22am UTC](https://discuss.prometheus.io/t/single-prometheus-restarted-after-it-ups-for-2-weeks/1648 "2023-05-16T07:22:35Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![yuxi](https://avatars.discourse-cdn.com/v4/letter/y/58f4c7/32.png) [@yuxi](https://discuss.prometheus.io/u/yuxi)\
**Post date:** [May 16, 2023, 7:22am UTC](https://discuss.prometheus.io/t/single-prometheus-restarted-after-it-ups-for-2-weeks/1648/1 "2023-05-16T07:22:35Z")

</div>

Hello！  
I’m using docker-compose to run a single Prometheus, everything goes well in the first two weeks;  
then I saw the Prometheus container restart while grafana and alertmanager container were up in a few days.  
Everytime I delete the wal, The memory usage occpied by Prometheus will increase from 4.4g to 5.3g then crashed  
If there’s anyway to reduce memory usage of Prometheus or my Prometheus instance needs more RAM?

My total RAM is 7.6g

Thanks in advanced!

---

<div class="post-metadata">

**Author:** ![stuart](https://dub1.discourse-cdn.com/flex017/user_avatar/discuss.prometheus.io/stuart/32/18_2.png) [@stuart](https://discuss.prometheus.io/u/stuart)\
**Post date:** [May 16, 2023, 9:50am UTC](https://discuss.prometheus.io/t/single-prometheus-restarted-after-it-ups-for-2-weeks/1648/2 "2023-05-16T09:50:46Z")

</div>

Memory usage is mostly determined by the amount of metrics being scraped. So to reduce the memory usage you can scrape less hosts, reduce scrape frequency, have less labels or less metrics.

One way that this is commonly achieved is to shard by functional areas and run multiple servers.

Alternatively if you want all the metrics you will need to increase the amount of memory available.

---

<div class="post-metadata">

**Author:** ![yuxi](https://avatars.discourse-cdn.com/v4/letter/y/58f4c7/32.png) [@yuxi](https://discuss.prometheus.io/u/yuxi)\
**Post date:** [May 16, 2023, 10:17am UTC](https://discuss.prometheus.io/t/single-prometheus-restarted-after-it-ups-for-2-weeks/1648/3 "2023-05-16T10:17:57Z")

</div>

Thanks for reply,  
Yes! I do want all of the metrics  
Maybe deleteing the expired data periodically can help?

---

<div class="post-metadata">

**Author:** ![stuart](https://dub1.discourse-cdn.com/flex017/user_avatar/discuss.prometheus.io/stuart/32/18_2.png) [@stuart](https://discuss.prometheus.io/u/stuart)\
**Post date:** [May 16, 2023, 11:56am UTC](https://discuss.prometheus.io/t/single-prometheus-restarted-after-it-ups-for-2-weeks/1648/4 "2023-05-16T11:56:19Z")

</div>

The length of time metrics are stored for has minimal impact on memory usage (mostly it will change how much disk you are using). As mentioned the main contributor for memory usage will be the scrapes, so in this case you’d need to allocate more memory to Prometheus.

---

<div class="post-metadata">

**Author:** ![yuxi](https://avatars.discourse-cdn.com/v4/letter/y/58f4c7/32.png) [@yuxi](https://discuss.prometheus.io/u/yuxi)\
**Post date:** [May 19, 2023, 8:56am UTC](https://discuss.prometheus.io/t/single-prometheus-restarted-after-it-ups-for-2-weeks/1648/5 "2023-05-19T08:56:45Z")

</div>

You are right, I checked my Prometheus profile then I found there are more than 2000 requests in my job eg. \* [kube-apiserver (0 / 2259 active targets)] , but we only have 3 apiserver pods in our cluster, It’s weird.  
Here is my job for ‘kube-apiserver’ ,any suggestion？

- job\_name: ‘kube-apiserver’  
kubernetes\_sd\_configs:
- role: endpoints  
api\_server: [https://10.1.60.140:6443](https://10.1.60.140:6443)  
bearer\_token\_file: /etc/prometheus/k8s.token  
tls\_config:  
insecure\_skip\_verify: true  
bearer\_token\_file: /etc/prometheus/k8s.token  
scheme: https  
tls\_config:  
insecure\_skip\_verify: true  
relabel\_configs:
- source\_labels: [\_\_meta\_kubernetes\_namespace, \_\_meta\_kubernetes\_service\_name, \_\_meta\_kubernetes\_endpoint\_port\_name]  
action: keep  
regex: default;kubernetes;https
