Hi everyone,
I’m having a strange problem. I had Prometheus setup on a host machine (Windows), which I ran with docker run -p 9090:9090 prom/prometheus and it was working fine, collecting metrics as it should and presenting them at [ip]:[port]/metrics.
The goal was to have multiple servers being monitored on the host. I put a node exporter into my test server, and it was collecting metrics but they weren’t showing up on the host. However, I decided to restart Prometheus on the host, and from that point onwards, running the docker run command written above always gets the setup stuck at “Starting rule manager”. I’ve already tried to reset everything, and it always hangs there.
The test server is still running fine, I can access its metrics page from the host without issue.
prometheus.yml:
# my global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
# - alertmanager:9093
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first_rules.yml"
# - "second_rules.yml"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: "prometheus"
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ["localhost:9090"]
A part that confused me in the log was this:
ts=2023-08-23T16:55:05.304Z caller=main.go:1231 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
since the host is Windows and not Linux…?
Strangely enough, this seems to function somehow, so I think it might be just looking for the .yml file in the root directory from which docker run is called?
ts=2023-08-23T16:55:05.305Z caller=main.go:1268 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=527.906µs
This is the part where it gets stuck on:
ts=2023-08-23T16:55:05.305Z caller=main.go:1011 level=info msg="Server is ready to receive web requests."
ts=2023-08-23T16:55:05.305Z caller=manager.go:1009 level=info component="rule manager" msg="Starting rule manager..."
It doesn’t abort the setup, it just hangs on this line.
No clue what to do, any help?