In this very helpful article:Prometheus Operator ServiceMonitor Troubleshooting | ManagedKube
ServiceMonitor troubleshoothing is very well explained…BUTI can’t find why Prometheus Operator is not picking up the new ServiceMonitor I’ve added.Here is my setup:a.- kubectl -n monitoring get prometheus
NAME VERSION REPLICAS AGE
k8s v2.19.2 1 3h44m b.- kubectl -n monitoring describe prometheus k8s | grep “Service Monitor Selector:” -C6
Role: alert-rules
Security Context:
Fs Group: 2000
Run As Non Root: true
Run As User: 1000
Service Account Name: prometheus-k8s
Service Monitor Selector:
Match Expressions:
Key: k8s-app
Operator: In
Values:
node-exporter
kube-state-metrics c.- ServiceMonitor YAML (I’m trying to scrape Prometheus Server itself!) apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
labels:
k8s-app: prometheus
name: prometheus
namespace: monitoring
spec:
endpoints:
- interval: 30s
port: web
selector:
matchLabels:
prometheus: k8s d.- And finally, the service configuration: apiVersion: v1
kind: Service
metadata:
labels:
prometheus: k8s
name: prometheus-k8s
namespace: monitoring
spec:
ports:
- name: web
port: 9090
targetPort: web
selector:
app: prometheus
prometheus: k8s
sessionAffinity: ClientIP After all this, when I go to Status->Targets in the Prometheus WebGUI I can’t see the new target being scraped. Any ideas?