Scraping from rabbitmq

Prometheus server
1

I’m trying to scrape the metrics from rabbitmq.

I defined the service monitor as shown below, but prom-operator isn’t picking it up.

Do I have to restart prometheus to pick up the change?

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  creationTimestamp: "2023-02-24T16:48:50Z"
  generation: 1
  labels:
    kustomize.toolkit.fluxcd.io/name: flux-system
    kustomize.toolkit.fluxcd.io/namespace: flux-system
    name: rabbitmq
  name: rabbitmq
  namespace: monitoring
  resourceVersion: "56280766"
  uid: 99a0c1a8-3152-435e-8f00-3fd6590d3628
spec:
  endpoints:
  - path: /metrics
    targetPort: 15692
  namespaceSelector:
    any: true
  selector:
    matchLabels:
      name: rabbitmq

The rabbitmq service looks like this:

apiVersion: v1
kind: Service
metadata:
  annotations:
    rabbitmq.com/queueRebalanceNeededAt: "2023-02-21T21:22:17Z"
  creationTimestamp: "2023-02-18T22:03:12Z"
  labels:
    app.kubernetes.io/component: rabbitmq
    app.kubernetes.io/name: rabbitmq
    app.kubernetes.io/part-of: rabbitmq
    kustomize.toolkit.fluxcd.io/name: flux-system
    kustomize.toolkit.fluxcd.io/namespace: flux-system
  name: rabbitmq
  namespace: default
  ownerReferences:
  - apiVersion: rabbitmq.com/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: RabbitmqCluster
    name: rabbitmq
    uid: 48884b05-a2bf-4b01-a346-6030dba21ad2
  resourceVersion: "54813830"
  uid: f2b41b31-26fb-4760-bc93-3f7d9f8c1131
spec:
  clusterIP: 10.43.47.82
  clusterIPs:
  - 10.43.47.82
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: amqp
    name: amqp
    port: 5672
    protocol: TCP
    targetPort: 5672
  - appProtocol: http
    name: management
    port: 15672
    protocol: TCP
    targetPort: 15672
  - appProtocol: prometheus.io/metrics
    name: prometheus
    port: 15692
    protocol: TCP
    targetPort: 15692
  - appProtocol: mqtt
    name: mqtt
    port: 1883
    protocol: TCP
    targetPort: 1883
  selector:
    app.kubernetes.io/name: rabbitmq
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}
2

I had to add:

metadata:
  labels:
    release: kube-prometheus-stack

Now it is showin in the service discovery but nothing but being added.

image
image797×73 21.8 KB

It is showing 0/56 active targets.

And all are being dropped?

serviceMonitor/default/rabbitmq/0

Discovered Labels Target Labels
address=“192.168.76.237:6443” Dropped
__meta_kubernetes_endpoint_port_name=“https”
__meta_kubernetes_endpoint_port_protocol=“TCP”
__meta_kubernetes_endpoint_ready=“true”

I have the feeling that I am so close but that last little bit is missing.

3

Changed it to:

spec:
  jobLabel: default-rabbitmq
  selector:
    matchLabels:
      app.kubernetes.io/name: rabbitmq
  namespaceSelector:
    matchNames:
      - default

I was looking at that a 100 times, I thought that app.kubernetes.io/name: rabbitmq and name: rabbitmq were equivalent.

Thank you so much to Joao on Slack for the pointer!