Gaps in prometheus scraping

Prometheus version 2.34.0

We use serverset SD to discover our spark targets

The spark service has two endpoints

They run on the same port

What’s weird is, we are noticing some gaps in /metrics/prometheus but no gaps in metrics from /metrics/executor/prometheus. The latter is much slower compared to the former scrape. We know for a fact we don’t reach the scrape timeouts

They are spawned as two separate jobs in prometheus. Can duplicate targets be a problem?

I see a lot of logs in Zk treecache package. How can we enable debug logs?