Hi, I am trying to understand why my PromQL query is not working as I expect.
This query works as I expect and returns many results.
sum by (job) (rate(traces_span_metrics_duration_milliseconds_count{"deployment.environment"="PROD","status.code"="STATUS_CODE_ERROR"}[10m]))
Every result has instance and http.method and span.name labels.
If I try to sum by (instance) it also works as I expect, but as soon as I try to sum by any label with a dot e.g. sum by ("span.name") I get only {} as the result.
In the process of trying to find an answer to this, I have been referred to UTF-8 in Prometheus many times. I have read this document end to end at least twice, and to the best extent of my limited ability, I don’t believe it contains any information that explains why sum by ("span.name") does not work, and implies quoted label values should work in queries.
How can I use sum by () with span.name without renaming or mutating the original metric?