I’d like to read test results into Prometheus as metrics. Other systems generally seem to do this via JUnit formatted XML, which I can generate, but I haven’t figured out how to get that data into Prometheus. I feel like maybe a pushgateway would be required, since these are short lived test containers? But beyond that I’m a bit stuck. Anyone have any suggestions on this or any other ways to track test results short of rewriting everything to
Thank you!
You need to be very careful about what type of data it is and what you are trying to achieve.
Prometheus is a metrics system rather than an event store, so it might be that you need a different solution.
Could you give a bit more detail about what you are trying to achieve?
I’d like to keep metrics on how long the various tests take, and how often they fail, essentially.
This is one of the cases where you may need both an event store and metrics.
For Prometheus having a metric that has “number of failures” makes perfect sense & is a good usage. Generating that metric however might need you to store the actual individual test runs to allow you to calculate that information.
The test runs are already stored in our CI, which is then where I was hoping to export them from for these metrics (since the CI system doesn’t have this capability). Just needing to figure out the formatting. Looking, I see I could maybe try to write the output in openmetrics format, but was hoping to stick to jUnit because we have multiple systems in multiple languages, but they all have the ability to generate jUnit so there’d be consistency there.
did you manage to do that? I have the same requirement and issue.
I want to get test results out to prometheus, I’m using pytest and ctest, any solution in mind would be great
I didn’t find a way, no. Ended up having to leave it for now to work on some other things, but I hope to get back to it one of these days. :-/