Prometheus support ECS discovery from APP Mesh or something else?
I can see Cloudwatch agent have ecs_service_discovery but they didn’t publish that to Prometheus and i don’t want to use Cloudwatch Prometheus.
I find this but the last update is two years.
How do you export your metrics from ECS ?
What are you trying to get? Where to scrape the metrics from your containerized applications or to where to scrape ECS-specific metrics?
The latter - for example: desired containers or CPU utilization by an ECS service - are only exposed through AWS CloudWatch and there’s no other option than to use a cloudwatch exporter.
What the Prometheus ECS Discovery plugin will be helpful is to be able to dynamically find where to scrape the /metrics endpoint for your apps that expose Prometheus metrics.
I haven’t used it, but I know there can be some wrinkles to iron if the number of containers you run in your environment is really large or it changes really frequently.
@salessandri Did you find a solution to this?
@dburrows-poloniex Related to the OP, I was not able to understand what he was trying to achieve.
For what I had to do, extract CloudWatch metrics into a Prometheus consumer, I ended up developing my own exporter leveraging CloudWatch Stream.
What is it that you are trying to get done?
@salessandri I have several ECS tasks that have there own metrics endpoints that I need to scrape like haproxy an well as many custom services. So I looking for a way to have my Prometheus server in a separate account be able to identify and scrape the containers endpoints.
So I am looking for a discovery / exporter that will be able return to Prometheus the container IP at the very least.
I appreciate any thoughts or insights.
I haven’t had to do that, yet. But it seems that the CloudWatch Agent has the capability of leveraging the API to generate a file that’s compatible with Prometheus’ file service discovery scraping. This link is probably a good starting point. What is not clear to me after a quick look is whether it will parse and process all ECS tasks in a single host.
The option I had found a while back is what the OP linked. It seemed to work fine but if you have a large number of ECS tasks then the discovery process would take too long to finish.
If anyone is looking for dynamic ECS service discovery of ECS Clusters - check out this blog post:
It provides dynamic service discovery of your ECS Tasks running in AWS ECS Cluster and exposes HTTP endpoint that is compatible with http_sd_target of Prometheus.
Also available on GitHub: