# Unable to remote write to AWS managed Prometheus workspace

**URL:** <https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277>\
**Category:** General Help/Support\
**Created:** [November 3, 2022, 5:38pm UTC](https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277 "2022-11-03T17:38:01Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![kingtaco94](https://avatars.discourse-cdn.com/v4/letter/k/9f8e36/32.png) [@kingtaco94](https://discuss.prometheus.io/u/kingtaco94)\
**Post date:** [November 3, 2022, 5:38pm UTC](https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277/1 "2022-11-03T17:38:01Z")

</div>

I’m unable to write to AWS managed prometheus workspace. I’m using vanilla prometheus running in a pod, and attempting to write to the managed aws prometheus workspace. I’m using sig4 for authentication to the remote write. It looks like i am authenticated but the request is too large. Im just scraping prometheus itself and sending the metrics via the remote write. What am i doing wrong? is there an optional setting i need to add?

```auto
    remote_write:
        - url: https://aps-workspaces.<region>.amazonaws.com/workspaces/<workspaceid>/api/v1/remote_write
          sigv4:
            region: us-west-2
            access_key: 
            secret_key: 
          queue_config:
            max_samples_per_send: 10000
            max_shards: 2000
            capacity: 25000

```

```auto
msg="non-recoverable error" count=10000 err="server returned HTTP status 400 Bad Request: series has too many labels (actual: 138, limit: 70) series: 

```

```auto
"server returned HTTP status 413 Request Entity Too Large: <RequestEntityTooLargeException>"

```

---

<div class="post-metadata">

**Author:** ![stuart](https://dub1.discourse-cdn.com/flex017/user_avatar/discuss.prometheus.io/stuart/32/18_2.png) [@stuart](https://discuss.prometheus.io/u/stuart)\
**Post date:** [November 3, 2022, 6:14pm UTC](https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277/2 "2022-11-03T18:14:32Z")

</div>

I’ve not had any issues sending data to AMP using the defaults (i.e. not setting things within the queue\_config block). The error suggests that 10,000 samples per send is too high, so I’d suggest looking to reduce that.

---

<div class="post-metadata">

**Author:** ![kingtaco94](https://avatars.discourse-cdn.com/v4/letter/k/9f8e36/32.png) [@kingtaco94](https://discuss.prometheus.io/u/kingtaco94)\
**Post date:** [November 3, 2022, 6:18pm UTC](https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277/3 "2022-11-03T18:18:47Z")

</div>

I tried to reduce that to below, and im still seeing issues. How can i get past this?

server returned HTTP status 400 Bad Request: series has too many labels (actual: 121, limit: 70) series:

```auto
         queue_config:
            max_samples_per_send: 100
            max_shards: 200
            capacity: 300

```

---

<div class="post-metadata">

**Author:** ![stuart](https://dub1.discourse-cdn.com/flex017/user_avatar/discuss.prometheus.io/stuart/32/18_2.png) [@stuart](https://discuss.prometheus.io/u/stuart)\
**Post date:** [November 3, 2022, 6:55pm UTC](https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277/4 "2022-11-03T18:55:09Z")

</div>

You would need to exclude the series with over 70 labels as the error suggests. Take a look at relabling rules. However even 70 labels is a lot, so I’d suggest looking into why there are so many labels attached to some of your metrics.

---

<div class="post-metadata">

**Author:** ![kingtaco94](https://avatars.discourse-cdn.com/v4/letter/k/9f8e36/32.png) [@kingtaco94](https://discuss.prometheus.io/u/kingtaco94)\
**Post date:** [November 3, 2022, 7:03pm UTC](https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277/5 "2022-11-03T19:03:53Z")

</div>

i commented out some of the labels, im deploying via helm manifest. I only enabled 1 job: prometheus to scrape itself, and then the remote write. GOing to test to see if data got sent to the remote write. How can we confirm from logs if the remote write is being processed. Right now in logs i see “remote\_write msg=“Done replaying WAL” duration=34.762575077s”

---

<div class="post-metadata">

**Author:** ![kingtaco94](https://avatars.discourse-cdn.com/v4/letter/k/9f8e36/32.png) [@kingtaco94](https://discuss.prometheus.io/u/kingtaco94)\
**Post date:** [November 3, 2022, 7:14pm UTC](https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277/6 "2022-11-03T19:14:44Z")

</div>

do you happen to have a sample prometheus.yml file that i can refer to. I dont have kube-state-exporter, or node exporter running, just the prometheus server.

---

<div class="post-metadata">

**Author:** ![stuart](https://dub1.discourse-cdn.com/flex017/user_avatar/discuss.prometheus.io/stuart/32/18_2.png) [@stuart](https://discuss.prometheus.io/u/stuart)\
**Post date:** [November 3, 2022, 8:32pm UTC](https://discuss.prometheus.io/t/unable-to-remote-write-to-aws-managed-prometheus-workspace/1277/7 "2022-11-03T20:32:01Z")

</div>

How do you mean you have commented out some of the labels?

If you go to the Prometheus UI what is the configuration listed?