Prometheus federation between 2 different kubernetes clusters in GKE using haproxy as IC

We are using 2 clusters(cluster A and cluster B) and also aiming to use prometheus federation to collect monitoring data. Cluster A is trying to reach out to cluster B through haproxy ingress controller but unfortunately http:///federate call is failing with 503 Service Unavailable. We have used http scheme.

where as when we used prometheus service type as LB in cluster B and then trying to access that from cluster A, it was successful.
To optimise resource, we are targeting not to use prometheus service type as LB.

Thanks a lot in advance.