I started to use checkpoint’s skyline application, i have it installed on a virtual Linux server with Prometheus and Grafana running on the same instance. I started to test it out with 5 nodes and it was working fine. As I started to roll it out noticed that some devices were not showing up. This was due to a few factors not being on the Prometheus side (Anti-spoof drops and mismatch on time). Once those issues were resolved the nodes re-appeared and I continued to add the rest of my environment. Suddenly all those nodes about 15-20 once again disappeared. I thought there was an issue with the version of Prometheus I was running 2.48 so I tried to upgrade to 2.53. I am very limited with my knowledge so I messed up the upgrade and Prometheus would not start up. I had to completely remove Prometheus and install 2.53 brand new. The install was successful and when I looked at the Prometheus server I saw all the devices I have sending their data to it. However it was for about 4-5 mins before they disappeared again.
When i look at the active connections i see several nodes connecting to the server over port 9090. I am wondering if anyone has seen this or a similar behavior.
