Hello Community,
i have node_exporter running on low latency servers on core0.
our application running on non core0 showing performance degradation while agent is running. if we shutdown agent performance improves drastically.
i have monitored agent and it’s thread and it’s all running on core0 and takes cpu cycle from 6% to 7%.
while doing perf trace on events it’s spending maximum overhead on runtime.mallocgc and nmi
6.06% node_exporter [.] runtime.mallocgc
6.05% [kernel] [k] nmi
to isoloate the node_exporter traffic i moved them to scrap through dedicated interface and i dont see much memory consumption here.
i’m not able to figure out why would node_Exporter would cause slowness on application side while application is not running on core0.
can someone please able to help or shed some light/give clues what can be done to improve this
Host operating system: output of uname -a
Linux hostname 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8 19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
node_exporter version: output of node_exporter --version
node_exporter, version 1.3.1 (branch: HEAD, revision: a2321e7b940ddcff26873612bccdf7cd4c42b6b6)
build user: root@243aafa5525c
build date: 20211205-11:09:49
go version: go1.17.3
platform: linux/amd64
node_exporter command line flags
/usr/local/bin/node_exporter
–collector.systemd
–collector.ethtool
–collector.qdisc
–collector.zoneinfo
–collector.buddyinfo
–collector.interrupts
–collector.meminfo_numa
–collector.mountstats
–collector.processes
–collector.textfile
–collector.textfile.directory=/var/lib/node_exporter
–web.listen-address=0.0.0.0:9100
–web.telemetry-path=/metrics