Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cAdvisor is not starting #3612

Open
jura43 opened this issue Oct 23, 2024 · 1 comment
Open

cAdvisor is not starting #3612

jura43 opened this issue Oct 23, 2024 · 1 comment

Comments

@jura43
Copy link

jura43 commented Oct 23, 2024

I am trying to run cAdvisor in Docker container, but every time 2 minutes after creating container is up, but state is unhealthy.

sudo docker run \
  -idt \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:ro \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --volume=/dev/disk/:/dev/disk:ro \
  --restart unless-stopped \
  --net=host \
  --name=cadvisor \
  gcr.io/cadvisor/cadvisor-amd64:v0.49.1 \
  -disable_metrics=disk,network,tcp,udp \
  -enable_load_reader=false \
  -housekeeping_interval=55s \
  -docker_only=true \
  -v=4
CONTAINER ID   IMAGE                                                                           COMMAND                  CREATED         STATUS                     PORTS       NAMES
ee1b9443812d   gcr.io/cadvisor/cadvisor-amd64:v0.49.1                                          "/usr/bin/cadvisor -…"   2 minutes ago   Up 2 minutes (unhealthy)               cadvisor
[email protected]:~$ curl http://localhost:8080/metrics -vvv
*   Trying 127.0.0.1:8080...
* TCP_NODELAY set
* connect to 127.0.0.1 port 8080 failed: Connection refused
*   Trying 127.0.0.1:8080...
* TCP_NODELAY set
* connect to 127.0.0.1 port 8080 failed: Connection refused
* Failed to connect to localhost port 8080: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 8080: Connection refused
[email protected]:~$ ss -tulpn | grep 8080
[email protected]:~$ docker -v
Docker version 20.10.5, build 55c4c88
[email protected]:~$ uname -a
Linux trans1.itc 5.4.0-70-generic #78-Ubuntu SMP Fri Mar 19 13:29:52 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

This are all logs I am getting:

I1023 13:33:02.470580       1 cadvisor.go:123] enabled metrics: advtcp,app,cpu,cpuLoad,cpu_topology,cpuset,diskIO,hugetlb,memory,memory_numa,oom_event,percpu,perf_event,process,referenced_memory,resctrl,sched
I1023 13:33:02.470667       1 storagedriver.go:55] Caching stats in memory for 2m0s
I1023 13:33:02.470818       1 manager.go:169] cAdvisor running in container: "/docker/ee1b9443812db9d6ee29cb5944e25b141e78682a1ffe015a00219ab6f8981a40"
I1023 13:33:02.642212       1 fs.go:133] Filesystem UUIDs: map[07c73ab5-b553-456e-bb02-4656f6ae2fdf:/dev/dm-0 29088e98-7091-495b-8116-151e12ddec51:/dev/md127 3e457501-12a8-492b-bdb5-541cead4d734:/dev/md0 6ffc5d26-98d7-4130-9e36-43e9aa01120f:/dev/dm-3 9d0f01a5-ceda-447b-ab39-9074c08750b8:/dev/dm-2 d3617fa4-9a37-487a-8aaa-a2f9eea0c4db:/dev/dm-1]
I1023 13:33:02.642296       1 fs.go:134] Filesystem partitions: map[/dev:{mountpoint:/dev major:0 minor:75 fsType:tmpfs blockSize:0} /dev/mapper/server1.local-root:{mountpoint:/rootfs major:253 minor:0 fsType:ext4 blockSize:0} /dev/mapper/server1.local-var:{mountpoint:/var/lib/docker major:253 minor:1 fsType:ext4 blockSize:0} /dev/mapper/server1.local-var--log:{mountpoint:/rootfs/var/log major:253 minor:2 fsType:ext4 blockSize:0} /dev/md0:{mountpoint:/rootfs/boot major:9 minor:0 fsType:ext2 blockSize:0} /dev/md127:{mountpoint:/rootfs/media major:9 minor:127 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:78 fsType:tmpfs blockSize:0} /proc/acpi:{mountpoint:/proc/acpi major:0 minor:79 fsType:tmpfs blockSize:0} /proc/kcore:{mountpoint:/proc/kcore major:0 minor:75 fsType:tmpfs blockSize:0} /proc/keys:{mountpoint:/proc/keys major:0 minor:75 fsType:tmpfs blockSize:0} /proc/sched_debug:{mountpoint:/proc/sched_debug major:0 minor:75 fsType:tmpfs blockSize:0} /proc/scsi:{mountpoint:/proc/scsi major:0 minor:80 fsType:tmpfs blockSize:0} /proc/timer_list:{mountpoint:/proc/timer_list major:0 minor:75 fsType:tmpfs blockSize:0} /rootfs/dev/shm:{mountpoint:/rootfs/dev/shm major:0 minor:26 fsType:tmpfs blockSize:0} /rootfs/run:{mountpoint:/rootfs/run major:0 minor:25 fsType:tmpfs blockSize:0} /rootfs/run/lock:{mountpoint:/rootfs/run/lock major:0 minor:27 fsType:tmpfs blockSize:0} /rootfs/run/user/1000:{mountpoint:/rootfs/run/user/1000 major:0 minor:82 fsType:tmpfs blockSize:0} /rootfs/sys/fs/cgroup:{mountpoint:/rootfs/sys/fs/cgroup major:0 minor:28 fsType:tmpfs blockSize:0} /rootfs/var/lib/docker/containers/50abac060d19a1af3202de289bacd1afc74734b41df2df52e89c8687b0459d58/mounts/secrets:{mountpoint:/rootfs/var/lib/docker/containers/50abac060d19a1af3202de289bacd1afc74734b41df2df52e89c8687b0459d58/mounts/secrets major:0 minor:109 fsType:tmpfs blockSize:0} /rootfs/var/lib/docker/containers/851fc8be2c7a3b5ca4656c567b6636129a318813641947aecfc59368949327eb/mounts/secrets:{mountpoint:/rootfs/var/lib/docker/containers/851fc8be2c7a3b5ca4656c567b6636129a318813641947aecfc59368949327eb/mounts/secrets major:0 minor:121 fsType:tmpfs blockSize:0} /rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev:{mountpoint:/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev major:0 minor:75 fsType:tmpfs blockSize:0} /rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev/shm:{mountpoint:/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev/shm major:0 minor:78 fsType:tmpfs blockSize:0} /rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/sys/fs/cgroup:{mountpoint:/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/sys/fs/cgroup major:0 minor:77 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:25 fsType:tmpfs blockSize:0} /run/lock:{mountpoint:/run/lock major:0 minor:27 fsType:tmpfs blockSize:0} /run/user/1000:{mountpoint:/run/user/1000 major:0 minor:82 fsType:tmpfs blockSize:0} /sys/firmware:{mountpoint:/sys/firmware major:0 minor:81 fsType:tmpfs blockSize:0} /sys/fs/cgroup:{mountpoint:/sys/fs/cgroup major:0 minor:28 fsType:tmpfs blockSize:0} /var/lib/docker/containers/50abac060d19a1af3202de289bacd1afc74734b41df2df52e89c8687b0459d58/mounts/secrets:{mountpoint:/var/lib/docker/containers/50abac060d19a1af3202de289bacd1afc74734b41df2df52e89c8687b0459d58/mounts/secrets major:0 minor:109 fsType:tmpfs blockSize:0} /var/lib/docker/containers/851fc8be2c7a3b5ca4656c567b6636129a318813641947aecfc59368949327eb/mounts/secrets:{mountpoint:/var/lib/docker/containers/851fc8be2c7a3b5ca4656c567b6636129a318813641947aecfc59368949327eb/mounts/secrets major:0 minor:121 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev major:0 minor:75 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev/shm:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev/shm major:0 minor:78 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/dev/shm:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/dev/shm major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/run:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/run major:0 minor:25 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/run/lock:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/run/lock major:0 minor:27 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/run/user/1000:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/run/user/1000 major:0 minor:82 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/sys/fs/cgroup:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/sys/fs/cgroup major:0 minor:28 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/containers/50abac060d19a1af3202de289bacd1afc74734b41df2df52e89c8687b0459d58/mounts/secrets:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/containers/50abac060d19a1af3202de289bacd1afc74734b41df2df52e89c8687b0459d58/mounts/secrets major:0 minor:109 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/containers/851fc8be2c7a3b5ca4656c567b6636129a318813641947aecfc59368949327eb/mounts/secrets:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/containers/851fc8be2c7a3b5ca4656c567b6636129a318813641947aecfc59368949327eb/mounts/secrets major:0 minor:121 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev major:0 minor:75 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev/shm:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/dev/shm major:0 minor:78 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/sys/fs/cgroup:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/sys/fs/cgroup major:0 minor:77 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/run:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/run major:0 minor:25 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/run/lock:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/run/lock major:0 minor:27 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/run/user/1000:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/run/user/1000 major:0 minor:82 fsType:tmpfs blockSize:0} /var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/sys/fs/cgroup:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/sys/fs/cgroup major:0 minor:28 fsType:tmpfs blockSize:0} 172.22.1.10:/media/input:{mountpoint:/rootfs/media/input major:0 minor:96 fsType:nfs4 blockSize:0} 172.22.1.10:/media/transcoder:{mountpoint:/rootfs/media/transcoder major:0 minor:96 fsType:nfs4 blockSize:0} nfsd:{mountpoint:/rootfs/proc/fs/nfsd major:0 minor:120 fsType:nfsd blockSize:0} overlay_0-107:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/383910581a2e4db6ebfc0382ffda040efedfe8dcd1085758a62b9dc241ce52ce/merged major:0 minor:107 fsType:overlay blockSize:0} overlay_0-108:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/dec4560c75b4f0426aee2d5cdf0f0fe8da323b71d27c599a63c0ee79bca30112/merged major:0 minor:108 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/1507f43bbb6b3f4f4b7d1d4eaddbbc1214586f988136b781dcc236222c13404c/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-52:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/5e312edd2ba284611aa651ce3801e5d313ee0bd88ed516a905e83a419dd70c8d/merged major:0 minor:52 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/3118521020b357b1f4bf98fd4a7744d98594d02f955e4a5e76b51ad3a8437b05/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-95:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/21cfc62cf07fc065175b274b95328007c9b9f725c43b9969e19be6e870ca8d0a/merged major:0 minor:95 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/docker/overlay2/fafe331a5fce5bea4b6e7aae00ec06bb162b6a0ffe37a1d41d047f62c1206400/merged/rootfs/var/lib/docker/overlay2/0b3c31efacc2132099d436907e2e5ed1bd21792c0757a2e65a05b77496c238b1/merged major:0 minor:97 fsType:overlay blockSize:0}]

I also tried to run cAdvisor without container, but result is the same.

@jura43 jura43 changed the title cAdvisor not starting cAdvisor is not starting Oct 23, 2024
@jura43
Copy link
Author

jura43 commented Oct 24, 2024

With version v0.50.0 cAdvsior starts without issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant