Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible issue with postgres process closing, runaways. #199

Closed
NOVACyclist opened this issue Jul 10, 2023 · 11 comments
Closed

Possible issue with postgres process closing, runaways. #199

NOVACyclist opened this issue Jul 10, 2023 · 11 comments

Comments

@NOVACyclist
Copy link

I have been using the container for a while and just updated it to :latest on Ubuntu and it seems to be spinning a lot of processes off to postgres and not closing them, leaving me in a condition where I have no sockets left to do any actual scanning. It keeps spinning off processes and then it hits the 100 connection limit and I am unable to navigate around the front-end UI, delete reports that are having errors, etc. It then runs away with all of the system resources and I am forced to kill the container. I've tried leaving the instance idle to let any background processes finish

Symptoms:
log:
==> /usr/local/var/log/gvm/gvmd.log <==
md manage:WARNING:2023-07-10 16h27.08 utc:1819: sql_open: PQconnectPoll failed
md manage:WARNING:2023-07-10 16h27.08 utc:1819: sql_open: PQerrorMessage (conn): connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: remaining connection slots are reserved for non-replication superuser connections
md manage:WARNING:2023-07-10 16h27.08 utc:1819: init_manage_open_db: sql_open failed
md manage:WARNING:2023-07-10 16h27.16 utc:1822: sql_open: PQconnectPoll failed
md manage:WARNING:2023-07-10 16h27.16 utc:1822: sql_open: PQerrorMessage (conn): connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: remaining connection slots are reserved for non-replication superuser connections
md manage:WARNING:2023-07-10 16h27.16 utc:1822: init_manage_open_db: sql_open failed
md manage:WARNING:2023-07-10 16h27.27 utc:1826: sql_open: PQconnectPoll failed
md manage:WARNING:2023-07-10 16h27.27 utc:1826: sql_open: PQerrorMessage (conn): connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: remaining connection slots are reserved for non-replication superuser connections

==> /usr/local/var/log/gvm/healthchecks.log <==
HEALTHECHECK FAILED !
These services failed
openvas

ps -ef | grep gvmd | egrep -v "post|grep"

docker-user 25218 24268 54 12:07 ? 00:03:43 gvmd -a 0.0.0.0 -p 9390 --listen-group=gvm --osp-vt-update=/var/run/ospd/ospd-openvas.sock --max-email-attachment-size=64000000 --max-email-include-size=64000000 --max-email-message-size=64000000 --
docker-user 25265 24268 53 12:07 ? 00:03:30 gvmd -a 0.0.0.0 -p 9390 --listen-group=gvm --osp-vt-update=/var/run/ospd/ospd-openvas.sock --max-email-attachment-size=64000000 --max-email-include-size=64000000 --max-email-message-size=64000000 --
docker-user 25267 24268 54 12:07 ? 00:03:34 gvmd -a 0.0.0.0 -p 9390 --listen-group=gvm --osp-vt-update=/var/run/ospd/ospd-openvas.sock --max-email-attachment-size=64000000 --max-email-include-size=64000000 --max-email-message-size=64000000 --
docker-user 25272 24268 54 12:07 ? 00:03:36 gvmd -a 0.0.0.0 -p 9390 --listen-group=gvm --osp-vt-update=/var/run/ospd/ospd-openvas.sock --max-email-attachment-size=64000000 --max-email-include-size=64000000 --max-email-message-size=64000000 --
(repeated about 100x)

ps -ef | egrep "gvm|UID" | egrep "idle|UID" | grep -v grep | cat -n

 1	UID          PID    PPID  C STIME TTY          TIME CMD
 2	_apt       28223   28110  0 12:16 ?        00:00:00 postgres: gvm gvmd [local] idle
 3	_apt       28954   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
 4	_apt       28959   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
 5	_apt       28960   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
 6	_apt       28967   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
 7	_apt       28980   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
 8	_apt       28987   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
 9	_apt       28988   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
10	_apt       28989   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
11	_apt       28990   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
12	_apt       28991   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
13	_apt       28996   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
14	_apt       29022   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
15	_apt       29023   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
16	_apt       29024   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
17	_apt       29035   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
18	_apt       29036   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
19	_apt       29037   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
20	_apt       29038   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
21	_apt       29049   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
22	_apt       29050   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
23	_apt       29051   28110  0 12:18 ?        00:00:02 postgres: gvm gvmd [local] idle
24	_apt       29062   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
25	_apt       29063   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
26	_apt       29064   28110  0 12:18 ?        00:00:01 postgres: gvm gvmd [local] idle
27	_apt       29096   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
28	_apt       29098   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
29	_apt       29099   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
30	_apt       29100   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
31	_apt       29101   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
32	_apt       29102   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
33	_apt       29111   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
34	_apt       29115   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
35	_apt       29161   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
36	_apt       29169   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
37	_apt       29170   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
38	_apt       29171   28110  0 12:18 ?        00:00:02 postgres: gvm gvmd [local] idle
39	_apt       29285   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
40	_apt       29293   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
41	_apt       29294   28110  0 12:18 ?        00:00:02 postgres: gvm gvmd [local] idle
42	_apt       29295   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
43	_apt       29383   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
44	_apt       29391   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
45	_apt       29392   28110  0 12:18 ?        00:00:03 postgres: gvm gvmd [local] idle
46	_apt       29393   28110  0 12:18 ?        00:00:00 postgres: gvm gvmd [local] idle
47	_apt       29469   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
48	_apt       29477   28110  0 12:19 ?        00:00:03 postgres: gvm gvmd [local] idle
49	_apt       29478   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
50	_apt       29479   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
51	_apt       29519   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
52	_apt       29527   28110  0 12:19 ?        00:00:03 postgres: gvm gvmd [local] idle
53	_apt       29529   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
54	_apt       29530   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
55	_apt       29531   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
56	_apt       29537   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
57	_apt       29544   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
58	_apt       29547   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
59	_apt       29567   28110  1 12:19 ?        00:00:05 postgres: gvm gvmd [local] idle
60	_apt       29568   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
61	_apt       29578   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
62	_apt       29579   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
63	_apt       29590   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
64	_apt       29591   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
65	_apt       29592   28110  2 12:19 ?        00:00:10 postgres: gvm gvmd [local] idle
66	_apt       29593   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
67	_apt       29602   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
68	_apt       29654   28110  0 12:19 ?        00:00:00 postgres: gvm gvmd [local] idle
69	_apt       29716   28110  3 12:20 ?        00:00:10 postgres: gvm gvmd [local] idle
70	_apt       29836   28110  0 12:20 ?        00:00:00 postgres: gvm gvmd [local] idle
71	_apt       29876   28110  3 12:20 ?        00:00:09 postgres: gvm gvmd [local] idle
72	_apt       29882   28110  0 12:20 ?        00:00:00 postgres: gvm gvmd [local] idle
73	_apt       29890   28110  0 12:20 ?        00:00:00 postgres: gvm gvmd [local] idle
74	_apt       29893   28110  0 12:20 ?        00:00:00 postgres: gvm gvmd [local] idle
75	_apt       29916   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
76	_apt       29920   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
77	_apt       29929   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
78	_apt       29930   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
79	_apt       29939   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
80	_apt       29940   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
81	_apt       29955   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
82	_apt       29956   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
83	_apt       29957   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
84	_apt       29958   28110  3 12:21 ?        00:00:09 postgres: gvm gvmd [local] idle
85	_apt       29968   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
86	_apt       29969   28110 40 12:21 ?        00:01:57 postgres: gvm gvmd [local] idle
87	_apt       30141   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
88	_apt       30142   28110 45 12:21 ?        00:02:03 postgres: gvm gvmd [local] idle
89	_apt       30192   28110  0 12:21 ?        00:00:00 postgres: gvm gvmd [local] idle
90	_apt       30217   28110  4 12:21 ?        00:00:10 postgres: gvm gvmd [local] idle
91	_apt       30278   28110  0 12:22 ?        00:00:00 postgres: gvm gvmd [local] idle
92	_apt       30428   28110  6 12:23 ?        00:00:10 postgres: gvm gvmd [local] idle
93	_apt       30489   28110  0 12:23 ?        00:00:00 postgres: gvm gvmd [local] idle
94	_apt       30677   28110 11 12:24 ?        00:00:10 postgres: gvm gvmd [local] idle
95	_apt       30753   28110  0 12:24 ?        00:00:00 postgres: gvm gvmd [local] idle
96	_apt       30885   28110 37 12:25 ?        00:00:02 postgres: gvm gvmd [local] idle

OS:
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

OpenVAS Container Versions:

Before Upgrade:
pg_gvm=v22.4.0
notus_scanner=v22.5.0
gvmd=v22.4.2
openvas=v22.6.2
openvas_smb=v22.5.0
gvm_libs=v22.5.2
openvas_scanner=v22.6.2
gsa=v22.4.1
ospd=v21.4.4
ospd_openvas=v22.5.1
python_gvm=v23.4.2
gvm_tools=v23.4.0
greenbone_feed_sync=v23.4.0

After:
pg_gvm=v22.5.1
notus_scanner=v22.5.0
gvmd=v22.5.3
openvas=v22.7.2
openvas_smb=v22.5.3
gvm_libs=v22.6.3
openvas_scanner=v22.7.2
gsa=v22.5.0
ospd=v21.4.4
ospd_openvas=v22.5.1
python_gvm=v23.5.1
gvm_tools=v23.4.0
greenbone_feed_sync=v23.6.0

@xenago
Copy link

xenago commented Jul 10, 2023

I believe it's related to this issue also affecting the official community containers:

greenbone/gvmd#2038

I installed 22.4.18 instead and it's working for now.

@immauss
Copy link
Owner

immauss commented Jul 10, 2023 via email

@NOVACyclist
Copy link
Author

Just to confirm it was not upgrade related I installed a new VM and a clean copy of the image, and the same issue appeared.

unrelated: immauss, Thank you so much for putting this container together. It has been super solid for me and as someone who has fought the OpenVAS install many times, the number of hours and headaches you've saved me since I've switched to your container is immeasurable. Your service to the community is most appreciated!!!

@immauss
Copy link
Owner

immauss commented Jul 11, 2023

@NOVACyclist Thanks!

I do think this is code based. If I'm following the thred provided by @xenago correctly, this should be in gvmd 22.5.4, which I'm building now as 22.4.20-beta. Once I've tested it, I'll push all 3 archs to docker hub.

-Scott

@nitsewg
Copy link

nitsewg commented Jul 11, 2023

Just updated the container this morning, and I am running into the same issue. It's been really solid up until now. Looking forward to getting it going again. Thanks a bunch for all of your work on this. So far, it has made running greenbone / openvas a breeze.

@DRIgnazGortngschirl
Copy link
Contributor

@immauss FYI: I have had the same issue and used the 22.4.20-beta and seems to run smoothly, will let you know when I encounter something.

Anyway, thanks for maintaining this project so well, Scott!

@immauss
Copy link
Owner

immauss commented Jul 12, 2023

@DRIgnazGortngschirl
Thanks!
The Datacenter fire in NJ last night ate up all my time, so didn't get a chance to finalize.
I'm going to kick off the build/push for final on 22.4.20 now, so assuming it builds smoothly, should be available as latest in a few hours.

@ciclonite
Copy link

Hi! same problem here.
@DRIgnazGortngschirl have you found a workaround?
Thanks.

@DRIgnazGortngschirl
Copy link
Contributor

@ciclonite

You can use the 22.4.20-beta version as mentioned by Scott

I'm going to kick off the build/push for final on 22.4.20 now, so assuming it builds smoothly, should be available as latest in a few hours.

or main release 22.4.20 version $ docker pull immauss/openvas:22.4.20 (both work for me fine) thx for the quick build / image from Scott.

I think you can close this issue @immauss.

@ciclonite
Copy link

@DRIgnazGortngschirl : thank you and sorry for the question, i've missed the 22.4.20 version tag.
Thank to you and @immauss

@NOVACyclist
Copy link
Author

immauss/openvas:22.4.20 has resolved the issue.

Thanks!!!!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants