Error when filtering by Users. "AttributeError: 'NoneType' object has no attribute 'users'"

[digdilem]

Description of problem

Under the menu item: "Jobs" if we select Filters and then "Users" we get the following and no data is shown;

/var/log/messages reports this

10:02:51 headnode_1 uwsgi[2414774]: uWSGIWorker4Core0: [ERROR] Exception on /api/users [GET]
Aug  6 10:02:51 headnode_1 uwsgi[2414774]: Traceback (most recent call last):
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:  File "/usr/lib/python3.9/site-packages/flask/app.py", line 2073, in wsgi_app
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:    response = self.full_dispatch_request()
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1518, in full_dispatch_request
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:    rv = self.handle_user_exception(e)
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1516, in full_dispatch_request
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:    rv = self.dispatch_request()
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:  File "/usr/lib/python3.9/site-packages/flask/app.py", line 1502, in dispatch_request
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:  File "/usr/lib/python3.9/site-packages/rfl/web/tokens.py", line 55, in wrapped
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:    return view(*args, **kwargs)
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:  File "/usr/lib/python3.9/site-packages/slurmweb/views/gateway.py", line 159, in users
Aug  6 10:02:51 headnode_1 uwsgi[2414774]:    for user in current_app.authentifier.users()
Aug  6 10:02:51 headnode_1 uwsgi[2414774]: AttributeError: 'NoneType' object has no attribute 'users'
Aug  6 10:02:51 headnode_1 uwsgi[2414774]: [pid: 2414774|app: 0|req: 193/968] 172.21.205.12 () {76 vars in 1524 bytes} [Tue Aug  6 10:02:51 2024] GET /api/users => generated 215 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 91 bytes (2 switches on core 0)

Other Filters work as expected, it's just Users that shows an error.

We thought it might be because we didn't have any users in Slurm's Db (we use AD for users to authenticate with), so we tried adding a normal user into slurmdbd on our TEST cluster, and that didn't help any.

We're using htttpd (apache2) as a reverse proxy for slurm-web with https passing to the local socks, and this seems to be working fine.

Using our TEST cluster to diagnose this. One headnode, 4 fairly beefy (40 cores, 384Gb each) work nodes with 13 partitions.

These are new installs, we have not previously looks at Slurm-Web and have not had this Filter working before.

Versions

Running on Rocky Linux 9.3, software versions:

python3-slurm-web.noarch                    3.1.0-1.el9                      @rackslab
slurm.x86_64                                23.02.5-1.el9                    @@commandline
slurm-perlapi.x86_64                        23.02.5-1.el9                    @@commandline
slurm-slurmctld.x86_64                      23.02.5-1.el9                    @@commandline
slurm-slurmd.x86_64                         23.02.5-1.el9                    @@commandline
slurm-slurmdbd.x86_64                       23.02.5-1.el9                    @@commandline
slurm-slurmrestd.x86_64                     23.02.5-1.el9                    @@commandline
slurm-web-agent.noarch                      3.1.0-1.el9                      @rackslab
slurm-web-gateway.noarch                    3.1.0-1.el9                      @rackslab

Thank you for any thoughts.

[rezib]

Oh this is an interesting bug indeed 🙂 Thanks for reporting!

Is authentication enabled in your gateway configuration? If not, that could explain, I may have missed this case in my tests.

[digdilem]

Thanks for the quick response, @rezib .

In /etc/slurm-web/gateway.ini ? Then no - this is that file

[agents]
url=https://headnode_1/agent

[ui]
host=https://headnode_1 

We don't have any authentication enabled that I can see (Sorry, I didn't set up the test instance)

There is an apache reverse proxy that does the certs and maps these back to the socks, like so;

    ProxyPass /agent unix:/run/slurm-web-agent/uwsgi.sock|uwsgi://agent/
    ProxyPass / unix:/run/slurm-web-gateway/uwsgi.sock|uwsgi://gateway/

[rezib]

Thanks for the quick response, @rezib .

In /etc/slurm-web/gateway.ini ? Then no - this is that file

[agents]
url=https://headnode_1/agent

[ui]
host=https://headnode_1 

Yes, it is in this file.

We don't have any authentication enabled that I can see (Sorry, I didn't set up the test instance)

Slurm-web extracts users from LDAP (or AD) so you have to configure authentication and [ldap] section for this feature to work.

Slurm-web should not fail as reported in this case though, something has to be fixed in some way.

[digdilem]

Thanks for your help - we weren't aware that users were gathered that way, and had assumed LDAP was used purely for authentication. It's helped us understand.

We tried enabling LDAP but weren't able to get it working against AD. But we didn't really want to enable authentication anyway, so didn't spend much time resolving that. (The machine is only reachable internally)

So we'll continue, but warn our users about that alert on the Users filter.

Suggestion: If [ldap] isn't configured, then Users filter could either

A) Show a more descriptive error, perhaps worded so it doesn't startle the users and make them raise tickets.
B) Hide the Users section entirely from filters.

Thank you for your help, and your work on this project.

(Fine to close this as resolved, or leave open as a suggestion - your call)

[rezib]

Yes, I prefer this issue to stay open until I implement either A/ or B/.

I chose to use LDAP as the source of users because it contains users fullname in addition to their logins, which is convenient when you want to find one user's job and you know its firstname/lastname but not its login. Another option is to use SlurmDBD as a fallback for users source but fullname are missing there.

I still have to think more about it.

[rezib]

@digdilem FYI I finally chose B) (hide users section entirely) when authentication is disabled, it has been merged with #337 and it will be released with upcoming v3.2.0.