logging.shutdown() should clear logging._handlerList if that's what it's called with

[cjw296]

Bug report

Bug description:

In the wild, I've encountered situations where logging.shutdown gets called more than once in a process.

The current implementation is called with logging._handlerList by default, meaning that handlers can have their flush() and close() methods called more than once, which can cause bad things to happen if those handlers are not amenable to being closed more than once.

If logging.shutdown() is processing logging._handlerList, it should clear that list once it finished to prevent handlers having their flush() and close() methods called more than once.

CPython versions tested on:

3.12, CPython main branch

Operating systems tested on:

Linux, macOS

[ZeroIntensity]

Would you like to submit a PR?

[cjw296]

Would be good to get a second opinion as to why this is the way it is. @vsajip - you around on here?

[vsajip]

Well, the API documentation for logging.shutdown says:

This should be called at application exit and no further use of the logging system should be made after this call.

When the logging module is imported, it registers this function as an exit handler (see atexit), so normally there’s no need to do that manually.

So, calling shutdown() in user code is mildly questionable practice (as not needed, due to the atexit handler), and calling it more than once perhaps more strongly so (given the "no further use should be made ...").

logging.shutdown also catches and ignores OSError and ValueError exceptions raised when flushing/closing, so perhaps those bad things happening are due to handlers being badly written.

The logging.shutdown API is used internally in logging.config and in the logging test suite, so changing it could have ramifications there, too.

Since the case you mention involves calling shutdown more than once and perhaps involves handlers that don't just throw exceptions if called inappropriately, surely any changes to the stdlib in this area would be more of a band-aid, rather than addressing the root cause of the problems? I appreciate you may not have direct control of the code calling shutdown or of the handlers.

If the handlers throw some other stdlib exception than OSError and ValueError, I can look at broadening that catch clause.

[cjw296]

@vsajip - what reason could there be to not logging._handlerList.clear() once the processing is done during shutdown?

[picnixz]

Similar but not a duplicate: #129143

[vsajip]

what reason could there be to not

Don't know yet if there is one, but I'm generally not keen on making changes to cover for what might be bad practice in user code. I gave some specific reasons why I thought there might be bad practice, but those haven't been addressed.

As for #129143 - it just looks like a documentation inaccuracy, as the docstring matches the behaviour of the code.