[4.3] temporarily increase query timeout during database warmup

[iAlex97]

Nominatim 4.3 introduced a new mechanism for setting query timeouts osm-search/Nominatim#3172

For a large import like planet, this resulted in timeouts while running the warmup queries:

+ sudo -H -E -u nominatim nominatim admin --warm
2023-09-30 05:50:48: Using project directory: /nominatim
2023-09-30 05:50:48: Warming database caches
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sqlalchemy/ext/asyncio/engine.py", line 457, in execute
    result = await greenlet_spawn(
  File "/usr/lib/python3/dist-packages/sqlalchemy/util/_concurrency_py3k.py", line 136, in greenlet_spawn
    result = context.switch(value)
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20
    return meth(self, args_10style, kwargs_10style, execution_options)
  File "/usr/lib/python3/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1481, in _execute_clauseelement
    ret = self._execute_context(
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1845, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2030, in _handle_dbapi_exception
    util.raise_(exc_info[1], with_traceback=exc_info[2])
  File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_
    raise exception
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1802, in _execute_context
    self.dialect.do_execute(
  File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute
    cursor.execute(statement, parameters)
  File "/usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 460, in execute
    self._adapt_connection.await_(
  File "/usr/lib/python3/dist-packages/sqlalchemy/util/_concurrency_py3k.py", line 76, in await_only
    return current.driver.switch(awaitable)
  File "/usr/lib/python3/dist-packages/sqlalchemy/util/_concurrency_py3k.py", line 129, in greenlet_spawn
    value = await result
  File "/usr/lib/python3/dist-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 423, in _prepare_and_execute
    self._rows = await prepared_stmt.fetch(*parameters)
  File "/usr/lib/python3/dist-packages/asyncpg/prepared_stmt.py", line 176, in fetch
    data = await self.__bind_execute(args, 0, timeout)
  File "/usr/lib/python3/dist-packages/asyncpg/prepared_stmt.py", line 241, in __bind_execute
    data, status, _ = await self.__do_execute(
  File "/usr/lib/python3/dist-packages/asyncpg/prepared_stmt.py", line 230, in __do_execute
    return await executor(protocol)
  File "asyncpg/protocol/protocol.pyx", line 201, in bind_execute
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/tasks.py", line 456, in wait_for
    return fut.result()
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/nominatim", line 12, in <module>
    exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module',
  File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 225, in nominatim
    return get_set_parser().run(**kwargs)
  File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 121, in run
    return args.command.run(args)
  File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/admin.py", line 60, in run
    return self._warm(args)
  File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/admin.py", line 106, in _warm
    api.search(word)
  File "/usr/local/lib/nominatim/lib-python/nominatim/api/core.py", line 709, in search
    return self._loop.run_until_complete(
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/usr/local/lib/nominatim/lib-python/nominatim/api/core.py", line 223, in search
    return await geocoder.lookup(phrases)
  File "/usr/local/lib/nominatim/lib-python/nominatim/api/search/geocoder.py", line 151, in lookup
    results = await self.execute_searches(query, searches[:50])
  File "/usr/local/lib/nominatim/lib-python/nominatim/api/search/geocoder.py", line 86, in execute_searches
    for result in await search.lookup(self.conn, self.params):
  File "/usr/local/lib/nominatim/lib-python/nominatim/api/search/db_searches.py", line 734, in lookup
    for row in await conn.execute(sql, _details_to_bind_params(details)):
  File "/usr/local/lib/nominatim/lib-python/nominatim/api/connection.py", line 63, in execute
    return await asyncio.wait_for(self.connection.execute(sql, params), self.query_timeout)
  File "/usr/lib/python3.10/asyncio/tasks.py", line 458, in wait_for
    raise exceptions.TimeoutError() from exc
asyncio.exceptions.TimeoutError

To overcome this, I am temporarily setting the query timeout to 600 seconds while running the warmup command and revert back to 10 seconds after it completes.

A further improvement idea would be to create different env variables for those timeouts (something like DOCKER_NOMINATIM_QUERY_TIMEOUT instead of NOMINATIM_QUERY_TIMEOUT) to allow users to specify different timeouts for their use-cases.

Fixes #478

[what-the-diff]

PR Summary

• Addition of New Environment Variables
  New environment variables named NOMINATIM_QUERY_TIMEOUT and NOMINATIM_REQUEST_TIMEOUT have been introduced in the scripts 4.3/init.sh and 4.3/start.sh.

• Commentary and Guidance for Query Types
  Detailed commentary has been added in both 4.3/init.sh and 4.3/start.sh scripts explaining the use of --search-only query type and also suggesting to revert back to the --reverse query type when a new version of Nominatim is released.

• Modification of Environment Variables
  The newly added environment variables, NOMINATIM_QUERY_TIMEOUT and NOMINATIM_REQUEST_TIMEOUT, have been adjusted with new values in the 4.3/init.sh and 4.3/start.sh scripts.

• Readiness Alert for Nominatim
  Introduced a new comment "Nominatim is ready to accept requests" in the 4.3/start.sh script which signifies that the Nominatim application is fully prepared to accept and process incoming requests.

[leonardehrenfried]

I would be ok with this solution. Lets see what @philipkozeny thinks.