Installation failed: Precomputing word tokens

[MikeSteglich]

Installation failed: Precomputing word tokens
I tried to install Nominatim following the instruction on https://nominatim.org/release-docs/latest/admin/Installation/.

Unfortunately, the installation stops while Precomputing word tokens . The log looks as followed:

2022-04-13 09:43:16: Using project directory: /data/nominatim/nominatim-planet
2022-04-13 09:43:16: Creating database
2022-04-13 09:43:23: Setting up country tables
2022-04-13 09:43:26: Importing OSM data file
2022-04-13 09:43:26  osm2pgsql version 1.5.1
2022-04-13 09:43:26  Database version: 12.9 (Ubuntu 12.9-0ubuntu0.20.04.1)
2022-04-13 09:43:26  PostGIS version: 3.0
2022-04-13 09:43:34  Parsing gazetteer style file '/usr/local/etc/nominatim/import-extratags.style'.
NOTICE:  table "place" does not exist, skipping
2022-04-13 14:45:06  Reading input files done in 18091s (5h 1m 31s).                      
2022-04-13 14:45:06    Processed 7587737772 nodes in 2724s (45m 24s) - 2786k/s
2022-04-13 14:45:06    Processed 847173340 ways in 10873s (3h 1m 13s) - 78k/s
2022-04-13 14:45:06    Processed 9773277 relations in 4494s (1h 14m 54s) - 2k/s
2022-04-13 14:45:09  Done postprocessing on table 'planet_osm_nodes' in 0s
2022-04-13 14:45:09  Done postprocessing on table 'planet_osm_ways' in 0s
2022-04-13 14:45:09  Done postprocessing on table 'planet_osm_rels' in 0s
2022-04-13 14:45:11  osm2pgsql took 18105s (5h 1m 45s) overall.
2022-04-13 14:45:11: Create functions (1st pass)
2022-04-13 14:45:12: Create tables
2022-04-13 14:47:06: Create functions (2nd pass)
2022-04-13 14:47:06: Create table triggers
2022-04-13 14:47:06: Create partition tables
2022-04-13 14:48:11: Create functions (3rd pass)
2022-04-13 14:48:11: Importing wikipedia importance data
2022-04-13 14:51:30: Initialise tables
2022-04-13 14:52:34: Load data into placex table
...................................................................... (this part is shortened to save space) 
2022-04-13 20:08:32: Setting up tokenizer
2022-04-13 20:08:33: Precomputing word tokens

When I check the database, I get the following results:

2022-04-14 12:13:38: Using project directory: /data/nominatim/nominatim-planet
2022-04-14 12:13:38: Checking database
Checking database connection ... OK
Checking for placex table ... OK
Checking for placex content ... OK
2022-04-14 12:13:39: Tokenizer was not set up properly. Database property missing.
Checking that tokenizer works ... Failed
Cannot load tokenizer. Did the import finish sucessfully?

Any advice on how I can find more information to analyse or fix the problem is appreciated.

Software Environment

• Nominatim version: 4.0.1
• Postgresql version: 12.9
• OS: Ubuntu 20.04

Hardware Configuration (please complete the following information):

• RAM: 512G
• number of CPUs: 20
• size of disks: >1T

Postgresql Configuration (only changed entries)
data_directory = '/data/postgres-data/main'
shared_buffers = 2GB
maintenance_work_mem = (10GB)
autovacuum_work_mem = 2GB
work_mem = (50MB)
effective_cache_size = (24GB)
synchronous_commit = off
max_wal_size = 1GB
checkpoint_timeout = 10min
checkpoint_completion_target = 0.9
fsync = off
full_page_writes = off

[lonvia]

Is there any activity on the database while token precomputation runs? Please check for messages in the PostgreSQL logs and run select * from pg_stat_activity on the database. Do you have any custom configuration in your .env file?

[MikeSteglich]

I have ran the import via ssh with nominatim import --osm-file planet-latest.osm.pbf > setup.log 2>&1 &.

My .env only contains NOMINATIM_FLATNODE_FILE="planet-flatnode"

The last entries in /var/log/postgresql/postgresql-12-main.log are as follows:

2022-04-13 15:08:12.344 UTC [37678] LOG:  checkpoints are occurring too frequently (28 seconds apart)
2022-04-13 15:08:12.344 UTC [37678] HINT:  Consider increasing the configuration parameter "max_wal_size".
2022-04-14 00:26:38.216 UTC [61499] nominatim@nominatim LOG:  could not send data to client: Broken pipe
2022-04-14 00:26:38.218 UTC [61499] nominatim@nominatim FATAL:  connection to client lost
2022-04-14 12:20:58.342 UTC [70426] postgres@postgres LOG:  provided user name (postgres) and authenticated user name (nominatim) do not match
2022-04-14 12:20:58.342 UTC [70426] postgres@postgres FATAL:  Peer authentication failed for user "postgres"
2022-04-14 12:20:58.342 UTC [70426] postgres@postgres DETAIL:  Connection matched pg_hba.conf line 89: "local   all             postgres                        
        peer"
2022-04-14 12:50:47.047 UTC [70188] nominatim@nominatim LOG:  could not send data to client: Broken pipe
2022-04-14 12:50:47.047 UTC [70188] nominatim@nominatim FATAL:  connection to client lost

psql -c "select * from pg_stat_activity" results in:

  datid   |  datname  |  pid  | usesysid |  usename  | application_name | client_addr | client_hostname | client_port |         backend_start         |          xact_start           |          query_start          |         state_change          | wait_event_type |     wait_event      | state  | backend_xid | backend_xmin |             query              |         backend_type         
----------+-----------+-------+----------+-----------+------------------+-------------+-----------------+-------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------+-----------------+---------------------+--------+-------------+--------------+--------------------------------+------------------------------
          |           | 37681 |          |           |                  |             |                 |             | 2022-04-11 12:36:15.334353+00 |                               |                               |                               | Activity        | AutoVacuumMain      |        |             |              |                                | autovacuum launcher
          |           | 37683 |       10 | postgres  |                  |             |                 |             | 2022-04-11 12:36:15.335687+00 |                               |                               |                               | Activity        | LogicalLauncherMain |        |             |              |                                | logical replication launcher
 15332726 | nominatim | 72642 |    16384 | nominatim | psql             |             |                 |          -1 | 2022-04-14 16:43:28.458749+00 | 2022-04-14 16:43:28.480503+00 | 2022-04-14 16:43:28.480503+00 | 2022-04-14 16:43:28.480506+00 |                 |                     | active |             |          877 | select * from pg_stat_activity | client backend
          |           | 37679 |          |           |                  |             |                 |             | 2022-04-11 12:36:15.332986+00 |                               |                               |                               | Activity        | BgWriterMain        |        |             |              |                                | background writer
          |           | 37678 |          |           |                  |             |                 |             | 2022-04-11 12:36:15.332381+00 |                               |                               |                               | Activity        | CheckpointerMain    |        |             |              |                                | checkpointer
          |           | 37680 |          |           |                  |             |                 |             | 2022-04-11 12:36:15.333647+00 |                               |                               |                               | Activity        | WalWriterMain       |        |             |              |                                | walwriter
(6 rows)

[lonvia]

The lost connections are not normal but I can't say what would cause that other than that you cancelled the import at that time.

[MikeSteglich]

Thank you for your reply. I will look into the matter again.

I have seen several Docker containers for Nominatim. Maybe that is a good idea. Do you have a suggestion for this?

[mtmail]

https://github.com/mediagis/nominatim-docker Is the most well maintained