Adding country to existing nominatim installation do not work

[b-michal]

Hello,

I have a strange issue when importing additional data to existing nominatim installation and I am looking for help. I am running nominatim 4.3.0 on Ubuntu 22.04.2 LTS with PostgreSQL 15.4 database, installed by following the official documentation. During the initial import we imported only Europe and Africa data (using files from Geofabrik website), and the current installation seems to be working fine.
However, now we wanted to add 3 more countries to our installation (Jordan, Israel and Palestine, all available on the Geofabrik website), so we followed the steps from here: https://github.com/osm-search/Nominatim/blob/master/docs/admin/Advanced-Installations.md#importing-multiple-regions-without-updates but we are unable to do so. It crashes already at the add-data command and I can't figure out how to make it work. Hereunder the output:
`2024-07-31 17:19:29: Using project directory: /mnt/data/nominatim-import-data
2024-07-31 17:19:29 osm2pgsql version 1.9.2
2024-07-31 17:19:29 Database version: 15.4 (Ubuntu 15.4-1.pgdg22.04+1)
2024-07-31 17:19:29 PostGIS version: 3.4
2024-07-31 17:19:29 Loading properties from table '"public"."osm2pgsql_properties"'.
2024-07-31 17:19:29 Using flat node file '/mnt/data/nominatim-import-data/flatnode.file' (same as on import).
2024-07-31 17:19:29 Using prefix 'planet_osm' (same as on import).
2024-07-31 17:19:29 Using style file '/usr/local/etc/nominatim/import-address.lua' (same as on import).
2024-07-31 17:19:29 ERROR: Database error: ERROR: relation "public.planet_osm_ways" does not exist
LINE 1: SELECT max(id) FROM "public"."planet_osm_ways"
^

Traceback (most recent call last):
File "/usr/local/bin/nominatim", line 12, in
exit(cli.nominatim(module_dir='/usr/local/lib/nominatim/module',
File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 225, in nominatim
return get_set_parser().run(**kwargs)
File "/usr/local/lib/nominatim/lib-python/nominatim/cli.py", line 121, in run
return args.command.run(args)
File "/usr/local/lib/nominatim/lib-python/nominatim/clicmd/add_data.py", line 79, in run
return add_osm_data.add_data_from_file(args.config.get_libpq_dsn(),
File "/usr/local/lib/nominatim/lib-python/nominatim/tools/add_osm_data.py", line 36, in add_data_from_file
_run_osm2pgsql(dsn, options)
File "/usr/local/lib/nominatim/lib-python/nominatim/tools/add_osm_data.py", line 21, in _run_osm2pgsql
run_osm2pgsql(options)
File "/usr/local/lib/nominatim/lib-python/nominatim/tools/exec_utils.py", line 76, in run_osm2pgsql
subprocess.run(cmd, cwd=options.get('cwd', '.'),
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/lib/nominatim/osm2pgsql', '--hstore', '--latlon', '--slim', '--log-progress', 'true', '--number-processes', '1', '--cache', '1000', '--style', '/usr/local/etc/nominatim/import-address.lua', '--output', 'flex', '--append', '--flat-nodes', '/mnt/data/nominatim-import-data/flatnode.file', 'jordan-latest.osm.pbf']' returned non-zero exit status 1.
`
From what I understand the table public.planet_osm_ways is missing, but I have no idea on how to properly create it and what data it should contain to make the import work as expected.

Could you please help me with that?

Thanks in advance

[mtmail]

Make sure you use the documentation for Nominatim 4.3 instead of 'master'. https://nominatim.org/release-docs/4.3/admin/Advanced-Installations/ In this case though both are still the same, it hasn't changed.

Using project directory: /mnt/data/nominatim-import-data might be wrong. Do you have a directory ~/nominatim-project or ~/nominatim-planet ?

[mtmail]

Ah, ok. That's just an unusual location because the installation documentation calls it differently. But any directory works.

During the initial import did you remove data for dynamic updates? That would remove the planet_osm_* tables. https://nominatim.org/release-docs/4.3/admin/Import/#dropping-data-required-for-dynamic-updates

[b-michal]

Yes, I used the --no-updates parameter on the initial import as I was not aware it would also prevent me from importing more data "manually" in the database.
So my only option now is to drop the whole database and re-import all the data? (It would be perfect if I could avoid dropping everything and importing 2 big continents again just to add 2 small countries to the existing data...)

[mtmail]

With--no-updates some database tables are deleted and can't be recreated easily. I will test that in the next days and come back with a better answer.

[mtmail]

That didn't work. I setup a test server with Nominatim 4.3 and imported a country with --no-updates. Of course saw the same error. Then I tried to work around it inside the Nominatim source code (hacking!). There's too many tables and database trigger functions missing in the database. If an old flatnode file is used (containing outdated OSM data data) risk of data corruption is also to big.

We will add a warning to the add-data command improve the documentation.

I'm afraid you'll have to do a full new import.

[b-michal]

Ok, too bad but we will manage.

Regarding the warning, it's a good point, adding this little precision will make it more clear on how the --no-updates parameter affects the other database manipulations.

Thank you very much for your time and your help!