Info for Maintainers

This page contains useful information for maintainers, including how to clone and build the project.

You don't need to know all of this if you just want to contribute to the index!

• Prerequisites
• Project Setup
• Building the index
• Syncing with Wikidata
• Releasing
• Building nsi.guide
• Other commands
• Collecting names from the OSM planet

 

Prerequisites

• Node.js version 10 or newer
• git for your platform

 

Project Setup

Installing

• Clone this project, for example: git clone [email protected]:osmlab/name-suggestion-index.git
• cd into the project folder,
• Run npm install to install libraries

Updates

• git pull origin --rebase is a good way to keep your local copy of the code updated
• rerun npm install whenever dependencies are updated in package.json

 

Building the index

• npm run build
  • Takes a few seconds and should be run whenever the data/* or config/* files change
  • Processes custom locations under features/**/*.geojson into dist/featureCollection.json
  • Sorts dist/collected/* name lists into dist/filtered/* "keep" and "discard" name lists
  • Merges new items found in the "keep" lists into the data/* files
  • Generates ids
  • Outputs warnings to suggest updates to data/**/*.json
  • Make sure to check in code when done, with something like git add . && git commit -m 'npm run build'

 

Syncing with Wikidata

• npm run wikidata
  • Takes about 15 minutes and should be run occasionally to keep NSI in sync with Wikidata
  • Fetches related Wikidata names, descriptions, logos, then updates dist/wikidata.json
  • Updates the Wikidata pages to contain the current NSI identifiers
  • Outputs warnings to suggest fixes on Wikidata for missing social accounts, or other common errors
  • Make sure to check in code when done, with something like git add . && git commit -m 'npm run wikidata'
  • (We may try to automate more of this eventually)

 

Releasing

• npm run dist
  • Takes a few seconds and builds all the files in dist/*
  • The semantic version number of the project is updated automatically: major.minor.patch where patch is the date in yyyymmdd format
  • Rebuilds iD and JOSM presets, taginfo file, other output files
  • Should be run whenever the index is in a good state (build and wikidata sync has happened successfully)
  • Make sure to check in code when done, with something like git add . && git commit -m 'npm run dist'
  • Projects which pull NSI data from GitHub (such as https://nsi.guide/) will appear updated soon after npm run dist
  • Other downstream projects may pull from dist/* too

To publish an official release, follow the steps in the Release Checklist.

• Official releases are stable forever and available via NPM or on CDNs like JSDelivr
• Projects which pull name-suggestion-index from NPM or a CDN (such as iD) will appear updated soon after publishing
• Publishing the code to NPM requires rights to run npm publish

 

Building nsi.guide

https://nsi.guide/ is a web application written in ReactJS that lets anyone browse the index.

• npm run appbuild
  • Rebuilds the ReactJS code for https://nsi.guide/
  • The source code for this app can be found under app/*
  • Only need to rebuild this when the app code changes, not when the index changes

 

Other commands

• npm run lint - Checks the Javascript code for correctness
• npm run test - Runs tests agains the Javascript code
• npm run - Lists other available commands

 

Collecting names from the OSM planet

This takes a long time and a lot of disk space. It can be done occasionally by project maintainers.

• Install osmium command-line tool and node package (may only be available on some environments)
  • apt-get install osmium-tool or brew install osmium-tool or similar
  • npm install --no-save osmium
• Download the planet
  • curl -L -o planet-latest.osm.pbf https://planet.openstreetmap.org/pbf/planet-latest.osm.pbf
• Prefilter the planet file to only include named items with keys we are looking for:
  • osmium tags-filter planet-latest.osm.pbf -R name,brand,operator,network -o filtered.osm.pbf
• Run node scripts/collect_all.js /path/to/filtered.osm.pbf
  • results will go in dist/collected/*.json
• A new challenge:
  • Attempt an npm run build. Now that unique id properties are generated, it is possible that this command will fail.
  • This can happen if there are multiple new items that end up with the same id (e.g. "MetroBus" vs "Metrobus")
  • You'll need to just pick one to keep, then keep trying to run npm run build until the duplicate id issues are gone.
  • git add . && git commit -m 'Collected common names from latest planet'