diff --git a/docs/migrating_to_16.md b/docs/migrating_to_16.md index 76fb1a82..4f633a38 100644 --- a/docs/migrating_to_16.md +++ b/docs/migrating_to_16.md @@ -11,16 +11,52 @@ Key points: * If needed, the default service specific volume directories defined in [bento.env](../etc/bento.env) can be overriden in your `local.env`. * The WES schema has been updated, so the run database must be cleared. +* The reference service has been updated, so an SQL migration must be run. -## 1. Pre-update data config +## 1. Run reference service pre-migration step + +```bash +./bentoctl.bash shell reference-db +PGPASSWORD="${POSTGRES_PASSWORD}" +psql --user "${POSTGRES_USER}" reference +``` + +Then, begin a transaction: + +```sql +BEGIN TRANSACTION; +``` + +Then, paste the contents of +[migrate_v0_2.sql](https://github.com/bento-platform/bento_reference_service/blob/main/bento_reference_service/sql/migrate_v0_2.sql) +from the reference service repository into the SQL command line. + +Next, commit the results of the transaction: + +```sql +COMMIT; +``` + +Finally, exit the shell (via successive `^D`). + + +## 2. Stop Bento + +```bash +./bentoctl.bash stop +``` + + +## 3. Pre-update data config Before updating, perform the following steps: * Shut down Bento with `./bentoctl.bash stop` * In `local.env`, replace `BENTOV2_ROOT_DATA_DIR` with `BENTO_FAST_DATA_DIR` and `BENTO_SLOW_DATA_DIR` -To minimize side effects in local environments, we recommend that you use the same directory as before for both new variables. +To minimize side effects in local environments, we recommend that you use the same directory as before for both new +variables. ```bash # local.env @@ -33,28 +69,30 @@ BENTO_FAST_DATA_DIR=some_dir BENTO_SLOW_DATA_DIR=some_dir ``` -*IMPORTANT:* If you decide to split the data in distinct directories, make sure to migrate the data accordingly. +*IMPORTANT:* If you decide to split the data in distinct directories, **make sure to migrate the data accordingly**. -## 2. Update images +## 4. Update images ```bash ./bentoctl.bash pull ``` -## 3. Create new Docker volume directories - -This step is optional if you DIDN'T split the data directories in step 1. +## 5. Create new Docker volume directories ```bash -./bentoctl.bash init-dirs # new volume directory for DRS temporary files - -# migrate the old data +./bentoctl.bash init-dirs # new volume directory for DRS and Reference temporary files ``` -## 4. Restart Bento +## 6. Migrate data from old `BENTOV2_ROOT_DATA_DIR` location to `BENTO_FAST_DATA_DIR`/`BENTO_SLOW_DATA_DIR` + +This is only needed if the data was split into distinct directories in part 1. All service data should be copied into +the corresponding locations as specified in `etc/bento.env`. + + +## 7. Restart Bento ```bash ./bentoctl.bash start diff --git a/etc/bento.env b/etc/bento.env index b15606d7..b0f256bd 100644 --- a/etc/bento.env +++ b/etc/bento.env @@ -179,7 +179,7 @@ BENTOV2_EVENT_RELAY_CPUS=1 # Reference # - Service BENTO_REFERENCE_IMAGE=ghcr.io/bento-platform/bento_reference_service -BENTO_REFERENCE_VERSION=0.1.2 +BENTO_REFERENCE_VERSION=0.2.0 BENTO_REFERENCE_VERSION_DEV=${BENTO_REFERENCE_VERSION}-dev BENTO_REFERENCE_CONTAINER_NAME=${BENTOV2_PREFIX}-reference BENTO_REFERENCE_NETWORK=${BENTOV2_PREFIX}-reference-net @@ -187,6 +187,7 @@ BENTO_REFERENCE_INTERNAL_PORT=${BENTO_STD_SERVICE_INTERNAL_PORT} BENTO_REFERENCE_EXTERNAL_PORT=9510 BENTO_REFERENCE_DEBUGGER_INTERNAL_PORT=9511 BENTO_REFERENCE_DEBUGGER_EXTERNAL_PORT=9511 +BENTO_REFERENCE_TMP_VOL_DIR=${BENTO_FAST_DATA_DIR}/reference/tmp # - Database BENTO_REFERENCE_DB_IMAGE=postgres BENTO_REFERENCE_DB_VERSION=16 diff --git a/lib/reference/docker-compose.reference.yaml b/lib/reference/docker-compose.reference.yaml index 5a73def0..3242d78f 100644 --- a/lib/reference/docker-compose.reference.yaml +++ b/lib/reference/docker-compose.reference.yaml @@ -15,6 +15,9 @@ services: - SERVICE_URL_BASE_PATH=${BENTOV2_PUBLIC_URL}/api/reference - CORS_ORIGINS=${BENTO_CORS_ORIGINS} - BENTO_AUTHZ_SERVICE_URL + - FILE_INGEST_TMP_DIR=/reference/tmp + volumes: + - ${BENTO_REFERENCE_TMP_VOL_DIR}:/reference/tmp healthcheck: test: [ "CMD", "wget", "-q0-", "http://localhost:${BENTO_REFERENCE_INTERNAL_PORT}/service-info", "&&", "echo" ] timeout: ${BENTOV2_HEALTHCHECK_TIMEOUT} diff --git a/py_bentoctl/other_helpers.py b/py_bentoctl/other_helpers.py index addb76e7..40dbf6ac 100644 --- a/py_bentoctl/other_helpers.py +++ b/py_bentoctl/other_helpers.py @@ -212,6 +212,7 @@ def init_dirs(): "katsu-db": "BENTOV2_KATSU_DB_PROD_VOL_DIR" if c.DEV_MODE else "BENTOV2_KATSU_DB_DEV_VOL_DIR", "notification": "BENTOV2_NOTIFICATION_VOL_DIR", "redis": "BENTOV2_REDIS_VOL_DIR", + "reference": "BENTO_REFERENCE_TMP_VOL_DIR", "reference-db": "BENTO_REFERENCE_DB_VOL_DIR", "wes": "BENTOV2_WES_VOL_DIR", "wes-tmp": "BENTOV2_WES_VOL_TMP_DIR",