HARMONY-1499: Added ability to use conditional step in services.yml based on native file format info in umm-c.

[ygliuvt]

Jira Issue ID

HARMONY-1499

Description

Added ability to use conditional step in services.yml based on native file format info in umm-c. Here is the proposed conditional step that is requested by @owenlittlejohns :

- name: sds/HyBIG
    description: |
      The Harmony Browse Image Generator (HyBIG) supports the conversion of
      GeoTIFF inputs to Global Imagery Browse Services (GIBS) compatible PNG or
      JPEG outputs. This includes, where necessary, conversion to a GIBS
      supported Coordinate Reference System (CRS). Other projections can be
      requested, but will not be compatible with GIBS.
    data_operation_version: '0.17.0'
    type:
      <<: *default-turbo-config
      params:
        <<: *default-turbo-params
        env:
          <<: *default-turbo-env
          STAGING_PATH: public/sds/hybig
    umm_s: S1257776354-EEDTEST
    capabilities:
      subsetting:
        bbox: false
        shape: false
        temporal: false
      output_formats:
        - image/png
        - image/jpeg
      reprojection: true
    steps:
      - image: !Env ${QUERY_CMR_IMAGE}
      - image: !Env ${NET_2_COG_IMAGE}
        conditional:
          umm_c:
            native_format: ['netcdf-4']
      - image: !Env ${HYBIG_IMAGE}

The umm_c conditional with native_format compares with the value of the collection UMM-C field: ArchiveAndDistributionInformationType.FileArchiveInformation.Format when the sibling FormatType = 'Native'.

Since the NetCDF to COG service is still in development, we can't test the desired configuration in Harmony. But tests have been run with stubbed setup to prove that once the new service is available, the setup will work.

Local Test Steps

Run Harmony locally and verify a regular harmony request works.
Manually set up a service chain with the new conditional umm_c native_format step (see description for an example) in services.yml.
For example, I set up this configuration with a conditional bad step for harmony/service-example service:

- name: harmony/service-example
    description: |
      Reference service that can be used to perform most operations supported by Harmony.
      Useful for testing new API features end-to-end on example data but not meant for production use.
    data_operation_version: '0.17.0'
    type:
      <<: *default-turbo-config
      params:
        <<: *default-turbo-params
        env:
          <<: *default-turbo-env
          STAGING_PATH: public/harmony/service-example
    umm_s: S1257851197-EEDTEST
    collections:
      - id: C1243747507-EEDTEST
        granule_limit: 100 # added to test collection granule limits for HARMONY-795
    capabilities:
      subsetting:
        bbox: true
        variable: true
        multiple_variable: true
      output_formats:
        - image/tiff
        - image/png
        - image/gif
      reprojection: true
    steps:
      - image: !Env ${QUERY_CMR_IMAGE}
      - image: !Env ${HARMONY_NETCDF_TO_ZARR_IMAGE}
        conditional:
          umm_c:
            native_format: ['netcdf-4']
      - image: !Env ${HARMONY_SERVICE_EXAMPLE_IMAGE}

Verify the conditional step is not scheduled when the collection's native-format is not netcdf-4, the request has two workflow steps and the request runs successfully.
Verify the conditional step is scheduled when the collection's native-format is netcdf-4, the request has three workflow steps and the request fails on the bad netcdf to zarr step. (I replaced line 16 in cmr-umm-collection-reader.ts with req.operation.ummcollections = await getUmmCollectionsByIds(['C1239324659-POCUMULUS'], req.accessToken); to fake the collection native_format to netcdf-4 for the test.)

You don't have to follow exactly what I did to create the fake condition to test this PR. You could add the umm_c native_format conditional step to a real service chain and with real collection if you want.

PR Acceptance Checklist

• Acceptance criteria met
• Tests added/updated (if needed) and passing
• Documentation updated (if needed)

[chris-durbin]

Design and implementation looks good - only a couple of minor changes requested.