Basic Mapping Pipeline

[josephjclark]

This feature is to create a basic mapping pipeline for the Mapping service

The focus is on getting an end-to-end pipeline. Quality and UX are secondary at this stage.

The service needs to be exposed via REST on the apollo server.

Inputs

It should accept a payload like this:

{
  input: [], // array of strings to map. The first word in the string is the key
  vocabs: [], // string list of target vocabularies, by name, to map to.
            // I'm quite happy to just accept one string for the time being!
}

Implementation Notes

For each value in the input list:

• Use similarity analysis to find possible matches in the embeddings (ie, load N possible matches from the database)
• Call a model to figure out how the best mapping based on the shortlist of possibilities
• Fit the mapped value into the correct data structure

Repeat this process until all inputs are mapped.

Return a JSON structure.

Limitations

Ironically this service isn't supposed to be particularly smart. Here are some big limitations:

• You can only use vocabularies that are already defined in the vector database. It's a hard-coded, limited list
• We expect a rigid input format. The difficult problem of how to take eg a list of commcare inputs and build them into the correct data structure is completely ignored.Sorry users, you've gotta do this bit yourself.
• The output is fixed to a fhir coding { display, system, value }
• There is no validation
• There is no ambiguity handling