ADDING_NEURAL_NETWORK.md

How to add a neural network

Adding a new neural network is not hard from the architectural point of view, but you need to take into account that every new neural network has another output format, and your api-point should convert this responce to base64 segments which are readable by the frontend. This is the guide on how to add a new neural network:

1. Add Neural Network to the Database by calling the request like:

   INSERT INTO Neural_Network (nn_name)
   VALUES ("jonathandinu/face-parsing");
   

   with only adding the name. You can use the request from TablePlus 'Fill with data/Insert Neural Network'. Remember the id.

2. Create a new file in server/apiEndpoints with the name of the network

3. Write and export the async function processImageWithModel(imageUrl) that takes the Image URL as a parameter and returns the array of segments, each segment is an object and has the structure:

   {
    label: the name of the segment, f. e. hat or background,
    color: hexadecimal color of the segment, please take into account that segments are drown above the picture, so it should be half-transparent,
    score: the score of the label, is not mandatory
    base64: the base64 data of the segment, the data should only contain the segment without the original image
   }
   

   Use jonathandinu/face-parsing for reference.

4. Register it's id with the js-Endpoint in callModelProcessingFactory by calling the written function. Follow the structure:

   const [model variable] = require('[path to the model]');
   ...
   async function callProcessImageWithModel(neuralNetworkId, imageUrl) {
    ...
    else if(neuralNetworkId == [your id from the database from step 1]) {
     return await [model variable] (imageUrl);
    }
   }
   

5. Do not add any other logic like saving segments or returning an api endpoint as it is handeled centrally for all neural networks in apiEndpoints/processImage.js

6. Register the new neural network in the factory by going to apiEndpoints/callModelProcessingFactory.js by adding the futher if-statement with the new neural network id and reference to the newly create function.