Generating many mappings with python and json

[AlbertoV5]

Generating many mappings with python and json

Just adding this here for posterity, I haven't tried the new LUA mappings but for now, this is how I've been generating hundreds of mappings for TouchOSC.

Step 1. Creating a reference mapping in Realearn so source and target are configured as you want all the mappings to be set:

In this particular case, I am creating aliases for my FX Parameters and letting Realearn find the "Channel" alias FX and then the "LPF", "HPF"... etc aliases for FX Parameters. These are all the aliases I would use for whatever channelstrip I load:

paramNames = [
    "LPF", "1QF", "2QF", "3QF", "4QF", 
    "HPF", "1FQ", "2FQ", "3FQ", "4FQ", 
    "INP", "1DB", "2DB", "3DB", "4DB",
    "CTR", "CRA", "CAT", "CRE", "CMO", 
    "GTR", "GRN", "GAT", "GRE", "GMO", 
    "OUT", "MIX", "PRE", "DRI", "BYP"]

I don't want to do it all by hand so:

Step 2. Copy the reference mapping and save it as .json:

Do it for as many variants you want to have, so if you are sending a float value, then a string formatted as {{target.text_value}} and another string formatted as {{target.fx_parameter.name}} you would need 3 different references because the there would be too many changes to the JSON file and it's easier to set it once on Realearn and then not touching the JSON much on the script.

You could do it without a reference but that's just to much JSON shenanigans so it's easier to start from a reference mapping.

Step 3. This is how the mapping would look, this example has the formatted string {{target.fx_parameter.name}}:

{
  "kind": "Mapping",
  "value": {
    "id": "3Ta3MALJD322sJqkulzdI",
    "name": "name",
    "groupId": "BxdtiVP5sEuRAC2_hgMHD",
    "source": {
      "category": "osc",
      "isRegistered": false,
      "is14Bit": false,
      "oscAddressPattern": "/channel/track/1/fxparam/1/name",
      "oscArgIndex": 0,
      "oscArgType": "string"
    },
    "mode": {
      "maxStepSize": 0.05,
      "eelFeedbackTransformation": "{{target.fx_parameter.name}}",
      "feedbackColor": null,
      "feedbackType": 1
    },
    "target": {
      "trackExpression": "selected_track_index + (p[9] * 1000)",
      "fxAnchor": "name",
      "fxName": "Channel",
      "paramType": "name",
      "paramName": "LP",
      "useProject": true,
      "moveView": true,
      "seekPlay": true,
      "oscArgIndex": 0,
      "pollForFeedback": false
    }
  }
}

Then you save all the references as something like "value.json", "name.json" etc, then config and run the following python script:

CODE HERE:

  from pathlib import Path
  from uuid import uuid4
  import json
  
  
  class Mappings:
      """Make sure to change the mapping attributes and the internal method's json keys to fit your needs!
  
      Attributes
      ----------
      directory : Defaults to current working directory
      trackExpression : This is the dynamic track target to work with, leave {0} where the track index would be
      numberOfTracks : This is the number of times the same expression will repeat for different track indexes
      referenceJsonFiles : This is the original mapping reference you created in Realearn first
      targetFxName : Creating mappings for one plugin at a time
      paramNames : These are all the parameters you want to map, which repeat across tracks and references
      oscControl : {0} is track number, {1} is fxName, {2} is reference mapping / jsonFile
  
      Methods
      ----------
      modifyJson : change this method's json keys to fit your needs!
      """
  
      directory = Path.cwd()
      trackExpression = "selected_track_indexes[{0}]"
      numberOfTracks = 2
      
      referenceJsonFiles = ["value", "name", "str"]
      targetFxName = "Channel"
      paramNames = [
      "LPF", "1QF", "2QF", "3QF", "4QF", 
      "HPF", "1FQ", "2FQ", "3FQ", "4FQ", 
      "INP", "1DB", "2DB", "3DB", "4DB",
      "CTR", "CRA", "CAT", "CRE", "CMO", 
      "GTR", "GRN", "GAT", "GRE", "GMO", 
      "OUT", "MIX", "PRE", "DRI", "BYP"]
      oscControl = "/channel/track/{0}/fxparam/{1}/{2}"
  
  
      def modifyJson(self, jsonOutput, filePath, ref, trackNum, expression):
          control = ref
          for i in range(len(self.paramNames)):
  
              with open(filePath, "r") as file:
                  jsm = json.loads(file.read())["value"]
  
              jsm["id"] = str(uuid4())
              jsm["name"] = self.paramNames[i] + " {0}".format(control)
              jsm["source"]["oscAddressPattern"] = self.oscControl.format(str(trackNum), str(i+1), control)
  
              jsm["target"]["trackExpression"] = expression
              jsm["target"]["fxName"] = self.targetFxName
              jsm["target"]["paramName"] = self.paramNames[i]
  
              jsonOutput["value"].append(jsm)
  
          return jsonOutput
  
  
      def generate(self):
          """ construct a json output that contains ALL the mappings
          """
          jsonOutput = {"kind": "Mappings", "value":[]}   
          
          for i in range(self.numberOfTracks):
  
              for ref in self.referenceJsonFiles:
                  path = self.directory / (ref+".json")
                  jsonOutput = self.modifyJson(jsonOutput, path, ref, str(i+1), self.trackExpression.format(str(i)))
  
           with open (self.directory / "output.json", "w") as file:
               file.write(json.dumps(jsonOutput, indent=2))
  
  
  
  mappings = Mappings()
  
  mappings.directory = Path.cwd() / "scripts" / "realearn-python"
  
  mappings.generate()

What you want to change is the Attributes of your "mappings" instance as:

mappings = Mappings()
mappings.paramNames = ["Threshold", "Ratio", "Attack", "Release"]
mappings.targetFxName = "VCS-1"
mappings.generate()

If you want to change how the JSON is addressed, just change the keys at modifyJson():

jsm["target"]["fxName"] = self.targetFxName

You can customize whatever target you want even after having created the reference mappings. As I said, I am using aliases for my FX and FX Param but you could use the real param names or whatever is your pleasure.

Step 4. The result will be in "output.json" and you can copy the contents of the file and then paste it into a new group in Realearn:

You can tweak and re-generate the output.json and replace all mappings in the group if you need to.

Then the TouchOSC side of things is another story but it works great:

Hope this made sense, haven't seen much of these shenanigans around.

This really works great with TouchOSC because the mappings are "infinite" so no matter what you want to control you can create hundreds of mappings and Realearn can handle it all really well. Make sure to enable/disble groups through Realearn parameters so you don't have thousands of mappings active at a time, but rather they get enabled as you page through your controller.

If y'all got any other ideas or ways of automating more of this stuff feel free to share them!

Thanks. Realearn is cool.

[helgoboss]

Cool stuff, thanks for this writeup.

BTW, even if you prefer Python over Lua, you might benefit from the new data format that came up with the introduction of ReaLearn Script (Lua). This new data format is just a plain data structure really, there's nothing Lua-specific in there! So you could easily use Python to generate a JSON that conforms to that new data format, ReaLearn will accept it. ReaLearn's integrated Lua import does more or less the same.

Here are some advantages of using the new format (directly written in JSON or generated by Python, Python, JavaScript or any language really):

• Much more self-explaining and consistent than ReaLearn's old format (the current native preset format). Reason: The old format is - well - very old. And it needs to be backward compatible until ReaLearn 1.0.
• The new format has an auto-generated JSON Schema definition.
• I will eventually document it.
• At some point, the new format might becomes ReaLearn's preferred native preset format.

The advantage of specifically using Lua to generate data in this new format (optional, on top of that):

• It executes directly within ReaLearn, so you don't need anything else to generate the presets.
• At some point, ReaLearn will probably be able to save these Lua preset snippets, so they get "first class citizens".
• If I can convince Justin to expose ReaScript's Lua context as part of the REAPER SDK, one could get access to ReaScript functions when building presets (I can add partial support for this already now but I prefer not to because I don't want to recreate ReaScript).

[AlbertoV5]

Awesome! I'll slowly get into it. Having access to Reascript from Realearn would be legendary, even if it takes a while.

[vonglan]

I can confirm that generation with Lua is very easy, as is the language itself.

[tompad2]

Please could anyone make a Youtube tutorial or a written tutorial for a noob explaining how to make
presets this way! I am not good at python/json, a bit better at lua but no pro.
Everytime I hear people talking about making presets this way I wished I understood how to do this.
I have looked at exported code from ReaLearn but I dont get it.

[tompad2]

:-) Ok, here comes a picture.

The problem is, after importing from clipboard the lua-script I got from you earlier, all mappings
is Multi. Its ok for the first 18, but there is 32 buttons on controller (XTmini) that I want to map also.

I thought I had to select f.ex. Multi 19 mapping and click Learn Source to make it a Button 19 and do so
for every mapping I want to be Button, but you say that's definitely wrong ;-)

I cant see how I can do other way.

[helgoboss]

You mean like this!?

local mappings = {}

for i = 0, 8 do
    local mapping = {
        source = {
            kind = "Virtual",
            character = "Multi",
            id = i,
        },
        target = {
            kind = "FxParameterValue",
            parameter = {
                address = "ByIndex",
                fx = {
                    address = "ByIndex",
                    chain = {
                        address = "Track",
                        track = {
                            address = "ByIndex",
                            index = 0,
                        },
                    },
                    index = 0,
                },
                index = i,
            },
        },
    }
    table.insert(mappings, mapping)
end

for i = 0, 8 do
    local mapping = {
        source = {
            kind = "Virtual",
            character = "Button",
            id = i,
        },
        target = {
            kind = "FxParameterValue",
            parameter = {
                address = "ByIndex",
                fx = {
                    address = "ByIndex",
                    chain = {
                        address = "Track",
                        track = {
                            address = "ByIndex",
                            index = 0,
                        },
                    },
                    index = 0,
                },
                index = i,
            },
        },
    }
    table.insert(mappings, mapping)
end

return {
    kind = "MainCompartment",
    value = {
        mappings = mappings,
    },
}

[tompad2]

Hahahaa! Yes sort of :-)

Now I get 8 Rotarys and 8 RotaryPush and thats great!
But how do I get 8 ordinarie buttons? Do I need to make
another For-loop?

Aaahaa - is this right? I make another For-loop from 9 to 17
to get the next ordinarie buttons!?

[tompad2]

Here we go!!

I think I got it - definitely?? ;-)

local mappings = {}

for i = 0, 16 do
    local mapping = {
        source = {
            kind = "Virtual",
            character = "Multi",
            id = i,
        },
        target = {
            kind = "FxParameterValue",
            parameter = {
                address = "ByIndex",
                fx = {
                    address = "ByIndex",
                    chain = {
                        address = "Track",
                        track = {
                            address = "ByIndex",
                            index = 0,
                        },
                    },
                    index = 0,
                },
                index = i,
            },
        },
    }
    table.insert(mappings, mapping)
end

for i = 0, 17 do
    local mapping = {
        source = {
            kind = "Virtual",
            character = "Button",
            id = i,
        },
        target = {
            kind = "FxParameterValue",
            parameter = {
                address = "ByIndex",
                fx = {
                    address = "ByIndex",
                    chain = {
                        address = "Track",
                        track = {
                            address = "ByIndex",
                            index = 0,
                        },
                    },
                    index = 0,
                },
                index = i,
            },
        },
    }
    table.insert(mappings, mapping)
end

for i = 18, 47 do
    local mapping = {
        source = {
            kind = "Virtual",
            character = "Button",
            id = i,
        },
        target = {
            kind = "FxParameterValue",
            parameter = {
                address = "ByIndex",
                fx = {
                    address = "ByIndex",
                    chain = {
                        address = "Track",
                        track = {
                            address = "ByIndex",
                            index = 0,
                        },
                    },
                    index = 0,
                },
                index = i,
            },
        },
    }
    table.insert(mappings, mapping)
end

return {
    kind = "MainCompartment",
    value = {
        mappings = mappings,
    },
}

[helgoboss]

Not sure if you got it. But if this yields the main compartment mappings that you were looking for, then you did at least something right ;)

They key points to understand ReaLearn's Lua support is:

• It's super simple, much simpler than ReaScript.
• All you need to do is to return a table of mappings (embedded into a parent table as shown in the return statement`). How you come up with those mappings is completely up to you.

In above example you are starting with an empty table of mappings. Then you gradually add mappings, at first 17, then 18 and so on. Using for loops.