Versioning How to modify code without breaking user scripts
When a script created in one of our supported UI is saved, all the BHoM components save information about themselves so they can initialise properly when the script is re-opened. That information is simply kept in a string format (more precisely Json format) and contains details such as the component/method name, it's argument types, output types, ...
If someone modifies a method definition in the code, it will become impossible to find that method based on outdated information and the initialisation of the component will fail. Unless, of course, we provide to the system a way to update that old json before using it to find the method.
The same logic applies for saved types (e.g. types of input/output of a component) and saved objects (e.g. objects stored in a database or file).
Alongside the dlls installed in AppData\Roaming\BHoM\Assemblies, you can find in the bin sub-folder a series of BHoMUpgrader exe programs. When a type/method/object fails to deserialise from its string representation (json), those upgrader are called to the rescue.
Every quarter, when we release a new beta installer, we also produce a new upgrader named BHoMUpgrader with the version number attached at the end (e.g. BHoMUpgrader32 for version 3.2). That upgrader contains all the changes to the code that occurred during the quarter.
When deserialisation fails in the BHoM, the most recent upgrader is called. If that upgrader contains all the information it needs to return a valid version of the object, it will return an updated version of the json string. There will, however, be cases where the json string is old enough that the current installer doesn't contain a record of all the changes. There might even be cases where multiple changes on a method/class happened across multiple quarter. This means we will need to combine the information from multiple upgraders. If that is the case, the upgrader will first call the previous upgrader before trying to upgrade the json string again.
We will go in details on how the upgrade information is stored inside an upgrader in the remaining sections. There is however on aspect worth mentioning already. Once a quarter is finished, an upgrader is never modified again and simply redistributed alongside the others. During that quarter however, the current upgrader is constantly updated to reflect the new changes. For everyone working on the BHoM to have to modify the exact same files inside the Versioning_Toolkit would be inconvenient and a frequent source of clashes. For that reason, the information related to the upgraded of the current quarter are stored locally at the root of each project where the change occurred.
Notice that the file name ends with the version of the BHoM it applies to.
The content of an empty Versioning_XX.json file is as follow:
{
"Namespace": {
"ToNew": {
},
"ToOld": {
}
},
"Type": {
"ToNew": {
},
"ToOld": {
}
},
"Property": {
"ToNew": {
},
"ToOld": {
}
}
}When the UI_PostBuild process that copies all the BHoM assemblies to the Roaming folder is ran (i.e. when BHoM_UI is compiled), the information from all the Versioning_XX.json files is collected and compiled in to a single json file copied to the roaming folder next to the BHoMUpgrader executable. It's content will look similar to the local json files with an extra section for the methods (more onto that later):
{
"Namespace": {
"ToNew": {
"BH.Engine.XML": "BH.Engine.External.XML",
"BH.oM.XML": "BH.oM.External.XML"
},
"ToOld": {
"BH.Engine.External.XML": "BH.Engine.XML",
"BH.oM.External.XML": "BH.oM.XML"
}
},
"Type": {
"ToNew": {
"BH.oM.Base.IBHoMFragment": "BH.oM.Base.IFragment",
"BH.oM.Adapters.ETABS.EtabsConfig": "BH.oM.Adapters.ETABS.EtabsSettings",
},
"ToOld": {
"BH.oM.Base.IFragment": "BH.oM.Base.IBHoMFragment",
"BH.oM.Adapters.ETABS.EtabsSettings":"BH.oM.Adapters.ETABS.EtabsConfig"
}
},
"Method": {
"ToNew": {
"BH.Adapter.XML.XMLAdapter(BH.oM.Adapter.FileSettings, BH.oM.XML.Settings.XMLSettings)": {
"_t": "System.Reflection.MethodBase",
"TypeName": "{ \"_t\" : \"System.Type\", \"Name\" : \"BH.Adapter.XML.XMLAdapter, XML_Adapter, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null\" }",
"MethodName": ".ctor",
"Parameters": [ "{ \"_t\" : \"System.Type\", \"Name\" : \"BH.oM.Adapter.FileSettings\" }" ]
},
"BH.Engine.Geometry.Compute.ClipPolylines(BH.oM.Geometry.Polyline, BH.oM.Geometry.Polyline)": {
"_t": "System.Reflection.MethodBase",
"TypeName": "{ \"_t\" : \"System.Type\", \"Name\" : \"BH.Engine.Geometry.Compute, Geometry_Engine, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null\" }",
"MethodName": "BooleanIntersection",
"Parameters": [ "{ \"_t\" : \"System.Type\", \"Name\" : \"BH.oM.Geometry.Polyline\" }", "{ \"_t\" : \"System.Type\", \"Name\" : \"BH.oM.Geometry.Polyline\" }", "{ \"_t\" : \"System.Type\", \"Name\" : \"System.Double, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089\" }" ]
}
},
"ToOld": {
}
},
"Property": {
"ToNew": {
"BH.oM.Structure.Elements.Bar.StartNode": "BH.oM.Structure.Elements.Bar.Start",
"BH.oM.Structure.Elements.Bar.EndNode": "BH.oM.Structure.Elements.Bar.End"
},
"ToOld": {
"BH.oM.Structure.Elements.Bar.Start": "BH.oM.Structure.Elements.Bar.StartNode",
"BH.oM.Structure.Elements.Bar.End": "BH.oM.Structure.Elements.Bar.End",
}
}
}Let's now go into details on how to record a change on the code for the various possible aspects that can be modified.
This applies to the case where an entire namespace is renamed. This means all the elements inside that namespace will now belong to a new namespace.
To record that change, simply provide the old namespace as key and teh new namespace as value to the Namspace.ToNew section of the json file. If you want the change to be backward compatible, you can also fill the ToOld section with the mirrored information.
Example:
{
"Namespace": {
"ToNew": {
"BH.oM.XML": "BH.oM.External.XML",
},
"ToOld": {
"BH.oM.External.XML": "BH.oM.XML",
}
},
...
}
Modifying the name of a type works very much the same way. Provide the full name of the old type (namespace + type name) as key and the full name of the new type as value. If you want the change to be backward compatible, you can also fill the ToOld section with the mirrored information.
Example:
{
...
"Type": {
"ToNew": {
"BH.oM.XML.Settings.XMLSettings": "BH.oM.External.XML.Settings.GBXMLSettings",
"BH.oM.XML.Environment.DocumentBuilder": "BH.oM.External.XML.GBXML.GBXMLDocumentBuilder"
},
"ToOld": {
"BH.oM.External.XML.Settings.GBXMLSettings": "BH.oM.XML.Settings.XMLSettings",
"BH.oM.External.XML.GBXML.GBXMLDocumentBuilder": "BH.oM.XML.Environment.DocumentBuilder"
}
}
}
Technically, we could the exact same thing for methods as we have done for types and namespaces. The content to provide is a bit more complex though. See for example
"Method": {
"ToNew": {
"BH.Adapter.XML.XMLAdapter(BH.oM.Adapter.FileSettings, BH.oM.XML.Settings.XMLSettings)": {
"_t": "System.Reflection.MethodBase",
"TypeName": "{ \"_t\" : \"System.Type\", \"Name\" : \"BH.Adapter.XML.XMLAdapter, XML_Adapter, Version=3.0.0.0, Culture=neutral, PublicKeyToken=null\" }",
"MethodName": ".ctor",
"Parameters": [
"{ \"_t\" : \"System.Type\", \"Name\" : \"BH.oM.Adapter.FileSettings\" }"
]
}
},
"ToOld": {
}
}
IF you want to go that route, you can simply provide a Method section in the VersioningXX.json file and it will be picked up with the rest during the UI_PostBuild process. To create the key, you can use the VersionKey component before doing the change on your method:
If you update a constructor, just leave the methodName input empty.
The representation of the new method is simply the json string.
But that's messy and admittedly difficult to read of you need to come back to it and check what is in the upgraded methods section.
SO, instead we recommend you use a PreviousVersion attribute on the method you have modified. For example, here's what it looks like for a constructor and a regular method:
public partial class XMLAdapter : BHoMAdapter
{
[Description("Specify XML file and properties for data transfer")]
[Input("fileSettings", "Input the file settings to get the file name and directory the XML Adapter should use")]
[Input("xmlSettings", "Input the additional XML Settings the adapter should use. Only used when pushing to an XML file. Default null")]
[Output("adapter", "Adapter to XML")]
[PreviousVersion("3.2", "BH.Adapter.XML.XMLAdapter(BH.oM.Adapter.FileSettings, BH.oM.XML.Settings.XMLSettings)")]
public XMLAdapter(BH.oM.Adapter.FileSettings fileSettings = null)
{
//....
}public static partial class Create
{
[PreviousVersion("3.2", "BH.Engine.Adapters.Revit.Create.FilterFamilyTypesOfFamily(BH.oM.Base.IBHoMObject)")]
[Description("Creates an IRequest that filters Revit Family Types of input Family.")]
[Input("bHoMObject", "BHoMObject that contains ElementId of a correspondent Revit element under Revit_elementId CustomData key - usually previously pulled from Revit.")]
[Output("F", "IRequest to be used to filter Revit Family Types of a Family.")]
public static FilterTypesOfFamily FilterTypesOfFamily(IBHoMObject bHoMObject)
{
//....
}Notice that you still have to create the key using the VersionKey component but at least you don't have to deal with raw json.
So far, we have focused on upgrading items that are used to save and restore components in the UI. But what about actual objects stored in a database or a file ? Well, if only their namespace or type name was modified, the solutions above will be enough. But what if you completely redesigned that type of object and changed the Properties that define it ?
This case cannot be solved by a simple replacement of a string and will most likely require some calculations to go from the old object to the new one. This means we need a method that takes the old object in and return the new. Two things about that:
- The old object definition will not exist anymore so we cannot use that as the input of the conversion method. Instead we will use a Dictionary containing the properties for both input and output of that conversion method. The other benefit is that the upgrader will not have to depend on BHoM dlls to be able to do the conversion.
- The conversion method needs to be compile and the upgrader needs to be able to access it. While there are ways to keep the conversion method decentralised, it is way simpler to have it in the versioning toolkit directly. This means this is the only case where you cannot just write the upgrade from your own repo. Luckily, this case is less frequent than the others.
So what do you need to do to cover the upgrade then ?
- First, locate the
Converter.csfile int the project of the current upgrader. - In that file, write a conversion method with the following signature:
public static Dictionary<string, object> UpgradeOldClassName(Dictionary<string, object> old). - In the
Converterconstructor, add that method to theToNewObjectDictionary. the key is that object type full name (namespace + type name) and the value is the method. - If you want to cover backward compatibility, you can also write a
DowngradeNewClassNamemethod and add it to theToOldObjectdictionary.
Here's an example:
public class Converter : Base.Converter
{
/***************************************************/
/**** Constructors ****/
/***************************************************/
public Converter() : base()
{
PreviousVersion = "";
ToNewObject.Add("BH.oM.Versioning.OldVersion", UpgradeOldVersion);
}
/***************************************************/
/**** Private Methods ****/
/***************************************************/
public static Dictionary<string, object> UpgradeOldVersion(Dictionary<string, object> old)
{
if (old == null)
return null;
double A = 0;
if (old.ContainsKey("A"))
A = (double)old["A"];
double B = 0;
if (old.ContainsKey("B"))
B = (double)old["B"];
return new Dictionary<string, object>
{
{ "_t", "BH.oM.Versioning.NewVersion" },
{ "AplusB", A + B },
{ "AminusB", A - B }
};
}
/***************************************************/
}A few things to notice:
- You are working from a Dictionary so make sure that the properties exist before using them
- You will also need to cast them since the dictionary values are all objects
- Make sure to provide the new object type in the dictionary by defining the "_t" property.
For the case where an object type was only modified by renaming some of its property, we have a simpler solution. One very similar to what was done for namespaces and type names actually.
As a key, provide the full name of the containing type (namespace + type name) followed by the old property name. As a value as key, do the same but with the new property name. If you want the change to be backward compatible, you can also fill the ToOld section with the mirrored information.
Example:
"Property": {
"ToNew": {
"BH.oM.Structure.Elements.Bar.StartNode": "BH.oM.Structure.Elements.Bar.Start",
"BH.oM.Structure.Elements.Bar.EndNode": "BH.oM.Structure.Elements.Bar.End"
},
"ToOld": {
"BH.oM.Structure.Elements.Bar.Start": "BH.oM.Structure.Elements.Bar.StartNode",
"BH.oM.Structure.Elements.Bar.End": "BH.oM.Structure.Elements.Bar.End",
}
}