Backwards compatible connection table comparisons between different versions

[philipstarkey]

Original report (archived issue) by Philip Starkey (Bitbucket: pstarkey, GitHub: philipstarkey).

---

As discussed in labscript_devices pull request 27 and labscript issue 45 (labscript-suite/labscript#45) we would like a way to ensure that new connection_table_properties in devices does not invalidate connection tables if possible.

I think that this may be best solved by a new class in each labscript_devices device file that can be invoked by connection table comparison code in order to determine whether a connection_table_properties dictionary is identical if it was compiled with a different version of labscript_devices.

This would also require storing the labscript suite versions used when producing the connection table as attributes to the connection table.

It also requires development of some sort of syntax (maybe a dictionary structure of some kind?) that allows us to keep track of changes between different versions of labscript_devices and informs how comparisons should be made. This will probably have to evolve with time as we come across new use cases, but I don't see a problem with that. It might sound complicated, but I think it's doable (as similar/tangential example, the Django project has an automated tool that keeps track of database schema changes and can roll database schemas forward/backward in time which sounds way more complicated than this!)

[philipstarkey]

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

---

Since all properties get saved regardless of whether the user set them (they are keyword arguments that always have defaults), whenever a new property is introduced, it will definitely be present in all compilations.

Can connections.py simply declare a connection table valid if it has additional properties not present in the lab connection table? Or you want a way to declare that some of them should invalidate the connection table, but some shouldn't, on a case by case basis?

[philipstarkey]

Original comment by Philip Starkey (Bitbucket: pstarkey, GitHub: philipstarkey).

---

Ignoring additional properties only works if you expect one of the connection tables to always be newer or equal in version to the other. However we have the possibility that:

• newly compiled shots use the new labscript_devices but the BLACS connection table still uses the old one
• historical HDF5 files will have used an older version of labscript_devices to make the connection table where as the BLACS connection table uses the newer one.

Enforcing that BLACS recompile the connection table, when labscript_devices is updated, doesn't work because BLACS and runmanager/labscript may be running on different PCs with different versions of labscript_devices. I guess if we refused to run shots compiled with a new version of labscript_devices (compared to BLACS) and forced BLACS to ensure the connection table was compiled with the current version of labscript_devices this would break the above symmetry....no that only works for added properties. It doesn't work if you remove or rename a property. I guess such an instance would be a major version bump and be discouraged though? (And we could refuse all connection table matches for differing major versions)