You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some of our data has dates that have been scrubbed to de-identify, and it would be really nice to be able to load them with EDF.jl even with non-compliant dates! Perhaps we could error by default and have an ignore_malformed_date=false kwarg.
This does seem to be the one part of the spec that gets ignored the most...
I'm not sure what the most useful way of dealing with these cases would be. Should the field be set to missing and malformed date string discarded or should whatever was actually in the file be preserved in some way in the object? We opt for the latter for EDF+ extensions since those are optional but the date is supposed to always be present according to the EDF spec.
I agree that complying to the spec is great, but I think also the need to open files prepared by some other tool and repair to best ability overrides that priority. In my example the original data has gone through several processing steps so it's not as much of an issue - I'm not handling patient data so I'm not needing stricter adherence for my own user story.
Whilst inserting dummy values is often not great, cause it causes silent problems further down the chain, for dates I think the unix epoch is not a bad option. If a date isn't specified in the file and technically it should be, then I think it's no worse and if making the field nullable would cause other issues in the library then it might be worth considering.
Some of our data has dates that have been scrubbed to de-identify, and it would be really nice to be able to load them with EDF.jl even with non-compliant dates! Perhaps we could error by default and have an
ignore_malformed_date=false
kwarg.cc @palday
The text was updated successfully, but these errors were encountered: