Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow noncompliant dates #66

Open
josephsdavid opened this issue Jan 23, 2023 · 2 comments
Open

Allow noncompliant dates #66

josephsdavid opened this issue Jan 23, 2023 · 2 comments

Comments

@josephsdavid
Copy link

Some of our data has dates that have been scrubbed to de-identify, and it would be really nice to be able to load them with EDF.jl even with non-compliant dates! Perhaps we could error by default and have an ignore_malformed_date=false kwarg.

cc @palday

@ararslan
Copy link
Member

This does seem to be the one part of the spec that gets ignored the most...

I'm not sure what the most useful way of dealing with these cases would be. Should the field be set to missing and malformed date string discarded or should whatever was actually in the file be preserved in some way in the object? We opt for the latter for EDF+ extensions since those are optional but the date is supposed to always be present according to the EDF spec.

@stellarpower
Copy link

I agree that complying to the spec is great, but I think also the need to open files prepared by some other tool and repair to best ability overrides that priority. In my example the original data has gone through several processing steps so it's not as much of an issue - I'm not handling patient data so I'm not needing stricter adherence for my own user story.

Whilst inserting dummy values is often not great, cause it causes silent problems further down the chain, for dates I think the unix epoch is not a bad option. If a date isn't specified in the file and technically it should be, then I think it's no worse and if making the field nullable would cause other issues in the library then it might be worth considering.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants