Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track whether truncated records were encountered on read #65

Open
ararslan opened this issue Nov 3, 2022 · 0 comments
Open

Track whether truncated records were encountered on read #65

ararslan opened this issue Nov 3, 2022 · 0 comments
Labels
enhancement New feature or request

Comments

@ararslan
Copy link
Member

ararslan commented Nov 3, 2022

When reading a file with one or more truncated data records, we currently emit a warning and store the corrected number of records in the header as though that's what we had been given all along. However, this is not programmatically discoverable. Knowing after the fact that the input was truncated is worthwhile when auditing incoming data and when converting to a different format (cf. beacon-biosignals/OndaEDF.jl#59).

Some ways to go about this:

  • Store the number of unread bytes somewhere, e.g. as a field in the File or FileHeader type. This would be 0 for untruncated files.
  • Add a field to the FileHeader alongside record_count to denote the original, incorrect record count that got corrected on read.

There are probably other ways too.

@ararslan ararslan added the enhancement New feature or request label Nov 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant