Fix calculation of dataset size

[Infernaught]

We previously calculated a dataset's size from the length of whatever the "dataset" parameter was that was passed in. However, this parameter could have been a filepath, meaning that the dataset.size parameter would be set incorrectly, potentially causing index out of bound errors if the actual length of the dataset was shorter than the length of the filepath string.

This PR fixes this problem by calculating the size after the filepath has been processed into a dataset.

[github-actions]

Unit Test Results

  6 files  ±       0    6 suites  ±0   20m 4s ⏱️ - 37m 28s
12 tests  - 2 785    9 ✔️  - 2 775    3 💤  -   9  0 ❌  - 1 
60 runs   - 5 546  42 ✔️  - 5 532  18 💤  - 12  0 ❌  - 2 

Results for commit 797c003. ± Comparison against base commit 92d0e0c.

♻️ This comment has been updated with latest results.

[arnavgarg1]

Really nice work figuring this one out @Infernaught! If it doesn't balloon the scope, can we add some tests to this PR (or even as a fast follow) to make sure we catch these kinds of errors in the future incase we make changes to the batcher? Some ideas: Check that the number of rows being returned by the batcher is the same as what's in the dataset when using a Pandas DataFrame, Dask DataFrame, and a file path as the dataset.

[arnavgarg1]

You may need to get rid of the print statement!

[justinxzhao]

Really nice work figuring this one out @Infernaught! If it doesn't balloon the scope, can we add some tests to this PR (or even as a fast follow) to make sure we catch these kinds of errors in the future incase we make changes to the batcher? Some ideas: Check that the number of rows being returned by the batcher is the same as what's in the dataset when using a Pandas DataFrame, Dask DataFrame, and a file path as the dataset.

+1. Great find. Another test we should add is one that verifies that one pass through the batcher up to last_batch() actually uses each individual example exactly once.

[justinxzhao]

Thanks for adding the test!