Skip to content

Commit

Permalink
LaplaceOrdering, fix when num_seqs multiple of bin size (#1681)
Browse files Browse the repository at this point in the history
  • Loading branch information
dorian-K authored Jan 27, 2025
1 parent b01684e commit 755ec60
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion returnn/datasets/postprocessing.py
Original file line number Diff line number Diff line change
Expand Up @@ -364,11 +364,11 @@ def __call__(self, iterator: Iterator[TensorDict], **kwargs) -> Iterator[TensorD
is_down_phase = False

seq_buffer = list(islice(iterator, self.num_seqs_per_bin))
has_ended = False
while True:
seq_buffer.sort(key=self._get_seq_len, reverse=is_down_phase)

next_seq_buffer = []
has_ended = False

# Yield items to trainer while gradually pulling more data from PP function.
# This optimizes CPU load when multiple workers are used.
Expand Down

0 comments on commit 755ec60

Please sign in to comment.