Skip to content

Commit

Permalink
Fix a bug where retrying an export RQ job may break scheduling (#8584)
Browse files Browse the repository at this point in the history
`_patched_retry` tries to schedule a copy of the current job. In
particular, it copies the dependencies of the old job using
`current_rq_job.dependency_ids`.

Unfortunately, `dependency_ids` does not return IDs of dependency jobs,
as one might expect. It actually returns the Redis _keys_ corresponding
to those jobs, as bytestrings. The RQ job creation code does not support
bytestrings as dependency specifiers, so it unintentionally treats them
as sequences, saving the individual bytes (as integers) as the
dependency job IDs. But since IDs have to be strings, the scheduler
quickly crashes when it tries to use those integer "IDs" to construct
Redis keys.

Thankfully, we don't actually need to get the dependency IDs.
`_patched_retry` is only used inside running jobs, and if a job is
running, it means that all its dependencies are already completed. Thus,
the newly scheduled job doesn't need to have any dependencies at all.
  • Loading branch information
SpecLad authored Oct 22, 2024
1 parent 5045f6a commit bcd0627
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
5 changes: 5 additions & 0 deletions changelog.d/20241022_191618_roman_fix_integer_dependencies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
### Fixed

- Fixed a bug where an export RQ job being retried may break scheduling
of new jobs
(<https://github.com/cvat-ai/cvat/pull/8584>)
1 change: 0 additions & 1 deletion cvat/apps/dataset_manager/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,6 @@ def _patched_retry(*_1, **_2):
**current_rq_job.kwargs,
job_id=current_rq_job.id,
meta=current_rq_job.meta,
depends_on=current_rq_job.dependency_ids,
job_ttl=current_rq_job.ttl,
job_result_ttl=current_rq_job.result_ttl,
job_description=current_rq_job.description,
Expand Down

0 comments on commit bcd0627

Please sign in to comment.