Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(job-runner): update migration status job #6339

Merged
merged 7 commits into from
Oct 3, 2024

Conversation

onewland
Copy link
Contributor

@onewland onewland commented Sep 24, 2024

Transcript of happy-path CLI usage:

➜  snuba git:(connect-admin-to-snuba-manual-jobs) ✗ docker exec sentry_clickhouse clickhouse-client --query "SELECT status,version FROM migrations_local FINAL WHERE migration_id='0020_transactions_add_codecs' FORMAT Vertical;"
Row 1:
──────
status:  not_started
version: 3

➜  snuba git:(connect-admin-to-snuba-manual-jobs) ✗ snuba jobs run --job_type=UpdateMigrationStatus --job_id=abcd1234 migration_id=0020_transactions_add_codecs old_status=not_started new_status=completed group=transactions
2024-09-24 13:19:46,659 Initializing Snuba...
2024-09-24 13:19:48,321 Snuba initialization took 1.6635732220020145s
2024-09-24 13:19:48,371 abcd1234: executing INSERT INTO migrations_local FORMAT JSONEachRow with data = [{'group': 'transactions', 'migration_id': '0020_transactions_add_codecs', 'timestamp': datetime.datetime(2024, 9, 24, 13, 19, 48, 370999), 'status': 'completed', 'version': 4}]
2024-09-24 13:19:48,394 ClickhouseResult(results=[1], meta=None, profile={'bytes': 0, 'progress_bytes': 0, 'blocks': 0, 'rows': 0, 'elapsed': 0.023443222045898438}, trace_output='')

➜  snuba git:(connect-admin-to-snuba-manual-jobs) ✗ docker exec sentry_clickhouse clickhouse-client --query "SELECT status,version FROM migrations_local FINAL WHERE migration_id='0020_transactions_add_codecs' FORMAT Vertical;"
Row 1:
──────
status:  completed
version: 4

➜  snuba git:(connect-admin-to-snuba-manual-jobs) ✗ snuba jobs run --job_type=UpdateMigrationStatus --job_id=abcd1234 migration_id=0020_transactions_add_codecs old_status=completed new_status=not_started group=transactions
2024-09-24 13:20:04,859 Initializing Snuba...
2024-09-24 13:20:06,557 Snuba initialization took 1.7010006950004026s
2024-09-24 13:20:06,606 abcd1234: executing INSERT INTO migrations_local FORMAT JSONEachRow with data = [{'group': 'transactions', 'migration_id': '0020_transactions_add_codecs', 'timestamp': datetime.datetime(2024, 9, 24, 13, 20, 6, 606748), 'status': 'not_started', 'version': 5}]
2024-09-24 13:20:06,620 ClickhouseResult(results=[1], meta=None, profile={'bytes': 0, 'progress_bytes': 0, 'blocks': 0, 'rows': 0, 'elapsed': 0.01300191879272461}, trace_output='')

➜  snuba git:(connect-admin-to-snuba-manual-jobs) ✗ docker exec sentry_clickhouse clickhouse-client --query "SELECT status,version FROM migrations_local FINAL WHERE migration_id='0020_transactions_add_codecs' FORMAT Vertical;"
Row 1:
──────
status:  not_started
version: 5

and if you supply an wrong old_status (doing some basic sanity checking):

➜  snuba git:(connect-admin-to-snuba-manual-jobs) ✗ snuba jobs run --job_type=UpdateMigrationStatus --job_id=abcd1234 migration_id=0020_transactions_add_codecs old_status=completed new_status=not_started group=transactions
2024-09-24 13:20:17,523 Initializing Snuba...
2024-09-24 13:20:19,164 Snuba initialization took 1.6427798130025622s
Traceback (most recent call last):
  File "/Users/oliver/workplace/snuba/.venv/bin/snuba", line 33, in <module>
    sys.exit(load_entry_point('snuba', 'console_scripts', 'snuba')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/oliver/workplace/snuba/.venv/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/oliver/workplace/snuba/.venv/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/oliver/workplace/snuba/.venv/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/oliver/workplace/snuba/.venv/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/oliver/workplace/snuba/.venv/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/oliver/workplace/snuba/.venv/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/oliver/workplace/snuba/snuba/cli/jobs.py", line 51, in run
    job_to_run.execute()
  File "/Users/oliver/workplace/snuba/snuba/manual_jobs/update_migration_status.py", line 57, in execute
    assert status == self._old_status, f"actual status {status} does not match expected {self._old_status}, aborting"
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: actual status not_started does not match expected completed, aborting

@onewland onewland marked this pull request as ready for review September 24, 2024 21:37
@onewland onewland requested a review from a team as a code owner September 24, 2024 21:37
@xurui-c xurui-c self-requested a review September 24, 2024 21:55
@onewland onewland merged commit 997ec1e into master Oct 3, 2024
30 checks passed
@onewland onewland deleted the job-runner/create-migration-overwrite-job branch October 3, 2024 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants