Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

syntax error in migration: failed migration events_analytics_platform, 0001_spans , release 24.8 #6355

Closed
shcherbak opened this issue Sep 27, 2024 · 5 comments
Closed

syntax error in migration: failed migration events_analytics_platform, 0001_spans , release 24.8 #6355

shcherbak opened this issue Sep 27, 2024 · 5 comments

Comments

@shcherbak
Copy link
Contributor

shcherbak commented Sep 27, 2024 •
edited
Loading

Environment

release 24.8, self-hosted, clickhouse docker image clickhouse/clickhouse-server:21.8.13.6

Steps to Reproduce

  1. update sentry to 24.8
  2. run snuba migration migrate
  3. failed migration events_analytics_platform, 0001_spans

Expected Result

succeeded migration events_analytics_platform, 0001_spans

Actual Result

migrations list:

events_analytics_platform (readiness_state: limited)
[-]  0001_spans (IN PROGRESS)
[ ]  0002_spans_attributes_mv
[ ]  0003_eap_spans_project_id_index
[ ]  0004_modify_sampling_weight
[ ]  0005_remove_attribute_mv
[ ]  0006_drop_attribute_key_project_id_indexes

logs:


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/snuba", line 33, in <module>
    sys.exit(load_entry_point('snuba', 'console_scripts', 'snuba')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/snuba/snuba/cli/migrations.py", line 116, in migrate
    runner.run_all(
  File "/usr/src/snuba/snuba/migrations/runner.py", line 219, in run_all
    self._run_migration_impl(
  File "/usr/src/snuba/snuba/migrations/runner.py", line 297, in _run_migration_impl
    migration.forwards(context, dry_run, columns_states)
  File "/usr/src/snuba/snuba/migrations/migration.py", line 170, in forwards
    op.execute()
  File "/usr/src/snuba/snuba/migrations/operations.py", line 81, in execute
    connection.execute(self.format_sql(), settings=self._settings)
  File "/usr/src/snuba/snuba/clickhouse/native.py", line 283, in execute
    raise ClickhouseError(e.message, code=e.code) from e
snuba.clickhouse.errors.ClickhouseError: DB::Exception: Codec DoubleDelta does not accept any arguments. Stack trace:

0. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0x8febd9a in /usr/bin/clickhouse
1. ? @ 0xfe432b1 in /usr/bin/clickhouse
2. ? @ 0xfd919fb in /usr/bin/clickhouse
3. ? @ 0xff0cf64 in /usr/bin/clickhouse
4. DB::CompressionCodecFactory::validateCodecAndGetPreprocessedAST(std::__1::shared_ptr<DB::IAST> const&, DB::IDataType const*, bool, bool) const @ 0xfd8ecf6 in /usr/bin/clickhouse
5. DB::InterpreterCreateQuery::getColumnsDescription(DB::ASTExpressionList const&, std::__1::shared_ptr<DB::Context const>, bool) @ 0x102007f4 in /usr/bin/clickhouse
6. DB::InterpreterCreateQuery::setProperties(DB::ASTCreateQuery&) const @ 0x102019d4 in /usr/bin/clickhouse
7. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x10208164 in /usr/bin/clickhouse
8. DB::InterpreterCreateQuery::execute() @ 0x1020e79c in /usr/bin/clickhouse
9. ? @ 0x1081bf04 in /usr/bin/clickhouse
10. DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool) @ 0x1081a5a3 in /usr/bin/clickhouse
11. DB::TCPHandler::runImpl() @ 0x110c08be in /usr/bin/clickhouse
12. DB::TCPHandler::run() @ 0x110d3859 in /usr/bin/clickhouse
13. Poco::Net::TCPServerConnection::start() @ 0x13c4ca2f in /usr/bin/clickhouse
14. Poco::Net::TCPServerDispatcher::run() @ 0x13c4e4ba in /usr/bin/clickhouse
15. Poco::PooledThread::run() @ 0x13d80739 in /usr/bin/clickhouse
16. Poco::ThreadImpl::runnableEntry(void*) @ 0x13d7c9ca in /usr/bin/clickhouse
17. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
18. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
@getsantry getsantry bot moved this to Waiting for: Product Owner in GitHub Issues with 👀 3 Sep 27, 2024
@shcherbak
Copy link
Contributor Author

suppose there is syntax error in migration query:

sign Int8 CODEC (DoubleDelta(1)),

belive it is correct:


DROP table default.eap_spans_local ON CLUSTER 'sentry-clickhouse' SYNC;
CREATE TABLE IF NOT EXISTS eap_spans_local ON CLUSTER 'sentry-clickhouse' (
  organization_id UInt64,
  project_id UInt64,
  service String CODEC (ZSTD(1)),
  trace_id UUID,
  span_id UInt64,
  parent_span_id UInt64 CODEC (ZSTD(1)),
  segment_id UInt64 CODEC (ZSTD(1)),
  segment_name String CODEC (ZSTD(1)),
  is_segment UInt8 CODEC (T64, ZSTD(1)),
  _sort_timestamp DateTime CODEC (DoubleDelta, ZSTD(1)),
  start_timestamp DateTime64(6) CODEC (DoubleDelta, ZSTD(1)),
  end_timestamp DateTime64(6) CODEC (DoubleDelta, ZSTD(1)),
  duration_ms UInt32 CODEC (DoubleDelta, ZSTD(1)),
  exclusive_time_ms Float64 CODEC (ZSTD(1)),
  retention_days UInt16 CODEC (DoubleDelta, ZSTD(1)),
  name String CODEC (ZSTD(1)),
  sampling_factor Float64 CODEC (ZSTD(1)),
  sampling_weight Float64 CODEC (ZSTD(1)),
  sign Int8 CODEC (DoubleDelta, ZSTD(1)),
  attr_str_0 Map(String, String) CODEC (ZSTD(1)),
  attr_str_1 Map(String, String) CODEC (ZSTD(1)),
  attr_str_2 Map(String, String) CODEC (ZSTD(1)),
  attr_str_3 Map(String, String) CODEC (ZSTD(1)),
  attr_str_4 Map(String, String) CODEC (ZSTD(1)),
  attr_str_5 Map(String, String) CODEC (ZSTD(1)),
  attr_str_6 Map(String, String) CODEC (ZSTD(1)),
  attr_str_7 Map(String, String) CODEC (ZSTD(1)),
  attr_str_8 Map(String, String) CODEC (ZSTD(1)),
  attr_str_9 Map(String, String) CODEC (ZSTD(1)),
  attr_str_10 Map(String, String) CODEC (ZSTD(1)),
  attr_str_11 Map(String, String) CODEC (ZSTD(1)),
  attr_str_12 Map(String, String) CODEC (ZSTD(1)),
  attr_str_13 Map(String, String) CODEC (ZSTD(1)),
  attr_str_14 Map(String, String) CODEC (ZSTD(1)),
  attr_str_15 Map(String, String) CODEC (ZSTD(1)),
  attr_str_16 Map(String, String) CODEC (ZSTD(1)),
  attr_str_17 Map(String, String) CODEC (ZSTD(1)),
  attr_str_18 Map(String, String) CODEC (ZSTD(1)),
  attr_str_19 Map(String, String) CODEC (ZSTD(1)),
  attr_num_0 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_1 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_2 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_3 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_4 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_5 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_6 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_7 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_8 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_9 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_10 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_11 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_12 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_13 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_14 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_15 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_16 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_17 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_18 Map(String, Float64) CODEC (ZSTD(1)),
  attr_num_19 Map(String, Float64) CODEC (ZSTD(1))
)
ENGINE ReplicatedCollapsingMergeTree('/clickhouse/tables/events_analytics_platform/{shard}/default/eap_spans_local', '{replica}', sign)
PRIMARY KEY (organization_id, _sort_timestamp, trace_id)
ORDER BY (organization_id, _sort_timestamp, trace_id, span_id)
PARTITION BY (toMonday(_sort_timestamp)) TTL _sort_timestamp + toIntervalDay(retention_days) 
SETTINGS index_granularity=8192;

@shcherbak shcherbak changed the title failed migration events_analytics_platform, 0001_spans , release 24.8 syntax error in migration: failed migration events_analytics_platform, 0001_spans , release 24.8 Sep 27, 2024
This was referenced Sep 27, 2024
Open
Merged
@shcherbak
Copy link
Contributor Author

try to fix
#6356

phacops pushed a commit that referenced this issue Oct 3, 2024
@shcherbak
try to fix failed migration events_analytics_platform, 0001_spans (#6356
d960074 
)

fix for syntax error in migration :
Codec DoubleDelta does not accept any arguments.

#6355

<!--

  Sentry employees and contractors can delete or ignore the following.

-->

### Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated
in the State of Delaware in 2015 as Functional Software, Inc. and is
gonna need some rights from me in order to utilize my contributions in
this here PR. So here's the deal: I retain all rights, title and
interest in and to my contributions, and by keeping this boilerplate
intact I confirm that Sentry can use, modify, copy, and redistribute my
contributions, under Sentry's choice of terms.

---------

Co-authored-by: iops <[email protected]>
@onkar
Copy link
Member

onkar commented Oct 11, 2024

Thanks for the fix, @shcherbak!

@onkar onkar closed this as completed Oct 11, 2024
@shcherbak
Copy link
Contributor Author

@onkar Thanks

@lasermoth
Copy link

I've run into this issue and the fix looks like it is in 24.10.0, how ever 24.8.0 is listed as a hard stop.

Is it safe to move to 24.10.0 from 24.5.1 to get this migration fix?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
GitHub Issues with 👀 3
Archived in project
Milestone
No milestone
Development

No branches or pull requests
3 participants
@onkar @shcherbak @lasermoth