Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add iceberg sync column types support #304

Merged

Conversation

sashakcc
Copy link
Contributor

@sashakcc sashakcc commented May 15, 2023

Description

Add support of sync column type for iceberg tables using general method of altering column type with steps:

  1. Add a new temporary column with new type
  2. Copy data from column with old type
  3. Drop the existing column
  4. Rename the new column to existing column

Model example

{{
   config(
        materialized='incremental',
        incremental_strategy='append',
        on_schema_change='sync_all_columns',
        table_type='iceberg',
   )
}}

select cast(150.5 as decimal(26,15)) as amount
     , cast('10.1' as varchar) as user_id
     , cast('14.123456789' as double) as amount_double
     , cast('2023-05-01' as date) as reg_date
     , cast('2023-05-01 13:45:00.123456' as timestamp(6)) as ts
;

Checklist

  • You followed contributing section
  • You kept your Pull Request small and focused on a single feature or bug fix.
  • You added unit testing when necessary
  • You added functional testing when necessary

@sashakcc sashakcc marked this pull request as ready for review May 15, 2023 17:58
@nicor88 nicor88 added the enable-functional-tests Label to trigger functional testing label May 15, 2023
…ature/add_iceberg_sync_column_types_support
@Jrmyy Jrmyy modified the milestones: v1.5.1, v1.5.2 Jun 9, 2023
Oleksandr Kot added 2 commits June 14, 2023 20:59
@sashakcc sashakcc force-pushed the feature/add_iceberg_sync_column_types_support branch from bcc23be to 54a526e Compare June 21, 2023 11:40
@sashakcc sashakcc force-pushed the feature/add_iceberg_sync_column_types_support branch from cdfb9f7 to 4a4f98d Compare June 22, 2023 09:29
@svdimchenko
Copy link
Contributor

svdimchenko commented Jun 22, 2023

@sashakcc the code looks good to me, great job done 💪 @Jrmyy pls take a look at this PR
is it possible to cover these changes with functional tests ?

@Jrmyy Jrmyy merged commit 3c1c6c1 into dbt-labs:main Aug 4, 2023
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enable-functional-tests Label to trigger functional testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants