dbt-maxcompute v1.8.0-a13

New Features

1. Support for Creating Ordinary Partition Tables via partition_py

Users can now utilize partition_py to create ordinary partition tables for better data management and querying.

2. Support for Creating Materialized Views

Support for materialized views has been introduced, enabling users to query and manage data more efficiently.

Usage

Creating an Ordinary Partition Table

To create an ordinary partition table, you can use the following configuration:

{{ config(
    materialized='table',
    partition_by={"fields": "name,some_date", "data_types": "string,string"}
) }}
select id, name, some_date from {{ source('raw', 'seed') }}

Schema of the materialized table like:

create table model(id bigint) partitioned by(name string, some_date string)

Creating an Auto-Partitioned Table

Here is an example configuration for creating an auto-partitioned table:

{{ config(
  materialized='table',
  partition_by={"fields": "some_date", "data_types": "timestamp", "granularity": "day"}
) }}
select id, name, some_date from {{ source('raw', 'seed') }}

Schema of the materialized table like:

create table model(id bigint, name string, some_date timestamp) auto partitioned by trunc_time(some_date, "day");

Creating a Materialized View

The configuration for creating a materialized view is as follows:

{{ config(
    materialized='materialized_view',
    lifecycle=1,
    build_deferred=True,
    columns=["id", "name", "some_date"],
    column_comment={"id": "this is id.", "name": "this is name."},
    disable_rewrite=True,
    table_comment="this is a materialized view.",
    tblProperties={"compressionstrategy":"normal"}
) }}
select * from {{ source('raw', 'seed') }}

For the meanings of each field, please refer to the Alibaba Cloud MaxCompute User Guide.

Limitations

1. The current version does not support specifying partitions when refreshing materialized views.
2. Materialized views do not support the rename operation.
3. It is not possible to switch to another materialization method without dropping the materialized table.