Skip to content

Commit

Permalink
[Enhancement](delete) Modify some p2 delete cases' property (#37689)
Browse files Browse the repository at this point in the history
In #35917 and #37151, we changed MOW table default delete command from
delete predicate to delete sign. It makes sure the correctness during
partial update but leads to slowdowns. Actually, if there is no partial
update, delete predicate will not lead to data fault. Delete data by
delete predicate or delete sign can be controlled by a table property
"enable_light_delete". If "enable_light_delete=true", we execute delete
command by delete predicate. Otherwise, we execute delete command by
delete sign.

In p2 cases, there are lots of cases with large data need to delete and
do not perform partial column update operations. Therefore, in order to
make it faster, we change some cases default create table clause.
  • Loading branch information
Yukang-Lian committed Oct 12, 2024
1 parent 9a9eaa7 commit 3060673
Show file tree
Hide file tree
Showing 95 changed files with 95 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ CREATE TABLE IF NOT EXISTS `customer` (
UNIQUE KEY (`c_custkey`)
DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ CREATE TABLE IF NOT EXISTS `customer` (
UNIQUE KEY (`c_custkey`)
DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"compression"="zstd",
"replication_num" = "1"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ CREATE TABLE IF NOT EXISTS `date` (
UNIQUE KEY (`d_datekey`)
DISTRIBUTED BY HASH(`d_datekey`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ CREATE TABLE IF NOT EXISTS `date` (
UNIQUE KEY (`d_datekey`)
DISTRIBUTED BY HASH(`d_datekey`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"compression"="zstd",
"replication_num" = "1"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ PARTITION p1997 VALUES [("19970101"), ("19980101")),
PARTITION p1998 VALUES [("19980101"), ("19990101")))
DISTRIBUTED BY HASH(`lo_orderkey`) BUCKETS 48
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ PARTITION p1997 VALUES [("19970101"), ("19980101")),
PARTITION p1998 VALUES [("19980101"), ("19990101")))
DISTRIBUTED BY HASH(`lo_orderkey`) BUCKETS 48
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"compression"="zstd",
"replication_num" = "1"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ CREATE TABLE IF NOT EXISTS `part` (
UNIQUE KEY (`p_partkey`)
DISTRIBUTED BY HASH(`p_partkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ CREATE TABLE IF NOT EXISTS `part` (
UNIQUE KEY (`p_partkey`)
DISTRIBUTED BY HASH(`p_partkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"compression"="zstd",
"replication_num" = "1"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ CREATE TABLE IF NOT EXISTS `supplier` (
UNIQUE KEY (`s_suppkey`)
DISTRIBUTED BY HASH(`s_suppkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ CREATE TABLE IF NOT EXISTS `supplier` (
UNIQUE KEY (`s_suppkey`)
DISTRIBUTED BY HASH(`s_suppkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"compression"="zstd",
"replication_num" = "1"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ CREATE TABLE IF NOT EXISTS `customer` (
UNIQUE KEY (`c_custkey`)
DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ CREATE TABLE IF NOT EXISTS `date` (
UNIQUE KEY (`d_datekey`)
DISTRIBUTED BY HASH(`d_datekey`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ PARTITION p1997 VALUES [("19970101"), ("19980101")),
PARTITION p1998 VALUES [("19980101"), ("19990101")))
DISTRIBUTED BY HASH(`lo_orderkey`) BUCKETS 48
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ CREATE TABLE IF NOT EXISTS `part` (
UNIQUE KEY (`p_partkey`)
DISTRIBUTED BY HASH(`p_partkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ CREATE TABLE IF NOT EXISTS `supplier` (
UNIQUE KEY (`s_suppkey`)
DISTRIBUTED BY HASH(`s_suppkey`) BUCKETS 10
PROPERTIES (
"enable_mow_light_delete" = "true",
"compression"="zstd",
"replication_num" = "1"
);
1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_p2/ddl/customer.sql
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ CREATE TABLE IF NOT EXISTS customer (
DUPLICATE KEY(C_CUSTKEY, C_NAME)
DISTRIBUTED BY HASH(C_CUSTKEY) BUCKETS 32
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "1"
)

1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_p2/ddl/lineitem.sql
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ CREATE TABLE IF NOT EXISTS lineitem (
DUPLICATE KEY(L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER)
DISTRIBUTED BY HASH(L_ORDERKEY) BUCKETS 32
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "1"
)

1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_p2/ddl/nation.sql
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ CREATE TABLE IF NOT EXISTS nation (
DUPLICATE KEY(N_NATIONKEY, N_NAME)
DISTRIBUTED BY HASH(N_NATIONKEY) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "1"
)

1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_p2/ddl/orders.sql
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ CREATE TABLE IF NOT EXISTS orders (
DUPLICATE KEY(O_ORDERKEY, O_CUSTKEY)
DISTRIBUTED BY HASH(O_ORDERKEY) BUCKETS 32
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "1"
)

1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_p2/ddl/part.sql
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ CREATE TABLE IF NOT EXISTS part (
DUPLICATE KEY(P_PARTKEY, P_NAME)
DISTRIBUTED BY HASH(P_PARTKEY) BUCKETS 32
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "1"
)

1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_p2/ddl/partsupp.sql
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ CREATE TABLE IF NOT EXISTS partsupp (
DUPLICATE KEY(PS_PARTKEY, PS_SUPPKEY)
DISTRIBUTED BY HASH(PS_PARTKEY) BUCKETS 32
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "1"
)

1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_p2/ddl/region.sql
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ CREATE TABLE IF NOT EXISTS region (
DUPLICATE KEY(R_REGIONKEY, R_NAME)
DISTRIBUTED BY HASH(R_REGIONKEY) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "1"
)

1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_p2/ddl/supplier.sql
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,6 @@ CREATE TABLE IF NOT EXISTS supplier (
DUPLICATE KEY(S_SUPPKEY, S_NAME)
DISTRIBUTED BY HASH(S_SUPPKEY) BUCKETS 32
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "1"
)
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ UNIQUE KEY(`c_custkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ UNIQUE KEY(`c_custkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"replication_num" = "3"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ UNIQUE KEY(`l_shipdate`, `l_orderkey`,`l_linenumber`,`l_partkey`,`l_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`l_orderkey`) BUCKETS 96
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ UNIQUE KEY(`l_shipdate`, `l_orderkey`,`l_linenumber`,`l_partkey`,`l_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`l_orderkey`) BUCKETS 96
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'DATE',
"replication_num" = "3"
)
Expand Down
1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_unique_p2/ddl/nation.sql
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ UNIQUE KEY(`N_NATIONKEY`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`N_NATIONKEY`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
);

Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ UNIQUE KEY(`N_NATIONKEY`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`N_NATIONKEY`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"replication_num" = "3"
);
Expand Down
1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_unique_p2/ddl/orders.sql
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@ UNIQUE KEY(`o_orderkey`, `o_orderdate`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`o_orderkey`) BUCKETS 96
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ UNIQUE KEY(`o_orderkey`, `o_orderdate`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`o_orderkey`) BUCKETS 96
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'bigint',
"replication_num" = "3"
)
1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_unique_p2/ddl/part.sql
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ UNIQUE KEY(`p_partkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`p_partkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ UNIQUE KEY(`p_partkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`p_partkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"replication_num" = "3"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ UNIQUE KEY(`ps_partkey`,`ps_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`ps_partkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ UNIQUE KEY(`ps_partkey`,`ps_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`ps_partkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"replication_num" = "3"
)
Expand Down
1 change: 1 addition & 0 deletions regression-test/suites/tpch_sf100_unique_p2/ddl/region.sql
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ UNIQUE KEY(`r_regionkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`r_regionkey`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ UNIQUE KEY(`r_regionkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`r_regionkey`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"replication_num" = "3"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,6 @@ UNIQUE KEY(`s_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`s_suppkey`) BUCKETS 12
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ UNIQUE KEY(`s_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`s_suppkey`) BUCKETS 12
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"replication_num" = "3"
)
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ UNIQUE KEY(`c_custkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ UNIQUE KEY(`l_shipdate`, `l_orderkey`,`l_linenumber`,`l_partkey`,`l_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`l_orderkey`) BUCKETS 96
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ UNIQUE KEY(`N_NATIONKEY`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`N_NATIONKEY`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
);

Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,6 @@ UNIQUE KEY(`o_orderkey`, `o_orderdate`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`o_orderkey`) BUCKETS 96
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ UNIQUE KEY(`p_partkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`p_partkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ UNIQUE KEY(`ps_partkey`,`ps_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`ps_partkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ UNIQUE KEY(`r_regionkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`r_regionkey`) BUCKETS 1
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,6 @@ UNIQUE KEY(`s_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`s_suppkey`) BUCKETS 12
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ UNIQUE KEY(`c_custkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ UNIQUE KEY(`c_custkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`c_custkey`) BUCKETS 24
PROPERTIES (
"enable_mow_light_delete" = "true",
"function_column.sequence_type" = 'int',
"replication_num" = "3"
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ UNIQUE KEY(`l_shipdate`, `l_orderkey`,`l_linenumber`,`l_partkey`,`l_suppkey`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`l_orderkey`) BUCKETS 96
PROPERTIES (
"enable_mow_light_delete" = "true",
"replication_num" = "3"
)

Loading

0 comments on commit 3060673

Please sign in to comment.