Skip to content

Commit

Permalink
[docs][hive-transactional-tables] Add hive transactional tables docum…
Browse files Browse the repository at this point in the history
…ents. (apache#29369)
  • Loading branch information
kaka11chen authored and HappenLee committed Jan 12, 2024
1 parent 12629b0 commit 2d7d051
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 0 deletions.
14 changes: 14 additions & 0 deletions docs/en/docs/lakehouse/multi-catalog/hive.md
Original file line number Diff line number Diff line change
Expand Up @@ -555,3 +555,17 @@ CREATE CATALOG hive PROPERTIES (
'yarn.resourcemanager.principal' = 'your-rm-principal'
);
```
## Hive Transactional Tables

Hive transactional tables are tables in Hive that support ACID (Atomicity, Consistency, Isolation, Durability) semantics. For more details, you can refer to: [Hive Transactions](https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions).

### Supported Operations for Hive Transactional Tables:
|Transactional Table Type|Supported Operations in Hive|Hive Table Properties|Supported Hive Versions|
|---|---|---|---|
|Full-ACID Transactional Table |Supports insert, update, delete operations|'transactional'='true', 'transactional_properties'='insert_only'|3.x, 2.x (requires major compaction in Hive before loading)|
|Insert-Only Transactional Table|Supports only Insert operations|'transactional'='true'|3.x, 2.x|

### Current Limitations:
Currently, it does not support scenarios involving Original Files.
When a table is transformed into a transactional table, subsequent newly written data files will use the schema of the Hive transactional table. However, existing data files will not be converted to the schema of the transactional table. These existing files are referred to as Original Files.

12 changes: 12 additions & 0 deletions docs/zh-CN/docs/lakehouse/multi-catalog/hive.md
Original file line number Diff line number Diff line change
Expand Up @@ -534,3 +534,15 @@ CREATE CATALOG hive_krb_ha PROPERTIES (
'dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider'
);
```
## Hive Transactional 表
Hive transactional 表是 Hive 中支持 ACID 语义的表。详情可见:https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions

### Hive Transactional 表支持情况:
|表类型|在 Hive 中支持的操作|Hive 表属性|支持的Hive 版本|
|---|---|---|---|
|Full-ACID Transactional Table |支持 Insert, Update, Delete 操作|'transactional'='true', 'transactional_properties'='insert_only'|3.x,2.x,其中 2.x 需要在 Hive 中执行完 major compaction 才可以加载|
|Insert-Only Transactional Table|只支持 Insert 操作|'transactional'='true'|3.x,2.x|

### 当前限制:
目前不支持 Original Files 的场景。
当一个表转换成 Transactional 表之后,后续新写的数据文件会使用 Hive Transactional 表的 schema,但是已经存在的数据文件是不会转化成 Transactional 表的 schema,这样的文件称为 Original Files。

0 comments on commit 2d7d051

Please sign in to comment.