Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs](hive-transactional-tables) Add hive transactional tables documents. #29369

Merged
merged 1 commit into from
Jan 3, 2024
Merged

[docs](hive-transactional-tables) Add hive transactional tables documents. #29369

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/en/docs/lakehouse/multi-catalog/hive.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -548,3 +548,17 @@ CREATE CATALOG hive PROPERTIES (
'yarn.resourcemanager.principal' = 'your-rm-principal'
);
```
## Hive Transactional Tables

Hive transactional tables are tables in Hive that support ACID (Atomicity, Consistency, Isolation, Durability) semantics. For more details, you can refer to: [Hive Transactions](https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions).

### Supported Operations for Hive Transactional Tables:
|Transactional Table Type|Supported Operations in Hive|Hive Table Properties|Supported Hive Versions|
|---|---|---|---|
|Full-ACID Transactional Table |Supports insert, update, delete operations|'transactional'='true', 'transactional_properties'='insert_only'|3.x, 2.x (requires major compaction in Hive before loading)|
|Insert-Only Transactional Table|Supports only Insert operations|'transactional'='true'|3.x, 2.x|

### Current Limitations:
Currently, it does not support scenarios involving Original Files.
When a table is transformed into a transactional table, subsequent newly written data files will use the schema of the Hive transactional table. However, existing data files will not be converted to the schema of the transactional table. These existing files are referred to as Original Files.

12 changes: 12 additions & 0 deletions docs/zh-CN/docs/lakehouse/multi-catalog/hive.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -528,3 +528,15 @@ CREATE CATALOG hive_krb_ha PROPERTIES (
'dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider'
);
```
## Hive Transactional 表
Hive transactional 表是 Hive 中支持 ACID 语义的表。详情可见：https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions

### Hive Transactional 表支持情况：
|表类型|在 Hive 中支持的操作|Hive 表属性|支持的Hive 版本|
|---|---|---|---|
|Full-ACID Transactional Table |支持 Insert, Update, Delete 操作|'transactional'='true', 'transactional_properties'='insert_only'|3.x，2.x，其中 2.x 需要在 Hive 中执行完 major compaction 才可以加载|
|Insert-Only Transactional Table|只支持 Insert 操作|'transactional'='true'|3.x，2.x|

### 当前限制：
目前不支持 Original Files 的场景。
当一个表转换成 Transactional 表之后，后续新写的数据文件会使用 Hive Transactional 表的 schema，但是已经存在的数据文件是不会转化成 Transactional 表的 schema，这样的文件称为 Original Files。
Loading