Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[temporary_table] add docs for temporary table #1135

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

[temporary_table] add docs for temporary table #1135

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b41f041
[temp_table] add docs for temp table
Yulei-Yang Sep 23, 2024
09ef86b
update
Yulei-Yang Sep 23, 2024
4a22ccd
update
Yulei-Yang Sep 30, 2024
9b06342
fix typo
Yulei-Yang Sep 30, 2024
c9da20f
update format
Yulei-Yang Sep 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...nual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ This statement creates the table structure by returning the results from the Sel
grammar：

```sql
CREATE TABLE table_name [( column_name_list )]
CREATE [TEMPORARY] TABLE table_name [( column_name_list )]
opt_engine:engineName
opt_keys:keys
opt_comment:tableComment
Expand Down
3 changes: 2 additions & 1 deletion ...ql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ This statement is used to create an empty table with the exact same table struct
grammar:

```sql
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]table_name [WITH ROLLUP (r1,r2,r3,...)]
CREATE [TEMPORARY | EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]table_name [WITH ROLLUP (r1,r2,r3,...)]
```

illustrate:
Expand All @@ -46,6 +46,7 @@ illustrate:
- The user needs to have `SELECT` permission on the copied original table
- Support for copying external tables such as MySQL
- Support the rollup of copying OLAP Table
- Temporary tables can only be internal tables. And they can only be created based on internal tables.

### Example

Expand Down
5 changes: 4 additions & 1 deletion docs/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ under the License.
This command is used to create a table. The subject of this document describes the syntax for creating Doris self-maintained tables.

```sql
CREATE TABLE [IF NOT EXISTS] [database.]table
CREATE [TEMPORARY] TABLE [IF NOT EXISTS] [database.]table
(
column_definition_list
[, index_definition_list]
Expand All @@ -48,6 +48,9 @@ distribution_desc
[extra_properties]
```

#### TEMPORARY
Indicates creating a temporary table, which only available in current session, and will be deleted after the session be closed.

#### column_definition_list

Column definition list:
Expand Down
60 changes: 60 additions & 0 deletions docs/table-design/temporary-table.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
title: Temporary Table
language: en
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

:::note
Supported since Doris version 2.1.8 / 3.0.3
:::

When handling complex data processing, saving intermediate calculation results as physical tables is a great method to significantly reduce SQL complexity and improve data debuggability. However, such tables need to be manually cleaned up after use. Currently, Doris only supports defining non-physical temporary tables through the WITH clause.

To address this issue, Doris introduces the concept of temporary tables. A temporary table is a materialized internal table that exists temporarily and is primarily different from internal tables in that it only exists in the session that created it. Its lifecycle is bound to the current session, and the temporary table will be automatically deleted when the session ends. Moreover, the visibility of a temporary table is restricted to the session in which it was created, meaning it is not visible to another session of the same user at the same time.


:::note

Similar to internal tables, temporary tables must be created under a Database within the Internal Catalog. However, since temporary tables are session-based, their naming is not subject to uniqueness constraints. You can create temporary tables with the same name in different sessions or create temporary tables with the same names as other internal tables.

If a temporary table and a non-temporary table with the same name exist simultaneously in the same Database, the temporary table has the highest access priority. Within that session, all queries and operations on the table with the same name will only affect the temporary table (except for creating materialized views).
:::

## Usage

### Creating a Temporay Table

Tables of various models can be defined as temporary tables, whether they are Unique, Aggregate, or Duplicate models. You can create temporary tables by adding the TEMPORARY keyword in the following SQL statements:
- [CREATE TABLE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE)
- [CREATE TABLE AS SELECT](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT)
- [CREATE TABLE LIKE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE)

The other uses of temporary tables are basically the same as regular internal tables. Except for the above-mentioned Create statement, other DDL and DML statements do not require adding the TEMPORARY keyword.

## Notes

- Temporary tables can only be created in the Internal Catalog.
- The ENGINE must be set to OLAP when creating a table.
- Alter statements are not supported for modifying temporary tables.
- Due to their temporary nature, creating views and materialized views based on temporary tables is not supported.
- Temporary tables cannot be backed up and are not supported for synchronization using CCR/Sync Job.
- Export, Stream Load, Broker Load, S3 Load, MySQL Load, Routine Load, and Spark Load are not supported.
- When a temporary table is deleted, it does not go to the recycle bin but is permanently deleted immediately.
2 changes: 1 addition & 1 deletion ...nual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ CREATE TABLE AS SELECT
语法：

```sql
CREATE TABLE table_name [( column_name_list )]
CREATE [TEMPORARY] TABLE table_name [( column_name_list )]
opt_engine:engineName
opt_keys:keys
opt_comment:tableComment
Expand Down
3 changes: 2 additions & 1 deletion ...ql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ CREATE TABLE LIKE
语法：

```sql
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]table_name [WITH ROLLUP (r1,r2,r3,...)]
CREATE [TEMPORARY | EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]table_name [WITH ROLLUP (r1,r2,r3,...)]
```

说明:
Expand All @@ -46,6 +46,7 @@ CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]ta
- 用户需要对复制的原表有`SELECT`权限
- 支持复制MySQL等外表
- 支持复制OLAP Table的rollup
- 临时表只能是内表。只能基于内表创建

### Example

Expand Down
5 changes: 4 additions & 1 deletion ...ent/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ under the License.
该命令用于创建一张表。本文档主要介绍创建 Doris 自维护的表的语法

```sql
CREATE TABLE [IF NOT EXISTS] [database.]table
CREATE [TEMPORARY] TABLE [IF NOT EXISTS] [database.]table
(
column_definition_list
[, index_definition_list]
Expand All @@ -48,6 +48,9 @@ distribution_desc
[extra_properties]
```

#### TEMPORARY
创建临时表。临时表仅在当前 Session 可见，会话结束自动删除

#### column_definition_list

列定义列表：
Expand Down
59 changes: 59 additions & 0 deletions i18n/zh-CN/docusaurus-plugin-content-docs/current/table-design/temporary-table.md
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for contribution

1.please use Chinese in docs and use :::info 备注 ::: instead of :::note :::
image

2.please fix the typo
image

3.please make sure the capitalization is unified, such as SQL (All caps) , Session(Sentence case) ， Rollup (Sentence case)

4.please add blank between Chinese and EN words or Numbers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
title: 临时表
language: zh-CN
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

:::info 备注
自 2.1.8 / 3.0.3 版本开始支持
:::

在进行复杂数据处理时，将中间计算结果保存为实体表是一种可显著降低 SQL 复杂度、提高数据可调试性的好方法。但这种表需要在使用完毕之后手动清理。目前 Doris 仅支持通过 with 从句定义非实体临时表。

为了解决上述问题，Doris 引入临时表类型。临时表是一种临时存在的物化内表，和内表的主要区别是其仅存在于创建它的 Session中，它的生命周期和当前 Session 绑定。当会话结束时，在其中创建的临时表会被自动删除。从另一方面讲，临时表的可见性也仅在创建它的会话中，即使同一时间同一个用户的另一个会话也不可见。


:::info 备注

与内表类似，临时表必须在 Internal Catalog 内的某个 Database 下创建。但由于临时表基于 Session，因此其命名不受唯一性约束。您可以在不同 Session 中创建同名临时表，或创建与其他内表同名的临时表。

如果同一 Database 中同时存在同名的临时表和非临时表，临时表具有最高访问优先级。在该 Session 内，所有针对同名表的查询和操作仅对临时表生效（除创建物化视图外）。
:::

## 用法

### 创建临时表
各种模型的表都可以被定义为临时表， 不论是 Unique、Aggregate 或是 Duplicate 模型。可以在下列 SQL 中添加 TEMPORARY 关键字创建临时表：
- [CREATE TABLE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE)
- [CREATE TABLE AS SELECT](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT)
- [CREATE TABLE LIKE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE)

临时表的其它用法基本和普通内表相同。除上述 Create 语句外， 其它 DDL 及 DML 语句无需添加 TEMPORARY 关键字。

## 注意事项

- 临时表只能在 Internal Catalog 中创建
- 建表时 `ENGINE` 必须为 `OLAP`
- 不支持使用 Alter 语句修改临时表
- 由于临时性，不支持基于临时表创建视图和物化视图
- 不支持备份临时表，不支持使用 CCR / Sync Job 同步临时表
- 不支持导出、Stream Load、Broker Load、S3 Load、Mysql Load、Routine Load、Spark Load
- 删除临时表时，不进回收站，直接彻底删除
2 changes: 1 addition & 1 deletion ...nual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ CREATE TABLE AS SELECT
语法：

```sql
CREATE TABLE table_name [( column_name_list )]
CREATE [TEMPORARY] TABLE table_name [( column_name_list )]
opt_engine:engineName
opt_keys:keys
opt_comment:tableComment
Expand Down
3 changes: 2 additions & 1 deletion ...ql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ CREATE TABLE LIKE
语法：

```sql
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]table_name [WITH ROLLUP (r1,r2,r3,...)]
CREATE [TEMPORARY | EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]table_name [WITH ROLLUP (r1,r2,r3,...)]
```

说明:
Expand All @@ -46,6 +46,7 @@ CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]ta
- 用户需要对复制的原表有`SELECT`权限
- 支持复制MySQL等外表
- 支持复制OLAP Table的rollup
- 临时表只能是内表。只能基于内表创建

### Example

Expand Down
5 changes: 4 additions & 1 deletion ...2.1/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ under the License.
该命令用于创建一张表。

```sql
CREATE TABLE [IF NOT EXISTS] [database.]table
CREATE [TEMPORARY] TABLE [IF NOT EXISTS] [database.]table
(
column_definition_list
[, index_definition_list]
Expand All @@ -48,6 +48,9 @@ distribution_desc
[extra_properties]
```

#### TEMPORARY
创建临时表。临时表仅在当前 Session 可见，会话结束自动删除

#### column_definition_list

列定义列表：
Expand Down
60 changes: 60 additions & 0 deletions ...h-CN/docusaurus-plugin-content-docs/version-2.1/table-design/temporary-table.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
title: 临时表
language: zh-CN
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

:::info 备注
自 2.1.8 / 3.0.3 版本开始支持
:::

在进行复杂数据处理时，将中间计算结果保存为实体表是一种可显著降低 SQL 复杂度、提高数据可调试性的好方法。但这种表需要在使用完毕之后手动清理。目前 Doris 仅支持通过 with 从句定义非实体临时表。

为了解决上述问题，Doris 引入临时表类型。临时表是一种临时存在的物化内表，和内表的主要区别是其仅存在于创建它的 Session中，它的生命周期和当前 Session 绑定。当会话结束时，在其中创建的临时表会被自动删除。从另一方面讲，临时表的可见性也仅在创建它的会话中，即使同一时间同一个用户的另一个会话也不可见。


:::info 备注

与内表类似，临时表必须在 Internal Catalog 内的某个 Database 下创建。但由于临时表基于 Session，因此其命名不受唯一性约束。您可以在不同 Session 中创建同名临时表，或创建与其他内表同名的临时表。

如果同一 Database 中同时存在同名的临时表和非临时表，临时表具有最高访问优先级。在该 Session 内，所有针对同名表的查询和操作仅对临时表生效（除创建物化视图外）。
:::

## 用法

### 创建临时表
各种模型的表都可以被定义为临时表， 不论是 Unique、Aggregate 或是 Duplicate 模型。可以在下列 SQL 中添加 TEMPORARY 关键字创建临时表：
- [CREATE TABLE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE)
- [CREATE TABLE AS SELECT](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT)
- [CREATE TABLE LIKE](../sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE)

临时表的其它用法基本和普通内表相同。除上述 Create 语句外， 其它 DDL 及 DML 语句无需添加 TEMPORARY 关键字。

## 注意事项

- 临时表只能在 Internal Catalog 中创建
- 建表时 `ENGINE` 必须为 `OLAP`
- 不支持使用 Alter 语句修改临时表
- 由于临时性，不支持基于临时表创建视图和物化视图
- 不支持备份临时表，不支持使用 CCR / Sync Job 同步临时表
- 不支持导出、Stream Load、Broker Load、S3 Load、Mysql Load、Routine Load、Spark Load
- 删除临时表时，不进回收站，直接彻底删除
-
2 changes: 1 addition & 1 deletion ...nual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-AS-SELECT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ CREATE TABLE AS SELECT
语法：

```sql
CREATE TABLE table_name [( column_name_list )]
CREATE [TEMPORARY] TABLE table_name [( column_name_list )]
opt_engine:engineName
opt_keys:keys
opt_comment:tableComment
Expand Down
3 changes: 2 additions & 1 deletion ...ql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE-LIKE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ CREATE TABLE LIKE
语法：

```sql
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]table_name [WITH ROLLUP (r1,r2,r3,...)]
CREATE [TEMPORARY | EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]table_name [WITH ROLLUP (r1,r2,r3,...)]
```

说明:
Expand All @@ -46,6 +46,7 @@ CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [database.]table_name LIKE [database.]ta
- 用户需要对复制的原表有`SELECT`权限
- 支持复制MySQL等外表
- 支持复制OLAP Table的rollup
- 临时表只能是内表。只能基于内表创建

### Example

Expand Down
5 changes: 4 additions & 1 deletion ...3.0/sql-manual/sql-statements/Data-Definition-Statements/Create/CREATE-TABLE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ under the License.
该命令用于创建一张表。本文档主要介绍创建 Doris 自维护的表的语法

```sql
CREATE TABLE [IF NOT EXISTS] [database.]table
CREATE [TEMPORARY] TABLE [IF NOT EXISTS] [database.]table
(
column_definition_list
[, index_definition_list]
Expand All @@ -48,6 +48,9 @@ distribution_desc
[extra_properties]
```

#### TEMPORARY
创建临时表。临时表仅在当前 Session 可见，会话结束自动删除

#### column_definition_list

列定义列表：
Expand Down
Loading