-
Notifications
You must be signed in to change notification settings - Fork 666
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improve the performance of CitusHasBeenLoaded function for a database…
… that does not do CREATE EXTENSION citus but load citus.so. (#7123) For a database that does not create the citus extension by running ` CREATE EXTENSION citus;` `CitusHasBeenLoaded ` function ends up querying the `pg_extension` table every time it is invoked. This is not an ideal situation for a such a database. The idea in this PR is as follows: ### A new field in MetadataCache. Add a new variable `extensionCreatedState `of the following type: ``` typedef enum ExtensionCreatedState { UNKNOWN = 0, CREATED = 1, NOTCREATED = 2, } ExtensionCreatedState; ``` When the MetadataCache is invalidated, `ExtensionCreatedState` will be set to UNKNOWN. ### Invalidate MetadataCache when CREATE/DROP/ALTER EXTENSION citus commands are run. - Register a callback function, named `InvalidateDistRelationCacheCallback`, for relcache invalidation during the shared library initialization for `citus.so`. This callback function is invoked in all the backends whenever the relcache is invalidated in one of the backends. (This could be caused many DDLs operations). - In the cache invalidation callback,` InvalidateDistRelationCacheCallback`, invalidate `MetadataCache` zeroing it out. - In `CitusHasBeenLoaded`, perform the costly citus is loaded check only if the `MetadataCache` is not valid. ### Downsides Any relcache invalidation (caused by various DDL operations) will case Citus MetadataCache to get invalidated. Most of the time it will be unnecessary. But we rely on that DDL operations on relations will not be too frequent.
- Loading branch information
1 parent
1d540b6
commit a849570
Showing
5 changed files
with
134 additions
and
60 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
import psycopg | ||
import pytest | ||
|
||
|
||
def test_create_drop_citus(coord): | ||
with coord.cur() as cur1: | ||
with coord.cur() as cur2: | ||
# Conn1 drops the extension | ||
# and Conn2 cannot use it. | ||
cur1.execute("DROP EXTENSION citus") | ||
|
||
with pytest.raises(psycopg.errors.UndefinedFunction): | ||
# Conn1 dropped the extension. citus_version udf | ||
# cannot be found.sycopg.errors.UndefinedFunction | ||
# is expected here. | ||
cur2.execute("SELECT citus_version();") | ||
|
||
# Conn2 creates the extension, | ||
# Conn1 is able to use it immediadtely. | ||
cur2.execute("CREATE EXTENSION citus") | ||
cur1.execute("SELECT citus_version();") | ||
cur1.execute("DROP EXTENSION citus;") | ||
|
||
with coord.cur() as cur1: | ||
with coord.cur() as cur2: | ||
# A connection is able to create and use the extension | ||
# within a transaction block. | ||
cur1.execute("BEGIN;") | ||
cur1.execute("CREATE TABLE t1(id int);") | ||
cur1.execute("CREATE EXTENSION citus;") | ||
cur1.execute("SELECT create_reference_table('t1')") | ||
cur1.execute("ABORT;") | ||
|
||
# Conn1 aborted so Conn2 is be able to create and | ||
# use the extension within a transaction block. | ||
cur2.execute("BEGIN;") | ||
cur2.execute("CREATE TABLE t1(id int);") | ||
cur2.execute("CREATE EXTENSION citus;") | ||
cur2.execute("SELECT create_reference_table('t1')") | ||
cur2.execute("COMMIT;") | ||
|
||
# Conn2 commited so Conn1 is be able to use the | ||
# extension immediately. | ||
cur1.execute("SELECT citus_version();") |