Skip to content

Commit

Permalink
Synchronize configuration parameters to Academy (#5188)
Browse files Browse the repository at this point in the history
I was tasked with documenting all configuration parameters in Academy.
The information I was given to document those parameters was out of
date. I have since compiled the responses from the rest of the core team
about the missing configuration parameters and compiled it here. Note
the following parameters have been left undocumented:

- `sm.skip_est_size_partitioning`
- `sm.skip_unary_partitioning_budget_check`
- `sm.merge_overlapping_ranges_experimental`
- `vfs.gcs.endpoint`

---
TYPE: NO_HISTORY
DESC: Update all configuration parameters to reflect what's available in
2.24.1

---------

Co-authored-by: Luc Rancourt <[email protected]>
  • Loading branch information
nickvigilante and KiterLuc authored Jul 17, 2024
1 parent 93d2909 commit 78e2185
Show file tree
Hide file tree
Showing 2 changed files with 107 additions and 57 deletions.
41 changes: 34 additions & 7 deletions tiledb/api/c_api/config/config_api_external.h
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* - `sm.consolidation.steps` <br>
* The number of consolidation steps to be performed when executing
* the consolidation algorithm.<br>
* **Default**: 1
* **Default**: UINT32_MAX
* - `sm.consolidation.purge_deleted_cells` <br>
* **Experimental** <br>
* Purge deleted cells from the consolidated fragment or not.<br>
Expand All @@ -207,6 +207,25 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* `sm.consolidation.timestamp_start` and this value (inclusive). <br>
* Only for `fragments` and `array_meta` consolidation mode. <br>
* **Default**: UINT64_MAX
* - `sm.encryption_key` <br>
* The key for encrypted arrays. <br>
* **Default**: ""
* - `sm.encryption_type` <br>
* The type of encryption used for encrypted arrays. <br>
* **Default**: "NO_ENCRYPTION"
* - `sm.enumerations_max_size` <br>
* Maximum in memory size for an enumeration. If the enumeration is <br>
* var sized, the size will include the data and the offsets. <br>
* **Default**: 10MB
* - `sm.enumerations_max_total_size` <br>
* Maximum in memory size for all enumerations. If the enumeration <br>
* is var sized, the size will include the data and the offsets. <br>
* **Default**: 50MB
* - `sm.max_tile_overlap_size` <br>
* Maximum size for the tile overlap structure which holds <br>
* information about which tiles are covered by ranges. Only used <br>
* in dense reads and legacy reads. <br>
* **Default**: 300MB
* - `sm.memory_budget` <br>
* The memory budget for tiles of fixed-sized attributes (or offsets for
* var-sized attributes) to be fetched during reads.<br>
Expand All @@ -230,14 +249,21 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* - `sm.query.dense.reader` <br>
* Which reader to use for dense queries. "refactored" or "legacy".<br>
* **Default**: refactored
* - `sm.query.dense.qc_coords_mode` <br>
* Dense configuration that allows to only return the coordinates of <br>
* the cells that match a query condition without any attribute data. <br>
* **Default**: "false"
* - `sm.query.sparse_global_order.reader` <br>
* Which reader to use for sparse global order queries. "refactored"
* or "legacy".<br>
* **Default**: legacy
* **Default**: refactored
* - `sm.query.sparse_unordered_with_dups.reader` <br>
* Which reader to use for sparse unordered with dups queries.
* "refactored" or "legacy".<br>
* **Default**: refactored
* - `sm.skip_checksum_validation` <br>
* Skip checksum validation on reads for the md5 and sha256 filters. <br>
* **Default**: "false"
* - `sm.mem.malloc_trim` <br>
* Should malloc_trim be called on context and query destruction? This might
* reduce residual memory usage. <br>
Expand Down Expand Up @@ -270,6 +296,7 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* `sm.mem.consolidation.reader_weight` and
* `sm.mem.consolidation.writer_weight`. <br>
* **Default**: 3
* - `sm.mem.consolidation.writer_weight` <br>
* Weight used to split `sm.mem.total_budget` and assign to the
* writer query. The budget is split across 3 values,
* `sm.mem.consolidation.buffers_weight`,
Expand Down Expand Up @@ -352,7 +379,7 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* **Default**: 10MB
* - `vfs.max_batch_size` <br>
* The maximum number of bytes in a VFS read operation<br>
* **Default**: UINT64_MAX
* **Default**: 100MB
* - `vfs.min_batch_size` <br>
* The minimum number of bytes in a VFS read operation<br>
* **Default**: 20MB
Expand Down Expand Up @@ -533,7 +560,7 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* **Default**: ""
* - `vfs.s3.connect_timeout_ms` <br>
* The connection timeout in ms. Any `long` value is acceptable. <br>
* **Default**: 3000
* **Default**: 10800
* - `vfs.s3.connect_max_tries` <br>
* The maximum tries for a connection. Any `long` value is acceptable. <br>
* **Default**: 5
Expand All @@ -549,7 +576,7 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* The AWS SDK logging level. This is a process-global setting. The
* configuration of the most recently constructed context will set
* process state. Log files are written to the process working directory.
* **Default**: ""
* **Default**: "Off"
* - `vfs.s3.request_timeout_ms` <br>
* The request timeout in ms. Any `long` value is acceptable. <br>
* **Default**: 3000
Expand Down Expand Up @@ -631,7 +658,7 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* `config_files` (forces SDK to only consider options found in aws
* config files),
* `sts_profile_with_web_identity` (force SDK to consider assume roles/sts
* from config files with support for web tokens, commonly used by EKS/ECS).
* from config files with support for web tokens, commonly used by EKS/ECS).
* **Default**: auto
* <br>
* - `vfs.s3.install_sigpipe_handler` <br>
Expand Down Expand Up @@ -693,7 +720,7 @@ TILEDB_EXPORT void tiledb_config_free(tiledb_config_t** config) TILEDB_NOEXCEPT;
* **Default**: "503"
* - `rest.retry_count` <br>
* Number of times to retry failed REST requests <br>
* **Default**: 3
* **Default**: 25
* - `rest.retry_initial_delay_ms` <br>
* Initial delay in milliseconds to wait until retrying a REST request <br>
* **Default**: 500
Expand Down
Loading

0 comments on commit 78e2185

Please sign in to comment.