Configuration
Configuration objects can be used when creating a TileDB context, or when using consolidation and virtual filesystem (VFS) functionality. See Configuration Parameters for a summary of all TileDB configuration options.
Basic Usage
You can create and set configuration objects as follows:
Saving To / Loading From File
You can save a configuration object into a text file, as well as load a configuration object from a text file, as follows:
Configuration Iterator
You can also use a configuration iterator as follows:
Configuration Parameters
Below we provide a table with all the TileDB configuration parameters, along with their description and default values. See Configuration for information on how to set them.
Parameter | Default Value | Description |
|
| This is applicable only if |
|
| If |
|
| If |
|
| The factor by which the size of the dense fragment resulting from consolidating a set of fragments (containing at least one dense fragment) can be amplified. This is important when the union of the non-empty domains of the fragments to be consolidated have a lot of empty cells, which the consolidated fragment will have to fill with the special fill value (since the resulting fragment is dense). |
|
| The size (in bytes) of the attribute buffers used during consolidation. |
|
| The maximum number of fragments to consolidate in a single step. |
|
| The minimum number of fragments to consolidate in a single step. |
|
| The size ratio of two (“adjacent”) fragments must be larger than this value to be considered for consolidation in a single step. |
|
| The number of consolidation steps to be performed when executing the consolidation algorithm. |
|
| The consolidation mode, one of |
|
| The vacuuming mode, one of |
|
| If |
|
| Determines whether or not TileDB will install signal handlers. |
|
| The memory budget for tiles of fixed-sized attributes (or offsets for var-sized attributes) to be fetched during reads. This is applicable to dense reads, sparse ordered reads and "legacy" reader modes. |
|
| The memory budget for tiles of var-sized attributes to be fetched during reads. This is applicable to dense reads and sparse ordered reads and "legacy" reader modes. |
|
| Should malloc_trim be called on context and query destruction? This might reduce residual memory usage. |
|
| Memory budget for refactored readers and writers. This includes sparse unordered reads, and dense reads with refactored reader. |
|
| Ratio of the budget allocated for coordinates in the sparse global order reader. |
|
| Ratio of the budget allocated for the query condition in the sparse global order reader. |
|
| Ratio of the budget allocated for tile ranges in the sparse global order reader. |
|
| Ratio of the budget allocated for array data in the sparse global order reader. |
|
| Ratio of the budget allocated for coordinates in the sparse unordered with duplicates reader. |
|
| Ratio of the budget allocated for the query condition in the sparse unordered with duplicates reader. |
|
| Ratio of the budget allocated for tile ranges in the sparse unordered with duplicates reader. |
|
| Ratio of the budget allocated for array data in the sparse unordered with duplicates reader. |
|
| The memory budget used by the read algorithm to force partition the query range in case sorting is much slower than the partitioning overhead. |
| # of cores | Upper-bound on number of threads to allocate for compute-bound tasks. |
| # of cores | Upper-bound on number of threads to allocate for IO-bound tasks. |
| TBB automatic | The number of threads allocated for the TBB thread pool. Note: this is a whole-program setting. Usually this should not be modified from the default. See also the documentation for TBB's |
|
| The tile cache size in bytes. |
|
| The maximum number of parallel operations on objects with |
|
| If set to |
|
| Permissions to use for posix file system with file creation. |
|
| Permissions to use for posix file system with directory creation. |
|
| The minimum number of bytes between two VFS read batches. |
|
| The minimum number of bytes in a VFS read operation. |
|
| The minimum number of bytes in a parallel VFS operation, except parallel S3 writes, which are controlled by parameter |
|
| The maximum byte size to read-ahead from the backend. |
|
| The the total maximum size of the read-ahead cache, which is an LRU. |
|
| The maximum tries for a connection. Any |
|
| The scale factor for exponential backoff when connecting to S3. Any |
|
| The connection timeout in ms. Any |
|
| The S3 endpoint, if S3 is enabled. |
|
| The maximum number of S3 backend parallel operations. |
|
| The part size (in bytes) used in S3 multipart writes. Any |
|
| The S3 proxy host. |
|
| The S3 proxy password. |
|
| The S3 proxy port. |
|
| The S3 proxy scheme. |
|
| The S3 proxy username. |
|
| The S3 region. |
|
| The AWS access key id ( |
|
| The AWS access secret ( |
|
| The AWS session token to use |
|
| The Amazon Resource Name (ARN) of the role to assume. |
|
| A unique identifier that might be required when you assume a role in another account |
|
| The duration, in minutes, of the role session |
|
| An identifier for the assumed role session. |
|
| The AWS SDK logging level (OFF, DEBUG, TRACE) |
|
| The request timeout in ms. Any |
|
| The S3 scheme. |
|
| Determines whether to use virtual addressing or not. |
|
| The S3 use of multi-part upload requests ( |
|
| The path to a cURL-compatible certificate file. |
|
| The path to a cURL-compatible certificate directory. |
|
| Enable certificate verification for HTTPS connections. |
|
| Set the GCS project id. |
|
| The part size (in bytes) used in GCS multi part writes. Any |
|
| The maximum number of GCS backend parallel operations. |
|
| The maximum amount of time to retry network requests to GCS. |
|
| Determines if the GCS backend can use chunked part uploads. |
|
| Set the Azure Storage Account name. |
|
| Set the Azure Storage Account key. |
|
| Overrides the default Azure Storage Blob endpoint. If empty, the endpoint will be constructed from the storage account name. This should not include an |
| 5242880 | The block size (in bytes) used in Azure blob block list writes. Any |
|
| Determines if the blob endpoint should use HTTP or HTTPS. |
|
| The maximum number of Azure backend parallel operations. |
|
| Determines if the Azure backend can use chunked block uploads. |
|
| Path to the Kerberos ticket cache when connecting to an HDFS cluster. |
|
| Optional namenode URI to use (TileDB will use |
|
| Username to use when connecting to the HDFS cluster. |
|
| URL for REST server to use for remote arrays. |
|
| Serialization format to use for remote array requests (CAPNP or JSON). |
|
| Username for login to REST server. |
|
| Password for login to REST server. |
|
| Authentication token for REST server (used instead of username/password). |
|
| If true, incomplete queries received from server are automatically resubmitted before returning to user control. |
|
| Have curl ignore ssl peer and host validation for REST server. |
|
| Compression used in HTTP requests. |
|
| The name of the registered access key to use for creation of the REST server. |
|
| CSV list of http status codes to automatically retry a REST request for. |
|
| Number of times to retry failed REST requests |
|
| Initial delay in milliseconds to wait until retrying a REST request |
|
| The delay factor to exponentially wait until further retries of a failed REST request |
|
| Prefix of environmental variables for reading configuration parameters. |
Last updated