Table Parameters

These are set upon table creation as follows:

create table test_array (
  ...
) engine=MyTile <table-parameters>; // e.g., array_type='SPARSE'

Column Parameters

The column parameters are set upon table creation as follows:

create table test_array (
  <column-parameters> // e.g., dim0 integer DIMENSION=1 lower_bound="0" upper_bound="100" tile_extent="1"
) engine=MyTile <table-parameters>; 

Filter Options

For filter parameters either table or column the following are the list of supported filters. Any number of filters can be passed in a csv string. If a filter takes a parameters it can be set using key=value. I.Efilters="GZIP=-1,LZ4=4"

TileDB Parameters

You can set any of the TileDB configuration parameters as follows:

set mytile_tiledb_config=<list-of-params>;
// E.g., set mytile_tiledb_config='vfs.num_threads=12,vfs.s3.aws_region=us-east-1';

MariaDB System Parameters

mytile_read_buffer_size

• Description: Size in bytes of the buffers to use for each attribute/coordinates when performing a read.

• Command line: --mytile-read-buffer-size

• Scope: Global

• Dynamic: Yes

• Data Type: long

• Default Value: 100M

mytile_write_buffer_size

• Description: Size in bytes of the buffers to use for each attribute/coordinates when performing a write.

• Command line: --mytile-write-buffer-size

• Scope: Global

• Dynamic: Yes

• Data Type: long

• Default Value: 100M

mytile_delete_arrays

• Description: Controls if a delete table statement causes the array to be deleted on disk or just deregistered from MariaDB. A true value causes actual deletions of data.

• Command line: --mytile-delete-arrays

• Scope: Global

• Dynamic: Yes

• Data Type: boolean

• Default Value: false

tiledb_config

• Description: Set TileDB configuration parameters, this is in csv form of key1=value1,key2=value2. Example: set mytile_tiledb_config = 'vfs.s3.aws_access_key_id=abc,vfs.s3.aws_secret_access_key=123';

• Command line: --mytile-tiledb-config

• Scope: Session

• Dynamic: Yes

• Data Type: string

• Default Value: ""

reopen_for_every_query

• Description: Closes and reopen the array for every query, this allows tiledb_config parameters to take effect without forcing a table flush, but any TileDB caching is removed.

• Command line: --mytile-reopen-for-every-query

• Scope: Session

• Dynamic: Yes

• Data Type: boolean

• Default Value: true

ready_query_layout

• Description: Set layout for ready queries, valid values are row-major, col-major, unordered or global-order

• Command line: --mytile-read-query-layout

• Scope: Session

• Dynamic: Yes

• Data Type: enum

• Default Value: unordered

dimensions_are_primary_keys

• Description: Should dimensions be treated and registered as primary keys.

• Command line: --mytile-dimensions-are-primary-keys

• Scope: Session

• Dynamic: Yes

• Data Type: boolean

• Default Value: true

enable_pushdown

• Description: Pushdown predicates where possible

• Command line: --mytile-enable-pushdown

• Scope: Session

• Dynamic: Yes

• Data Type: boolean

• Default Value: true

compute_table_records

• Description: Compute size of table (record count) on table opening. This is useful when you are performing a large join or group by and knowing the table size can affect the optimizer. Computing the table records can take several seconds (worst case) so this should not be enabled for small queries.

• Command line: --mytile-compute-table-records

• Scope: Session

• Dynamic: Yes

• Data Type: boolean

• Default Value: false

TileDB integrates with through a pluggable storage engine called MyTile, which can be found in the repo. MyTile is written to take advantage of query predicate pushdown to TileDB. MyTile allows a user to dynamically access any TileDB array, whether it is local to the machine, on S3 or on . This provides a powerful C++ based SQL engine for TileDB arrays.

A docker image is provided to allow for quick testing of the MyTile storage engine. The docker image starts a MariaDB server and connects to it from the shell for you.

Prerequisites

MyTile requires TileDB to be installed on the system (see ). You can install all the other dependencies as follows.

Depending on your platform there may be other dependencies needed. See for details.

Compiling

When compiling from source, you will need to fetch the MariaDB server and then build MariaDB with the MyTile source included. You must build MariaDB and MyTile together in one source tree, as MariaDB requires that all compilation flags and optimizations be the same between the server and the plugins.

For simplicity the cmake command below disables many of the optional storage engines to reduce the build size. You can toggle each storage engine as you see fit.

The CMAKE_INSTALL_PREFIX can also be set to switch locations of the installation. By default, we set it to $HOME/mytile_server

After the compilation has completed, in $HOME/mytile_server you will have MariaDB 10.4 with a working MyTile plugin.

Configuring

By default optional plugins are not loaded. You will need to edit the MariaDB configuration to load the plugin. Full configuration of MariaDB is beyond our scope as there is a plethora of options outside of MyTile settings.

Edit your ~/.my.cnf, under the [mysqld] section add plugin-load-add=mytileand plugin-maturity=experimental . If you do not /.my.cnf file, a sample one is provided below:

Running

Now that you have installed and configured MariaDB, you can run the server. First you need to run MariaDB's setup script to initialize the server:

After the database has been initialized, in a terminal start the server with:

This will launch the server in your terminal. After the server is successfully started, you can connect with: