Creating Attributes

Creating an attribute requires specifying a (optional) name and a (required) datatype. In the example below we create an int32 attribute called attr.

C
C++
Python
R
Java
Go
C
#include <tiledb/tiledb.h>
// Create context
tiledb_ctx_t* ctx;
tiledb_ctx_alloc(NULL, &ctx);
// Create attribute
tiledb_attribute_t* attr;
tiledb_attribute_alloc(ctx, "attr", TILEDB_INT32, &attr);
// ...
// Make sure to free the attribute and context
tiledb_attribute_free(&attr);
tiledb_ctx_free(&ctx);
C++
#include <tiledb/tiledb>
// Create context
Context ctx;
// Create attribute
auto attr = Attribute::create<int32>(ctx, "attr")
// Or alternatively:
Attribute attr(ctx, "attr", TILEDB_INT32);
Python
import tiledb
import numpy as np
attr = tiledb.Attr(name="attr", dtype=np.int32)
R
# Create attribute
attr <- tiledb_attr("attr", "INT32")
Java
// Create context
try(Context ctx = new Context()) {
// Create attribute
Attribute attr = new Attribute(ctx, "attr", TILEDB_INT32);
}
Go
import tiledb "github.com/TileDB/TileDB-Go"
// Create context
ctx, _ := tiledb.NewContext(nil)
// Create attribute
attr, _ := tiledb.NewAttribute(ctx, "attr", tiledb.TILEDB_INT32)

An attribute can also store a fixed number of values (of the same datatype) in a single cell, or a variable number of values. You can specify this as follows:

C
C++
Python
R
Java
Go
C
// Setting a fixed number of values
tiledb_attribute_set_cell_val_num(ctx, attr, 3);
// Setting a variable number of values
tiledb_attribute_set_cell_val_num(ctx, attr, TILEDB_VAR_NUM);
C++
// Setting a fixed number of values
attr.set_cell_val_num(3);
// Setting a variable number of values
attr.set_cell_val_num(TILEDB_VAR_NUM);
Python
# Setting a fixed number of values
attr = tiledb.Attr(name="attr", dtype=np.dtype('i4, i4, i4'))
# Setting a variable number of values
attr = tiledb.Attr(name="attr", dtype=np.int32, var=True)
R
# NOTE: Currently the R API supports only single-value attributes
Java
// Setting a fixed number of values
attr.setCellValNum(3);
// Setting a variable number of values
attr.setCellValNum(TILEDB_VAR_NUM);
Go
// Setting a fixed number of values
attr.SetCellValNum(3)
// Setting a variable number of values
attr.SetCellValNum(tiledb.TILEDB_VAR_NUM)

Supported Attribute Datatypes:

C/C++/Java/Go
Python
R
C/C++/Java/Go

Datatype

Description

TILEDB_INT8

8-bit integer

TILEDB_UINT8

8-bit unsigned integer

TILEDB_INT16

16-bit integer

TILEDB_UINT16

16-bit unsigned integer

TILEDB_INT32

32-bit integer

TILEDB_UINT32

32-bit unsigned integer

TILEDB_INT64

64-bit integer

TILEDB_UINT64

64-bit unsigned integer

TILEDB_FLOAT32

32-bit floating point

TILEDB_FLOAT64

64-bit floating point

TILEDB_DATETIME_YEAR

Years

TILEDB_DATETIME_MONTH

Months

TILEDB_DATETIME_WEEK

Weeks

TILEDB_DATETIME_DAY

Days

TILEDB_DATETIME_HR

Hours

TILEDB_DATETIME_MIN

Minutes

TILEDB_DATETIME_SEC

Seconds

TILEDB_DATETIME_MS

Milliseconds

TILEDB_DATETIME_US

Microseconds

TILEDB_DATETIME_NS

Nanoseconds

TILEDB_DATETIME_PS

Picoseconds

TILEDB_DATETIME_FS

Femtoseconds

TILEDB_DATETIME_AS

Attoseconds

TILEDB_DATETIME_CHAR

Single character

TILEDB_STRING_ASCII

ASCII string

TILEDB_STRING_UTF8

UTF-8 string

TILEDB_STRING_UTF16

UTF-16 string

TILEDB_STRING_UTF32

UTF-32 string

TILEDB_STRING_UCS2

UCS-2 string

TILEDB_STRING_UCS4

UCS-4 string

TILEDB_ANY

(datatype, bytelength, value)

Python

Datatype

Description

np.int8

8-bit integer

np.uint8

8-bit unsigned integer

np.int16

16-bit integer

np.uint16

16-bit unsigned integer

np.int32

32-bit integer

np.uint32

32-bit unsigned integer

np.int64

64-bit integer

np.uint64

64-bit unsigned integer

np.float32

32-bit floating point

np.float64

64-bit floating point

np.dtype('S1')

Character

np.dtype('U1')

Unicode UTF-8

"datetime64[Y]"

Years

"datetime64[M]"

Months

"datetime64[W]"

Weeks

"datetime64[D]"

Days

"datetime64[h]"

Hours

"datetime64[m]"

Minutes

"datetime64[s]"

Seconds

"datetime64[ms]"

Milliseconds

"datetime64[us]"

Microseconds

"datetime64[ns]"

Nanoseconds

"datetime64[ps]"

Picoseconds

"datetime64[fs]"

Femtoseconds

"datetime64[as]"

Attoseconds

R

Datatype

Description

"CHAR"

Single character

"INT8"

8-bit integer

"UINT8"

8-bit unsigned integer

"INT16"

16-bit integer

"UINT16"

16-bit unsigned integer

"INT32"

32-bit integer

"UINT32"

32-bit unsigned integer

"INT64"

64-bit integer

"UINT64"

64-bit unsigned integer

"FLOAT32"

32-bit floating point

"FLOAT64"

64-bit floating point

Setting a Compressor

The following example shows how to set a GZIP compressor to an attribute, with compression level 10.

C
C++
Python
R
Java
Go
C
#include <tiledb/tiledb.h>
// Create context
tiledb_ctx_t* ctx;
tiledb_ctx_alloc(nullptr, &ctx);
// Create compressor as a filter
tiledb_filter_t* filter;
tiledb_filter_alloc(ctx, TILEDB_FILTER_GZIP, &filter);
int level = 10;
tiledb_filter_set_option(ctx, filter, TILEDB_COMPRESSION_LEVEL, &level);
// Create filter list
tiledb_filter_list_t* filter_list;
tiledb_filter_list_alloc(ctx, &filter_list);
// Add compressor to filter list
tiledb_filter_list_add_filter(ctx, filter_list, filter);
// Create attribute
tiledb_attribute_t* attr;
tiledb_attribute_alloc(ctx, "attr", TILEDB_INT32, &attr);
// Set filter list to attribute
tiledb_attribute_set_filter_list(ctx, attr, filter_list);
// Clean up
tiledb_attribute_free(&attr);
tiledb_filter_free(&filter);
tiledb_filter_list_free(&filter_list);
tiledb_ctx_free(&ctx);
C++
#include <tiledb/tiledb>
// Create context
Context ctx;
// Create compressor as a filter
Filter filter(ctx, TILEDB_FILTER_GZIP);
filter.set_option(ctx, TILEDB_COMPRESSION_LEVEL, 10);
// Create filter list
FilterList filter_list(ctx);
// Add compressor to filter list
filter_list.add_filter(filter)
// Create attribute
Attribute attr(ctx, "attr", TILEDB_INT32);
// Set filter list to attribute
attr.set_filter_list(filter_list);
Python
import tiledb
import numpy as np
attr = tiledb.Attr(name="attr", dtype=np.int32,
filters=tiledb.FilterList([tiledb.GzipFilter(level=10)]))
R
# Create a compressor as a filter
comp <- tiledb_filter("GZIP")
tiledb_filter_set_option(comp,"COMPRESSION_LEVEL", 10)
# Create a filter list with the compressor
filter_list <- tiledb_filter_list(c(comp))
# Create attribute with the filter list
attr <- tiledb_attr("attr", "INT32", filter_list = filter_list)
Java
// Create context
Context ctx = new Context();
// Create filter list
FilterList filter_list = new FilterList(ctx);
// Add compressor as filter to the filter list
filter_list.add(new GzipFilter(ctx, 10));
// Create attribute
Attribute attr = new Attribute(ctx, "attr", TILEDB_INT32);
// Set filter list to attribute
attr.setFilterList(fl);
Go
import tiledb "github.com/TileDB/TileDB-Go"
// Create context
ctx, _ := tiledb.NewContext(nil)
// Create a compressor as a filter
filter, _ := tiledb.NewFilter(ctx, tiledb.TILEDB_FILTER_GZIP)
filter.SetOption(tiledb.TILEDB_COMPRESSION_LEVEL, 10)
// Create filter list
filter_list, _ := tiledb.NewFilterList(ctx)
// Add compressor to filter list
filter_list.AddFilter(filter)
// Create attribute
attr, _ := tiledb.NewAttribute(ctx, "attr", tiledb.TILEDB_INT32)
// Set filter list to attribute
attr.SetFilterList(filter_list)

Supported attribute compressors:

C/C++/Go
Python
R
Java
C/C++/Go

Compressor

Description

Option

TILEDB_FILTER_GZIP

GZIP

TILEDB_COMPRESSION_LEVEL (int)

TILEDB_FILTER_ZSTD

ZStandard

TILEDB_COMPRESSION_LEVEL (int)

TILEDB_FILTER_LZ4

LZ4

TILEDB_COMPRESSION_LEVEL (int)

TILEDB_FILTER_RLE

RLE

TILEDB_COMPRESSION_LEVEL (int)

TILEDB_FILTER_BZIP2

BZIP2

TILEDB_COMPRESSION_LEVEL (int)

TILEDB_FILTER_DOUBLE_DELTA

Double Delta

None

Python

Compressor

Description

Option

tiledb.GzipFilter()

GZIP

level (np.int32)

tiledb.ZstdFilter()

ZStandard

level (np.int32)

tiledb.LZ4Filter()

LZ4

level (np.int32)

tiledb.RleFilter()

Run-length Encoding

level (np.int32)

tiledb.Bzip2Filter()

BZIP2

level (np.int32)

tiledb.DoubleDeltaFilter()

Double Delta

None

R

Compressor

Description

Option

"GZIP"

GZIP

"COMPRESSION_LEVEL" (int32)

"ZSTD"

ZStandard

"COMPRESSION_LEVEL" (int32)

"LZ4"

LZ4

"COMPRESSION_LEVEL" (int32)

"RLE"

Run-length Encoding

"COMPRESSION_LEVEL" (int32)

"BZIP2"

BZIP2

"COMPRESSION_LEVEL" (int32)

"DOUBLE_DELTA"

Double Delta

None

Java

Compressor

Description

Option

GzipFilter

GZIP

Level (int)

ZstdFilter

ZStandard

Level (int)

LZ4Filter

LZ4

Level (int)

RleFilter

Run-length encoding

Level (int)

Bzip2Filter

BZIP2

Level (int)

DoubleDeltaFilter

Double delta

None

Setting Other Filters

You can set a filter list to an attribute as follows:

C
C++
Python
R
Java
Go
C
#include <tiledb/tiledb.h>
// Create context
tiledb_ctx_t* ctx;
tiledb_ctx_alloc(nullptr, &ctx);
// Create filters
tiledb_filter_t *f1, *f2;
tiledb_filter_alloc(ctx, TILEDB_FILTER_BITWIDTH, &f1);
tiledb_filter_alloc(ctx, TILEDB_FILTER_ZSTD, &f2);
// Create filter list
tiledb_filter_list_t* filter_list;
tiledb_filter_list_alloc(ctx, &filter_list);
// Add filters to filter list
tiledb_filter_list_add_filter(ctx, filter_list, f1);
tiledb_filter_list_add_filter(ctx, filter_list, f2);
// Create attribute
tiledb_attribute_t* attr;
tiledb_attribute_alloc(ctx, "attr", TILEDB_INT32, &attr);
// Set filter list to attribute
tiledb_attribute_set_filter_list(ctx, attr, filter_list);
// Clean up
tiledb_attribute_free(&attr);
tiledb_filter_free(&f1);
tiledb_filter_free(&f2);
tiledb_filter_list_free(&filter_list);
tiledb_ctx_free(&ctx);
C++
#include <tiledb/tiledb>
// Create context
Context ctx;
// Create filters
Filter f1(ctx, TILEDB_FILTER_BITWIDTH);
Filter f2(ctx, TILEDB_FILTER_ZSTD);
// Create filter list
FilterList filter_list(ctx);
// Add filters to filter list
filter_list.add_filter(f1)
.add_filter(f2)
// Create attribute
Attribute attr(ctx, "attr", TILEDB_INT32);
// Set filter list to attribute
attr.set_filter_list(filter_list);
Python
import tiledb
import numpy as np
filter_list = tiledb.FilterList([tiledb.BitWidthReductionFilter(), tiledb.ZstdFilter()])
attr = tiledb.Attr(name="attr", dtype=np.int32, filters=filter_list)
R
# Create filters
f1 <- tiledb_filter("BIT_WIDTH_REDUCTION")
f2 <- tiledb_filter("ZSTD")
# Create a filter list with the two filters
filter_list <- tiledb_filter_list(c(f1,f2))
# Create attribute with the filter list
attr <- tiledb_attr("attr", "INT32", filter_list = filter_list)
Java
// Create context
Context ctx = new Context();
// Create filter list
FilterList filter_list = new FilterList(ctx);
// Add filters to the filter list
filter_list.add(new BitWidthReductionFilter(ctx));
filter_list.add(new ZstdFilter(ctx));
// Create attribute
Attribute attr = new Attribute(ctx, "attr", TILEDB_INT32);
// Set filter list to attribute
attr.setFilterList(fl);
Go
import tiledb "github.com/TileDB/TileDB-Go"
// Create context
ctx, _ := tiledb.NewContext(nil)
// Create filters
f1, _ := tiledb.NewFilter(ctx, tiledb.TILEDB_BIT_WIDTH_REDUCTION)
f2, _ := tiledb.NewFilter(ctx, tiledb.TILEDB_ZSTD)
// Create filter list
filter_list, _ := tiledb.NewFilterList(ctx)
// Add filters to filter list
filter_list.AddFilter(f1)
.AddFilter(f2)
// Create attribute
attr, _ := tiledb.NewAttribute(ctx, "attr", tiledb.TILEDB_INT32)
// Set filter list to attribute
attr.SetFilterList(filter_list)

Supported attribute filters (beyond compressors):

C/C++/Go
Python
R
Java
C/C++/Go

Filter

Description

Option

TILEDB_BIT_WIDTH_REDUCTION

Bit width reduction

TILEDB_BIT_WIDTH_MAX_WINDOW (uint32)

TILEDB_BITSHUFFLE

Bit shuffle

None

TILEDB_BYTESHUFFLE

Byte shuffle

None

TILEDB_POSITIVE_DELTA

Positive Delta

TILEDB_POSITIVE_DELTA_MAX_WINDOW (uint32)

Python

Filter

Description

Option

tiledb.BitWidthReductionFilter()

Bit width reduction

window (np.uint32)

tiledb.BitShuffleFilter()

Bit shuffle

None

tiledb.ByteShuffleFilter()

Byte shuffle

None

tiledb.PositiveDeltaFilter()

Positive delta

window (np.uint32)

R

Filter

Description

Option

BIT_WIDTH_REDUCTION

Bit width reduction

BIT_WIDTH_MAX_WINDOW (int)

BITSHUFFLE

Bit shuffle

None

BYTESHUFFLE

Byte shuffle

None

POSITIVE_DELTA

Positive Delta

POSITIVE_DELTA_MAX_WINDOW (int)

Java

Filter

Description

Option

BitWidthReductionFilter

Bit width reduction

window (int)

BitShuffleFilter

Bit shuffle

None

ByteShuffleFilter

Byte shuffle

None

PositiveDeltaFilter

Positive Delta

window (int)

Setting the Tile Chunk Size

The default tile chunk size used by TileDB is 64KB, which is the size of many common processor L1 caches. You can control the chunk size by changing the option on a filter list:

C
C++
Python
R
Java
Go
C
// ... create context and filter list
tiledb_filter_list_set_max_chunk_size(ctx, filter_list, 10000);
C++
// ... create filter list
filter_list.set_max_chunk_size(10000)
Python
import tiledb
import numpy as np
filter_list = tiledb.FilterList([tiledb.GzipFilter()], chunksize=10000)
R
# ... create filter list
set_max_chunk_size(filter_list, 10000)
Java
// ... create filter list
filter_list.setMaxChunkSize(10000)
Go
// ... create filter list
filter_list.SetMaxChunkSize(10000)