Array Access

For reads, writes, embedded SQL, any integration, and any API, you can use TileDB Open Source with only two changes:

• Set the TileDB configuration parameters rest.username and rest.passwordwith your TileDB Cloud username and password, or alternatively rest.tokenwith the API token you created.

• Every array registered with TileDB Cloud must be accessed using a URI of the form tiledb://<namespace>/<array-name>, where<namespace> is the user or organization who owns the array and <array-name> is the array name set by the owner upon array registration. This URI is displayed on the console when viewing the array details.

Accessing arrays by setting an API token is typically faster than using your username and password.

Here are some Python/R/Java examples, although the above changes will work with any TileDB API or integration:

Python

import tiledb, tiledb.sql
import pandas

# Create the configuration parameters
config = tiledb.Config()
config["rest.username"] = "xxx"
config["rest.password"] = "yyy"
# or, more preferably, config["rest.token"] = "ttt"

# This is the array URI format in TileDB Cloud
array_name = "tiledb://TileDB-Inc/quickstart_sparse"

# Write code exactly as in TileDB Developer
with tiledb.open(array_name, 'r', ctx=tiledb.Ctx(config)) as A:
    print (A.df[:])

# A helper function tiledb.cloud.Ctx() exists to create a context
# automatically based on a previous call to tiledb.cloud.login()
with tiledb.open(array_name, 'r', ctx=tiledb.cloud.Ctx()) as A:
    print (A.df[:])
    
# Using embedded SQL, you need to pass the username/password or token
# as config parameters in `init_command`
db = tiledb.sql.connect(db="test",
        init_command="set mytile_tiledb_config='rest.username=xxx,rest.password=yyy'")
pandas.read_sql(sql="select * from `tiledb://TileDB-Inc/quickstart_sparse`", con=db)

R

library(tiledb)

# Create the configuration parameters
config <- tiledb_config()
config["rest.username"] <- "xxx"
config["rest.password"] <- "yyy"
# or, more preferably, config["rest.token"] <- "ttt"
ctx <- tiledb_ctx(config)

# This is the array URI format in TileDB Cloud
array_name <- "tiledb://TileDB-Inc/quickstart_sparse"

# Write code exactly as in TileDB Developer
arr <- tiledb_array(array_name, query_type="READ", ctx=ctx, return_as='data.frame')
df <- arr[]
str(df)

Java

//set the config
Config config = new Config();
config.set("rest.username", "xxx");
config.set("rest.password", "yyy");
Context ctx = new Context(config);

//open the array
Array array = new Array(ctx, "tiledb://TileDB-Inc/quickstart_sparse");

//print the array schema
System.out.println(array.getSchema());

//this will print:
//ArraySchema<TILEDB_SPARSE io.tiledb.java.api.Domain@51462713 Attr<a,TILEDB_UINT32,1>>

You can create an array inside or outside TileDB Cloud. The benefit of creating an array with TileDB Cloud is that it will be logged for auditing purposes. Moreover, it will be registered automatically with your account upon creation.

Create New Arrays

To instruct TileDB Open Source that you are creating an array through the TileDB Cloud service, you just need a single change:

• Instead of using <array-uri> as you would typically in TileDB Open Source, you must use tiledb://<username>/<array-uri>. For example, if you wish to create an array at s3://my_bucket/my_array, you need to set the array URI to tiledb://my_username/s3://my_bucket/my_array.

Python

import tiledb
import numpy as np

#####################
# Define the Schema #
#####################
# The array will be 4x4 with dimensions "rows" and "cols", with domain [1,4].
dom = tiledb.Domain(
    tiledb.Dim(name="rows", domain=(1, 4), tile=4, dtype=np.int32),
    tiledb.Dim(name="cols", domain=(1, 4), tile=4, dtype=np.int32),
)

# The array will be sparse with a single attribute "a" so each (i,j) cell can store an integer.
schema = tiledb.ArraySchema(
    domain=dom, sparse=True, attrs=[tiledb.Attr(name="a", dtype=np.int32)]
)

####################
# Create the Array #
####################
array_uri = "tiledb://my_username/s3://my_bucket/my_array"
# Create the (empty) array on disk.
tiledb.SparseArray.create(array_uri, schema)

R

library(tiledb)

#####################
# Define the Schema #
#####################
dom <- tiledb_domain(dims = c(
  tiledb_dim("rows", c(1L, 4L), type = "INT32"),
  tiledb_dim("cols", c(1L, 4L), type = "INT32")
))
attrs <- c(
  tiledb_attr("a",  type = "INT32")
)
schema <- tiledb_array_schema(domain=dom, sparse=TRUE, attrs=attrs)

####################
# Create the Array #
####################
array_uri <- "tiledb://my_username/s3://my_bucket/my_array"
# Create the (empty) array on disk.
tiledb_array_create(array_uri, schema)

Java

// Create getDimensions
Dimension<Long> d1 =
new Dimension<Long>(ctx, "d1", Long.class, new Pair<Long, Long>(1l, 4l), 2l);
Dimension<Long> d2 =
    new Dimension<Long>(ctx, "d2", Long.class, new Pair<Long, Long>(1l, 4l), 2l);
// Create getDomain
Domain domain = new Domain(ctx);
domain.addDimension(d1);
domain.addDimension(d2);

// Create and add getAttributes
Attribute a1 = new Attribute(ctx, "a1", Integer.class);

// Define the schema
ArraySchema schema = new ArraySchema(ctx, TILEDB_SPARSE);
schema.setTileOrder(TILEDB_ROW_MAJOR);
schema.setCellOrder(TILEDB_ROW_MAJOR);
schema.setCapacity(2);
schema.setDomain(domain);
schema.addAttribute(a1);

// Check the array schema
schema.check();
String arrayURI = "tiledb://my_username/s3://my_bucket/my_array"
Array.create(arrayURI, schema);

Register Existing Array

It is possible to programmatically register an existing array. To do that you will need to use one of our cloud clients. See: Installation

Python

tiledb.cloud.array.register_array(
    uri="s3://my_bucket/my_array",
    namespace="my_organization",
    array_name="my_array",
    description=None, # Optional string for markdown description
    access_credentials_name=None, # Optional access credential name. Use this to specify a credential that is not the namespace default
)

R

tiledbcloud::register_array(
    uri="s3://my_bucket/my_array",
    namespace="my_organization",
    array_name="my_array",
    description="my array", # Optional string for markdown description
    access_credentials_name="my creds" # Optional access credential name. Use this to specify a credential that is not the namespace default
)

Java

String namespace = "<NAMESPACE>"; // String | namespace array is in (an organization name or user's username)
String array = "s3://<S3_BUCKET>/<ARRAY_NAME>"; // String | name/uri of s3 array that is url-encoded
ArrayInfoUpdate arrayMetadata = new ArrayInfoUpdate(); // ArrayInfoUpdate | metadata associated with array
arrayMetadata.setUri("s3://<S3_BUCKET>/<ARRAY_NAME>");
arrayMetadata.setName("<ARRAY_NAME>");
try {
    ArrayInfo result = arrayApi.registerArray(namespace, array, arrayMetadata);
    System.out.println(result);
} catch (ApiException e) {
    System.err.println("Exception when calling ArrayApi#registerArray");
    System.err.println("Status code: " + e.getCode());
    System.err.println("Reason: " + e.getResponseBody());
    System.err.println("Response headers: " + e.getResponseHeaders());
    e.printStackTrace();
}