Array Access
For reads, writes, embedded SQL, any integration and any API, you can simply use TileDB Embedded with only two changes:
- Set the TileDB configuration parameters
rest.usernameandrest.passwordwith your TileDB Cloud username and password, or alternativelyrest.tokenwith the API token you created. - Every array registered with TileDB Cloud must be accessed using a URI of the form
tiledb://<namespace>/<array-name>, where<namespace>is the user or organization who owns the array and<array-name>is the array name set by the owner upon array registration. This URI is displayed on the console when viewing the array details.
Accessing arrays by setting an API token is typically faster than using your username and password.
Here are some Python/R/Java examples, although the above changes will work with any TileDB API or integration:
Python
R
Java
import tiledb, tiledb.sql
import pandas
# Create the configuration parameters
config = tiledb.Config()
config["rest.username"] = "xxx"
config["rest.password"] = "yyy"
# or, more preferably, config["rest.token"] = "ttt"
# This is the array URI format in TileDB Cloud
array_name = "tiledb://TileDB-Inc/quickstart_sparse"
# Write code exactly as in TileDB Developer
with tiledb.open(array_name, 'r', ctx=tiledb.Ctx(config)) as A:
print (A.df[:])
# A helper function tiledb.cloud.Ctx() exists to create a context
# automatically based on a previous call to tiledb.cloud.login()
with tiledb.open(array_name, 'r', ctx=tiledb.cloud.Ctx()) as A:
print (A.df[:])
# Using embedded SQL, you need to pass the username/password or token
# as config parameters in `init_command`
db = tiledb.sql.connect(db="test",
init_command="set mytile_tiledb_config='rest.username=xxx,rest.password=yyy'")
pandas.read_sql(sql="select * from `tiledb://TileDB-Inc/quickstart_sparse`", con=db)
library(tiledb)
# Create the configuration parameters
config <- tiledb_config()
config["rest.username"] <- "xxx"
config["rest.password"] <- "yyy"
# or, more preferably, config["rest.token"] <- "ttt"
ctx <- tiledb_ctx(config)
# This is the array URI format in TileDB Cloud
array_name <- "tiledb://TileDB-Inc/quickstart_sparse"
# Write code exactly as in TileDB Developer
arr <- tiledb_array(array_name, query_type="READ", ctx=ctx, return_as='data.frame')
df <- arr[]
str(df)
//set the config
Config config = new Config();
config.set("rest.username", "xxx");
config.set("rest.password", "yyy");
Context ctx = new Context(config);
//open the array
Array array = new Array(ctx, "tiledb://TileDB-Inc/quickstart_sparse");
//print the array schema
System.out.println(array.getSchema());
//this will print:
//ArraySchema<TILEDB_SPARSE [email protected] Attr<a,TILEDB_UINT32,1>>
You can create an array inside or outside TileDB Cloud. The benefit of creating an array with TileDB Cloud is that it will be logged for auditing purposes. Moreover, it will be registered automatically with your account upon creation.
To instruct TileDB Embedded that you are creating an array through the TileDB Cloud service, you just need a single change:
- Instead of using
<array-uri>as you would typically in TileDB Embedded, you must usetiledb://<username>/<array-uri>. For example, if you wish to create an array ats3://my_bucket/my_array, you need to set the array URI totiledb://my_username/s3://my_bucket/my_array.
Python
R
Java
import tiledb
import numpy as np
#####################
# Define the Schema #
#####################
# The array will be 4x4 with dimensions "rows" and "cols", with domain [1,4].
dom = tiledb.Domain(
tiledb.Dim(name="rows", domain=(1, 4), tile=4, dtype=np.int32),
tiledb.Dim(name="cols", domain=(1, 4), tile=4, dtype=np.int32),
)
# The array will be sparse with a single attribute "a" so each (i,j) cell can store an integer.
schema = tiledb.ArraySchema(
domain=dom, sparse=True, attrs=[tiledb.Attr(name="a", dtype=np.int32)]
)
####################
# Create the Array #
####################
array_uri = "tiledb://my_username/s3://my_bucket/my_array"
# Create the (empty) array on disk.
tiledb.SparseArray.create(array_uri, schema)
library(tiledb)
#####################
# Define the Schema #
#####################
dom <- tiledb_domain(dims = c(
tiledb_dim("rows", c(1L, 4L), type = "INT32"),
tiledb_dim("cols", c(1L, 4L), type = "INT32")
))
attrs <- c(
tiledb_attr("a", type = "INT32")
)
schema <- tiledb_array_schema(domain=dom, sparse=TRUE, attrs=attrs)
####################
# Create the Array #
####################
array_uri <- "tiledb://my_username/s3://my_bucket/my_array"
# Create the (empty) array on disk.
tiledb_array_create(array_uri, schema)
// Create getDimensions
Dimension<Long> d1 =
new Dimension<Long>(ctx, "d1", Long.class, new Pair<Long, Long>(1l, 4l), 2l);
Dimension<Long> d2 =
new Dimension<Long>(ctx, "d2", Long.class, new Pair<Long, Long>(1l, 4l), 2l);
// Create getDomain
Domain domain = new Domain(ctx);
domain.addDimension(d1);
domain.addDimension(d2);
// Create and add getAttributes
Attribute a1 = new Attribute(ctx, "a1", Integer.class);
// Define the schema
ArraySchema schema = new ArraySchema(ctx, TILEDB_SPARSE);
schema.setTileOrder(TILEDB_ROW_MAJOR);
schema.setCellOrder(TILEDB_ROW_MAJOR);
schema.setCapacity(2);
schema.setDomain(domain);
schema.addAttribute(a1);
// Check the array schema
schema.check();
String arrayURI = "tiledb://my_username/s3://my_bucket/my_array"
Array.create(arrayURI, schema);
It is possible to programmatically register an existing array. To do that you will need to use one of our cloud clients. See: Installation
Python
R
Java
tiledb.cloud.array.register_array(
uri="s3://my_bucket/my_array",
namespace="my_organization",
array_name="my_array",
description=None, # Optional string for markdown description
access_credentials_name=None, # Optional access credential name. Use this to specify a credential that is not the namespace default
)
tiledbcloud::register_array(
uri="s3://my_bucket/my_array",
namespace="my_organization",
array_name="my_array",
description="my array", # Optional string for markdown description
access_credentials_name="my creds" # Optional access credential name. Use this to specify a credential that is not the namespace default
)
String namespace = "<NAMESPACE>"; // String | namespace array is in (an organization name or user's username)
String array = "s3://<S3_BUCKET>/<ARRAY_NAME>"; // String | name/uri of s3 array that is url-encoded
ArrayInfoUpdate arrayMetadata = new ArrayInfoUpdate(); // ArrayInfoUpdate | metadata associated with array
arrayMetadata.setUri("s3://<S3_BUCKET>/<ARRAY_NAME>");
arrayMetadata.setName("<ARRAY_NAME>");
try {
ArrayInfo result = arrayApi.registerArray(namespace, array, arrayMetadata);
System.out.println(result);
} catch (ApiException e) {
System.err.println("Exception when calling ArrayApi#registerArray");
System.err.println("Status code: " + e.getCode());
System.err.println("Reason: " + e.getResponseBody());
System.err.println("Response headers: " + e.getResponseHeaders());
e.printStackTrace();
}
Last modified 11mo ago