MinIO
MinIO is a lightweight S3-compliant object-store. Although it has many nice features, here we focus only on local deployment, which is very useful if you wish to quickly test your TileDB-S3 programs locally. See the MinIO quickstart guide for installation instructions. Here is what we do to run MinIO on port 9999:
$ mkdir -p /tmp/minio-data
$ docker run -e MINIO_ACCESS_KEY=minio -e MINIO_SECRET_KEY=miniosecretkey \
-d -p 9999:9000 minio/minio server /tmp/minio-data
$ export AWS_ACCESS_KEY_ID=minio
$ export AWS_SECRET_ACCESS_KEY=miniosecretkey
Once you get MinIO server running, you need to set the S3 configurations as follows (below, <port> stands for the port on which you are running the MinIO server, equal to 9999 if you run the MinIO docker as shown above):
Parameter
Value
"vfs.s3.scheme"
"http"
"vfs.s3.region"
""
"vfs.s3.endpoint_override"
"localhost:<port>"
"vfs.s3.use_virtual_addressing"
"false"
The Configuration page explains in detail how to set configuration parameters in TileDB. Below is a quick example for the minio specific parameters.
C
C++
Python
R
Java
Go
// Create a configuration object
tiledb_config_t *config;
tiledb_config_alloc(&config, NULL);
// Set configuration parameters
tiledb_config_set(config, "vfs.s3.scheme", "http", &error);
tiledb_config_set(config, "vfs.s3.region", "", &error);
tiledb_config_set(config, "vfs.s3.endpoint_override", "<minio address>", &error);
tiledb_config_set(config, "vfs.s3.use_virtual_addressing", "false", &error);
// Create contex
tiledb_ctx_t* ctx;
tiledb_ctx_alloc(&config, &ctx);
// Pass CTX to read/write of array
// Clean up
tiledb_config_free(&config);
tiledb_ctx_free(&ctx);
// Create a configuration object
Config config;
// Set a configuration parameter
config["vfs.s3.scheme"] = "http";
config["vfs.s3.region"] = "";
config["vfs.s3.endpoint_override"] = "<minio address>";
config["vfs.s3.use_virtual_addressing"] = "false";
// Create contex
Ctx ctx(config);
// Pass CTX to read/write of array
# Create a configuration object
config = tiledb.Config()
# Set configuration parameters
config["vfs.s3.scheme"] = "http"
config["vfs.s3.region"] = ""
config["vfs.s3.endpoint_override"] = "<minio address>"
config["vfs.s3.use_virtual_addressing"] = "false"
# Create contex
ctx = tiledb.Ctx(config)
# Pass CTX to read/write of array
with tiledb.open(array_uri, ctx=ctx) as A:
# Read or write and context with minio is applied
# Create a configuration object
config <- tiledb_config()
# Set a configuration parameter
config["vfs.s3.scheme"] = "http"
config["vfs.s3.region"] = ""
config["vfs.s3.endpoint_override"] = "<minio address>"
config["vfs.s3.use_virtual_addressing"] = "false"
ctx <- tiledb_ctx(config)
# Pass CTX to read/write of array
try(Config config = new Config()) {
// Set configuration parameters
config.set("vfs.s3.scheme", "http");
config.set("vfs.s3.region", "");
config.set("vfs.s3.endpoint_override", "<minio address>");
config.set("vfs.s3.use_virtual_addressing", "false");
// set config on context
try(Context ctx = new Context(config)) {
// Pass CTX to read/write of array
}
}
// Create a configuration object
config, _ := tiledb.NewConfig()
// Set configuration parameters
config.Set("vfs.s3.scheme", "http")
config.Set("vfs.s3.region", "")
config.Set("vfs.s3.endpoint_override", "<minio address>")
config.Set("vfs.s3.use_virtual_addressing", "false")
// set config on context
ctx, _ := tiledb.NewCtx(&config)
// Pass CTX to read/write of array
// Clean up
config.Free()
ctx.Free()
Export as PDF
Copy link