Consolidating Arrays

You can consolidate an array as follows. See Consolidation for how to fine-tune the consolidation algorithm with configuration parameters, and Configuration for more details on creating configuration objects.

C
C++
Python
R
Java
Go
C
// Create TileDB context
tiledb_ctx_t* ctx;
tiledb_ctx_alloc(NULL, &ctx);
// Consolidate, using ctx's configuration
tiledb_array_consolidate(ctx, array_name, NULL);
// Alteratively, you can create and pass a configuration object
tiledb_config_t* config;
tiledb_config_alloc(&config, NULL);
tiledb_config_set(config, "sm.consolidation.steps", "3", &error);
tiledb_array_consolidate(ctx, array_name, config);
// Clean-up
tiledb_ctx_free(&ctx);
tiledb_config_free(&config);
C++
Context ctx;
// Consolidate, using ctx's configuration
Array::consolidate(ctx, "<array-uri>");
// Alteratively, you can create and pass a configuration object
Config config;
config["sm.consolidation.steps"] = "3";
Array::consolidate(ctx, "<array-uri>", config);
Python
# Consolidate array at array_uri
tiledb.consolidate(array_uri)
# Alteratively, you can create and pass a configuration object
config = tiledb.Config({"sm.consolidation.steps": 3})
ctx = tiledb.Ctx(config)
tiledb.consolidate(array_uri, ctx=ctx)
R
# TODO: Consolidation is currently not supported in R
Java
try(Context ctx = new Context() {
// Consolidate, using ctx's configuration
consolidate(ctx, "<array-uri>");
// Alteratively, you can create and pass a configuration object
Config config = new Config();
config.set("sm.consolidation.steps") = "3";
consolidate(ctx, "<array-uri>", config);
}
Go
// Create configuration
config, _ := NewConfig()
// Test context with config
ctx, _ := NewContext(config)
// Consolidate, using ctx's configuration
config.Set("sm.consolidation.steps", "3")
array.Consolidate(ctx, config)

If you do not set a configuration object to the consolidation function, the consolidation algorithm will inherit the (default or set) configuration of the context. Otherwise, the set options in the passed configuration object will override those of the context's, but the rest of the options will still be inherited from the context's configuration.

Post-consolidation vacuuming

After consolidation, you may optionally remove the consolidated fragments with the "vacuum" step. This step is controlled by the sm.vacuum.mode Configuration Parameter (fragments, fragment_meta, array_meta). Note that vacuuming must not be run if you intend to use the time-traveling feature (opening array at a specific prior timestamp).

C
C++
Python
R
Java
Go
C
// Create TileDB context
tiledb_ctx_t* ctx;
tiledb_ctx_alloc(NULL, &ctx);
// Vacuum, using ctx's configuration
tiledb_array_vacuum(ctx, array_uri, nullptr);
// Alteratively, you can create and pass a configuration object
tiledb_config_t* config;
tiledb_config_alloc(&config, NULL);
tiledb_config_set(config, "sm.vacuum.mode", "fragments", &error);
tiledb_array_vacuum(ctx, array_name, config);
// Clean-up
tiledb_ctx_free(&ctx);
tiledb_config_free(&config);
C++
Context ctx;
// Consolidate, using ctx's configuration
Array::vacuum(ctx, "<array-uri>");
// Alteratively, you can create and pass a configuration object
Config config;
config["sm.vacuum.mode"] = "fragments";
Array::vacuum(ctx, "<array-uri>", config);
Python
# Consolidate array at array_uri
tiledb.vacuum(array_uri)
# Alteratively, you can create and pass a configuration object
config = tiledb.Config({"sm.vacuum.mode": "fragments"})
tiledb.consolidate(array_uri, config=config)
R
# TODO: Consolidation is currently not supported in R
Java
# Vacuuming is not currently supported in the Java API
Go
// Vacuuming is not currently supported in the Go API

As with consolidation above, vacuuming inherits the context configuration if no configuration override is specified.