Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
TileDB Cloud Enterprise is a license-based version of TileDB Cloud. It offers the same features, but it is packaged for on-prem or private cloud deployments.
MapProxy server is available for installation along side TileDB Cloud Enterprise in Kubernetes clusters though a helm chart. Installation of MapProxy assumes you have already configured and have a running instance of TileDB Cloud Enterprise. If you do not have TileDB Cloud Enterprise running please see Installation Instructions.
In order to use MapProxy server with TileDB Cloud Enterprise Edition you will need to get access to the private docker registry and the private helm registry. Please contacts your TileDB, Inc account representative for credentials to these services.
Before you install MapProxy server it is important to setup and customize your installation. This involves creating a custom values file for helm. Below is a sample file you can save and edit.
Save this value as values.yaml
. There are several required changes, all sections which require changes are prefixed with a comment of # REQUIRED:
. Examples of the changes needed including setting access token for TileDB Cloud Enterprise, and setting the array URI.
Once you have created the values.yaml
file you can install MapProxy by running the following helm command.
When new releases of MapProxy server are announced you can easily upgrade your installation by first updating the helm repository:
After the repository is updated you can run the helm upgrade:
When using the hosted TileDB Cloud service, you can set up corporate SSO to TileDB Cloud. Users associated with the domain name you specify (a domain that you control) will be able to log in to the service without having to separately register or create a new password.
TileDB Cloud connects to your login provider with OpenID Connect, which is supported by most SSO systems, including Google Cloud, Okta, Microsoft, PingIdentity, and more.
To enable TileDB Cloud login, you will need to create an OpenID Connect integration with your SSO provider and configure it to accept requests from TileDB Cloud. Then, you can register this application within the TileDB Cloud web interface to connect it to TileDB Cloud SaaS. Alternatively after release of Enterprise Chart Version 2.10.0 it is possible to use configuration values for TileDB Cloud Enterprise.
TileDB Cloud supports most standard OpenID Connect identity providers. These basic steps are shared across all identity providers. For more detailed instructions on how to configure a specific provider with these settings, see the identity provider–specific walkthroughs below.
Create an OpenID Connect integration.
Within your OpenID Connect integration:
Add the redirect URL (sometimes called a callback URL) of https://cloud.tiledb.com/auth/sso/callback/perdomain
. This allows login details for this integration to be sent to TileDB.
Enable required scopes (if needed):
openid
(should already be enabled)
email
(allows TileDB Cloud to access and verify the user’s email address)
profile
(allows TileDB Cloud to see the user’s name and basic information)
Overview of how to perform TileDB Cloud Enterprise upgrades
When new releases of TileDB Cloud Enterprises are announced you can easily upgrade your installation by first updating the helm repository:
After the repository is updated you can run the helm upgrade:
Starting from version 2.x.x a newer version of JupyterHub is used. JupyterHub Chart version 1.2.0 is used including support for JupyterHub 1.5.0. With the new version of JupyterHub there are a number of changes needed in your values.yaml
file. Below we highlight the specific sections requiring changes.
Add imagePullSecret
key under jupyterhub
section.
Remove imagePullSecret
key from singleuser
section.
Please copy and page in a notepad the contents of jupyterhub.auth
key, then remove it completely. You will need the values in client_secret
and cryptoKey
fields.
Replace jupyterhub.ingress.tls
key with the following.
Replace initial contents of jupyterhub.hub
key with the following:
In this step you need to fill-in the Keys
and client_secret fields with values from the field jupyterhub.auth
that earlier were saved in a notepad.
In hydra
section please replace the following fields as noted here:
Starting from version 2.4.x a newer version of JupyterHub is used. JupyterHub Chart version 2.0.0 is used including support for JupyterHub 3.0.0.
We also introduced a new version of JupyterHub Authenticator which is used automatically. This means that the authenticator_class
is no longer needed in values.yaml
.
Below we highlight the specific section that has to be removed.
Furthermore if https
is not used, it must be disabled explicitly, as in the following example:
As part of JupyterHub 3.0.0, new version of Kubespawner is used, 4.2.0. Users who have overridden jupyterhub.singleuser.profileList
in values.yaml
should update to latest structure. Here is and example usage of profileList
section which is also used in TileDB Enterprise Helm Chart
Starting with helm chart release 2.7, python is officially deprecated and scheduled for removal in January, 2024.
Replace
with
in all ingress sections. Annotations are formally deprecated since Kubernetes 1.18, like in the following example:
Support for GPUs is added in notebooks. Here is and example addition to the profileList
section which is also used in TileDB Enterprise Helm Chart that includes a GPU entry:
Corporate SSO Configuration values are supported starting from this version. Users can define a list of
This section will define a handful of example configurations for common scenarios. These examples serve as a basis for you to modify for your specific deployment.
The following configuration is a reduced set that brings up services with minimum features enabled. This includes no configuration for email (SMTP), only local accounts (no SSO/LDAP/AD), no TLS certificates, etc. This should NOT be used for production deployments but it helpful for getting an initial deployment that can be then modified.
MinIO is commonly used as a self-hosted object store. MinIO works well with TileDB Cloud.
Provided that after MinIO installation you have created:
A bucket called minio-tiledb-cloud-intermediate-result-storage
The configuration below shows only the section that needs to be modified for use with MinIO.
TileDB Cloud Enterprise edition is available for installation in Kubernetes clusters though a helm chart. The helm chart will install all components of TileDB Cloud. The instructions below will walk you though getting the helm chart, getting access to the private docker registry and setting up the installation.
In order to use the enterprise edition you will need to get access to the private docker registry and the private helm registry. Please contact your TileDB, Inc account representative for credentials to these services.
A Kubernetes cluster is required for installation. Setting up a Kubernetes cluster is outside the scope of this document, please contact your account representative if you need assistance with this.
TileDB Cloud has been officially tested with the following Kubernetes environments:
EKS (AWS)
AKS (Azure)
ACK (Alibaba)
kOps (Generic Deployment, agnostic to cloud vendor)
The minimum Kubernetes version support is v1.23.0
. If your cluster is older than this you will need to upgrade. The latest tested version is v1.28
. Newer Kubernetes version are not yet tested for compatibility and you may encounter issues. Please contact your account representative if you need a newer Kubernetes version.
You will also need the following components configured in your cluster:
Metric Server for auto-scaling
Ingress for exposing the service
We recommend a default of worker nodes with 24GB of memory at atleast 8 CPUs. This is based on the default configurations of TileDB Cloud for allowing up to 2GB of ram for computation requests. If you adjust the configurations for TileDB Cloud memory usage, its likely the optimal k8s worker nodes sizes will change.
Helm charts are used for the installation of TileDB Cloud Enterprise services in the Kubernetes cluster. You will need to have helm v3 installed on your local machine to facilitate the installation. Helm v3 does not require any components inside the Kubernetes cluster.
Helm v3.6
or newer is required.
MariaDB 10.3 or newer is required. This is used for persistent storage of user account details, organizations, tasks and more. While MySQL should be compatible only MariaDB 10.3 or newer are officially supported.
It is strongly recommended to enable SSL and at-rest encryption with MariaDB.
To get started with you will need to add the TileDB helm chart repository. This repository requires authentication, please use the username/password provided to you by your account representative.
TileDB cloud will be installed into a dedicated namespace, tiledb-cloud
Before you install TileDB Cloud Enterprise it is important to setup and customize your installation. This involves creating a custom values file for helm. Below is a sample file you can save and edit.
Save this value as values.yaml
. There are several required changes, all sections which require changes are prefixed with a comment of # REQUIRED:
. Examples of the changes needed including setting your docker registry authentication details, updating the domain names you would like to deploy TileDB Cloud too.
Once you have created the values.yaml
file you can install TileDB Cloud by running the following helm command.
After you have installed TileDB Cloud you can verify the installation works by performing the following procedure.
First step is to login to the web UI. The URL is dependent on your installation, in the values.yaml
you should have replaced console.tiledb.example.com
with the domain to access it on. Navigate in your web browser and create an account.
This step has verified that both the TileDB Cloud UI and TileDB Cloud REST components are working.
Now that you have an account we will create your first array. This array will show you that creating, writing and reading it functioning as well as give you an array and a task to view in the UI.
For this section we will use a python script. This script will create, write to and read from an array. Please note there are two sections where you need to adjust the configuration for your TileDB Cloud instance and set the array storage location.
This section requires the TileDB-Py api installed. You can get this from pip or conda. Once you have TileDB-Py, copy the following script to check_installation.py
and modify the first few lines as required.
Run this script with:
If this script ran and printed out the output, then your installation is working successfully for creating, reading and writing TileDB arrays.
The newly created array, quickstart_sparse
should now be viewable in the web console. If you navigate to the arrays
page you will see it listed.
See Upgrades
Configuring TileDB Cloud Enterprise for SSO and user management
TileDB Cloud can connect to Okta for SSO. You can also optionally configure Okta’s SCIM functionality to enable automatically provisioning users and adding them to organizations.
To enable SSO, you need to first create an Okta OpenID Connect integration for your installation.
In the Okta administrative dashboard, go to Applications and click Create App Integration. A dialog box will appear to initially set up the application.
Select the sign-in method OIDC - OpenID Connect, and the Web Application type, and click Next. You will be taken to the “New Web App Integration” screen.
Give your application a name (and optionally a logo). Next to Grant type, check the Refresh Token box.
Add a Sign-in redirect URI. This should be the URL of your TileDB Cloud Console followed by /auth/sso/callback/okta
. For instance, if your console lives at https://console.tiledb.example.com/
, the redirect URI should be https://console.tiledb.example.com/auth/sso/callback/okta
. You may optionally add your console’s home page as a sign-out redirect URI, for example https://console.tiledb.example.com/
. Leave Trusted origins empty.
If you have multiple installations that will be using the same Okta instance for authentication, or you have another instance that you use for testing or deployment testing, you can add that installation’s URL as an additional sign-in redirect URI.
At the bottom of the page, decide which users in your Okta organization should have access to TileDB Cloud. Only those selected users will be able to log in. (If you are planning on using provisioning with SCIM, you should limit this to only the same users you provision to avoid account conflicts.) Click Save to create the integration.
Copy the values from this page into your TileDB configuration in values.yaml
. The Client ID and Client Secret go in the tiledb-cloud-ui
section, and the domain also goes in the tiledb-cloud-rest
section.
After updating the config and redeploying, an Okta sign-in button will appear on your installation’s login page, and users will be able to authenticate to TileDB Cloud with their Okta credentials.
Okta uses SCIM (System for Cross-domain Identity Management) to provision users into external applications. To handle automatic creation and management of users, you can configure Okta to connect to your TileDB Enterprise installation’s SCIM endpoints. (This is not available to TileDB Cloud SaaS users.)
The first step in the setup is to configure TileDB Cloud’s SCIM support. TileDB Cloud uses HTTP Basic authentication to protect its SCIM endpoints, and a password needs to be configured in the values.yaml
file:
With a SCIM password set, TileDB Cloud is now serving SCIM endpoints at the base path of .../v1/scim/2.0/
. For example, if your TileDB Cloud API server is located at https://api.tiledb.example.com/
, its SCIM endpoint is https://api.tiledb.example.com/v1/scim/2.0/
.
Ordinarily, there will only be one password set. Multiple passwords are supported for seamless credential rotation:
Add a new password to the SCIMPasswords
list, so it contains ['old password', 'new password']
Update the password in Okta's SCIM configuration to new password
Remove the old password from the SCIMPasswords
list.
Once you’ve set SCIMPasswords
and deployed your TileDB Cloud installation, the SCIM endpoints are available to Okta. You then need to configure your Okta installation to provision users and groups in TileDB Cloud as desired.
In the Okta administration panel, visit the Applications page and click Browse App Catalog. Search for “SCIM 2.0” and select the SCIM 2.0 Test App (Basic Auth). (There are several different SCIM integrations with different authentication types; ensure you select Basic.) Click Add Integration.
Give the application a memorable name. On the second page (Sign-On Options), leave all values at their default. Click Done. You will be taken to the newly-created integration.
At the newly-created integration, click the Provisioning tab, and click Configure API Integration (the only button in the tab). Check the Enable API integration box (once again the only visible UI element) and you will be greeted with the configuration for SCIM.
After you check that box, the SCIM configuration will appear.
SCIM 2.0 Base Url: The URL to your TileDB installation’s SCIM endpoint. This is /v1/scim/2.0/
on your TileDB API server, as described earlier. The result should look like https://api.tiledb.example.com/v1/scim/2.0/
.
Username: This is always __scim
.
Password: The password you configured in the SCIMPasswords
section of your TileDB Cloud configuration.
Click Test API Credentials and, if you entered in all the right values, you should see a message of success:
Click Save and you will be taken to a screen with settings for the integration.
On this page, click Edit and check Create Users, Update User Attributes, and Deactivate Users. (Sync Password is not currently supported.) Click Save (located below Sync Password) to enable provisioning.
After these steps, provisioning is enabled, but no users or groups are currently configured to be provisioned.
Now that provisioning is enabled, you need to set up users and groups to be provisioned into TileDB Cloud.
Under the Assignments tab of your SCIM integration, click Assign. You can then assign users or groups to be provisioned into TileDB Cloud.
The simplest thing to is to use Assign to Groups. When you select a group or groups, it will automatically provision all users from that group with accounts on TileDB Cloud. When new users are added to the group, they will also be provisioned with an account. Likewise, when users are removed from the group, their access to TileDB Cloud will be suspended. This works whether the group is manually or automatically managed.
This step only pushes the members of the groups into TileDB Cloud; it does not create the groups themselves. To create groups (represented as Organizations) in TileDB Cloud, you have to enable pushing groups.
By visiting the Push Groups page and selecting the Push Groups button, you can select groups, either individually or using a search rule, to be created in TileDB Cloud.
After searching for a group by name, ordinarily you want to leave all the defaults as-is: Push group memberships immediately should be checked, and the Create Group push action should be selected. Click Save and your group will be pushed to TileDB Cloud as an organization.
The Push Status column is a menu where you can forcibly update the group (thoug this happens automatically) or disconnect the group from TileDB Cloud.
This page contains common administrative tasks and how to perform them.
Typically a user can reset their own password from the login page. In some cases, common during testing, email support via SMTP might not be setup. A user's self-service for password reset requires email support to be enabled. It is possible for a system admin to perform the reset for the user.
Bundled with the deployment is a utility command. This can be accessed from inside the Kubernetes cluster deployment inside the tiledb-cloud-tiledb-cloud-rest
pod.
The output will contain a link for reset which can be passed directly to the user to allow them to set a new password.
In some environments there might be shared NFS drives that contain data you are interested in accessing from inside a notebook environment or from a batch task graph. To support this we provide the ability to mount any volume into notebook environments.
The first step is to create a volume in the cluster that defines the NFS mount. Save the file below to a file called nfs-pv.yaml
. Modify to replace the section marked as REQUIRED
.
Apply this with kubectl.
Next copy the following file to nfs-jupyter.yaml
Modify the sections marked as REQUIRED
Apply this file to the helm chart by performing an upgrade and adding the additional values file.
After you have set up the integration on your identity provider, you can configure it within TileDB Cloud.
Switch to your newly-created organization by opening your user menu in the top-right corner of the screen and selecting the organization in the pop-up.
In the main navigation bar on the left side of the page, open the organization’s profile.
Select “SSO connections” in the main toolbar on the page, and then click the “Add SSO connection” button to add your SSO connection.
Enter the information from the OpenID Connect integration you just created. Click Submit to create the connection.
Once this is complete, you will see your newly-created integration in the list.
To protect the security of TileDB Cloud accounts, TileDB uses DNS to verify that the user claiming a domain actually has control over it. The final step in setting up your SSO connection is to set up the DNS to verify your domain ownership.
To start, click on the SSO connection you just created in the table of SSO connections. This will open a page with information on that specific connection.
Click the “DNS information” button to open up a dialog showing the information you need to set up in DNS.
You can set up either a TXT record or a CNAME record to verify ownership of the domain. Create the new record at your DNS provider.
Approximately every hour, TileDB Cloud will run DNS checks to verify your domain. After you complete setup, you can click the “Run check” button to immediately check for verification. A new entry will appear in the table once this completes. Due to the distributed nature of DNS, it may take a while for the new entry to propagate (though it is often immediate).
When your domain is verified, users can now log in to TileDB Cloud using corporate SSO.
Leave this record in place. TileDB Cloud continues to verify the domain to ensure continued ownership. If the record is not found, SSO will continue to work for 1 week before SSO is disabled. (This prevents transient errors or accidental DNS record removals from immediately breaking SSO.) Even if SSO is disabled, re-creating the DNS record will re-enable SSO with no new configuration necessary.
To enable SSO, you need to first create an Okta OpenID Connect integration for your installation.
In the Okta administrative dashboard, go to Applications and click Create App Integration. A dialog box will appear to initially set up the application. Create an OIDC - OpenID Connect integration with application type Web Application. Click Next once these are selected.
On the next page, give the integration a name (like “TileDB Cloud”) and set the sign-in redirect URI to https://cloud.tiledb.com/auth/sso/callback/perdomain
. You can also remove the sign-out redirect URI, which TileDB Cloud does not use. All the other settings on this page can remain the same.
At the bottom of the page, decide which users in your Okta organization should have access to TileDB Cloud. Only those selected users will be able to log in. Click Save to create the integration.
You will be taken to the page for your new integration.
You now have all the information you need to set up TileDB Cloud:
Issuer: Your Okta domain, for instance https://ingen.okta.com
, with no slash at the end.
Client ID: The client ID displayed on the page (in this case, 0oa90kw5r1pSVMyP85d7
).
Client Secret: The client secret (currently hidden; a longer string which looks something like Lby3LACsZewg_CzOCyG2CdGVWhXhZEfyDPcEKf30
).
From your PingIdentity administration dashboard, enter the appropriate environment and click Connections → Applications in the sidebar.
Click the + icon to add a new application. This will open a dialog box for you to set up the OpenID Connect connection for TileDB Cloud to use. Give the application a name (“TileDB Cloud”) and select OIDC Web App from the options at the bottom of the page. Click Save.
After creating the application, you should now be on the configuration panel for your new TileDB Cloud connection.
Click the Protocol: OpenID Connect button to open the OpenID Connect configuration dialog. Add the Redirect URL https://cloud.tiledb.com/auth/sso/callback/perdomain
, leave everything else unchanged, and click Save. This will allow TileDB Cloud to process logins.
Click Overview to return to the main tab, and click the Resource Access: 1 Scope button. In the dialog that pops up, add the email and profile scopes to the application. Click Save here as well.
Now the entire setup on the PingIdentity side is complete! Use the Access tab to configure who from your organization has access to TileDB Cloud (if desired) and enable the application.
Don’t close up PingIdentity yet, though; we still need the Client ID and Client Secret for TileDB Cloud.
Return to the Configuration tab of the TileDB Cloud application in PingIdentity and expand the General zippy. (You may need to scroll down.)
TileDB Cloud needs three pieces of information from this page to successfully connect to PingIdentity:
The Issuer, which is a URL that will look like https://auth.pingone.com/[some-uuid-goes-here]/as
. It does not have a /
on the end.
The Client ID, which identifies TileDB to PingIdentity. (For PingIdentity, this happens to be a UUID.)
the Client Secret, which allows TileDB to access PingIdentity resources (This is a random alphanumeric string.)
Starting from release of 2.10 or Enterprise Helm Chart it is possible to use configuration values to enable Corporate SSO. Please review the , relevant values in the configuration is displayed also here for convenience:
First, for your company or domain. Users who log in via corporate SSO will be added to this organization.
These walkthroughs provide detailed steps for the “” section above for a few providers. TileDB Cloud supports other standard OpenID Connect providers beyond the ones listed below; for providers where we don’t have detailed instructions, you can adapt the generic instructions above to your provider.
Continue the process in the “” section above.
Continue the process in the “” section above.