Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Configuring TileDB Cloud Enterprise for SSO and user management
TileDB Cloud can connect to Okta for SSO. You can also optionally configure Okta’s SCIM functionality to enable automatically provisioning users and adding them to organizations.
To enable SSO, you need to first create an Okta OpenID Connect integration for your installation.
In the Okta administrative dashboard, go to Applications and click Create App Integration. A dialog box will appear to initially set up the application.
Select the sign-in method OIDC - OpenID Connect, and the Web Application type, and click Next. You will be taken to the “New Web App Integration” screen.
Give your application a name (and optionally a logo). Next to Grant type, check the Refresh Token box.
Add a Sign-in redirect URI. This should be the URL of your TileDB Cloud Console followed by /auth/sso/callback/okta
. For instance, if your console lives at https://console.tiledb.example.com/
, the redirect URI should be https://console.tiledb.example.com/auth/sso/callback/okta
. You may optionally add your console’s home page as a sign-out redirect URI, for example https://console.tiledb.example.com/
. Leave Trusted origins empty.
If you have multiple installations that will be using the same Okta instance for authentication, or you have another instance that you use for testing or deployment testing, you can add that installation’s URL as an additional sign-in redirect URI.
At the bottom of the page, decide which users in your Okta organization should have access to TileDB Cloud. Only those selected users will be able to log in. (If you are planning on using provisioning with SCIM, you should limit this to only the same users you provision to avoid account conflicts.) Click Save to create the integration.
Copy the values from this page into your TileDB configuration in values.yaml
. The Client ID and Client Secret go in the tiledb-cloud-ui
section, and the domain also goes in the tiledb-cloud-rest
section.
After updating the config and redeploying, an Okta sign-in button will appear on your installation’s login page, and users will be able to authenticate to TileDB Cloud with their Okta credentials.
Okta uses SCIM (System for Cross-domain Identity Management) to provision users into external applications. To handle automatic creation and management of users, you can configure Okta to connect to your TileDB Enterprise installation’s SCIM endpoints. (This is not available to TileDB Cloud SaaS users.)
The first step in the setup is to configure TileDB Cloud’s SCIM support. TileDB Cloud uses HTTP Basic authentication to protect its SCIM endpoints, and a password needs to be configured in the values.yaml
file:
With a SCIM password set, TileDB Cloud is now serving SCIM endpoints at the base path of .../v1/scim/2.0/
. For example, if your TileDB Cloud API server is located at https://api.tiledb.example.com/
, its SCIM endpoint is https://api.tiledb.example.com/v1/scim/2.0/
.
Ordinarily, there will only be one password set. Multiple passwords are supported for seamless credential rotation:
Add a new password to the SCIMPasswords
list, so it contains ['old password', 'new password']
Update the password in Okta's SCIM configuration to new password
Remove the old password from the SCIMPasswords
list.
Once you’ve set SCIMPasswords
and deployed your TileDB Cloud installation, the SCIM endpoints are available to Okta. You then need to configure your Okta installation to provision users and groups in TileDB Cloud as desired.
In the Okta administration panel, visit the Applications page and click Browse App Catalog. Search for “SCIM 2.0” and select the SCIM 2.0 Test App (Basic Auth). (There are several different SCIM integrations with different authentication types; ensure you select Basic.) Click Add Integration.
Give the application a memorable name. On the second page (Sign-On Options), leave all values at their default. Click Done. You will be taken to the newly-created integration.
At the newly-created integration, click the Provisioning tab, and click Configure API Integration (the only button in the tab). Check the Enable API integration box (once again the only visible UI element) and you will be greeted with the configuration for SCIM.
After you check that box, the SCIM configuration will appear.
SCIM 2.0 Base Url: The URL to your TileDB installation’s SCIM endpoint. This is /v1/scim/2.0/
on your TileDB API server, as described earlier. The result should look like https://api.tiledb.example.com/v1/scim/2.0/
.
Username: This is always __scim
.
Password: The password you configured in the SCIMPasswords
section of your TileDB Cloud configuration.
Click Test API Credentials and, if you entered in all the right values, you should see a message of success:
Click Save and you will be taken to a screen with settings for the integration.
On this page, click Edit and check Create Users, Update User Attributes, and Deactivate Users. (Sync Password is not currently supported.) Click Save (located below Sync Password) to enable provisioning.
After these steps, provisioning is enabled, but no users or groups are currently configured to be provisioned.
Now that provisioning is enabled, you need to set up users and groups to be provisioned into TileDB Cloud.
Under the Assignments tab of your SCIM integration, click Assign. You can then assign users or groups to be provisioned into TileDB Cloud.
The simplest thing to is to use Assign to Groups. When you select a group or groups, it will automatically provision all users from that group with accounts on TileDB Cloud. When new users are added to the group, they will also be provisioned with an account. Likewise, when users are removed from the group, their access to TileDB Cloud will be suspended. This works whether the group is manually or automatically managed.
This step only pushes the members of the groups into TileDB Cloud; it does not create the groups themselves. To create groups (represented as Organizations) in TileDB Cloud, you have to enable pushing groups.
By visiting the Push Groups page and selecting the Push Groups button, you can select groups, either individually or using a search rule, to be created in TileDB Cloud.
After searching for a group by name, ordinarily you want to leave all the defaults as-is: Push group memberships immediately should be checked, and the Create Group push action should be selected. Click Save and your group will be pushed to TileDB Cloud as an organization.
The Push Status column is a menu where you can forcibly update the group (thoug this happens automatically) or disconnect the group from TileDB Cloud.
TileDB Cloud Enterprise is a license-based version of TileDB Cloud. It offers the same features, but it is packaged for on-prem or private cloud deployments.
This section will define a handful of example configurations for common scenarios. These examples serve as a basis for you to modify for your specific deployment.
The following configuration is a reduced set that brings up services with minimum features enabled. This includes no configuration for email (SMTP), only local accounts (no SSO/LDAP/AD), no TLS certificates, etc. This should NOT be used for production deployments but it helpful for getting an initial deployment that can be then modified.
MinIO is commonly used as a self-hosted object store. MinIO works well with TileDB Cloud.
Provided that after MinIO installation you have created:
A bucket called minio-tiledb-cloud-intermediate-result-storage
The configuration below shows only the section that needs to be modified for use with MinIO.
When using the hosted TileDB Cloud service, you can set up corporate SSO to TileDB Cloud. Users associated with the domain name you specify (a domain that you control) will be able to log in to the service without having to separately register or create a new password.
TileDB Cloud connects to your login provider with OpenID Connect, which is supported by most SSO systems, including Google Cloud, Okta, Microsoft, PingIdentity, and more.
To enable TileDB Cloud login, you will need to create an OpenID Connect integration with your SSO provider and configure it to accept requests from TileDB Cloud. Then, you can register this application within the TileDB Cloud web interface to connect it to TileDB Cloud SaaS. Alternatively after release of Enterprise Chart Version 2.10.0 it is possible to use configuration values for TileDB Cloud Enterprise.
TileDB Cloud supports most standard OpenID Connect identity providers. These basic steps are shared across all identity providers. For more detailed instructions on how to configure a specific provider with these settings, see the identity provider–specific walkthroughs below.
Create an OpenID Connect integration.
Within your OpenID Connect integration:
Add the redirect URL (sometimes called a callback URL) of https://cloud.tiledb.com/auth/sso/callback/perdomain
. This allows login details for this integration to be sent to TileDB.
Enable required scopes (if needed):
openid
(should already be enabled)
email
(allows TileDB Cloud to access and verify the user’s email address)
profile
(allows TileDB Cloud to see the user’s name and basic information)
MapProxy server is available for installation along side TileDB Cloud Enterprise in Kubernetes clusters though a helm chart. Installation of MapProxy assumes you have already configured and have a running instance of TileDB Cloud Enterprise. If you do not have TileDB Cloud Enterprise running please see Installation Instructions.
In order to use MapProxy server with TileDB Cloud Enterprise Edition you will need to get access to the private docker registry and the private helm registry. Please contacts your TileDB, Inc account representative for credentials to these services.
Before you install MapProxy server it is important to setup and customize your installation. This involves creating a custom values file for helm. Below is a sample file you can save and edit.
Save this value as values.yaml
. There are several required changes, all sections which require changes are prefixed with a comment of # REQUIRED:
. Examples of the changes needed including setting access token for TileDB Cloud Enterprise, and setting the array URI.
Once you have created the values.yaml
file you can install MapProxy by running the following helm command.
When new releases of MapProxy server are announced you can easily upgrade your installation by first updating the helm repository:
After the repository is updated you can run the helm upgrade:
Overview of how to perform TileDB Cloud Enterprise upgrades
When new releases of TileDB Cloud Enterprises are announced you can easily upgrade your installation by first updating the helm repository:
After the repository is updated you can run the helm upgrade:
Starting from version 2.x.x a newer version of JupyterHub is used. JupyterHub Chart version 1.2.0 is used including support for JupyterHub 1.5.0. With the new version of JupyterHub there are a number of changes needed in your values.yaml
file. Below we highlight the specific sections requiring changes.
Add imagePullSecret
key under jupyterhub
section.
Remove imagePullSecret
key from singleuser
section.
Please copy and page in a notepad the contents of jupyterhub.auth
key, then remove it completely. You will need the values in client_secret
and cryptoKey
fields.
Replace jupyterhub.ingress.tls
key with the following.
Replace initial contents of jupyterhub.hub
key with the following:
In this step you need to fill-in the Keys
and client_secret fields with values from the field jupyterhub.auth
that earlier were saved in a notepad.
In hydra
section please replace the following fields as noted here:
Starting from version 2.4.x a newer version of JupyterHub is used. JupyterHub Chart version 2.0.0 is used including support for JupyterHub 3.0.0.
We also introduced a new version of JupyterHub Authenticator which is used automatically. This means that the authenticator_class
is no longer needed in values.yaml
.
Below we highlight the specific section that has to be removed.
Furthermore if https
is not used, it must be disabled explicitly, as in the following example:
As part of JupyterHub 3.0.0, new version of Kubespawner is used, 4.2.0. Users who have overridden jupyterhub.singleuser.profileList
in values.yaml
should update to latest structure. Here is and example usage of profileList
section which is also used in TileDB Enterprise Helm Chart
Starting with helm chart release 2.7, python is officially deprecated and scheduled for removal in January, 2024.
Replace
with
in all ingress sections. Annotations are formally deprecated since Kubernetes 1.18, like in the following example:
Support for GPUs is added in notebooks. Here is and example addition to the profileList
section which is also used in TileDB Enterprise Helm Chart that includes a GPU entry:
Corporate SSO Configuration values are supported starting from this version. Users can define a list of
TileDB Cloud Enterprise edition is available for installation in Kubernetes clusters though a helm chart. The helm chart will install all components of TileDB Cloud. The instructions below will walk you though getting the helm chart, getting access to the private docker registry and setting up the installation.
In order to use the enterprise edition you will need to get access to the private docker registry and the private helm registry. Please contact your TileDB, Inc account representative for credentials to these services.
A Kubernetes cluster is required for installation. Setting up a Kubernetes cluster is outside the scope of this document, please contact your account representative if you need assistance with this.
TileDB Cloud has been officially tested with the following Kubernetes environments:
EKS (AWS)
AKS (Azure)
ACK (Alibaba)
kOps (Generic Deployment, agnostic to cloud vendor)
The minimum Kubernetes version support is v1.23.0
. If your cluster is older than this you will need to upgrade. The latest tested version is v1.28
. Newer Kubernetes version are not yet tested for compatibility and you may encounter issues. Please contact your account representative if you need a newer Kubernetes version.
You will also need the following components configured in your cluster:
Metric Server for auto-scaling
Ingress for exposing the service
We recommend a default of worker nodes with 24GB of memory at atleast 8 CPUs. This is based on the default configurations of TileDB Cloud for allowing up to 2GB of ram for computation requests. If you adjust the configurations for TileDB Cloud memory usage, its likely the optimal k8s worker nodes sizes will change.
Helm charts are used for the installation of TileDB Cloud Enterprise services in the Kubernetes cluster. You will need to have helm v3 installed on your local machine to facilitate the installation. Helm v3 does not require any components inside the Kubernetes cluster.
Helm v3.6
or newer is required.
MariaDB 10.3 or newer is required. This is used for persistent storage of user account details, organizations, tasks and more. While MySQL should be compatible only MariaDB 10.3 or newer are officially supported.
It is strongly recommended to enable SSL and at-rest encryption with MariaDB.
To get started with you will need to add the TileDB helm chart repository. This repository requires authentication, please use the username/password provided to you by your account representative.
TileDB cloud will be installed into a dedicated namespace, tiledb-cloud
Before you install TileDB Cloud Enterprise it is important to setup and customize your installation. This involves creating a custom values file for helm. Below is a sample file you can save and edit.
Save this value as values.yaml
. There are several required changes, all sections which require changes are prefixed with a comment of # REQUIRED:
. Examples of the changes needed including setting your docker registry authentication details, updating the domain names you would like to deploy TileDB Cloud too.
Once you have created the values.yaml
file you can install TileDB Cloud by running the following helm command.
After you have installed TileDB Cloud you can verify the installation works by performing the following procedure.
First step is to login to the web UI. The URL is dependent on your installation, in the values.yaml
you should have replaced console.tiledb.example.com
with the domain to access it on. Navigate in your web browser and create an account.
This step has verified that both the TileDB Cloud UI and TileDB Cloud REST components are working.
Now that you have an account we will create your first array. This array will show you that creating, writing and reading it functioning as well as give you an array and a task to view in the UI.
For this section we will use a python script. This script will create, write to and read from an array. Please note there are two sections where you need to adjust the configuration for your TileDB Cloud instance and set the array storage location.
This section requires the TileDB-Py api installed. You can get this from pip or conda. Once you have TileDB-Py, copy the following script to check_installation.py
and modify the first few lines as required.
Run this script with:
If this script ran and printed out the output, then your installation is working successfully for creating, reading and writing TileDB arrays.
The newly created array, quickstart_sparse
should now be viewable in the web console. If you navigate to the arrays
page you will see it listed.
It is recommended that you attempt to launch a jupyter notebook to confirm that the cluster and deployment is correctly configured. You can launch a notebook by going to the "monitoring" tab in the web console and selecting Launch server
.
See Upgrades
Starting from release of 2.10 of our Enterprise Helm Chart, it is possible to use configuration values to enable Corporate SSO using OpenID Connect. There are some differences compared to SSO in the TileDB Cloud SaaS product:
Domain verification is not required (since you are hosting and controlling your own installation).
Users are not automatically added to an organization upon login.
You can rewrite OpenID Connect claims if needed.
These configuration stanzas in the values.yaml
file contain the necessary values to enable corporate SSO with your TileDB Cloud enterprise installation. By configuring these settings, the TileDB Cloud backend will be able to connect to your company's OpenID Connect–based SSO service.
If your OpenID Connect implementation doesn't provide data in the necessary format, you can configure TileDB Cloud to rewrite the claims to get what it needs. This is configured by a mapping of {"target": "template string with {other}"}
, where target
is the claim that will be written to, and template string with {other}
is a string where the text {other}
will be replaced by the other
claim in the source claims.
In this case, performing the above substitution on an OpenID Connect token with the following claims:
will result in the addition of a target
claim:
For instance, if the OIDC doesn't include the email
claim, but it does include a preferred_username
claim with a bare username, you can configure the substitution:
This will transform a token like:
into
Alternately, if your preferred_username
field is already a full email address, you can omit the suffix:
Customize these parameters by replacing the example values listed below with details for your specific SSO service's OpenID Connect configuration.
After you have set up the integration on your identity provider, you can configure it within TileDB Cloud.
First, create a TileDB Cloud organization for your company or domain. Users who log in via corporate SSO will be added to this organization.
Switch to your newly-created organization by opening your user menu in the top-right corner of the screen and selecting the organization in the pop-up.
In the main navigation bar on the left side of the page, open the organization’s profile.
Select “SSO connections” in the main toolbar on the page, and then click the “Add SSO connection” button to add your SSO connection.
Enter the information from the OpenID Connect integration you just created. Click Submit to create the connection.
Once this is complete, you will see your newly-created integration in the list.
To protect the security of TileDB Cloud accounts, TileDB uses DNS to verify that the user claiming a domain actually has control over it. The final step in setting up your SSO connection is to set up the DNS to verify your domain ownership.
To start, click on the SSO connection you just created in the table of SSO connections. This will open a page with information on that specific connection.
Click the “DNS information” button to open up a dialog showing the information you need to set up in DNS.
You can set up either a TXT record or a CNAME record to verify ownership of the domain. Create the new record at your DNS provider.
Approximately every hour, TileDB Cloud will run DNS checks to verify your domain. After you complete setup, you can click the “Run check” button to immediately check for verification. A new entry will appear in the table once this completes. Due to the distributed nature of DNS, it may take a while for the new entry to propagate (though it is often immediate).
When your domain is verified, users can now log in to TileDB Cloud using corporate SSO.
Leave this record in place. TileDB Cloud continues to verify the domain to ensure continued ownership. If the record is not found, SSO will continue to work for 1 week before SSO is disabled. (This prevents transient errors or accidental DNS record removals from immediately breaking SSO.) Even if SSO is disabled, re-creating the DNS record will re-enable SSO with no new configuration necessary.
These walkthroughs provide detailed steps for the “identity provider setup” section above for a few providers. TileDB Cloud supports other standard OpenID Connect providers beyond the ones listed below; for providers where we don’t have detailed instructions, you can adapt the generic instructions above to your provider.
To enable SSO, you need to first create an Okta OpenID Connect integration for your installation.
In the Okta administrative dashboard, go to Applications and click Create App Integration. A dialog box will appear to initially set up the application. Create an OIDC - OpenID Connect integration with application type Web Application. Click Next once these are selected.
On the next page, give the integration a name (like “TileDB Cloud”) and set the sign-in redirect URI to https://cloud.tiledb.com/auth/sso/callback/perdomain
. You can also remove the sign-out redirect URI, which TileDB Cloud does not use. All the other settings on this page can remain the same.
At the bottom of the page, decide which users in your Okta organization should have access to TileDB Cloud. Only those selected users will be able to log in. Click Save to create the integration.
You will be taken to the page for your new integration.
You now have all the information you need to set up TileDB Cloud:
Issuer: Your Okta domain, for instance https://ingen.okta.com
, with no slash at the end.
Client ID: The client ID displayed on the page (in this case, 0oa90kw5r1pSVMyP85d7
).
Client Secret: The client secret (currently hidden; a longer string which looks something like Lby3LACsZewg_CzOCyG2CdGVWhXhZEfyDPcEKf30
).
Continue the process in the “TileDB Cloud setup” section above.
From your PingIdentity administration dashboard, enter the appropriate environment and click Connections → Applications in the sidebar.
Click the + icon to add a new application. This will open a dialog box for you to set up the OpenID Connect connection for TileDB Cloud to use. Give the application a name (“TileDB Cloud”) and select OIDC Web App from the options at the bottom of the page. Click Save.
After creating the application, you should now be on the configuration panel for your new TileDB Cloud connection.
Click the Protocol: OpenID Connect button to open the OpenID Connect configuration dialog. Add the Redirect URL https://cloud.tiledb.com/auth/sso/callback/perdomain
, leave everything else unchanged, and click Save. This will allow TileDB Cloud to process logins.
Click Overview to return to the main tab, and click the Resource Access: 1 Scope button. In the dialog that pops up, add the email and profile scopes to the application. Click Save here as well.
Now the entire setup on the PingIdentity side is complete! Use the Access tab to configure who from your organization has access to TileDB Cloud (if desired) and enable the application.
Don’t close up PingIdentity yet, though; we still need the Client ID and Client Secret for TileDB Cloud.
Return to the Configuration tab of the TileDB Cloud application in PingIdentity and expand the General zippy. (You may need to scroll down.)
TileDB Cloud needs three pieces of information from this page to successfully connect to PingIdentity:
The Issuer, which is a URL that will look like https://auth.pingone.com/[some-uuid-goes-here]/as
. It does not have a /
on the end.
The Client ID, which identifies TileDB to PingIdentity. (For PingIdentity, this happens to be a UUID.)
the Client Secret, which allows TileDB to access PingIdentity resources (This is a random alphanumeric string.)
Continue the process in the “TileDB Cloud setup” section above.
This page contains common administrative tasks and how to perform them.
Typically a user can reset their own password from the login page. In some cases, common during testing, email support via SMTP might not be setup. A user's self-service for password reset requires email support to be enabled. It is possible for a system admin to perform the reset for the user.
Bundled with the deployment is a utility command. This can be accessed from inside the Kubernetes cluster deployment inside the tiledb-cloud-tiledb-cloud-rest
pod.
The output will contain a link for reset which can be passed directly to the user to allow them to set a new password.
TileDB Cloud supports custom docker images for user defined functions when self hosted. This page outlines how to setup and configure these custom images.
TileDB Cloud docker images for user defined functions have several required packages and entrypoint script. As a result your custom docker images must be based on the official TileDB Cloud docker image.
The following are the default image names available:
Image | Image Name |
---|
Image | Image Name |
---|
The specific tag that is available depends on the version of TileDB Cloud you are deploying out. Your account representative can help you identify this.
The images need to be hosted in a place that is accessible to the kubernetes cluster. For an example of hosting docker images using ECR see .
After you have build the docker images they can be added them to your values.yaml
by adding a section.
TileDB Cloud Enterprise uses by default Dockerhub as the container registry for UDF Images, but ECR can also be used. This page outlines how to setup and configure ECR as UDF images repository.
Provided that TileDB Enterprise Helm Chart is installed in namespace tiledb-cloud
, the default service account of this namespace is used by REST Server. It has to be annotated as follows:
This annotation refers to an AWS Role, that has to include a policy statement to allow ECR access:
After you have verified ECR access, it is needed to configure values.yaml
to enable using it.
In some environments there might be shared NFS drives that contain data you are interested in accessing from inside a notebook environment or from a batch task graph. To support this we provide the ability to mount any volume into notebook environments.
The first step is to create a volume in the cluster that defines the NFS mount. Save the file below to a file called nfs-pv.yaml
. Modify to replace the section marked as REQUIRED
.
Apply this with kubectl.
Next copy the following file to nfs-jupyter.yaml
Modify the sections marked as REQUIRED
Apply this file to the helm chart by performing an upgrade and adding the additional values file.
Base python image |
|
Geospatial packages |
|
Biomedical Imaging |
|
Vector Search |
|
Base R image |
|