Jupyter notebooks are a common way to run Python code for data science or data exploration. You can find more information about how TileDB Cloud manages JupyterLab instances here.

You can launch a JupyterLab instance by clicking on Launch Notebook from the left menu, select and an image and the server size, and click on the Start button. After about 30 seconds, you are all set to start writing some code. To shut down the notebook server, simply click on the Shut down button (it also takes a few seconds).

You can also make a notebook public, effectively sharing it with every user on TileDB Cloud. This can make your notebook discoverable by other users that wish to explore public notebooks, or you can discover useful code that others wish to share with the world.

To make a notebook public, you just need to navigate to the Settings of that notebook and click on Make public as shown below. Similarly, you can always switch the notebook back to private mode at any time.

This page group contains simple recipes for managing your notebooks. You can find the contents below:

To create a Jupyter notebook, you first need to navigate to the notebooks tab from the sidebar under your assets and then click the "plus" icon. That will prompt a dialog asking you to specify the notebook name, the path of the cloud storage space where the physical notebook will live and your cloud credentials that will allow TileDB Cloud to access that storage space. Once you create the notebook, you can launch it, edit and save it as you would otherwise do for any Jupyter notebook.

Alternatively you can create a notebook from the launch a JupyterLab notebook instance. Once you are in the Jupyter notebook environment, navigate to File -> New -> TileDB notebook.

You can deregister a notebook by navigating to its Settings and clicking on Deregister Notebook.

You can share a registered notebook with any other user on TileDB Cloud. To share a notebook, find it on Assets -> Notebooks and either click on the sharing button located on the right end of the notebook card in the list, or click on the notebook card and navigate to the Sharing tab. The added member will appear in the notebook members list, where you will be able to change the access policy or revoke access from the user. Users get notified by email when someone shares a notebook with them.

TileDB Cloud notebook servers include a persistent home directory for you to install custom packages.

TileDB Cloud supports pip, conda , cran, and other methods of installing packages. When installing a package, you should install it in your home directory if you want it persisted after reboots.

Pip

For Python, you can use pip to install packages into your home directory with the --user option.

pip install --user <package>

Conda

Both conda and mamba are available in the notebook environment to install any available packages. Creating a custom environment in your home directory will allow you to install and persist any packages.

mamba create -n my_env 
mamba activate my_env
mamba install -c conda-forge <package>

CRAN

R packages can be installed from CRAN by setting the `lib location to the home directory for persistence.

install.packages("package_name", lib="~/R")

You can explore public notebooks, adding a variety of filters, from the Explore page.

TileDB Cloud support uploading notebooks from git repositories. Its common that you might have notebooks stored in source control such as github or gitlab and wish to automatically upload them to TileDB Cloud for usage.

Github Actions

TileDB provides public github actions that let you upload a notebook.

Example usage includes:

   - uses: TileDB-Inc/github-actions/upload-notebooks@main
      with:
        notebooks_local: >-
          path/to/notebook-1.ipynb
          path/to/notebook-2.ipynb
          path/to/notebook-3.ipynb
        notebooks_remote: >-
          tiledb://namespace/s3://cloud-path/notebook-1
          tiledb://namespace/s3://cloud-path/notebook-2
          tiledb://namespace/s3://cloud-path/notebook-3
        TILEDB_CLOUD_TOKEN: ${{ secrets.TILEDB_CLOUD_TOKEN }}
        TILEDB_CLOUD_STORAGE_CREDENTIAL_NAME: <name-of-credential-saved-on-tiledb-cloud>

Gitlab CI

Gitlab CI doesn't offer a marketplace or public template support currently. Instead find the bellow example for setting up a gitlab ci setup.

Secrets

An API Token is required to access TileDB Cloud. Its recommended to set this as a secret in gitlab. This should be set as an environmental variable for TILEDB_REST_TOKEN.

upload_notebook:
  stage: upload
  image: python:3.9
  variables:
    TILEDB_STORAGE_CREDENTIAL_NAME: "example-credential-name"
    TILEDB_NAMESPACE: "namespace-to-upload-to"
  script:
    # Install tiledb-cloud
    - pip install tiledb-cloud
    # Define notebook to upload
    - export NOTEBOOK_FILE="example.ipynb"
    # Invoke tiledb-cloud's upload functionality
    - python -c "import tiledb, os; tiledb.cloud.upload_notebook_from_file(
        ipynb_file_name=${NOTEBOOK_FILE},
        namespace=${TILEDB_NAMESPACE},
        array_name=os.path.basename(${NOTEBOOK_FILE}),
        on_exists=OnExists.OVERWRITE,
    )"