Convinced by Onyxia? Let's see how you can get your own instance today!
If you are already familiar with Kubernetes and Helm, here's how you can get an Onyxia instance up and running in just a matter of seconds.
With this minimal configuration, you'll have an Onyxia instance operating in a degraded mode, which lacks features such as authentication, S3 explorer, secret management, etc. However, you will still retain the capability to launch services from the catalog.
Whether you are a Kubernetes veteran or a beginner with cloud technologies, this guide aims to guide you through the instantiation and configuration of an Onyxia instance with it's full range of features enabled. Let's dive right in! 🤿
First let's make sure we have a suitable deployment environement to work with!
Let's use hashicorp Vault for storing the user secrets.
Vault is also used by Onyxia as the persistance layer for all saved configuration. If you don't have a vault all user settings are stored in the local storage.
Onyxia-web use vault as a storage for two kinds of secrets : 1. secrets or information generate by Onyxia to store differents values (ui preferences for example) 2. user secrets Vault must be configured with JWT or OIDC authentification methods.
As vault needs to be initialized with a master key, it can't be directly configured with all parameters such as oidc or access policies and roles. So first step we create a vault with dev mode (do not use this in production and do your initialization with any of the recommanded configuration : shamir, gcp, another vault)
Create a client called "vault"
Root URL: https://vault.lab.my-domain.net/
Valid redirect URIs: https://vault.lab.my-domain.net/*
Web origins: *
Let's install ArgoCD to manage and monitor our Onyxia Datalab deployment!
At this stage of this installation process we assumes that:
You have a Kubernetes cluster and kubectl configured
datalab.my-domain.net and *.lab.my-domain.net's DNS are pointing to your cluster's external address. my-domain.net being a domain that you own.
Your ingress-nginx is set up with a default TLS certificate that covers both datalab.my-domain.net and *.lab.my-domain.net, processing all ingress objects, .
We can proceed with manually installing various services via Helm to set up the datalab. However, it's more convenient and reproducible to maintain a Git repository that outlines the required services that we need for our datalab, allowing ArgoCD to handle the deployment for us.
To clarify, using ArgoCD is merely an approach that we recommend, but it is by no means a requirement. Feel free to manually helm install the different services using the values.yaml from InseeFrLab/onyxia-ops!
Let's install ArgoCD on the our cluster.
Now you have to get the password that have been automatically generated to protect ArgoCD's admin console.
Allow some time for ArgoCD to strart, you can follow the progress by running kubectl get pods and making sure that all pod are ready 1/1. After that running this command will print the password:
You can now login to https://argocd.lab.my-domain.net using:
username: admin
password: <the output of the previous command (without the % at the end)>
Now that we have an ArgoCD we want to connect it to a Git repository that will describe what services we want to be running on our cluster.
Let's fork the onyxia-ops GitHub repo and use it to deploy an Onyxia instance!
Note that in this guide, we use GitHub, but feel free to fork the InseeFrLab/onyxia-ops repository on GitLab or any other forge. You'll need to slightly adapt the instructions, but you should be able to follow along!
At this point you should have a very bare bone Onyxia instance that you can use to launch services.
What's great, is that now, if you want to update the configuration of your Onyxia instance you only have to commit the change to your GitOps repo, ArgoCD will takes charge of restarting the service for you with the new configuration. To put that to the test try to modify your Onyxia configuration by setting up a global alert that will be shown as a banner to all users!
After a few seconds, if you reload https://datalab.my-domain.net you should see the message!
Next step is to see how to enable your user to authenticate themselvs to your datalab!
Enable S3 storage via MinIO S3
Onyxia uses AWS Security Token Service API to obtain S3 tokens on behalf of your users. We support any S3 storage compatible with this API. In this context, we are using MinIO, which is compatible with the Amazon S3 storage service and we demonstrate how to integrate it with Keycloak.
Before configuring MinIO, let's create a new Keycloak client (from the previous existing "datalab" realm).
Before deploying MinIO on the cluster let's set, in the MinIO configuration file, the OIDC client secret we have copied in the previous step.
Once you've done that you can deploy MinIO!
Before configuring the onyxia region to create tokens we should go back to Keycloak and create a new client to enable onyxia-web to request token for MinIO. This client is a little bit more complex than other if you want to manage durations (here 7 days) and this client should have a claim name policy and with a value of stsonly according to our last deployment of MinIO.
Now let's update our Onyxia configuration to let it know that there is now a S3 server available on the cluster.
Diff of the changes applied to the Onyxia configuration:
Congratulation, all the S3 related features of Onyxia are now enabled in your instance! Now if you navigate to your Onyxia instance you should have My Files in the left menu.
Next step in the installation process is to setup Vault to provide a way to your user so store secret and also to provide something that Onyxia can use as a persistance layer for user configurations.
Using Keycloak to enable user authentication
Let's setup Keycloak to enable users to create account and login to our Onyxia.
Note that in this instalation guide we make you use Keycloak but you can use any identity server that is Open ID Connect compliant.
We're going to install Keycloak just like we installed Onyxia.
Before anything open apps/keycloak/values.yaml in your onyxia-ops repo and . Also write down the keycloak.auth.adminPassword, you'll need it to connect to the Keycloak console.
Try to remember, when you update Onyxia in apps/onyxia/Chart.yaml to also update the Onyxia theme in apps/keycloak/values.yaml.
You can now login to the administration console of https://auth.lab.my-domain.net/auth/ and login using username: keycloak and password: <the one you've wrote down earlier>.
Create a realm called "datalab" (or something else), go to Realm settings
On the tab General
User Profile Enabled: On
On the tab login
User registration: On
Forgot password: On
Remember me: On
On the tab email, we give an example with AWS SES, if you don't have a SMTP server at hand you can skip this by going to Authentication (on the left panel) -> Tab Required Actions -> Uncheck "set as default action" Verify Email. Be aware that with email verification disable, anyone will be able to sign up to your service.
From: noreply@lab.my-domain.net
Host: email-smtp.us-east-2.amazonaws.com
Port: 465
Authentication: enabled
Username: **************
Password: ***************************************
When clicking "save" you'll be asked for a test email, you have to provide one that correspond to a pre-existing user or you will get a silent error and the credentials won't be saved.
On the tab Themes
Login theme: onyxia-web (you can also select the login theme on a per client basis)
Email theme: onyxia-web
On the tab Localization
Internationalization: Enabled
Supported locales: <Select the languages you wish to support>
On the tab Session.
SSO Session Idle:
SSO Session Max:
SSO Session Idle Remember Me:
SSO Session Max Remember Me: 14 days
Create a client with client ID "onyxia"
Root URL: https://datalab.my-domain.net/
Valid redirect URIs: https://datalab.my-domain.net/*
Web origins: *
Login theme: onyxia-web
In Authentication (on the left panel) -> Tab Required Actions enable and set as default action Therms and Conditions.
Now you want to ensure that the username chosen by your users complies with Onyxia requirement (only alphanumerical characters) and define a list of email domain allowed to register to your service.
Go to Realm Settings (on the left panel) -> Tab User Profile (this tab shows up only if User Profile is enabled in the General tab and you can enable user profile only if you have started Keycloak with -Dkeycloak.profile=preview) -> JSON Editor.
Now you can edit the file as suggested in the following DIFF snippet. Be mindful that in this example we only allow emails @gmail.com and @hotmail.com to register you want to edit that.
Now our Keycloak server is fully configured we just need to update our Onyxia deployment to let it know about it.
In your GitOps repo you now want to update your onyxia configuration.
Here is the DIFF of the onyxia configuration:
Now your users should be able to create account, log-in, and start services on their own Kubernetes namespace.
Next step in the installation proccess it to enable all the S3 related features of Onyxia:
Provision a Kubernetes cluster
First you'll need a Kubernetes cluster. If you have one already you can skip and directly go to the Onyxia instalation section.
Hashicorp maintains great tutorials for terraforming Kubernetes clusters on AWS, GCP or Azure.
Pick one of the three and follow the guide.
You can stop after the configure kubectl section.
Ingress controller
Let's install ingress-ngnix on our newly created cluster:
DNS
Let's assume you own the domain name my-domain.net, for the rest of the guide you should replace my-domain.net by a domain you actually own.
Now you need to get the external address of your cluster, run the command
and write down the External IP assigned to the LoadBalancer.
Depending on the cloud provider you are using it can be an IPv4, an IPv6 or a domain. On AWS for example, it will be a domain like xxx.elb.eu-west-1.amazonaws.com.
If you see <pending>, wait a few seconds and try again.
Once you have the address, create the following DNS records:
If the address you got was an IPv4 (x.x.x.x), create a A record instead of a CNAME.
If the address you got was ans IPv6 (y:y:y:y:y:y:y:y), create a AAAA record.
https://datalab.my-domain.net will be the URL for your instance of Onyxia. The URL of the services created by Onyxia are going to look like: https://<something>.lab.my-domain.net
You can customise "datalab" and "lab" to your liking, for example you could chose onyxia.my-domain.net and *.kub.my-domain.net.
SSL
In this section we will obtain a TLS certificate issued by LetsEncrypt using the certbot commend line tool then get our ingress controller to use it.
If you are already familiar with certbot you're probably used to run it on a remote host via SSH. In this case you are expected to run it on your own machine, we'll use the DNS chalenge instead of the HTTP chalenge.
The obtained certificate needs to be renewed every three month.
To avoid the burden of having to remember to re-run the certbot command periodically you can setup cert-manager and configure a DNS01 challenge provider on your cluster but that's out of scope for Onyxia.
You may need to delegate your DNS Servers to one of the supported DNS service provider.
Now we want to create a Kubernetes secret containing our newly obtained certificate:
Lastly, we want to tell our ingress controller to use this TLS certificate, to do so run:
This command will open your configured text editor, go to containers -> args and add:
If you are on a Mac or Window computer you can install Docker desktop then enable Kubernetes.
WARNING: If you are folowing this installating guide on an Apple Sillicon Mac, be aware that many of the services that comes by default with Onyxia like Jupyter RStudio and VSCode won't run because we do not yet compile our datacience stack for the ARM64 architecture. If you would like to see this change please sumit an issue about it.
Docker desktop isn't available on Linux, you can use Kind instead.
Port Forwarding
You'll need to forward the TCP ports 80 and 443 to your local machine. It's done from the administration panel of your domestic internet Box. If you're on a corporate network you'll have to test onyxia on a remote Kubernetes cluster.
DNS
Let's assume you own the domain name my-domain.net, for the rest of the guide you should replace my-domain.net by a domain you actually own.
Get your internet box routable IP and create the following DNS records:
If you have DDNS domain you can create CNAME instead example:
https://datalab.my-domain.net will be the URL for your instance of Onyxia.
The URL of the services created by Onyxia are going to look like: https://xxx.lab.my-domain.net
You can customise "datalab" and "lab" to your liking, for example you could chose onyxia.my-domain.net and *.kub.my-domain.net.
SSL
In this section we will obtain a TLS certificate issued by LetsEncrypt using the certbot commend line tool.
The obtained certificate needs to be renewed every three month.
To avoid the burden of having to remember to re-run the certbot command periodically you can setup cert-manager and configure a DNS01 challenge provider on your cluster but that's out of scope for Onyxia.
You may need to delegate your DNS Servers to one of the supported DNS service provider.
Now we want to create a Kubernetes secret containing our newly obtained certificate:
Ingress controller
We will install ingress-nginx in our cluster, although any other ingress controller would be suitable as well. The configuration will be set up to handle all ingress objects, including those without a specified class, and to utilize our SSL certificate for our wildcard certificate. This approach ensures a straightforward SSL termination managed by the reverse proxy for both *.lab.my-domain.net and datalab.my-domain.net, eliminating any further concerns regarding SSL setup.
Now that we have a Kubernetes cluster ready to use let's levrage ArgoCD and GitOps practices to deploy and monitor the core services of our Onyxia Datalab.
Save and quit. Done We installed the ingress-nginx in our cluster, (but note that any other ingress controller could have been used as well). The configuration was adjusted to handle all ingress objects, even those lacking a specified class, and to employ our SSL certificate for our wildcard certificate. This strategy facilitated an effortless SSL termination, managed by the reverse proxy for both *.lab.my-domain.net and datalab.my-domain.net, thus removing any additional SSL configuration concerns.
