Install Kubeflow anywhere

Tutorials
1

Kubeflow is a novel open-source tool for Machine Learning workflow orchestration on Kubernetes. It has great powers, but deploying it may not be so easy, depending on how and where you deploy your Kubernetes.

This tutorial will show you an easy way to deploy Kubeflow on any Kubernetes, provided you have kubectl installed, in a few simple steps.

What you’ll learn

  • How to deploy MicroK8s on Ubuntu, Windows or MacOS
  • How to deploy Kubeflow on top of MicroK8s
  • How to access your Kubeflow dashboard

What you’ll need

  • Desktop or Virtual Machine with Ubuntu (16.04 LTS or above), Windows or MacOS
  • A minimum of 4 CPU, 16GB RAM, 50GB Disk (recommended 8 CPU, 32GB RAM, 60GB Disk)
  • Some basic command-line knowledge

Install the Juju OLM

The Juju Operator Lifecycle Manager leverages the concept of Kubernetes charm operators and provides easy operations (install, upgrade, integrate, remove) for your Kubeflow applications.

The following steps allow you to install Kubeflow on any Kubernetes, provided that you have kubectl.

snap install juju --classic

Alternatively, you can brew install juju on macOS or download the Juju installer for Windows.

Point Juju to your Kubernetes cluster

juju add-k8s myk8scloud --cluster=foo --kubeconfig=path/to/config

If you are on AKS, EKS, or GKE, append --aks, --eks, or --gke.

For more, see Juju docs.

Create a Juju controller and boostrap to your cluster

juju bootstrap myk8scloud my-controller

further reading on this step can be found in the juju docs.

Create a Juju model

A Juju model is a blank canvas where your charm operators will be deployed. Create a model called ‘kubeflow’, and your Kubeflow applications will be deployed into a Kubernetes namespace with the name you define at this point (N.B. you must use this name for some components to work).

Create a Juju model with the command

juju add-model kubeflow

Deploy Kubeflow

Once you have a model, it is possible to juju deploy any of the Kubeflow bundles into your cluster.

To deploy the full kubeflow bundle, it is recommended that you have at least 2 CPUs, 14Gb of free RAM, and 50Gb of disk available on your Linux machine or VM. If you have fewer resources consider deploying kubeflow-liteor kubeflow-edge.

For the Kubeflow lite bundle, run:

juju deploy kubeflow-lite

Congratulations, Kubeflow is now installing!

You can observe your Kubeflow deployment getting spun-up with the command:

watch -c juju status --color

Note:

There is currently a manual step to apply the correct role for the istio-ingressgateway. You can use the following command to create it:

kubectl patch role -n kubeflow istio-ingressgateway-operator -p '{"apiVersion":"rbac.authorization.k8s.io/v1","kind":"Role","metadata":{"name":"istio-ingressgateway-operator"},"rules":[{"apiGroups":["*"],"resources":["*"],"verbs":["*"]}]}'

Once you have deployed Kubeflow, it is time to access the Kubeflow dashboard.

Access the dashboard

1. Get authentication credentials

To display your access credentials, run the following commands:

juju config dex-auth static-username
juju config dex-auth static-password

2. Find the IP address of the Kubeflow dashboard

To find the IP address of the Kubeflow dashboard for your deployment run:

kubectl get services kubeflow

3. Access the dashboard

For local Kubeflow deployments, such as in a workstation, you can simply access the link found in the previous step, appending xip.io, for example: http://10.64.140.43.xip.io.

However, for remote deployments, or running on a virtual machine, creating a SOCKS proxy is required to access the dashboard. This can be done as follows:

  1. Logout from the current session with the exit command

  2. Re-establish connection to the machine using ssh with SOCKS proxy enabled through the -D9999 parameter. As in the example below:

    ssh -D9999 ubuntu@<machine_public_ip>

  3. On your computer, go to Settings > Network > Network Proxy, and enable SOCKS proxy pointing to: 127.0.0.1:9999

  4. On a new browser window, access the link given in the previous step, appended by .xip.io, for example: http://10.64.140.43.xip.io

Troubleshooting:

If you face any difficulties following these instructions, please create an issue here.

2 Likes