Table of Contents
vcluster is an open source tool for creating and managing virtual Kubernetes clusters. A virtual Kubernetes cluster runs within a namespace on a host cluster, but it appears to the user as if it’s a full-blown, standalone cluster.
But what happens once the virtual cluster is created? Like any new cluster, there’s usually a lot more that needs to happen before it’s usable. The gap between a newly provisioned cluster and a working dev environment for one of your developers may be very large. You may need to install tools to manage other setups or configurations, for example.
In this hands-on tutorial, we’ll look at how to configure newly provisioned virtual clusters with init manifests (Kubernetes YAML) and Helm charts that are applied automatically after the virtual cluster is created. This feature lets you configure your virtual clusters fully or install other tools that will configure them, depending on your needs.
If you’re new to vcluster, you may want to go through our introductory hands-on tutorial first.
#Prerequisites
The only prerequisite is a Kubernetes cluster that you can run the virtual cluster in. Any Kubernetes cluster, local or remote, should work.
#Install the vcluster CLI
If you use a Mac with Homebrew, you can install the CLI with this command:
brew install vcluster
For other platforms, see the quickstart in the docs. The CLI is supported on macOS, Linux, and Windows.
#Apply a manifest on initialization
The manifests we’re referring to are any valid Kubernetes YAML that you could apply with `kubectl apply -f`. Being able to apply Kubernetes YAML immediately after the cluster is created allows us to automate just about anything we’d want to do when creating or configuring a cluster.
Let’s look at an example. Create a file called values.yaml with the following contents:
init:
manifests: |-
apiVersion: v1
kind: Namespace
metadata:
name: nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: nginx
spec:
selector:
matchLabels:
app: nginx
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.25.1
ports:
- containerPort: 80
Let’s break down what’s happening here.
The first two lines specify that this is the `init.manifest` section of the vcluster YAML file and that what follows are the manifests to be applied.
The next four lines create a namespace in the virtual cluster called nginx.
Then there is a separator, which needs to be present between each manifest.
The rest of the file creates a deployment to launch two NGINX pods in the nginx namespace that was just created. It will launch two pods in the virtual cluster with a label of nginx.
Next, let’s create a virtual cluster that will automatically apply those manifests. The manifests will be applied in the order they appear in the file.
Make sure you are pointed at the kube context for the host cluster you use. Then create the virtual cluster.
vcluster create init-tutorial -f values.yaml
This will create a virtual cluster with the name init-tutorial, and a namespace to run it in.
You should see output like:
info Creating namespace vcluster-init-tutorial
info failed to find IPv6 service CIDR: couldn't find host cluster Service CIDR ("Service "test-service-9wpzn" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")
info Detected local kubernetes cluster docker-desktop. Will deploy vcluster with a NodePort & sync real nodes
info Create vcluster init-tutorial...
info execute command: helm upgrade init-tutorial /var/folders/gy/d3_c4t1x731_hl8qtrfkhr_h0000gn/T/vcluster-0.15.2.tgz-86186483 --kubeconfig /var/folders/gy/d3_c4t1x731_hl8qtrfkhr_h0000gn/T/3759283775 --namespace vcluster-init-tutorial --install --repository-config='' --values /var/folders/gy/d3_c4t1x731_hl8qtrfkhr_h0000gn/T/1562320995 --values values.yaml
done √ Successfully created virtual cluster init-tutorial in namespace vcluster-init-tutorial
info Waiting for vcluster to come up...
done √ Switched active kube context to vcluster_init-tutorial_vcluster-init-tutorial_docker-desktop
- Use `vcluster disconnect` to return to your previous kube context
- Use `kubectl get namespaces` to access the vcluster
The exact output will depend a bit on how your host cluster is set up.
If your host cluster is local (Docker Desktop, Minikube, etc.), vcluster will configure it to use a NodePort and connect to it automaticaly.
For remote clusters, vcluster will connect using port forwarding.
Now that we’re connected to the virtual cluster, we can run kubectl commands against it like any other cluster. Let’s look at the pods that are running.
kubectl get pods -n nginx
You should see output like:
NAMESPACE NAME READY STATUS RESTARTS AGE
nginx nginx-deployment-775b6549b5-nr8jh 1/1 Running 0 68s
nginx nginx-deployment-775b6549b5-vcx7w 1/1 Running 0 68s
In this example the two NGINX pods are running in the nginx namespace. The pods and the namespace were created automatically as soon as the vcluster was created.
Now let’s disconnext from the virtual cluster and delete it to clean up.
vcluster disconnect
Disconnecting will switch you back to the context of the host cluster. Then run:
vcluster delete init-tutorial
#Apply a Helm chart on initialization
Now let’s create a NGIX namespace and deployment again, but this time instead of creating the deployment manually, we’ll install NGINX with a public Helm chart.
First, edit the values.yaml. Remove the previous contents and set them to:
init:
helm:
- chart:
name: nginx
repo: https://charts.bitnami.com/bitnami
# optional field
values: |-
replicaCount: 2
release:
name: nginx
namespace: nginx
As you can see, we specify the chart name and repo first. Then we can optionally include Helm values for the chart inline (we’ve set the replica count to 2). And last, the release section contains the name (the pods will use this name) and the namespace that’s being used.
Create the vcluster:
vcluster create init-tutorial-2 -f values.yaml
Once again you should be automatically connected to the vcluster. And we can view the NGINX pods with:
kubectl get pods -n nginx
Your output should look like:
NAME READY STATUS RESTARTS AGE
nginx-769c898b4f-g922n 1/1 Running 0 108s
nginx-769c898b4f-vwh6w 1/1 Running 0 108s
And that’s the process for automatically applying a Helm chart when the vcluster is initialized.
We can clean up once again with:
vcluster disconnect
vcluster delete init-tutorial-2
You can also apply private Helm charts when vclusters are initialized from a private repo or your local filesystem. For more info, see the docs.
#Troubleshooting
A quick note on troubleshooting. When a virtual cluster is initialized with manifests or Helm charts, you won’t see any errors that result in the output from the `vcluster create` command. You can see more information in the host cluster’s logs, though.
First, if you are connected to the virtual cluster, disconnect with the vcluster disconnect command. Then run this command against the host cluster:
kubectl logs -n vcluster-NAME -l app=vcluster,release=NAME -c syncer
Substitute the name you gave your virtual cluster in the vcluster connect command for NAME in both places. By default, vcluster prepends the string vcluster- to the name of the virtual cluster’s namespace.
#More resources
We’ve walked through how to apply Kubernetes manifests and Helm charts to virtual clusters when they’re initialized. This is a great way to bootstrap new virtual clusters and configure them.
Here are some resources to learn more:
- The documentation for this feature
- Our Intro to vcluster tutorial, if you missed it
- The #vcluster channel in the Loft Community Slack, where the vcluster maintainers and many other users are around to answer questions
