Table of Contents
#What is vcluster?
vcluster is an open source tool for creating and managing virtual Kubernetes clusters. A virtual cluster lives inside a namespace on a host cluster, but it appears to the user as if it’s a full-blown, standalone Kubernetes cluster. Think of it as a Kubernetes control plane in a namespace.
In this tutorial, we’ll look at how to get started using vcluster, some use cases for virtual clusters, and resources to learn more.
- Introduction to Virtual Clusters in Kubernetes
- Kubernetes Namespaces vs. Virtual Clusters
- vcluster Hands-on Tutorial
- High Availability with vcluster
- Virtual Clusters For Kubernetes - Benefits & Use Cases
- Development Environments with vcluster
- How Virtual Kubernetes Clusters Can Speed Up Your Local Development
- Using Virtual Clusters for Development and CI/CD Workflows
- Kubernetes: Virtual Clusters For CI/CD & Testing
- How Codefresh Uses vcluster to Provide Hosted Argo CD
Sharing Kubernetes clusters is hard. If you want multiple individuals or teams to use the same cluster, it can be painful to set up and maintain. But giving all of those people or teams their own clusters is also hard to manage and can be very expensive. vcluster lets you share clusters while letting users feel like they have their own cluster. Virtual clusters are also very quick to provision and destroy, so they’re even great for things like local development.
For this tutorial, you will need the following:
- The kubectl binary. Installation instructions are here.
- A Kubernetes cluster to use as the host cluster and a kube context that points at it. vcluster works both with local and remote Kubernetes clusters. This tutorial will assume you’re using a local cluster like Minikube, Docker Desktop, or kind.
#1. Install the vcluster CLI
If you’re on a Mac and using Homebrew, you can install the vcluster CLI with this command:
brew install vcluster
For other platforms, see the installation instructions.
#2. Create a virtual cluster
Run this command with the vcluster CLI:
vcluster create my-vcluster
This will create a virtual cluster and a namespace for it. As you can see, creating virtual clusters is super fast.
Next, let’s list the namespaces in the virtual cluster. When you run vcluster create the vcluster CLI automatically switches your kube context to point at the virtual cluster. So you can just run kubectl like you normally would.
kubectl get namespaces
Your output should look like this:
NAME STATUS AGE default Active 83s kube-system Active 83s kube-public Active 83s kube-node-lease Active 83s
Note that we don’t see any other namespaces that exist in the host cluster. We’re connected to the API server for the virtual cluster, so we will only see namespaces that have been created inside of it.
#3. Create an Nginx deployment
Let’s create a namespace and an Nginx deployment inside the virtual cluster.
kubectl create namespace nginx kubectl create deployment nginx --image=nginx -n nginx -r 2
This creates the deployment in the nginx namespace with two Nginx pods. We can then view the pods with kubectl.
kubectl get pods -n nginx
You should see two nginx pods in the Running state.
NAME READY STATUS RESTARTS AGE nginx-76d6c9b8c-j62h7 1/1 Running 0 8s nginx-76d6c9b8c-57xk5 1/1 Running 0 8s
If the pods are in the ContainerCreating state, continue to run the
get pods command until they come up.
#4. Examine the host cluster
Now that our deployment is running, let’s disconnect from the virtual cluster and look at what’s happening under the hood in the host cluster.
This automatically switches our kube context back to the previous one, the one for the host cluster. Let’s look at the namespaces there in the host cluster.
kubectl get namespaces
You will see the default Kubernetes namespaces and any others you created, plus a newer namespace called vcluster-my-vcluster. The virtual cluster is running inside that namespace. By default, vcluster prepends the string vcluster- to the name of the virtual cluster, but you can choose a different name for the namespace if you’d like by passing the -n flag to vcluster create.
Now let’s look at what’s running inside the namespace for the virtual cluster.
kubectl get pods -n vcluster-my-vcluster
Your output should look something like this:
NAME READY STATUS RESTARTS AGE coredns-56d44fc4b4-zrqjf-x-kube-system-x-my-vcluster 1/1 Running 0 17m my-vcluster-0 2/2 Running 0 17m nginx-76d6c9b8c-57xk5-x-nginx-x-my-vcluster 1/1 Running 0 12m nginx-76d6c9b8c-j62h7-x-nginx-x-my-vcluster 1/1 Running 0 12m
Four pods are running in this example. The vcluster pod, which contains the virtual cluster’s API server and some additional tools, is called my-vcluster-0. There’s also a CoreDNS pod, which vcluster uses, and our two Nginx pods.
Notice that this time the pods names are much longer than when we viewed them from inside the virtual cluster. Here’s where we get into a big part of what makes vcluster work, the syncer.
The virtual cluster doesn’t contain a scheduler. Instead, the vcluster syncer process tells the underlying cluster to schedule workloads. This syncer process communicates with the API server of the host cluster to schedule the pods and keep track of their state. vcluster appends the name of the virtual cluster namespace the pods are running in and the name of the virtual cluster, to prevent collisions.
The state of most objects running in the virtual cluster is stored in a database inside it. vcluster uses SQLite by default for that DB, but it can also use etcd or a few other options like PostgreSQL. But pods are scheduled in the host cluster.
See the docs if you’d like to learn more about the vcluster architecture.
#5. Other helpful commands
Let’s look at a few more things you can do with the vcluster CLI.
First, let’s list the virtual clusters running in our host cluster.
NAME NAMESPACE STATUS CONNECTED CREATED AGE CONTEXT my-vcluster vcluster-my-vcluster Running 2023-02-17 16:21:50 -0800 PST 26s docker-desktop
You’ll see the my-vcluster virtual cluster and some additional details about it. This is handy for times when you might not remember which virtual clusters you’ve launched.
We can also pause the vcluster.
vcluster pause my-vcluster
You should see output like this:
info Scale down statefulSet vcluster-my-vcluster/my-vcluster... info Delete 3 vcluster workloads done √ Successfully paused vcluster vcluster-my-vcluster/my-vcluster
The vcluster pause command changes the number of replicas in StatefulSets and ReplicaSets to zero. This suspends all of your running pods while preserving all of the other configurations and state in the virtual cluster.
Let’s look at what’s running in the namespace for the virtual cluster now.
kubectl get pods -n vcluster-my-vcluster
You should see this output:
No resources found in vcluster-my-vcluster namespace.
The vcluster itself is a StatefulSet, so even the my-vcluster-0 pod and the CoreDNS pod are scaled down, as well as the Nginx pods we had launched.
Before vcluster sets the replicas to zero, though, it makes an annotation of the current number of replicas. We can spin everything back up with this command:
vcluster resume my-vcluster
The pods will spin back up. This is a very fast and easy way to suspend and resume your workloads in virtual clusters
Virtual clusters are rapid to provision and destroy, though, so you may prefer to delete the virtual cluster entirely. You can do that with:
vcluster delete my-vcluster
It should just take a few seconds, and your output will look like this:
info Delete vcluster my-vcluster... done √ Successfully deleted virtual cluster my-vcluster in namespace vcluster-my-vcluster done √ Successfully deleted virtual cluster namespace vcluster-my-vcluster
All of the resources running in the virtual cluster are cleaned up with vcluster delete.
#vcluster Demo Video
So what can you do with vcluster? Lots of things. Some popular use cases we’ve seen are:
Dev environments for local or remote development. The fact that virtual clusters are so lightweight and fast makes them very suitable for remote development in shared clusters, but you can use them on your laptop too. Rather than having to reset/rebuild your local cluster repeatedly, you can delete a virtual cluster and create a new one in seconds.
CI/CD pipelines. Sometimes in a test suite, you reach a point where you want to wipe out the current test environment and start with a fresh, new one. If you are provisioning clusters with your cloud provider’s managed Kubernetes offering, you may wait 10-20 minutes for that new cluster. That time adds up and interferes with your developers getting fast feedback. Running those tests in virtual clusters can be much quicker and more efficient.
Testing against multiple versions of Kubernetes. Are you developing an operator or an app that you’d like to test against several different versions of Kubernetes? You can spin up multiple virtual clusters running different Kubernetes versions very quickly using vcluster’s –version flag.
Training and workshops. Do you need to provision many Kubernetes clusters for a technical training session or workshop? You can do that in a shared Kubernetes cluster with vcluster in a fraction of the time and cost it would take to give every attendee their own cluster.
Can you think of other use cases? If you have a workflow where you must provision and destroy clusters quickly and cheaply, vcluster can help. vcluster is also a certified Kubernetes distribution, which means it passes a suite of conformance tests from the CNCF. So it can be relied on.
#To learn more
We also host a community Slack with a vcluster channel, which is a great place to chat with the maintainers and other users if you have any questions. You can sign up for it here.
#Additional Articles You May Like:
- How to Build an Internal Kubernetes Platform
- The Definitive Guide to Development Environments
- Kubernetes Virtual Clusters
- Kubernetes Cost Optimization with Virtual Clusters
- 10 Essentials For Kubernetes Multi-Tenancy
- Kubernetes Multitenancy: Why Namespaces aren’t Good Enough
- Kubernetes Multi-Tenancy with Argo CD And Loft
- Kubernetes Multi-Tenancy: Why Virtual Clusters Are The Best Solution
- [Video] Beyond Namespaces: Virtual Clusters are the Future of Multi-Tenancy
- Virtualizing Kubernetes Is the Key to Cost-Effective Scale
- Best Practices for Achieving Isolation in Kubernetes Multi-Tenant Environments
- Achieving Network, Namespace, and Cluster Isolation in Kubernetes - Part 1