Virtual Clusters For Kubernetes - Benefits & Use Cases

Daniel Thiry

August 24, 2020

Minute Read

This is some text inside of a div block.

Virtual Kubernetes Clusters (vClusters) have the potential to bring Kubernetes adoption to the next level. They are running in a physical Kubernetes cluster and can be used in the same way as normal clusters, but still are just a virtual construct. (Learn more about how virtual Clusters work here). Similar to Virtual Machines that revolutionized the use of physical servers, virtual Kubernetes clusters have some benefits compared to physical clusters, which make them particularly useful for some scenarios.

In this article, I will describe the benefits of virtual Kubernetes clusters and provide some use cases in which vClusters are advantageous to other solutions such as many individual clusters or namespace-based multi-tenancy.

Benefits of Virtual Kubernetes Clusters

The benefits of virtual clusters for Kubernetes are mainly based on two characteristics: Sharing of a physical cluster and isolation.

Since vClusters are a virtual abstraction within Kubernetes, it is possible to run many vClusters on just a single physical cluster, which has the following advantages:

Reduced Management Effort. Since there is only one physical cluster to maintain, the administrative effort is significantly reduced by virtual clusters. This becomes especially clear when comparing it to another alternative that would lead to a similar outcome from a user perspective: Instead of running virtual clusters in one physical cluster, it is possible to run many physical clusters that all have to be maintained, which can become infeasible pretty fast even in only mid-sized teams. Additionally, the physical cluster can be configured in a pretty "basic" way without extensive additional installations as most of this will happen on the level of the vCluster.

Reduced Cost. Besides the reduced management effort that also results in cost reductions, virtual clusters are also more efficient in utilizing computing resources because the resources are shared by the tenants. Again, similar efficiency improvements were gained by introducing virtual machines to a physical server infrastructure. This cost efficiency is further increased by the disposable nature of virtual cluster, so they can be “thrown away” (shut off) when they are not needed. Alternatively, they can be “put to sleep” (scaled down), a process that can even be automized, e.g. with a sleep mode that is provided by solutions such as Loft.

Isolation / Hard Multi-Tenancy

Since virtual Kubernetes clusters are providing a harder form of multi-tenancy, i.e. users are strictly isolated from each other, they have some additional benefits especially in comparison to a namespace-based multi-tenancy approach:

Stable System. Using virtual Kubernetes clusters does not compromise the stability of the system. Even if a virtual cluster fails, the underlying physical cluster is usually not affected by this. (Only in some extreme cases, the failure of a virtual cluster would lead to a failure of the physical cluster, which is again similar to a VM that rarely can lead to a break in the underlying physical machine.) This is regardless of the source of the error which can come from within the cluster, e.g. an engineer accidentally breaks something, or from outside, e.g. a malicious attack on the system. This makes the whole system more resilient and helps to implement a reasonable microservice architecture with a true separation of concerns for the individual services.

Full Flexibility. While every engineer is working or every microservice is running on the same physical cluster, their virtual clusters are completely independent. This allows the vClusters to also be configured in very different ways. For example, the virtual clusters can have different Kubernetes versions or different API server configurations. This again enables engineers to work freely and to use whatever is best for their use case without having to consider other requirements or the underlying physical Kubernetes cluster.

Overall, virtual Kubernetes clusters are the only approach to combine efficiency with a stable, flexible system due to its hard multi-tenancy with Kubernetes-native resources. The alternative of using many individual clusters is only able to resolve the isolation issue but creates a huge cost burden, while the approach of namespace-based soft multi-tenancy keeps cost reasonable but can only provide limited stability and flexibility. Virtual clusters are thus so far the only option for companies that want to have the best of the two worlds.

Use Cases

Since virtual clusters are mostly behaving like regular Kubernetes clusters, their scope of application is similarly broad. For this, I will concentrate on three main use cases that are not covered well by alternative solutions with namespaces or a multitude of physical clusters.

CI/CD & Testing

Requirements: For CI/CD and testing scenarios in a Kubernetes environment, the engineers need to have access to Kubernetes whenever they need it. To keep cost low, the environment should only be running if it is actually used. Engineers should also be able to modify the Kubernetes configuration to get more realistic tests.

Problem: Creating a Kubernetes cluster takes some time, even in public clouds. Since computing resources are costly and many CI/CD pipelines are billed by the minute, it is often not feasible to wait 30 minutes until a cluster is started. This would also slow down the feedback loop for the engineer’s actions and interrupts their workflow. Therefore, a cluster is often simply shared by engineers for testing, which again can lead to waiting times if a colleague is using it at the same time. It is also highly inefficient as such a cluster is always running and costing money even if no tests or pipelines are running.

Virtual Cluster Solution: Virtual clusters have the advantage that they can be started in a few seconds, so it is possible to create them on-demand by the engineers. Since they are fully fledged clusters that are only used by one engineer or application at a time, they can be freely configured and adapted to the individual situation. After the completion of a test or CI/CD process, they can also be thrown away without any issues, which keeps the cost for computing resources as low as possible. For this, virtual Kubernetes clusters are a perfect fit for CI/CD and testing scenarios.

Cloud-Native Development

Requirements: More and more companies want to give developers a direct access to Kubernetes already during development. For this, the engineers need to have a Kubernetes access they can work and experiment with, while keeping the cost and management effort for these environments low.

Problem: Individual clusters for every engineer are often not feasible as this is an expensive solution that requires some Kubernetes knowledge and leads to high maintenance effort. For this, namespace-based multi-tenancy is already used sometimes. However, the lack of a strong isolation leaves the engineers with some risk of breaking the whole system. They further usually do not have an admin access to the cluster, so they are limited in terms of configuration options.

Virtual Cluster Solution: Again, virtual Kubernetes clusters can be started by the engineers on-demand and without much prior Kubernetes knowledge. The engineers are then isolated very well from each other and work in secure dev sandboxes without the fear of affecting others. At the same time, this approach is very cost-efficient because only a low management effort is needed with a single physical cluster, the resources can be shared, and the virtual clusters can be easily deleted. Instead of deleting the virtual clusters, it is also possible to put them to sleep with a sleep mode such as the one Loft provides. This allows developers to resume their work where they ended while no computing cost are created in the meantime.

AI+ML experiments

Requirements: For Artificial Intelligence and Machine Learning applications, a lot of computing resources are often needed. To run experiments, engineers thus need access to a cloud environment with a lot of power. They also need to be able to easily replicate these environments to rerun the experiment with different parameters or to run them in parallel.

Problem: Due to the high resource requirements, cost is a major issue for AI and ML experiments. It is thus not reasonable to have an experimentation environment always running as this would create high cost even if it is not used. From a workflow perspective, a shared environment leads to waiting times for engineers if another experiment is still running. The same goes for individual clusters that take long to start and so also interrupts the engineers’ workflows.

Virtual Cluster Solution: Virtual clusters are a great alternative for artificial intelligence and machine learning engineers who want to run experiments. They can be created within seconds when they are needed and are very well replicable, so it is even possible to run experiments in parallel, which can accelerate many workflows. Even though vClusters are almost instantly available, they are not running idle as they can be easily shut off after the experiment is over, so the cost for these expensive experiments are minimized.

Which Virtual Cluster Solutions exist

Virtual Kubernetes clusters are still a very new topic, but they are already available today. The multi-tenancy working group has presented an experimental solution in this area. k3v by Darren Shepherd is another proof of concept implementation in the open-source community.

A more advanced platform solution for virtual Kubernetes clusters is Loft. Loft is a commercial solution that provides some additional features on top of the virtual cluster solution such as a sleep mode and a user management in a complete platform that can be used off-the-shelf.

Conclusion

The concept of virtual Kubernetes clusters is still very new but could be a big advancement for Kubernetes and similarities to the groundbreaking move from physical servers to virtual machines are striking. The combination of a shared cluster with a strong isolation merges efficiency with resilience and flexibility. This makes Kubernetes also more attractive for use cases where it is currently still hard to implement, such as CI/CD, testing, cloud-native development and AI/ML experimentation, and could thus spur Kubernetes adoption even further.

Photo by Kelvin Ang on Unsplash

vCluster

Multi Tenancy

Use Cases