Table of Contents
When it came time to host software for a large public audience Codefresh turned to vCluster to provide the security and scalability needed. Codefresh is a Continuous Integration and Continuous Delivery (CD/CD) platform that allows organizations to deploy Kubernetes applications by following the GitOps principles and fulfill all the enterprise requirements such as scalability, security and ease of use.
The Codefresh platform is a unified control plane over all Argo projects that include:
- Argo CD for Continuous Deployment
- Argo Rollouts for Progressive Delivery
- Argo Workflows for Continuous Integration pipelines
- Argo Events for notifications and messaging
- Argo Autopilot for installation and maintenance of the platform.
- Codefresh runtime - A control plane that ties all the above components together
Codefresh is often deployed in on-prem or hybrid environments which sometimes presents a challenge for smaller organizations who don’t want to run their own software. In order to help organizations of all sizes adopt GitOps, Codefresh released a hosted version of GitOps/Argo CD where everything is managed by Codefresh engineers. Users can just connect their clusters as deployment destinations and start delivering their applications using the GitOps workflow.
While Argo CD does support multi-tenancy, it’s not really designed for the robust security requirements of a cloud service offered to anyone on the internet. For this hosted version of the platform, Codefresh engineering designed a new solution that would handle multiple customers in a unified way. The main challenge was how to isolate and secure the customer installations while still gaining all the advantages of cluster autoscaling.
The main architectural question was how to approach the customer runtimes.
- Use a single cluster that houses all runtimes for all tenants. This conserves resources but sacrifices isolation
- Use multiple clusters (one for each tenant). This guarantees isolation but is very expensive in regards to resource cost.
To overcome this challenge, Codefresh engineers adopted vcluster from Loft.sh which is explicitly designed for Kubernetes multi-tenancy and namespace isolation. The Codefresh hosted GitOps platform is based on multiple virtual clusters that run on a set of root clusters. These virtual clusters are managed in a fully GitOps way by Crossplane, the native Kubernetes solution for infrastructure provisioning:
Adopting vcluster is the optimal solution for both customers and Codefresh personnel as it brings the best of both worlds (tenant-per-namespace and tenant-per-cluster) with none of the disadvantages. Each customer gets a dedicated Codefresh runtime which itself runs on a virtual cluster with full cluster permissions. This ensures that all Argo components have the privileges they need. For example, all Argo CRDs are scoped in a single customer installation, something that is normally very hard to do in a single cluster.
The benefits of using virtual cluster are the following:
- Customers can use different versions of the Codefresh runtime without being forced to follow “global” updates
- The Codefresh runtimes for each customer are completely isolated from each other. There are no version conflicts or CRD installation issues
- Codefresh Ops can use native Kubernetes autoscaling for all customer installations. This helps for better resource utilization and cost management
- Customers can focus deploying their own applications, while Codefresh personnel take care of all maintenance and security issues
- Runtimes are secure and out-of-bounds templating references have no where to go
You can use Codefresh GitOps free at https://codefresh.io/codefresh-signup/
Learn more about Virtual Clusters https://loft.sh/features/kubernetes-virtual-clusters/
