Kubernetes RBAC: Simplifying Permissions and Security with vCluster

Role-Based Access Control (RBAC) is an important concept and way of Kubernetes security with the help of which we control what users and workloads can do within the cluster. However, in multi-tenant environments, managing permissions can become complex. Virtual clusters present an elegant solution to this problem, providing isolated RBAC environments while benefiting the scalability of Kubernetes.

Virtual clusters created by Loft Labs, are lightweight virtual Kubernetes clusters that run within namespaces of a host cluster. They isolate workloads and resources, allowing administrators to configure permissions independently for each tenant, experiment, or application. In this tutorial, you'll learn how to implement RBAC in both host and virtual clusters, and how to manage teams effectively using vCluster.

Understanding Kubernetes RBAC

Kubernetes RBAC provides a framework for defining what actions users or service accounts can perform within a cluster. This is achieved through four main objects: Roles, RoleBindings, ClusterRoles, and ClusterRoleBindings.

1. Roles:  This object defines a set of permissions within a namespace. A Role can specify resources (such as Pods or Deployments) and actions (like "create" or "delete") allowed on those resources.
2. RoleBindings:  This object connects a Role to a user or group. By creating a RoleBinding, you assign the Role's permissions to a specified user within a namespace.
3. ClusterRoles – Unlike Roles, ClusterRoles are not limited to a single namespace and can define permissions for cluster-wide resources or resources in any namespace.
4. ClusterRoleBindings – ClusterRoleBindings link a ClusterRole to a user or group at the cluster level, allowing cross-namespace or global access.

Setting Up RBAC in a Host Cluster

Let’s walk through a simple RBAC setup in a host Kubernetes cluster:

1. Create a new namespace

Namespaces isolate resources, and RBAC permissions are typically scoped to them. Now, create a namespace:

kubectl apply -f - <<EOF

apiVersion: v1
kind: Namespace
metadata:
  name: example-namespace

EOF

2. Define a Role:

Now, let’s define a Role that permits viewing Pods in the example-namespace:

kubectl apply -f - <<EOF

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: example-namespace
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]
EOF

3. Bind the Role to a User:

Now, Create a RoleBinding that links the pod-reader Role to the user johndoe:

kubectl apply -f - <<EOF

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: example-namespace
subjects:
- kind: User
  name: "johndoe"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

EOF

If the RoleBinding is applied successfully, the Kubernetes API server will acknowledge with a message like this:

rolebinding.rbac.authorization.k8s.io/read-pods created

This message confirms that the RoleBinding resource named read-pods has been successfully created in the specified namespace.

To verify johndoe's permissions, use:

kubectl auth can-i list pods --namespace=example-namespace --as=johndoe

If the permissions are correctly configured, the output should be:

yes

This setup allows the user "johndoe" to view pods in the "example-namespace" namespace but doesn’t grant any other permissions in the cluster. While this works well for simple setups, managing permissions in multi-tenant clusters can become complex.

Implementing RBAC in vCluster

Virtual clusters provide isolated environments within a single host cluster, allowing teams to operate in secure, independent spaces. One significant advantage of virtual clusters is their capability to provide isolated RBAC settings, meaning each virtual cluster can have its own set of roles and bindings, entirely separate from the host cluster.

In a virtual cluster environment, each virtual cluster runs as a Kubernetes namespace in the host cluster, but it presents itself as a fully independent Kubernetes API server. This allows for dedicated RBAC configurations within each virtual cluster, which are completely isolated from other clusters and the host cluster. Let’s see how to set up RBAC within a vCluster:

Setting Up RBAC in a vCluster

First, you have to install the vCluster CLI, and follow the instructions from the docs. 

1. Create a vCluster:

You can create a virtual cluster with this command:

vcluster create my-vcluster --namespace example-namespace

2. Access the vCluster:

Once the virtual cluster is created, you will automatically be in the virtual cluster context. If not, use the following command to connect manually:

vcluster connect my-vcluster -n example-namespace

After connecting, you’re now within the virtual cluster my-vcluster, which is completely isolated from the main Kubernetes cluster.

3. Create Roles and RoleBindings:

Now, Define the Roles and RoleBindings within the virtual cluster. This is similar to setting up RBAC in the host cluster, but now operates in the isolated vCluster environment. This isolation helps you to experiment, test permissions, or set up team-specific access without affecting the host.

kubectl apply -f - <<EOF

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: vcluster-pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: vcluster-read-pods
  namespace: default
subjects:
- kind: User
  name: "vclusteruser"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: vcluster-pod-reader
  apiGroup: rbac.authorization.k8s.io

EOF‎

This example allows a user to view Pods in the default namespace in the virtual cluster.

Before applying the above configuration, the default behavior is that a user (e.g., vclusteruser) will have no specific permissions to interact with Pods. To confirm this, use:

kubectl auth can-i list pods --namespace=default --as=vclusteruser

The response will likely be:

no

After applying the above configuration, test the permissions again:

kubectl auth can-i list pods --namespace=default --as=vclusteruser

The expected response is now:

yes

This clearly demonstrates how vCluster helps isolate and manage RBAC effectively.

By setting up RBAC within the vCluster, we ensure that any permissions granted here are isolated from the host cluster. This can be especially useful in environments where multiple teams share the same Kubernetes infrastructure but need isolated, secure environments.

Managing Teams with vClusters

When working with multiple teams, vClusters can simplify the process of assigning isolated Kubernetes environments. Each team can be provided with its own vCluster, allowing administrators to make RBAC settings for each team while maintaining centralized control in the host cluster.

Creating and Configuring vClusters for Multiple Teams

1. Create a dedicated virtual cluster:

For two teams (team1 and team2), create separate vClusters:

vcluster create team1-vcluster --namespace team1
vcluster create team2-vcluster --namespace team2

Each team gets its own namespace and virtual cluster, isolating workloads and permissions.

2. Assign appropriate roles and permissions:

For each vCluster, create Roles and RoleBindings to enforce permissions. For example:

For team1:

kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: team1-deployment-manager
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["create", "update", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team1-manage-deployments
  namespace: default
subjects:
- kind: User
  name: "team1-member"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: team1-deployment-manager
  apiGroup: rbac.authorization.k8s.io
EOF

For team 2:

kubectl apply -f - <<EOF

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: team2-read-only
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team2-view-pods
  namespace: default
subjects:
- kind: User
  name: "team2-member"
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: team2-read-only
  apiGroup: rbac.authorization.k8s.io

EOF

To verify that team1-member and team2-member have distinct permissions, you have to switch to the vCluster context for team1and team2.

First, Test team1-member permissions in their vCluster:

kubectl auth can-i create deployments --namespace=default --as=team1-member

Expected result:

yes

Then, Test team2-member permissions in their vCluster:

kubectl auth can-i create deployments --namespace=default --as=team2-member

Expected result:

no

This proves that team-specific permissions are respected, highlighting the power of vClusters in enforcing isolated RBAC configurations.

By giving each team its own virtual cluster, administrators can reduce the risk of accidental interference between teams.

Final Thoughts

In this tutorial, we covered the basics of Kubernetes RBAC, explored how to implement it in both host clusters and virtual clusters, and demonstrated how it can simplify RBAC management in multi-tenant environments.

By providing isolated environments for each team, virtual clusters not only enhance security but also improve flexibility and productivity within Kubernetes environments.

For organizations aiming to manage complex access requirements in Kubernetes, virtual clusters with dedicated RBAC configurations provide a scalable and secure way. To explore more, check out the vCluster documentation and join the Loft community here!