Table of Contents
Managing containerized applications allows platform engineers and DevOps team members to ensure scalable, reliable, and efficient deployment, operation, and orchestration of applications, thus maintaining system stability, security, and performance. Kubernetes manifest files play a prominent role in this process, as they define the desired state of each Kubernetes object in the cluster.
This comprehensive guide explores what Kubernetes manifests are, their main components, and how to write, manage, and effectively use them in your environment.
#What Are Manifest Files?
Kubernetes manifests are essentially files in YAML or JSON format that describe the desired state of Kubernetes API objects within the cluster. The three most important components within the structure of a manifest file are metadata, spec, and status. The metadata section includes essential information like the name and namespace of the object. spec, or object specification, outlines the desired state for the object, specifying its properties and behavior.
The status field is typically not included in the Kubernetes manifest files you create and apply, as it’s managed by the Kubernetes system itself to reflect the current state of the resources. However, once a resource is running, you can query its status using kubectl commands (more on this shortly).
All in all, manifests bring the power of declarative programming to infrastructure resource management. You tell Kubernetes what you want, and it handles the details of getting there, resulting in a more streamlined, efficient, and error-resilient infrastructure management process.
#Writing Your First Kubernetes Manifest File
Now that you know the theory, it’s time to put it into action. In this section, you’ll learn how to write your first Kubernetes manifest.
#Setting Up the Tooling
You can write a manifest file using a text editor of your choice. However, to apply these manifests to the Kubernetes cluster, you’ll need the following:
- A running Kubernetes cluster. You can use kind, minikube, K3s, Rancher Desktop, or any other local Kubernetes cluster.
- The kubectl command line tool properly configured to access the Kubernetes cluster.
#Writing the Manifest File
You’ll use a pod for the following example because it’s the smallest compute unit you can deploy in a Kubernetes cluster.
Writing a manifest for a pod is a straightforward process. First, you need to define the apiVersion, which indicates the Kubernetes API version that you’re using. Second, you need to specify the kind field to denote the type of object you want to create—in this case, a pod. Next, you need to write the metadata section, which contains information about the pod, such as its name and labels.
In other words, the first portion of your manifest should look similar to the following:
apiVersion: v1
kind: Pod
metadata:
name: my-pod
labels:
app: my-app
The second portion corresponds to the object specification. The spec section outlines the behavior of the pod. It’s here that you define the container(s) that the pod will run. Each container needs an image and a name, like so:
spec:
containers:
- name: my-container
image: nginx
ports:
- containerPort: 80
In this example, my-container is the name of the container, and nginx corresponds to the image it’s using. The ports field is a list that defines the network ports this container will expose, which is useful for services to communicate with the container. Here, it’s exposing containerPort 80, which is the standard port for serving HTTP traffic.
Overall, the Kubernetes manifest is the blueprint that dictates how the my-pod pod will behave in the Kubernetes cluster. All you have to do now is save it on your local machine:
apiVersion: v1
kind: Pod
metadata:
name: my-pod
labels:
app: my-app
spec:
containers:
- name: my-container
image: nginx
ports:
- containerPort: 80
The manifest in this example is called my-pod.yaml, but you can use any other name that suits your needs.
#Creating a Pod Using a Manifest File
Next, you’ll learn how to create a pod, verify that it runs, and perform other basic operations with it. Note that everything explained here can also be applied to other Kubernetes cluster resources.
#Applying the Manifest
The kubectl apply command is used for two tasks: to create and update resources in your Kubernetes cluster. So, to create your pod, simply run kubectl apply -f /path/to/your/file in your terminal:
kubectl apply -f my-pod.yaml
The -f flag stands for file name. In summary, when you run kubectl apply -f, Kubernetes reads the configuration from the my-pod.yaml file at the specified path and applies it to the cluster.
#Verifying the Pod
As soon as you apply the manifest, the image specified in it will be downloaded, and the pod will be launched. However, it’s important to verify that everything works as expected because the smallest mistakes—such as a typo when specifying the image or a syntax error in the manifest—can prevent the pod from functioning correctly.
You can perform such a check using the command kubectl get pod, as shown below:
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
my-pod 1/1 Running 1 (5m37s ago) 5h17m
The output shows the status field mentioned earlier in the tutorial, which is useful to verify that the resource is working.
The kubectl get command can also give you more information about any Kubernetes resource, both in JSON and YAML formats. For example, if you need an extended output in JSON, you can use the command kubectl get pods my-pod -o jsonpath='{.status}'. The output will be similar to the one shown below:
$ kubectl get pods my-pod -o jsonpath='{.status}'
{"conditions":[{"lastProbeTime":null,"lastTransitionTime":"2023-10-28T13:54:25Z","status":"True","type":"Initialized"},{"lastProbeTime":null,"lastTransitionTime":"2023-10-28T13:54:27Z","status":"True","type":"Ready"},{"lastProbeTime":null,"lastTransitionTime":"2023-10-28T13:54:27Z","status":"True","type":"ContainersReady"},{"lastProbeTime":null,"lastTransitionTime":"2023-10-28T13:54:25Z","status":"True","type":"PodScheduled"}],"containerStatuses":[{"containerID":"containerd://1ed1c11db5b5a662922dd841469228430b9ed65ae60aae65cdb30a5d70292084","image":"docker.io/library/nginx:latest","imageID":"docker.io/library/nginx@sha256:add4792d930c25dd2abf2ef9ea79de578097a1c175a16ab25814332fe33622de","lastState":{},"name":"my-container","ready":true,"restartCount":0,"started":true,"state":{"running":{"startedAt":"2023-10-28T13:54:26Z"}}}],"hostIP":"172.18.0.2","phase":"Running","podIP":"10.42.0.18","podIPs":[{"ip":"10.42.0.18"}],"qosClass":"BestEffort","startTime":"2023-10-28T13:54:25Z"}
You can also get the information in YAML format by running kubectl get pods my-pod -o yaml. Below is an excerpt of the output:
$ kubectl get pods my-pod -o yaml
apiVersion: v1
kind: Pod
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"app":"my-app"},"name":"my-pod","namespace":"default"},"spec":{"containers":[{"image":"nginx","name":"my-container","ports":[{"containerPort":80}]}]}}
creationTimestamp: "2023-10-28T13:54:25Z"
labels:
app: my-app
name: my-pod
namespace: default
resourceVersion: "14860"
uid: c335252c-63b7-42d0-9606-6857448c0594
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: my-container
ports:
- containerPort: 80
protocol: TCP
...
hostIP: 172.18.0.2
phase: Running
podIP: 10.42.0.18
podIPs:
- ip: 10.42.0.18
qosClass: BestEffort
startTime: "2023-10-28T13:54:25Z"
As you can see, all the information from the manifest is present. Furthermore, information on the status, IP address of the pod, and other general information of interest is also displayed.
While kubectl get is tremendously useful for quick checks of a pod’s status, there are situations where you may need to go further and check container logs or even run commands inside a container. In the next section, you’ll learn more about these situations.
#Interacting with the Pod
In the previous section, you verified that the pod my-pod is running. Suppose now that you need to check if the container my-container inside the pod is working correctly. For this, you can use the kubectl logs command:
$ kubectl logs my-pod -c my-container
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/10/28 13:54:26 [notice] 1#1: using the "epoll" event method
2023/10/28 13:54:26 [notice] 1#1: nginx/1.25.3
2023/10/28 13:54:26 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14)
2023/10/28 13:54:26 [notice] 1#1: OS: Linux 5.15.0-87-generic
2023/10/28 13:54:26 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2023/10/28 13:54:26 [notice] 1#1: start worker processes
2023/10/28 13:54:26 [notice] 1#1: start worker process 29
2023/10/28 13:54:26 [notice] 1#1: start worker process 30
2023/10/28 13:54:26 [notice] 1#1: start worker process 31
2023/10/28 13:54:26 [notice] 1#1: start worker process 32
The -c flag indicates which container you want to check. In this example, there is only one container, so if you run the command kubectl logs my-pod, you will get the same output.
Since you are checking my-container, you could use kubectl exec to go further and run commands inside the container:
kubectl exec -it my-pod -- /bin/bash
The -it and -- /bin/bash flags basically allow you to start a Bash session inside the container. Once inside, you could check if the Nginx server is running using the curl command:
root@my-pod:/# curl http://localhost
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
The above methods are useful for running containers. However, debugging an error in the manifest using only that data can be challenging. To find where the problem is, you would need to use the kubectl describe and kubectl events commands, as explained in the next section.
#Debugging the Pod
To illustrate how to debug problems in the pod, you’ll deliberately introduce an error in the manifest. Edit my-pod.yaml and rename the image to something like nginxX. Then, run kubectl apply again:
kubectl apply -f my-pod.yaml
Now, if you run kubectl get pods, you’ll see a result similar to the following:
NAME READY STATUS RESTARTS AGE
my-pod 0/1 ImagePullBackOff 1 (6m28s ago) 21h
The message shows the error ImagePullBackOff, which might lead you to think that the image path is incorrect or that there are authentication issues with the container registry. However, if you dig a little deeper using the kubectl describe command, you’ll find the root cause of the error. Here’s an excerpt of the output:
$ kubectl describe pod my-pod
Name: my-pod
Namespace: default
Priority: 0
Service Account: default
Node: k3d-rancher-server-0/172.18.0.2
Start Time: Sat, 28 Oct 2023 13:54:25 +0000
Labels: app=my-app
Annotations: <none>
Status: Running
IP: 10.42.0.18
IPs:
IP: 10.42.0.18
...
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Killing 5s kubelet Container my-container definition changed, will be restarted
Warning InspectFailed 4s (x2 over 5s) kubelet Failed to apply default image tag "nginxX": couldn't parse image reference "nginxX": invalid reference format: repository name must be lowercase
Warning Failed 4s (x2 over 5s) kubelet Error: InvalidImageName
The command expands on the information provided by kubectl get pods my-pod -o yaml. Moreover, towards the end, it shows each event since you applied the changes to the manifest. This sequence of events explains the real error, InvalidImageName, caused by using uppercase in the image name.
Alternatively, you could use the kubectl events command to see such events:
$ kubectl events pod my-pod
LAST SEEN TYPE REASON OBJECT MESSAGE
2m44s Normal Killing Pod/my-pod Container my-container definition changed, will be restarted
15s (x7 over 2m4s) Warning BackOff Pod/my-pod Back-off restarting failed container
2s (x7 over 2m44s) Warning InspectFailed Pod/my-pod Failed to apply default image tag "nginxX": couldn't parse image reference "nginxX": invalid reference format: repository name must be lowercase
2s (x7 over 2m44s) Warning Failed Pod/my-pod Error: InvalidImageName
As the name suggests, the kubectl events command displays recent events in the pod my-pod.
#Advanced Manifest Concepts
Up to this point, you’ve worked on a simple manifest with a pod and a container. However, real-life cases may involve deployments with hundreds of pods, services, and other resources running in the cluster. In these cases, understanding the following advanced concepts can be an advantage.
#Labels and Selectors
Labels and selectors in Kubernetes serve as an efficient way to organize and group resources. Labels are key-value pairs attached to objects like pods and services, and they act like tags. For their part, selectors enable users to filter and identify Kubernetes objects based on their labels.
For example, you can use the label app: my-app to extend the sample manifest with a service that routes the pod’s traffic:
apiVersion: v1
kind: Pod
metadata:
name: my-pod
labels:
app: my-app
spec:
containers:
- name: my-container
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
type: LoadBalancer
ports:
- port: 80
selector:
app: my-app
In this example, the service my-service will route traffic to the pod my-pod because they share the app: my-app label, so you can use the selector field to match them.
#Resource Limits and Requests
In a Kubernetes manifest, you can specify resource requests and limits for a container with the resources field under spec. The requests field indicates the minimum amount of resources the container needs, while limits set the maximum. Here’s a simple example using the same manifest as before:
apiVersion: v1
kind: Pod
metadata:
name: my-pod
labels:
app: my-app
spec:
containers:
- name: my-container
image: nginx
resources:
requests:
cpu: 200m
memory: 200Mi
limits:
cpu: 500m
memory: 500Mi
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
type: LoadBalancer
ports:
- port: 80
selector:
app: my-app
As you can see, the container my-container requests 200 m CPU units and 200 Mi of memory, and its usage is limited to 500 m CPU units and 500 Mi of memory.
#Environment Variables and ConfigMaps
ConfigMaps and environment variables can pass configuration data to pods. You declare environment variables inside the pod specification, and they can be directly accessed. ConfigMaps store configuration data as key-value pairs and can function as environment variables, command line arguments, or configuration files in a volume.
Here is a simple example:
apiVersion: v1
kind: ConfigMap
metadata:
name: my-app-config
data:
LOG_LEVEL: "info"
---
apiVersion: v1
kind: Pod
metadata:
name: my-app
spec:
containers:
- name: my-container
image: nginx
envFrom:
- configMapRef:
name: my-app-config
In this example, the pod uses the envFrom and configMapRef fields to pull the LOG_LEVEL value from the my-app-config ConfigMap. envFrom is a list type and allows you to define all environment variables in a ConfigMap or secret as container environment variables. The configMapRef field is a subfield of envFrom and refers to the name of the ConfigMap you want to use (in this case, my-app-config).
In short, labels and selectors are useful for organizing and identifying resources in the Kubernetes manifest. Resource limits and requests allow you to keep cluster resources under control. Finally, ConfigMaps and environment variables allow you to separate configuration values from the manifest that defines your application, service, or resource, which is considered a best practice as it improves maintainability. You’ll learn about a few more best practices in the next sections.
#Manifest Best Practices
The best practices below are no different than the ones that programmers use when writing code, which makes sense given that Kubernetes manifests are essentially infrastructure as code (IaC).
#Version Control
You should track changes in your Kubernetes manifests, just as you do with your codebase. Version controlling your manifests provides a clear history of changes, facilitates rollback in case of issues, and ensures the reproducibility of your application’s infrastructure. It also empowers teams to collaborate more effectively, enabling peer reviews of changes before they are applied to the environment.
#Modularity and Reusability
Creating modular manifests allows you to reuse and manage each component independently, promoting simplicity and maintainability. Use ConfigMaps, secrets, or persistent volumes to handle configuration or storage needs across multiple pods. Similarly, deployment or service manifests can be reused for different applications with minor tweaks. This approach not only streamlines your work but also reduces the chances of errors. Think of manifests as building blocks—once you have them, constructing and deconstructing becomes a breeze.
#Security Considerations
You need to ensure that you safeguard sensitive information within manifest files. Kubernetes secrets offer a secure way to store sensitive data like API keys or database credentials instead of including them directly in container images or environment variables.
#Validation and Dry Runs
Another best practice that is sometimes overlooked is to validate the manifest code and perform dry runs. The first can be achieved with a variety of tools, one of them being Kubeval, which makes it easy to verify YAML or JSON manifest files from the terminal. For dry runs, if Helm is part of your tooling, you could use the --dry-run flag to simulate the changes before applying them to the cluster.
#Conclusion
In this tutorial, you learned everything you need to know about Kubernetes manifests, including their structure and how to write and manage them on a day-to-day basis. Additionally, you explored some advanced concepts such as labels, selectors, resource limits, and ConfigMaps, as well as some best practices. With this newly acquired knowledge, you can now manage Kubernetes resources, applications, and services more efficiently.
