Table of Contents
Containers make application portability and delivery easier. They use resources more efficiently than bare metal or virtual machines. They're more portable, lighter, and smaller, making them easier to deploy anywhere, regardless of which cloud provider or on-premises center you need to run your code.
Kubernetes (k8s) supercharges container infrastructure with built-in scaling, reliability, and even more efficient resource usage. But, k8s adds an extra layer of abstraction, too. Many administrative tasks don't work exactly the way you'd expect. Sometimes you find it difficult to pierce the extra layers of abstraction and see what's going on inside your cluster.
In this post, we're going to look at kubectl exec. We'll see how you can use this tool to peek inside Kubernetes containers. We'll look at what exec has to offer and some examples of how to and how not to use it to manage your Kubernetes clusters.
If you want to follow the examples in this article, you'll need a system running Docker with Minikube installed and started.
Kubectl Series
What is Kubectl Exec?
Kubectl
Kubectl is a command line tool for communicating withKubernetes clusters via the Kubernetes API. You can use it to monitor Kubernetes status, apply manifest files, edit resources, and much more. It's a general admin tool for k8s clusters.
Depending on your operating system, you may need to install kubectl separately. The packages for Docker and Kubernetes may install it for you.
Kubectl Exec
Exec is one of kubectl's most useful tools. You can use it to execute a command inside a container. If you're familiar with Docker, kubectl's exec may remind you of Docker's exec command.
Let's start with an example.
Kubectl and Namespaces
Before we can execute a command in a container, we need to know its name, and before we can list containers, we need to know their namespaces. So, let's start by viewing the namespaces inside our cluster.
First, start with kubectl get namespace:
% kubectl get namespace
NAME STATUS AGE
default Active 2d23h
kube-node-lease Active 2d23h
kube-public Active 2d23h
kube-system Active 2d23h
We have four namespaces: default, kube-node-lease, kube-public, and kube-system.
Next, we'll retrieve a list of containers with get pods and a namespace by adding the --namespace command line argument.
% kubectl --namespace=default get pods
No resources found in default namespace.
I haven't run anything on my Minikube cluster yet, so the default namespace is empty. Let's look at kube-system:
% kubectl --namespace=kube-system get pods
NAME READY STATUS RESTARTS AGE
coredns-6d4b75cb6d-xsqnx 1/1 Running 0 3d
etcd-minikube 1/1 Running 0 3d
kube-apiserver-minikube 1/1 Running 0 3d
kube-controller-manager-minikube 1/1 Running 0 3d
kube-proxy-hqxbp 1/1 Running 0 3d
kube-scheduler-minikube 1/1 Running 0 3d
storage-provisioner 1/1 Running 287 (46h ago) 3d
There are a few containers in this namespace. So, let's look at them.
Exec a Shell in a Kubernetes Container
Now we can start a shell in one of the Kubernetes containers. Let's use kube-proxy-hqxbp:
% kubectl exec --namespace=kube-system kube-proxy-hqxbp -it -- sh
# ls
bin boot dev etc go-runner home lib media mnt opt proc root run sbin srv sys tmp usr var
# pwd
/
Kubectl started a Bourne shell inside the container and redirected the input and output back to my shell. Next, I listed the directory sh put me in, and then I used pwd to display its name.
Let's break down this command line next.
Namespaces in Kubernetes are an excellent way to manage your clusters. This is especially true when you have different teams working on a project. While namespaces are a good tool for some light isolation, they are not complete Kubernetes clusters on their own, which means that you run some serious risks and limitations when you rely on namespace-based isolation. Learn about Kubernetes namespaces vs. virtual clusters and why Kubernetes namespaces are not good enough
Kubectl Exec Examples
Now that we understand how to run exec, we can look at a few examples.
We didn't need to start and quit a shell to retrieve the contents of the root directory. We can pass the ls command to sh and have exec return the results to us.
% kubectl exec --namespace=kube-system kube-proxy-hqxbp -it -- sh -c ls
bin dev go-runner lib mnt proc run srv tmp var
Actually, this isn't an interactive command, so we don't need -it. Let's try it without it.
% kubectl exec --namespace=kube-system kube-proxy-hqxbp -- sh -c "ls"
bin
boot
dev
etc
go-runner
home
lib
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var
That's interesting! As a result of leaving out the -it ls' behavior changed. It doesn't use more than one column for a listing of the output isn't to a terminal. This makes parsing the results much easier in shell scripts.
Let's look in the /bin directory and see what else we can do in this container.
% kubectl exec --namespace=kube-system kube-proxy-hqxbp -- sh -c "ls -l /bin"
total 3384
-rwxr-xr-x 1 root root 39832 Sep 22 2020 cat
-rwxr-xr-x 1 root root 64512 Sep 22 2020 chgrp
-rwxr-xr-x 1 root root 60368 Sep 22 2020 chmod
-rwxr-xr-x 1 root root 64528 Sep 22 2020 chown
-rwxr-xr-x 1 root root 138896 Sep 22 2020 cp
-rwxr-xr-x 1 root root 129544 Dec 10 2020 dash
-rwxr-xr-x 1 root root 101384 Sep 22 2020 date
-rwxr-xr-x 1 root root 80984 Sep 22 2020 dd
-rwxr-xr-x 1 root root 89824 Sep 22 2020 df
-rwxr-xr-x 1 root root 143088 Sep 22 2020 dir
lrwxrwxrwx 1 root root 8 Nov 7 2019 dnsdomainname -> hostname
lrwxrwxrwx 1 root root 8 Nov 7 2019 domainname -> hostname
-rwxr-xr-x 1 root root 35632 Sep 22 2020 echo
-rwxr-xr-x 1 root root 28 Nov 9 2020 egrep
(trimmed)
This container has a limited set of command line tools installed, so there's not much we can do with it. Let's look at a better container.
Kubectl Exec With a User Container
First, start an Ubuntu container in the default namespace. The official container isn't bundled with a default command, so run it with an interactive shell:
% kubectl run --namespace=default -it exec-test-ubuntu --image=ubuntu bash
If you don't see a command prompt, try pressing enter.
root@exec-test-ubuntu:/# ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 18:39 pts/0 00:00:00 bash
root 11 1 0 18:39 pts/0 00:00:00 ps -ef
Note that you have to pass in the -it argument this time, or bash will exit. This puts us into a shell. The command prompt matches the container name. This container has the ps command, so we can see what's running inside the container. It shows the bash process we're interacting with and the ps command itself.
Next, in another window, exec a shell into the same container.
% kubectl exec -it exec-test-ubuntu -- bash
root@exec-test-ubuntu:/#
Now, go back to the first command prompt and run a process listing:
root@exec-test-ubuntu:/# ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 18:39 pts/0 00:00:00 bash
root 12 0 0 18:39 pts/1 00:00:00 bash
root 21 1 0 18:39 pts/0 00:00:00 ps -ef
There are two bash sessions now! So, it's clear we have two shells into the same container and an idea of how exec works.
Kubectl Exec Notes
As we've seen from the examples above, exec gives you the ability to do anything to a container, but that doesn't mean you should. Exec is a tool for inspecting containers, not for running applications or changing container configuration.
If you need to update a container's configuration or code, build and deploy a new image. Exec-ing into the container and installing new packages or copying in new files will cause reproducibility issues and violate the spirit of containerization.
Kubectl Exec for Container Inspection
We've examined kubectl's exec function and how it works. We saw how to figure out which namespace your containers are running in. Then we used exec to execute a program inside the desired container. We saw how you can use kubectl exec to run both interactive shells and commands that simply return results. Next, we started a container in the default namespace and used exec to inspect it.
Kubectl exec is a powerful tool for viewing the status and contents of containers in your Kubernetes clusters. Now that you're familiar with how it works, you can use it to keep your k8s clusters running smoothly!
This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).
