Python: Django Development on Kubernetes with DevSpace

The possibilities that Kubernetes has brought to the container orchestration space are vast. Kubernetes simplifies the deployment and operation of such systems. However, from a developer's point of view, it may not be as simple as their previous workflows; there are some nuances of the system to be learned. DevSpace is a tool that closes that gap and helps to provide a better developer experience. This article will use DevSpace to develop and deploy an application based on the popular and feature-rich Python framework Django.

Requirements and Setting Up Development Environment

You will need three tools to continue, namely Kubectl, Helm, and DevSpace. And access to a local or remote Kubernetes cluster, any flavors of Kubernetes cluster would work.

You can find the installation instructions of the three tools in the following links:

Developing with DevSpace

The first step for setting up DevSpace is to run the devspace init command to initialize the project. This command will ask you some questions so that it can generate the skeleton devspace.yaml configuration according to your project's needs. If you do not have a Dockerfile file, it will be created for you as well. It is recommended to review the devspace.yaml configuration to make sure the settings work for your project. There are many configuration options that you can set within the devspace.yaml. Please review the documentation for further info.

$ devspace init

     ____              ____
    |  _ \  _____   __/ ___| _ __   __ _  ___ ___
    | | | |/ _ \ \ / /\___ \| '_ \ / _` |/ __/ _ \
    | |_| |  __/\ V /  ___) | |_) | (_| | (_|  __/
    |____/ \___| \_/  |____/| .__/ \__,_|\___\___|
                            |_|


? How do you want to deploy this project? helm: Use Component Helm Chart [QUICKSTART] (https://devspace.sh/component-chart/docs)

? How should DevSpace build the container image for this project? Create a new Dockerfile for this project

? Select the programming language of this project python


[info]   DevSpace does *not* require pushing your images to a registry but let's assume you wanted to do that (optional)

? Which registry would you want to use to push images to? (optional, choose any) Use hub.docker.com => you are logged in as leventogut
[done] √ Great! You are authenticated with hub.docker.com

[info]   Configuration saved in devspace.yaml - you can make adjustments as needed
[done] √ Project successfully initialized

You can now run:
- `devspace use namespace` to pick which Kubernetes namespace to work in
- `devspace dev` to start developing your project in Kubernetes
- `devspace deploy -p production` to deploy your project to Kubernetes
- `devspace -h` to get a list of available commands
$

Set the correct context to deploy the application.

$ kubectl config use-context docker-desktop
Switched to context "docker-desktop".

Additionally, it is a good practice to select a namespace to develop in.

$ devspace use namespace myproject-namespace

Let's set the variables we need, run the following command. It will ask values of missing variables:

$ devspace list vars

? Please enter a value for DB_PASSWORD ***********

 Variable      Value
 DB_DATABASE   django
 DB_PASSWORD   db.password
 DB_PORT       5432
 DB_USERNAME   postgres
 DB_VERSION    11.11.0
 IMAGE         leventogut/app

Now we are ready to run devspace dev to start our development environment. This command will deploy all components that are defined in the devspace.yaml file. Also, by default, it will open an interactive console session to the application container. DevSpace will create a port forwarding so that you can reach the application using the local port. Lastly, it will set up file sync between your project directory and the application container.

$ devspace dev

[warn]   Deploying into the 'default' namespace is usually not a good idea as this namespace cannot be deleted

[info]   Using namespace 'default'
[info]   Using kube context 'docker-desktop'
[info]   Execute 'helm upgrade myproject /Users/logut/.devspace/component-chart/component-chart-0.8.0.tgz --namespace default --values /var/folders/3h/tq577p717mdccgjpcgtcvqv80000gn/T/478617250 --install --kube-context docker-desktop'
[info]   Execute 'helm list --namespace default --output json --kube-context docker-desktop'
[done] √ Deployed helm chart (Release revision: 1)
[done] √ Successfully deployed myproject with helm
[done] √ Scaled down Deployment default/myproject
[done] √ Successfully replaced pod default/myproject-86cf556785-txfj5
[done] √ Port forwarding started on 8080:8080 (default/myproject-86cf556785-txfj5-devspace)

#########################################################
[info]   DevSpace UI available at: http://localhost:8090
#########################################################

[0:sync] Waiting for pods...
[0:sync] Starting sync...
[0:sync] Sync started on /Users/logut/dev/loft/devspace-django/myproject <-> . (Pod: default/myproject-86cf556785-txfj5-devspace)
[0:sync] Waiting for initial sync to complete
[info]   Opening 'http://localhost:8080' as soon as application will be started (timeout: 4m0s)
[info]   Opening shell to pod:container myproject-86cf556785-txfj5-devspace:container-0
Installing Python Dependencies
Collecting asgiref==3.4.1
  Downloading asgiref-3.4.1-py3-none-any.whl (25 kB)
Collecting Django==3.2.7
  Downloading Django-3.2.7-py3-none-any.whl (7.9 MB)
     |████████████████████████████████| 7.9 MB 7.8 MB/s
Collecting pytz==2021.1
  Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)
     |████████████████████████████████| 510 kB 9.7 MB/s
Collecting sqlparse==0.4.1
  Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
     |████████████████████████████████| 42 kB 62 kB/s
Collecting psycopg2-binary==2.9.1
  Using cached psycopg2_binary-2.9.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)
Installing collected packages: sqlparse, pytz, asgiref, Django, psycopg2-binary
Successfully installed Django-3.2.7 asgiref-3.4.1 pytz-2021.1 sqlparse-0.4.1 psycopg2-binary-2.9.1
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.1.2; however, version 21.2.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.

   ____              ____
  |  _ \  _____   __/ ___| _ __   __ _  ___ ___
  | | | |/ _ \ \ / /\___ \| '_ \ / _` |/ __/ _ \
  | |_| |  __/\ V /  ___) | |_) | (_| | (_|  __/
  |____/ \___| \_/  |____/| .__/ \__,_|\___\___|
                          |_|

Welcome to your development container!

This is how you can work with it:
- Run `python main.py` to build the application
- Files will be synchronized between your local machine and this container
- Some ports will be forwarded, so you can access this container on your local machine via localhost:


 Image   ImageSelector    LabelSelector   Ports (Local:Remote)
         leventogut/app                   8080:8080

root@myproject-86cf556785-txfj5-devspace:/app#

In the output above, DevSpace executed several actions:

Lastly, we are dropped into a shell of our application container. From here, we can run commands within the container.

An important point here is that by default, DevSpace uses replacedPods feature for development. The ability to replace the image allows the developer to use a different image with the same code base. A developer can create an image with debugging tools installed to be used in the development process without affecting production deployment. If this is not desired, you can remove the replacedPods stanza in the devspace.yaml.

Adding a Database (PostgreSQL)

To add a PostgreSQL deployment, please add the following snippet under the deployments section.

- name: postgresql
  helm:
    componentChart: false
    chart:
      name: postgresql
      version: 10.4.3
      repo: https://charts.bitnami.com/bitnami
    values:
      image:
        tag: $!{DB_VERSION}
      postgresqlDatabase: $!{DB_DATABASE}
      postgresqlUsername: $!{DB_USERNAME}
      postgresqlPassword: $!{DB_PASSWORD}
      service:
        port: ${DB_PORT}

And modify the vars section, as follows:

vars:
- name: IMAGE
  value: leventogut/app
- name: DB_VERSION
  value: 11.11.0
- name: DB_DATABASE
  value: django
- name: DB_USERNAME
  value: postgres
- name: DB_PASSWORD
  password: true
- name: DB_PORT
  value: 5432
- name: DB_HOST

Adding Environment Values to a Deployment

To add the environment value to our pod, we need to define them in the deployments section; you only need the env section to be added to the existing devspace.yaml.

- name: myproject
  # This deployment uses `helm` but you can also define `kubectl` deployments or kustomizations
  helm:
    # We are deploying the so-called Component Chart: https://devspace.sh/component-chart/docs
    componentChart: true
    # Under `values` we can define the values for this Helm chart used during `helm install/upgrade`
    # You may also use `valuesFiles` to load values from files, e.g. valuesFiles: ["values.yaml"]
    values:
      containers:
        - image: ${IMAGE} # Use the value of our `${IMAGE}` variable here (see vars above)
          env:
            - name: DB_DATABASE
              value: $!{DB_DATABASE}
            - name: DB_HOST
              value: $!{DB_HOST}
            - name: DB_PORT
              value: $!{DB_PORT}
            - name: DB_USERNAME
              value: $!{DB_USERNAME}
            - name: DB_PASSWORD
              value: $!{DB_PASSWORD}
      service:
        ports:
        - port: 8080

Now re-run devspace dev for changes to take effect.

Modifying Django Database Settings

The database configuration can be found under myproject>settings.py. As we do not want to hardcore our sensitive data, we will use environment values.

In settings.py, first import os module so that we can use os.environ to get the values from the environment:

import os

Then go to the DATABASES section to add the database environment values as follows.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.environ['DB_DATABASE'],
        'USER': os.environ['DB_USERNAME'],
        'PASSWORD': os.environ['DB_PASSWORD'],
        'HOST': os.environ['DB_HOST'],
        'PORT': os.environ['DB_PORT'],

    }
}

Running Database Migrations

As we defined our PostgreSQL database, we can now run the migrations from our shell.

root@myproject-97954557-bzvp9-devspace:/app# ./manage.py migrate

Operations to perform:
  Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying admin.0003_logentry_add_action_flag_choices... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying auth.0012_alter_user_first_name_max_length... OK
  Applying sessions.0001_initial... OK
root@myproject-97954557-bzvp9-devspace:/app#

All migrations are successfully executed. These are Django's built-in admin and auth modules migrations, but the same process also applies to our migrations.

Lastly, we can run ./manage.py runserver 0.0.0.0:8080 to start the development server in our container.

root@myproject-97954557-bzvp9-devspace:/app# ./manage.py runserver 0.0.0.0:8080

Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
September 10, 2021 - 18:51:00
Django version 3.2.7, using settings 'myproject.settings'
Starting development server at http://0.0.0.0:8080/
Quit the server with CONTROL-C.

Let's visit the app via the URL http://localhost:8080 and see the requests in the logs.

[11/Sep/2021 18:51:08] "GET / HTTP/1.1" 200 10697
[11/Sep/2021 18:51:08] "GET /static/admin/css/fonts.css HTTP/1.1" 200 423
[11/Sep/2021 18:51:08] "GET /static/admin/fonts/Roboto-Regular-webfont.woff HTTP/1.1" 200 85876
[11/Sep/2021 18:51:08] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 200 86184
[11/Sep/2021 18:51:08] "GET /static/admin/fonts/Roboto-Light-webfont.woff HTTP/1.1" 200 85692
Not Found: /favicon.ico
[11/Sep/2021 18:51:08] "GET /favicon.ico HTTP/1.1" 404 2113
[11/Sep/2021 18:52:07] "GET /admin HTTP/1.1" 301 0
[11/Sep/2021 18:52:07] "GET /admin/ HTTP/1.1" 302 0
[11/Sep/2021 18:52:07] "GET /admin/login/?next=/admin/ HTTP/1.1" 200 2214
[11/Sep/2021 18:52:07] "GET /static/admin/css/base.css HTTP/1.1" 200 19513
[11/Sep/2021 18:52:07] "GET /static/admin/css/nav_sidebar.css HTTP/1.1" 200 2271
[11/Sep/2021 18:52:07] "GET /static/admin/css/login.css HTTP/1.1" 200 939
[11/Sep/2021 18:52:07] "GET /static/admin/js/nav_sidebar.js HTTP/1.1" 200 1360
[11/Sep/2021 18:52:07] "GET /static/admin/css/responsive.css HTTP/1.1" 200 18545
[11/Sep/2021 18:52:07] "GET /static/admin/css/fonts.css HTTP/1.1" 304 0
[11/Sep/2021 18:52:07] "GET /static/admin/fonts/Roboto-Regular-webfont.woff HTTP/1.1" 304 0
[11/Sep/2021 18:52:07] "GET /static/admin/fonts/Roboto-Light-webfont.woff HTTP/1.1" 304 0
[11/Sep/2021 18:52:12] "POST /admin/login/?next=/admin/ HTTP/1.1" 200 2375
[11/Sep/2021 18:52:12] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 304 0

Our application is deployed in development mode into our Kubernetes cluster. Now we can start developing the application locally, and DevSpace will sync the changes to the container.

Create a Sample Application and a View

We will use the django-admin tool to create our first app for the project. If you are unfamiliar with projects and applications, please refer here.

$ ../venv/bin/django-admin startapp myapp

Edit myproject/myapp/views.py file and add the following code:

from django.http import HttpResponse
from django.views import View

class MyView(View):

    def get(self, request, *args, **kwargs):
        return HttpResponse('pong')

This code creates a view and configures the view to return pong to all requests.

Edit myproject/urls.py and add the following code so that we glue ping/ path to MyView:

from myapp.views import MyView

urlpatterns += [
    path('ping/', MyView.as_view(), name='myview'),
]

This code adds a path to the urlpatterns; ping/ is linked to the previously created view.

Now please visit the http://localhost:8080/ping/, and you should see pong as a text response.

Please note that we didn't re-deploy the application into the Kubernetes cluster. The files are synced from your local folder into the container after the change so that we are able to develop our application as we do locally. This is one of the powerful features DevSpace brings.

Deploying to Production

DevSpace has a helpful feature called profiles. This feature allows you to change the configuration itself based on a given profile. The changes are configured within the profiles stanza and applied with the -p flag, for example, devspace dev -p staging.

devspace deploy -p production

Troubleshooting with DevSpace

Logs

In the default configuration, DevSpace opens a shell into the container; however, you can change this behavior and get a stream of logs of one or all components. To get the logs, you can also use the following command.

$ devspace logs

When you don't specify the container, DevSpace will ask you which container you would like to get the logs for.

? Select a container  [Use arrows to move, type to filter]
> myproject-97954557-bzvp9-devspace:container-0
  postgresql-postgresql-0:postgresql
[info]   Printing logs of pod:container postgresql-postgresql-0:postgresql
postgresql 18:44:31.47
postgresql 18:44:31.47 Welcome to the Bitnami postgresql container
postgresql 18:44:31.47 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-postgresql
postgresql 18:44:31.47 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-postgresql/issues
postgresql 18:44:31.48
postgresql 18:44:31.49 INFO  ==> ** Starting PostgreSQL setup **
postgresql 18:44:31.51 INFO  ==> Validating settings in POSTGRESQL_* env vars..
postgresql 18:44:31.51 INFO  ==> Loading custom pre-init scripts...
postgresql 18:44:31.52 INFO  ==> Initializing PostgreSQL database...

Entering into Containers

DevSpace allows you to enter containers with the container name instead of the usual way, using the long pod name and then the container name.

Without any parameters, it will ask you which pod/container you want to enter.

$ devspace enter

? Which pod do you want to open the terminal for?  [Use arrows to move, type to filter]
> myproject-97954557-bzvp9-devspace:container-0
  postgresql-postgresql-0:postgresql

Here we specifically select the container-0 to enter.

$ devspace enter -c container-0

[info]   Opening shell to pod:container myproject-97954557-bzvp9-devspace:container-0
root@myproject-97954557-bzvp9-devspace:/app#

Running Commands within a Container

DevSpace allows you to configure predefined commands in the devspace.yaml under commands stanza. Also, you have the option to use the following syntax to run a command in the container without a configuration.

$ devspace enter -c container-0 ./manage.py

We can see that manage.py command's help text here, as we didn't dive a subcommand or parameter.

[info]   Opening shell to pod:container myproject-97954557-bzvp9-devspace:container-0

Type 'manage.py help <subcommand>' for help on a specific subcommand.

Available subcommands:

[auth]
    changepassword
    createsuperuser
...
...

Running Tests

You have several options to start testing. Here is a simple example:

$ devspace enter -c container-0 ./manage.py test

[info]   Opening shell to pod:container myproject-97954557-bzvp9-devspace:container-0
System check identified no issues (0 silenced).----
Ran 0 tests in 0.000s

OK

Running tests is very similar to how you would run in the local environment. Also, note that this can be turned into a DevSpace command so that you can use it as devspace run test, and all specified tests will be run with a single command.

Clean Up

DevSpace provides a subcommand that deletes all the deployments and the resources (except PV/PVC) easily so you can remove the development environment with ease.

$ devspace purge

Conclusion

We have seen how to set up DevSpace in a basic Django environment. We have also seen that DevSpace seamlessly synced changed files to the running container so that we didn't need to re-deploy the application to Kubernetes cluster every time there was a change. This syncing also works for compiled languages.

DevSpace is highly configurable and is very feature-rich to make your life easier. Have a look at the documentation to learn how to create commands, attach event-based hooks, verify dependencies, and create profiles for different deployment environments such as staging, production, and CI/CD pipelines.

Further Reading

Photo by Chris Ried on Unsplash