Table of Contents
GitOps + Kubernetes Series
The rise of cloud-native applications and infrastructure has resulted in a shift in how software is developed and deployed, with a focus on scalability, flexibility, and resilience. The shift has necessitated DevOps practices that automate processes, leverage appropriate technologies and tools, and enable developers to work efficiently.
GitOps has emerged as a popular approach to implementing DevOps in a goal-oriented fashion. It allows you to manage infrastructure and your applications by leveraging Git as a source of truth when using container orchestrators like Kubernetes for your application layer. With various benefits like better rollbacks and self-service capabilities for developers, implementing GitOps can make teams more optimized, prevent drift, and force consistency in your environments.
In this article, you'll learn about the principles of GitOps, its benefits both for operational and development purposes, and how it can help you effectively manage infrastructure and applications with its declarative nature.
What Is GitOps?
GitOps, coined by Weaveworks CEO Alexis Richardson in 2017, is a modern operational model that expands on the traditional continuous delivery model for Kubernetes-related deployments.
GitOps enables developers to move away from manual deployment using scripts and kubectl. Instead, it uses a model that exists externally to the system in their source code management (SCM), acting as a single source of truth, and then uses a GitOps tool to implement and deploy the source manifests. Using an SCM tool such as Git that your developers are already familiar with allows you to version and review your infrastructure and applications similarly to your code and implement self-service capabilities.
The model focuses on infrastructure deployment, management, and monitoring, along with automatic and secure configuration management of applications. Access is limited to debugging or maintenance purposes, resulting in better overall security.
Principles of GitOps
With GitOps' seamless integration with Kubernetes and Git (the most common SCM), the paradigm is gaining popularity as a DevOps implementation. Understanding the principles that make this approach successful across various teams is important if you want to successfully implement it.
Declarative Configuration
GitOps uses declarative configurations like Terraform manifests, Kustomize configs, Helm charts, and others to manage infrastructure and applications. Instead of scripting everything manually, GitOps relies on a set of declarative configuration files that define the system's desired state, and any approved changes are automatically reconciled by GitOps tools.
GitOps uses versioned CI/CD on top of a declarative infrastructure, allowing your teams to "stop scripting and start shipping," as Kelsey Hightower put it. Moreover, the declarative configuration is helpful to define your entire stack as code and continuously deliver changes with confidence to different production or staging environments.
Git as the Single Source of Truth
In GitOps, Git is used as the single source of truth for all configurations. All changes are made through Git, and the Git repository is the authoritative source of information about the system's state. This provides an auditable history of all changes, and developers using a shared source can collaborate and discuss changes before shipping them.
For example, pull requests or merge requests can be used to discuss new infrastructure changes such as increasing the number of nodes in a Kubernetes cluster. This approach can include a declarative update for scaling nodes, avoiding the need for a new Slack thread in the infrastructure channel. When changes are approved and merged, an audit trail is created, which can easily help you reference the reasons and requirements in the future.
Versioned System State and Ability to Roll Back
Since Git is used as the single source of truth, it provides versioning for all changes made to the system's state. This means that the entire history of changes is available, allowing for straightforward rollbacks by reverting the commit to a previous operational configuration version. This approach avoids the need to identify cluster updates and navigate changes made by other developers.
Configuration Drift Detection
GitOps tools continuously monitor the system for configuration drifts, which occur when the system state deviates from the desired state defined in the configuration files. When drifts are detected, GitOps tools can automatically reconcile the system back to the desired state, ensuring that the system remains consistent and reliable and that it reflects what's stored in your SCM.
Developer Benefits of Using GitOps with Kubernetes
The principles underlying GitOps are crucial to its successful implementation. When properly implemented, GitOps offers significant advantages, such as self-service and a unified declaration store, extending from infrastructure to the application layer. These benefits are further amplified when combined with Kubernetes.
It Provides Self-Service Ability for Developers
Using Git as the SCM tool for GitOps enables a more seamless deployment experience and easier adoption, particularly for those already familiar with Git. Instead of figuring out new tools, developers can adapt their auditing and branching methods to meet the requirements of a better continuous delivery system. They won't need to focus on understanding Kubernetes in detail, since they only need to deal with the manifests and not with the real cluster.
New changes to infrastructure don't need interference from operations or the platform team; once there's a setup, developers can use a pull request to create a change and then ask for approvals from respective teams instead of waiting around for answers. Automatic checks can also be implemented, meaning you can find a cost estimation of your infrastructure changes and fix lint or security issues while in the discussion phase, increasing both speed and productivity.
A practical example of the benefits of GitOps with Kubernetes involves deploying a new API for end users. With GitOps, developers can create new services in the Kubernetes cluster using pull requests (PRs) instead of waiting for permissions to access the cluster or deploy, which can save time and improve efficiency.
Another advantage of using GitOps with Kubernetes is improved security. Because everything is managed through Git, developers don't need access tokens or kubeconfig files for the cluster, which reduces the risk of unauthorized access or security breaches. This helps to ensure that the infrastructure and application configurations remain secure, even as the development team scales up or changes over time.
It's a Sweet Declarative Mix
The benefits of declarative configuration and the ability to store the configuration in a source repository are some of the reasons why GitOps is so widely adopted. This advantage is further emphasized when GitOps is combined with Kubernetes, which is also declarative and uses manifests to deploy objects.
Storing manifests aligns with GitOps principles and simplifies the process of coding both infrastructure and the applications it supports. This would be challenging using imperative tools, as you would need to manage the configuration store and deployment parts separately.
Unless something goes wrong, you don't need to imperatively declare changes or touch the cluster. Even then, you can revert to the previous state of your manifest-backed changes. For new changes, your GitOps tool will create your objects once they are approved and merged, providing compliance and auditing capabilities in an enterprise setting.
Operational Benefits of Using GitOps with Kubernetes
GitOps also offers operational benefits that make it an attractive option for operations teams and contribute to its widespread adoption.
Improved CI/CD
Continuous integration and continuous delivery (CI/CD) is the backbone of any modern organization. The combination of GitOps and Kubernetes can improve CI/CD by providing effective ways to establish Git-cluster synchronization and management.
GitOps provides a central location for managing infrastructure and application configurations, making it easier to maintain consistency and version control during the CI/CD process and separate your CI and CD pipelines. The separation ensures that changes in one don't affect the other. New changes around all the components are also properly tested and deployed, reducing the risk of errors and downtime as your GitOps pipeline is only triggered once the artifact is built.
Another advantage is that your developers don't need to touch the infra layer and can focus on their tool kit, which contains Git.
IaC Makes Creating and Managing Infrastructure Easy
Infrastructure as code (IaC) helps organizations create environments in a repeatable manner. Through GitOps integration, you can declare both your infrastructure and application layers together, rather than treating them as distinct elements. This reduces the risk of configuration drift and human error.
The best part is that the manifest you declare can provide the required infrastructure by only stating what you need deployed and not how.
Disaster Recovery through Rollbacks
One of the most significant advantages is how easy rollbacks are with GitOps. If things go wrong, you don't need to search through cluster logs to fix issues and can just rely on reverting your changes using your SCM tool, reducing the mean time to recovery (MTTR) to mere minutes even when dealing with thousands of clusters.
If there's some configuration drift, your GitOps tool can manage and reconcile the application to a previously known working state.
Considerations While Adopting GitOps and Kubernetes
Despite all the benefits of using GitOps with Kubernetes, there are a few things you should consider before adopting this combination.
A GitOps Agent
As with any CI/CD practice, you need agents and tools to implement the pipeline and processes. GitOps can be implemented with the help of agents and tools like Argo CD, a Kubernetes operator that helps you continuously monitor the Git repository for any changes and automatically apply those changes to the target Kubernetes cluster to enforce GitOps.
There are many available options, like Jenkins X, Flux, and other agents. However, the most important consideration while moving to GitOps is to focus on how well it integrates with your existing security posture and whether it supports the desired declarative approach. Some of the tools have observability built in, which can help you assess the state of your cluster and your environment.
CI Process
GitOps primarily focuses on the continuous delivery (CD) phase of the software development cycle, which occurs after artifacts have been created. However, to adopt GitOps effectively, it's essential to have a robust continuous integration (CI) process like GitHub Actions, Jenkins, or GitLab in place.
A CI process enables teams to test, build, scan, and publish artifacts before deploying them through GitOps. Typically, the GitOps configuration repository is separate from the code repository, and the CI process ensures that the code changes are thoroughly tested and built into deployable artifacts that can be pushed to a registry.
The GitOps agent, like Argo CD, then pulls the images from the registry and deploys them to the Kubernetes cluster based on the declarative configuration in the Git repository. Having a CI process in place ensures the overall reliability and stability of the deployment pipeline.
Here's a hands-on tutorial you will find useful, how to implement GitOps in Kubernetes.
GitOps and Kubernetes Use Cases
GitOps is built on top of Kubernetes concepts, making it a perfect fit for organizations already using Kubernetes and wanting more security. It follows the principle of least access and performs compliance audits by scanning the source repository automatically. The ability to seamlessly integrate native features and automate software delivery in a human-readable way has made GitOps and Kubernetes a popular combination for many use cases.
Currently, GitOps principles are widely used to deploy applications to your Kubernetes cluster, as having a single source of truth helps you deploy the containers in a repeatable and consistent way. Automation speeds up the release process while dealing with a complex environment like Kubernetes without involving human error, which creeps in when dealing with a manual release process.
Moreover, multicluster management is an important aspect of modern applications to guarantee uptime, and with GitOps, you can make sure it's easier to deploy and manage applications across multiple infrastructures. This approach ensures consistency across cloud providers and on-premise infrastructure, making it easier to manage infrastructure and applications without figuring out specific configurations and tweaks.
Also, in big teams, when multiple teams manage a single infrastructure or Kubernetes cluster, knowing what's live in production is difficult, so having a single pane of glass across all your cluster and the deployment comes in very handy when checking your configs for security and compliance issues.
All in all, GitOps is a powerful option to increase developer productivity and expedite shipping to intricate environments while still upholding a disaster recovery plan. When trying to restore your cluster state manually to a working version, it's difficult without transparency. With help from GitOps, the state can be restored in case of failure or disaster, minimizing downtime and data loss.
Conclusion
Backed by the Cloud Native Computing Foundation (CNCF), GitOps and Kubernetes have been widely adopted together for complex environments and deployment across multiple clusters in a consistent and repeatable way. This approach also prioritizes disaster recovery and a strong security posture.
Tools like Argo CD and Flux make it possible for you to switch to GitOps with minimal modifications to your pipelines, providing easy access to benefits such as self-service, drift detection, and improved security. All you need to do is install the tools on your cluster and set up the processes.
The ecosystem is also expanding, thanks to conferences like GitOpsCon that have provided an environment where companies can share their experiences and help others adopt it. If you want to learn more, feel free to go through the GitOps book and the GitOps Days playlist.
