I propose that there are three fundamental and concrete practices DevOps and security teams can adopt to add security into the CI/CD pipeline and secure critical applications, involving:
- Infrastructure-as-Code (IaC).
- Kubernetes application manifests.
- Containers and container images.
But before discussing these technology-related components, it is important to highlight salient aspects pertaining to the people and process components of the application lifecycle.
People and Process
I've previously discussed solutions for the cultural stalemate that can arise between DevOps and security teams in the people and process part of cloud native adoption. For this article, I'll add a few additional statements.
DevOps Teams Do Not Need to Be Security Experts
It is not a secret that security is hard to get right. Security teams are resourced with highly skilled members with the domain expertise needed to secure the critical assets of an enterprise. But they cannot be expected to be experts on the code for every business application. Similarly, DevOps teams are highly skilled at building, deploying and running complex applications, but cannot be expected to be security experts.
However, as the principal authors and architects of infrastructure and applications, DevOps teams have the incentive to deliver quality products – and security can be a part of quality. DevOps teams just need to be armed with tools that help them build security in from the start.
Purpose-built security tools can provide DevOps teams with security failure and vulnerability information, including how to make fixes. Armed with rich and contextual information, DevOps teams can remediate and address security failures in the same manner they would address failures arising from the QA testing process.
Additionally, modern and purpose-built security tools integrate seamlessly with existing DevOps processes, making adoption less burdensome. For example, security scans and checks can be triggered within an integrated development environment (IDE), when a pull request (PR) is executed or as an additional step in the CI/CD pipeline.
Security Teams Require a Trust-But-Verify Posture
Traditionally, security teams don't obtain visibility into applications until the last minute. They must scramble to incorporate best practices for compliance, vulnerability management, configuration checks, network security and least-privileged access. This contributes to an undesirable reality where security is always "bolted-on" at the end.
In contrast, the ideal posture is to ensure that security is injected throughout the development process, in close collaboration with the DevOps teams. Such an approach enables DevOps to deliver applications at a rapid rate, while security teams have the requisite oversight to ensure applications are deployed in accordance with enterprise security policies and compliance requirements.
Importantly, the security teams must trust DevOps teams to incorporate the desired security capabilities. Even still, security teams can still verify that security has been incorporated by using a lightweight footprint, meaning, using tools that offer visibility into the CI/CD pipeline but remain decentralized – as it should be. This truly affords a win-win scenario for both teams, with the ultimate winner being the business.
Security Teams Need to Empower DevOps
To recap what has been stated so far, bolted-on security is undesirable and potentially insufficient to protect enterprise applications; the ideal scenario is one wherein security is incorporated as a non-functional requirement for DevOps teams; security teams should work to provide the tools and processes to satisfy those requirements in a DevOps-friendly manner.
The benefits of such an approach are that DevOps teams rightly assume responsibility for satisfying security requirements, while security teams are responsible for empowering DevOps with tools to achieve these requirements in a cloud and container native manner. There are powerful benefits when DevOps and security teams agree on how and where to incorporate security into the CI/CD pipeline:
- Security is not a bolted-on afterthought.
- Security is treated as a "first-class citizen," similar to other app components such as implementation errors and software bugs.
- Security testing and failures are identified early and addressed early on – and more importantly, prior to application deployment, when changes are more costly.
Putting Secure DevOps into Practice
You can see the interdependence of people, process and technology, and their effect on a successful cloud native security posture. Let us now pivot to the specific technology steps that can be taken by both teams in order to adopt and realize the ideal state for secure DevOps.
1. Scan Infrastructure-as-Code Templates
DevOps teams have adopted IaC templates – such as Amazon CloudFormation, Azure Resource Manager and Hashicorp Terraform – in conjunction with automation to rapidly deploy, tear down and manage infrastructure. However, Unit 42 Cloud Threat research on IaC templates has revealed that a large number of templates contain misconfigurations that do not comply with either best practice or enterprise security policies. This results in the deployment of insecure infrastructure that could potentially result in exploits and breaches.
This problem can be overcome by incorporating IaC template scanning, using purpose-built security tools that:
- Seamlessly integrate into existing DevOps processes such as IDEs, PR and CI/CD pipeline steps (as discussed above).
- Provide the results of a scan, the misconfiguration identified and relevant information needed to address the security failure.
Scanning IaC templates within the DevOps process, prior to the deployment of the infrastructure, greatly reduces the occurrence of security incidents in the runtime environment.
2. Scan Kubernetes Application Manifests
The predominant approach to deploying applications on Kubernetes platforms is by declaring all the relevant parameters in a YAML file called an application manifest. However, the same Unit 42 report cited above found that 42% of Docker containers leveraging open source software contain insecure defaults, which can result in the deployment of vulnerable containers and applications.
The Application Container Security Guide from the National Institute on Standards and Technology (NIST SP 800-190) lists controls for containers and containerized applications that should be incorporated in order to secure these applications. Inserting security tools that can scan for these controls in the DevOps IDE, PR workflows and CI/CD pipeline steps helps support compliance and other best practices.
The benefit to DevOps teams is that security scans can identify insecure and noncompliant configurations prior to deployment, where they still have the ability to make the necessary corrections and fixes. For example, a security scan can identify and alert an engineer if a container image is either run as the root user or with escalated privileges. Detecting vulnerabilities at this earlier point – rather than detecting them in runtime after the container has been deployed – also greatly reduces team friction.
3. Scan Container Images
This third and final step involves scanning the container image – which is a lightweight, standalone executable piece of software. Because container images are often pulled from public repositories – essentially untrusted sources – they can be a major threat vector for containerized applications. However, with purpose-built security tools, it is possible to provide detailed vulnerability information to DevOps teams at multiple points during the development and deployment cycle by performing an image scan when the image is built and again in the CI/CD pipeline.
The scan results provide a detailed account of the vulnerability, its severity, the status of a fix and the version of the software package in which the vulnerability has been fixed. A DevOps engineer can easily address and fix the vulnerabilities in a prioritized manner as determined by security policies. These steps help ensure only sanctioned and secure images are deployed.
Takeaways: How to Add Security Into the CI/CD Pipeline
By adopting these three concrete steps, people on DevOps teams can maintain lockstep with security requirements early in the build and deploy phases, greatly enhancing agility and the deployment of secure applications. This posture is achievable due to the availability of powerful security technology that DevOps teams can incorporate into existing processes such as PRs and CI/CD pipelines.
Concluding takeaways:
- Purpose-built security tools provide early, context-rich information to DevOps teams.
- These security tools seamlessly integrate with existing DevOps tooling and processes.
- Enterprise lines of businesses are secured as both DevOps and security achieve their domain objectives.
To learn more about implementing DevSecOps strategies and hardening the CI/CD pipeline, you can check out the on-demand webinar I recently hosted with CodeFresh.