5 ways you can help fulfill the promise of secure DevOps

DevOps got us to where we wanted—infrastructure that’s fast, easy to deploy, and requires a lot less maintenance. But one thing the industry is still searching for is secure infrastructure as code. Faster deployments and more hands-off configurations have led to less focus on properly securing deployments and less demand for security best practices from DevOps teams. This “cutting corners” approach, if you will, has meant more incidents, less visibility, and even more fatigue after chasing down alerts and alarms.

The reason for security teams to care about DevOps is simple: the faster and more easily infrastructure is deployed and the less visibility you have on configurations and risk management, the more likely you are to face threats. Add in more complex tools like containerization and you have a recipe for unmitigated security risks. Without the right controls and visibility, you open up a massive attack surface across your most critical systems, constantly enabling incidents that may never get noticed.

Turn the ship around

These five key investments can help security teams ride side by side with DevOps without sacrificing the speed and ease of infrastructure-as-code deployments. By making security a focus of an automated deployment pipeline, security operations teams and incident responders can increase their visibility into security risks and threats to better protect an organization’s most important applications and services.

1. Boost security awareness and training

Start with the basics. DevOps teams are not security teams, even with more “DevSecOps” type roles appearing. DevOps teams generally have wide and deep training on the intricacies of tools and platforms like AWS and Terraform, and bring that expertise to the table. What they may not have is full security awareness and training to guide how they develop and deploy.

Give them some help. Increasing security advocacy within DevOps—whether through trainings, pairing, or hiring—can help close the gap and ensure that security concerns can be addressed as part of the infrastructure management process. Programs like the SANS Cloud Security and DevOps Automation course can help close the gap and spring up passionate advocates for security overnight. Partnering with DevOps teams—instead of relying on brute force risk management policies and the annual pen test—gives your infrastructure leaders true ownership in the outcomes.

2. Set the standards through code

Infrastructure as code allows DevOps teams to define what a proper deployment looks like ahead of time. These early changes can include steps such as having the right operating system (OS) for the job, with the right applications and services enabled by default. This diligence upfront means that developers only need to follow the process to launch infrastructure that supports their needs.

We can do the same for security by investing the time upfront to create more secure, hardened images that remove the need for individuals to adjust a deployment to close a gap. These steps can include ensuring that only right roles and permissions are granted on a server, that ports and network connections are locked down to block unknown connections, or that the underlying OS and applications are up to date with no known open vulnerabilities. It also helps to make sure sensitive data like passwords and keys are properly stored and not shared in plaintext. Knocking out the basics of cloud posture management at this stage can reduce the attack surface and help prevent myriad security issues when infrastructure is deployed.

3. Automate as much as possible

One of the promises of infrastructure as code—automation—is also a benefit to better securing infrastructure. Once you’ve properly designed and hardened an image for your deployment, each future deployment should be consistent, with the same security controls and base image. However, when DevOps teams have to break automation to deploy manually or respond to an emergency, the same diligence to make sure images are secure and properly configured can be dropped.

Investing more into automating the entire infrastructure management lifecycle, including accounting for hotfixes or upgrades, can help prevent these security lapses ahead of time. That way, when DevOps teams or developers need to break the process to build out minimum viable products (MVP), new infrastructure, or release hotfixes, they have the right controls baked in from the start. While this approach cannot catch all cases, it can reduce the potential attack surface caused by even just one misconfigured instance or application.

4. Reimage and redeploy frequently

One of the benefits of infrastructure as code is that deployments are fast and can often be spun up and down as needed. That same ephemeral infrastructure has benefits for both better securing workloads as well as reducing the potential for a pervasive threat. Ephemeral workloads are spun down frequently, meaning their attack surface should be smaller, but without the right controls, they can introduce massive new security issues.

If your infrastructure-as-code program encourages regular redeployments as well as frequent patching and version updates on the base image and applications, your infrastructure will run the latest and most secure versions most of the time. This can mean setting up your infrastructure to regularly spin down unneeded or old instances (containers help) or just manually refreshing infrastructure after a set time period. Similarly, if your infrastructure is more ephemeral, attackers have less of a chance to move laterally or bring down your entire infrastructure before everything can be spun down or redeployed. Make sure containers (and all infrastructure) still have basic hardening practices such as locking down permissions and access to reduce risk during container lifespans.

5. Take runtime threats seriously

The big challenge in securing infrastructure deployments broadly has been the same for years: runtime threat protection. The concept of reducing risk—patching, properly configuring and hardening systems, or restricting access—is more common and serves as your first line of defense. However, even the most up to date and patched server can be vulnerable to zero days or other attack vectors. Monitoring for threats, along with the ability to catch things like suspicious network connections or rootkits as they take action, is an important emerging security challenge in your infrastructure. This second line of defense allows you to immediately take action should a threat appear and reduce the impact of malicious actors. The key is to have tools to detect runtime threats present on every piece of infrastructure you run, as a single gap without protection can mean compromise.

You don’t have to do it alone

Solutions like CWP or EDR platforms can help bring threat visibility to the infrastructure deployed as part of your automated deployment process. These tools look for signatures of malicious activity or specific behaviors associated with an attack. They serve as a second layer of protection behind properly configuring your infrastructure. They can also serve an inventory management role, helping you get eyes on the entire deployment. Services here can include everything from open source tools like osquery to inspect for suspicious behaviors, to managed services that review and triage alerts for you, so that you can focus on response. Including these types of sensors or services in your base image means that, as infrastructure is spun up, you get instant visibility into any signs of an attack and can take action. Things get more tricky when dealing with containers or Kubernetes deployments with limited lifespan and visibility, so ensure that your tool of choice supports all of the environments you run.