Automated remediation can be an effective tool for ensuring system security–provided remediation policies are configured in a way that is appropriate to a company’s software release process. There are few things more distressing than having a remediation tool that’s intended to avoid disaster inadvertently create one.
For example, imagine adding a new security policy to an automated remediation system that’s intended to restrict a container from having root access to its host. At first glance, this policy is a reasonable addition, but after deploying the policy into a production environment, the result is that a number of preexisting containers that require root access are made inoperative. Thus, system failure occurs. What started as a good idea turned into an IT disaster.
Clearly, the scenario described above this is one that needs to be avoided. The question is, how? After all, the scenario’s policy rule is sound. The problem is that the rule was introduced too late in the development cycle. The rule should have been introduced earlier in the software development lifecycle—for example, in the testing or staging phases of the release process. Of course, introducing the remediation rule early on will still wreak havoc, but the failure will occur in a “safe” environment in which the problem will be exposed and fixes can be put into place. Then, once the troublesome behavior in the software was corrected, both the new policy and the new code can be moved in tandem into production.
While the scenario illustrated above is a bit dramatic, it does provide a good example of the importance of establishing appropriate remediation policies throughout the entire software development process. A good set of remediation policies will react to security and best practices violations according to both the degree of severity and the release phase in which the violation occurs. Draconian severity responses might be appropriate to execute in testing and staging phases, yet completely unwarranted in production environments, and vice versa.
Getting started with automated remediation
One of the benefits of DivvyCloud by Rapid7 is that a response to a given violation is configurable according to the needs and maturity of the given IT organization. Companies that are just starting out with automated remediation might do well to respond to problems by sending out emails or notifications in a Slack channel and leaving physical remediation actions in the hands of a developer or system administrator.
Other companies that are further along with automated remediation and are more trusting of the technology will impose more stringent remediation behavior in response to a policy violation—for example, gracefully stopping a build or safely removing a container from a cluster. Adopting automated remediation is not an all-or-nothing undertaking. It can be done in an incremental manner by introducing more powerful remediation automation over time as companies become more skillful using the technology.
Few companies get remediation automation right at the beginning. It takes time to establish a set of remediation policies that work. The important first step to using remediation automation effectively is make sure that all members of a company’s IT staff are committed to using remediation automation. Once the commitment is made, a company then develops appropriate remediations policies in an iterative fashion that fit the needs of the enterprise’s day to day operations.
The world of ephemeral computing using the cloud, containers, and Kubernetes continues to evolve in ways that are both innovative and challenging. Change happens so fast it’s hard for Security and GRC professionals to keep up. But there is help available. DivvyCloud automation allows developers to engage in more experimentation and innovation while also providing the trust and verification that system administrators need to ensure that work is being done according to industry standard security guidelines and well-established best practices.
Interested in learning more? Speak with a DivvyCloud expert today!