Last updated at Tue, 26 Dec 2023 19:39:10 GMT
Series co-written with Jeremiah Dewey, Rapid7 Director of Incident Response.
Part 3: Putting the Incident Response Plan to the Test
Earlier in this series, we’ve walked through creating an incident response plan, and giving it a proper review. Now, it’s time for the fun stuff. While an IR plan review may feel like practicing moves on a Mook Yan Jong (wooden dummy), stress testing should feel more like Donnie Yen fighting ten people for bags of rice in Ip Man. Serious training is the only way to cement the proper mindset and process your moves into muscle memory.
Today, we’ll cover ways to test, tips for success, and how to evaluate the results.
Tabletop Exercises
Tabletop exercises are a great, lower-stress way of testing your incident response plan. They are typically half-day exercises that recap how a cross-functional team will respond to an attack scenario. While this may not involve simulating an incident on your network, it should be taken seriously. This is a dress rehearsal, not a training class or a review of the incident response plan.
A successful exercise hinges on the planning you bring to the table. Items to consider:
- Who are you bringing into the exercise? Is it an executive exercise or technical in nature?
- What is the tabletop scenario? Is this realistic, taking into account your expected adversary (opportunistic vs targeted) and current response capabilities?
- What should the exercise look like? Have a basic script and flow in mind so that the group can make the best use of time.
Red Team, Blue Team, Purple Team
With a general response workflow mapped out over tabletop conversations, your team can proceed to the next level. If you have internal teammates with penetration testing expertise, you can certainly consider red-blue team exercises, pitting one group of security pros in simulated attacks against another. Penetration tests are great events to plan around, to test both your detection and response capabilities and your coordinated response.
It’s important not to limit your vision to only the technical details, as a thorough incident response requires the help of teams across multiple disciplines. For example, the security team might begin by working with IT to properly isolate affected machines from the network. The PR team and other members of the task force should join the established communication channel to receive recurring updates. Set realistic expectations for the frequency and details of these updates so the team can focus on working through the incident instead of fielding a constant feed of questions.
Now, start the mock-attack.
Go All In: Testing Your Incident Response Plan
For best results, replicate an attack as fully as possible. For example, if you’re testing the IR plan during a penetration test with an outside firm, don’t tip off the company once you detect them on the network. Are you able to follow their movements and identify privilege escalations, lateral movement, and other malicious behavior? This may be a good time to try alternate communication channels so you don’t fall into disarray if your primary channel is compromised. Test your team’s ability to operate outside its comfort zone. Push your limits to see what breaks.
Preserve forensic data. While your first impulse upon identifying an incident may be to get compromised machines off the network, you should consider the counterintuitive: keep the machines running. You don't want to lose important forensic evidence, such as system memory.
Properly Document
Paper trails and electronic evidence are vital to managing the response to and aftermath of an attack. Evidence that has been tampered with, for example, may be viewed as inadmissible in a court case to determine culpability.
For those reasons and more, teams responding to attacks must ensure reliability of evidence about who in their organizations did what, when, and why.
Documentation also helps investigators get to the bottom of how an attack unfolded. Did human action or inaction contribute to events? Did a defect, malfunction, or absence of equipment play a role? Could the attack have been the result of insider threat, ranging from a negligent to a malicious employee? Proper documentation over the course of an incident will be the foundation to answering these questions.
Review the Performance of Your Incident Response Plan
Lastly, have an after-action review. This will help your organization see what worked and what didn’t. Then you can apply any lessons learned to the plan to better guard against future attacks. Successful review requires that you identify your true risks and be honest about how you address (or in some cases, accept) them.
A solid plan is a constantly moving target, and it must evolve to meet new attack vectors and an ever-changing network. Consider these three purposes as benchmarks for your plan:
- Evaluation of your detection and response program
- A way to find tangible recommendations for improvement
- Broaden the range of incidents the team can tackle with confidence
The plan review is also a good time to reevaluate what works best in-house and what should be done through external collaboration in executing the plan. And consider an incident response retainer with a trustworthy outside partner such as Rapid7.