Security Nation, Ep. 12

How Todd Beebe Used the MITRE ATT&CK Framework to Build His Own Automated Threat Simulation System

December 17, 2019


In our latest episode of Security Nation, we talk to Tod Beebe, the Information Security Officer for an oil and gas company in Texas. Todd breaks down how he leveraged the MITRE ATT&CK framework to build an automated threat simulation system that enabled his organization to conduct daily threat simulation to validate their detective and preventive controls. 

Be sure to stick around for our Rapid Rundown, in which we are joined by Rapid7 Director of Public Policy Harley Gieger to hear his take on two recently released proposals for privacy bills. 

If you like what you hear, please subscribe! We release a podcast every other Tuesday morning.  We’ll be taking a short break over the holidays, but be sure to tune back in on Jan. 7 for an inspirational interview with Chris Hadgnagy of the Innocent Lives Foundation. 


Appears on This Episode

Jen Ellis
Vice President, Community and Public Affairs

Jen Ellis is the vice president of community and public affairs at Rapid7. Jen’s primary focus is on creating positive social change to advance security for all. She believes that it is critical to build productive collaboration between those in the security community and those operating outside it, and to this end, she works extensively with security researchers, technology providers, operators, and influencers, and various government entities to help them understand and address cybersecurity challenges. She believes effective collaboration is our only path forward to reducing cyber attacks and protecting consumers and businesses. She has testified before Congress and spoken at a number of security industry events including SXSW, RSA, Derbycon, Shmoocon, SOURCE, UNITED, and various BSides.

Tod Beardsley
Research Director, Rapid7

Tod Beardsley is the director of research at Rapid7. He has over 20 years of hands-on security experience, stretching from in-band telephony switching to modern IoT implementations. He has held IT Ops and IT Security positions in large organizations such as 3Com, Dell, and Westinghouse, as both an offensive and defensive practitioner. Today, Tod directs the myriad security research programs and initiatives at Rapid7. He can be uniquely identified at

Todd Beebe.jpg
Todd Beebe
Information Security Officer

Todd has been in cybersecurity since the early '90s. He has nine U.S. patents focused on cybersecurity and founded multiple cybersecurity startups including telecom security and the first web application firewall. The security solutions he has built have been deployed at companies across the globe, including the Fortune 10 and critical government locations. One of his passions is building the next generation of cybersecurity talent, and he offers free hands-on training through his "HuntEvil" program. Interested individuals can contact him at 

Harley Geiger
Director, Public Policy, Rapid7

Harley Geiger is Director of Public Policy at Rapid7, leading the company's policy engagement with a focus on cybersecurity, privacy, computer crime, exports, and digital trade issues. He collaborates extensively with technical experts, trade groups, security researchers, and government officials to achieve workable policy solutions that advance security and protect consumers. Harley serves on the Industry Trade Advisory Committee on Digital Economies (ITAC8) at the U.S. Dept. of Commerce, where he advises on trade policy issues related to the information security industry. Prior to working at Rapid7, Harley was Advocacy Director at the Center for Democracy & Technology (CDT) and Senior Legislative Counsel for U.S. Representative Zoe Lofgren of California. Harley is a licensed attorney, CIPP/US certified, has testified before the U.S. House and Senate, and regularly speaks at events on technology policy.

About the Security Nation Podcast

Security Nation is a podcast dedicated to celebrating the champions in the cybersecurity community who are advancing security in their own ways. We also cover the biggest events in security that you should know about. In each episode, host Jen Ellis (@infosecjen) sits down with a guest so they can share their stories, what worked, what didn’t, and what you can learn from their initiative so maybe we can inspire you to do something new, while Tod Beardsley breaks down the biggest security headlines of the week. 

View all Security Nation episodes

Podcast Transcript

Jen Ellis: Hi, and welcome to this episode of Security Nation, the podcast where we talk to interesting people doing really cool things to advance cybersecurity. I'm your host, Jen Ellis. I'm Rapid7's VP of Community and Public Affairs. With me is my co-host, the amazing Tod Beardsley. Hey, Tod. How are you doing?

Show more Show less

Tod Beardsley: Hello! I'm great.

Jen Ellis: This week, we're going to be talking to Todd Beebe, who I'll introduce in a second. One thing to note, quickly, is that we're switching it up this time around. We're going to put the Rapid Rundown to the end of the episode, so you're all going to have to wait and see what it is that Tod's ranting about this week. That would be Tod Beardsley. Oh, this is going to get super complicated and confusing.

Jen Ellis: All right, so how am I going to manage this? I'm going to call you Tod OG, and then we have our amazing guest, Todd Beebe, who is the Information Security Officer for an oil and gas company in the Texas area. I believe there are one or two of those around in Texas. Ah, you're both Todds from Texas! It's like you're doing it intentionally to confuse me. Ah! Anyway, welcome, Todd Beebe. How are you doing?

Todd Beebe: Thank you. I'm doing well! Trying to stay warm, which is kind of odd for Texas.

Jen Ellis: Really? Is it chilly where you are?

Todd Beebe: Yeah. It's been, at times, down in the 20s. That's pretty chilly for Texas.

Jen Ellis: That actually sounds pretty chilly for anywhere, yeah. Like, that's legit cold. Obviously, I'm assuming you mean Fahrenheit. In Celsius, 20 is balmy.

Todd Beebe: Yeah. Fahrenheit it is.

Jen Ellis: All right. Why don't we jump into some getting-to-know-you stuff? You mentioned to me that you worked in security for a really long time. How long is a really long time?

Todd Beebe: Since the early '90s.

Jen Ellis: Okay, that is a legit while. What is one thing that you've done in all that time that you're, like, super proud of, or that really stands out to you, as you look back across the time?

Todd Beebe: As I look back, there's actually two. The first was, I was dealing a lot with penetration tests, early in my career. Of course, this will date me. A lot of the success we had was dial-up. We could get into just about any company through some kind of modem attached to their network, or attached to some server. Ended up starting a company that we built a solution, patented it, and sold it to the White House, Pentagon, Fortune 10s. That was pretty cool.

Todd Beebe: Then, while I was at that same company, because we were a security company, we were attacked a lot. So, in one of my roles, I was kind of the technical guy, plus the CIO, was to protect our web servers from attack, and built almost by mistake, what's now known as a web application firewall. We got hit with Code Red, Nimda. Again, kind of dates me.

Tod Beardsley: Nice.

Todd Beebe: And never compromised any of our servers.

Tod Beardsley: What? That's insane! That's amazing.

Todd Beebe: Those kind of things were, kind of ... I wouldn't say by mistake, but I was in the right place at the right time. I pretty proud of those.

Tod Beardsley: No, hats off to you, Todd. I mean, everyone I know got owned by Nimda and Code Red. So I am shocked.

Todd Beebe: Yeah, yeah. I mean, we used that as our calling card. It was a good introduction into how I thought to sell security to companies. We would brag, almost, of, hey, we haven't patched the servers in two years, and we continued not to patch them, to prove, hey, if you have the right security in place, you don't have to run around with a fire response, to the latest and greatest vulnerability. That you can manage vulnerabilities if you have the right controls.

Jen Ellis: Congratulations. Those sound like some pretty huge wins, and very understandable that you're proud of those. I suspect that Tod has a question he wants to ask you. He does have a question he loves to ask.

Tod Beardsley: I do, Todd. Since you're old as hell, like me, can you tell me about a scene in a movie that depicts hacking, that particularly speaks to you?

Todd Beebe: Ironically, it would be “War Games.”

Tod Beardsley: Nice.

Todd Beebe: Because, again, no mistake, or more of a coincidence that deals with dial-up. When they say, "The best choice is not to play," security is a never-ending kind of thing. If you get into the game, you're not going to get out of it and think, hey, I got this secure, now I can move on to other things. You're going to have to continue to try.

Tod Beardsley: That's true. A colleague of mine, Egypt, who you may have heard of, he often used to say, "Computer security is like physics. You don't get to not play, and you will never, ever win in the end."

Todd Beebe: Amen.

Jen Ellis: Wow. I'm so surprised that security people are so jolly all the time.

Todd Beebe: When you accept it, then you can just try to address what you can, instead of try to fight the impossible.

Jen Ellis: Yeah, I mean, ultimately, we are in the risk tolerance game. So, understanding the realities of what you can and can't control is a pretty important principle to figure out what your risk tolerance is going to be and how to prioritize. I agree with all of that stuff.

Jen Ellis: All right, cool. Thank you. So, let's talk about what you actually came on to talk about, though. You came on to talk to us about, you built an automated threat simulation system. This sounds very impressive. Tell us everything!

Todd Beebe: So, when I was brought on at the company I'm at, it was, "Hey, we need a security program, and we need you to build it." So, it was kind of like, first person in, then I had to build the team. Maybe, because of my background of focusing on pen testing, I knew that the philosophy that a lot of people now use is, trust but verify, is I needed to know where my controls were, where my controls weren't. More importantly, after that is, where are my detections? What can I detect that blows through, or is more advanced enough to get through a security control?

Todd Beebe: Slowly but surely ... That was, you know, a couple years ago. That was when the MITRE ATT&CK, was starting to, I wouldn't say replace, but become more common. So, we leveraged that to build daily threat simulation, to validate that both our detective controls and our preventative controls were working as they should. I'm still a big believer in bringing in third parties, like Rapid7, to again, trust but verify, but we couldn't do that every single day. We still wanted to test every single day. The solutions that we were seeing out there, there's now a breach detection marketplace, what I've seen is they're usually focused on the, we'll call it the exploitation side, or on the post-exploitation, or lateral movement, and stealthy mode, living off-the-land-kind of thing.

Todd Beebe: Since there wasn't a solution that did both, and we needed both, we slowly built that over time, to where, at least on the different categories of things, whether it be web attacks, whether it be emails, there's 500 to 1,000 attempts each day, of various categories. We make sure that the alerts match the attacks, so we can see if there's any new gaps. As we pay attention to threat intelligence to say, "Hey, now the attackers are starting to focus a little bit more on this kind of area," or, they're running these living-off-the-land commands right now. We'll add those to the solution.

Jen Ellis: That's awesome. I definitely want to get more into this, it sounds really interesting. I'm going to ask you, for any listeners who are not super familiar with some of these terms, let's start with the MITRE ATT&CK framework. Can you give us a one sentence explanation of what that is?

Todd Beebe: Yeah. It tries to break down into categories, the phases of an attack. The MITRE says, okay, first they're going to do initial access. Then, they're going to gain access. They'll do privilege escalation, lateral movement, data exfiltration. It's the different stages of an attack. When you look at it that way, attackers do similar things, even if they're in different nation states. They might all be focused, for a period of time, on PowerShell, might switch up and do Microsoft documents that are going after a vulnerability. Then, they're just going after a malicious macro.

Todd Beebe: So, when you take a step back and look at the different things an attacker does, then you can mimic those and see if your defenses block them, or if your detection solutions detect them. It helps a lot to have visibility into the threat.

Jen Ellis: Great. You mentioned living-off-the-land attacks, which is something that, I think, the Verizon Data Breach Report reported were on the rise last year. For anyone whose not familiar with what those kind of attacks are, can you give us a one-sentence explanation of those?

Todd Beebe: A one sentence? They're using commands that are already on the system. So, PowerShell, Wscript, those kind of things, that's what they use. They hope they don't trigger alerts, because those are expected executables to be running on a system.

Tod Beardsley: Yeah, the idea is the attacker doesn't bring along their own malware. They just use the stuff that's already there. No more Meterpreter shells, just PowerShell.

Jen Ellis: Awesome. I think, you know, you explained to us why the project was so important, and what the motivation was behind it. How long ago were you embarking on this?

Todd Beebe: Shortly after I got here. Again, I used it more initially, of getting a sense of where are my gaps in my detection or prevention controls. I essentially did pen test myself, doing the different areas of the MITRE, and seeing, okay, what got blocked, and what didn't get detected, or what didn't get blocked.

Todd Beebe: Then, I'd bring in—in fact, I think one of the first companies I brought in, ironically, was Rapid7, because it was like, they have a different view. PowerShell might catch my eye a little bit more, but they've been in environments where PowerShell was blocked, so they switched up their attack to use something else. It's good to bring in external resources that might have a different view.

Jen Ellis: Awesome. I feel like I should be like, and you can find us at www. ... No. So, thank you for that.

Jen Ellis: How did you go about it? What did you need to do to make this a reality? What was the process?

Todd Beebe: The process was, I like to not reinvent the wheel if I don't have to. It was finding existing tools out there, most likely on GitHub, that were already geared towards MITRE. So, they were already building ... And, living off the land scripts. So, they were already building, I wouldn't call them tools, but, essentially, tools that would do a subset of MITRE activity. Then, we could run them in our environment, and say, okay, it's testing out 67 different commands. How many did we detect with our EDR solution, after we ran the tool? Then, we would find gaps, and then we would close those gaps by building better detection rules in our tools. Then, we would go out, after we, call it rinse and repeat, after we could detect all the ones that were in that tool, we'd find another tool.

Todd Beebe: There's various companies out there, whether it be Uber, Red Canary, etc. But they've put tools out on GitHub that help people detect those activities that threat actors are doing, once they compromise the system. Now, they want to either maintain that system, like get persistence, or they want to start moving around to find something interesting in the environment that they can steal or disrupt.

Jen Ellis: I like the way that the security community really does act as a community, in terms of enriching each other.

Todd Beebe: Yeah. It's almost help educating people, and bringing them up to the next level quicker, instead of ... They don't have to be in it as long as I've been in it, to learn how to defend the network, or see if a network is compromised. They can leverage that knowledge with a tool out in GitHub, or whatever.

Jen Ellis: Right, which is great. It does create that rising tide that raises all ships thing, which I think is really important, as we all figure out how to chip away at the wall of attackers.

Todd Beebe: Yes.

Jen Ellis: I love that. Have you made your tool, now, available as well? Is that something you're planning on doing, or have done?

Todd Beebe: I guess, at this point, we could. We just haven't thought about ... I wouldn't say it's ugly, but it's not production code. I would say it that way.

Jen Ellis: It's for internal use.

Todd Beebe: But, it works.

Jen Ellis: Right.

Todd Beebe: I wouldn't say there's issues, but a lot of is repurposed, we'll call it for lack of a better word, signatures from those various other tools. The tools are out there. We could add ours. What we've done that's a little bit different is a lot of those tools that are, we'll call it MITRE based, are on the detection side. Almost, the post-exploitation. Somebody clicked on a link, and now they're on a box. Now, what are they doing to get to the next system?

Todd Beebe: We've also built a number of solutions where we've got a hosted Kali sitting out on the internet. We'll be sending inbound stuff, whether it be emails, or even do outbound stuff, where we have an internal system going to that system, to try to download malicious things, and see if our proxies are doing the right thing, if our email gateways are doing the right thing, seeing if our VPN is doing the right thing. That's where we have both sides of the fence, where we're testing both detections of activity, and prevention of malicious things. That, I don't know how we would replicate to put on GitHub, but it is a good idea. Maybe we'll think about that for the next year.

Tod Beardsley: Well, that's super advanced, too, right? I mean, I don't ... I think most organizations even just struggle with asset management.

Todd Beebe: Yeah.

Tod Beardsley: What do I have on the internet that's actually reachable? If you're going all the way down to the point of, we're just going to simulate some attacks from the internet, using Kali Linux or something, and see what we detect, if you're drilling on that regularly, you're way ahead of the game.

Todd Beebe: Like I said, I think other companies could get there, if they start to think more of, hey, we need somebody who can go to GitHub, and incorporate that in, instead of buying all this stuff off the shelf. It's like, where are the gaps where there's really not an off-the-shelf solution yet? There's probably something on GitHub where somebody else said, "Hey, I'm not going to wait, so I'll build it myself and I'll share it."

Tod Beardsley: Right.

Todd Beebe: Yeah, we definitely tried to go a little bit above and beyond, because we've even gotten to the point where, on a regular basis, we refresh that malicious malware with other scripts that we've found on GitHub. So, the stuff that's trying to be downloaded is brand new. We really know, does our AV catch stuff that's, I would call it zero day, but it's not six months old? To where, okay, that gives them plenty of time to detect, right?

Jen Ellis: I mean, it all sounds really great. Have you had any hiccups as you've gone through it? Any challenges you had to overcome?

Todd Beebe: So, the main challenge was, when we considered bringing in or purchasing an outside resource, because it does take time to maintain it. Then, when you find gaps, to troubleshoot it, and it's almost, now, a new software package. Security people aren't software developers, right? I mean, they usually can script, but they're not hardcore software developers, and doing all the things that you need to make something very bulletproof. That's been the biggest challenge, is maintaining it. Again, we don't want it that, hey, it's now detecting or testing the same stuff it did six months ago. We want it to be living, because the attackers are changing their stuff. That's probably the biggest challenge, is dedicating that time.

Todd Beebe: Luckily, I have a very security-oriented senior management board, so they allow me to focus my team on what I call ... I think of CISOs and ISOs, there's two different classes. There's those that are worried about fines, and people getting fired, compliance and governance. And there's another set, which I think is a subset, is really more geared around criminal activity or catastrophic activity. So, my team's less worried, and doesn't have to focus on all the compliance, and governance, and all that. We really focus on, what are the threats that are criminal, whether they're inside the company or outside, but they're criminal actors that are trying to do? What are they doing, and can we detect and block that? That helps us free up a little bit of time to manage our own tools, until the security industry builds something that does that we really need, versus covers one of two bases, which means we'll still have to build something on our own.

Jen Ellis: That's pretty awesome, that you're in an organization that takes security really seriously, and prioritizes it. What do you think the keys are to why your organization does that, or how that behavior or culture has been stimulated or nurtured through the security team's efforts?

Todd Beebe: That's a good question. I mean, I feel like I got lucky. Maybe it was because they didn't have a security program before I got here. That, when I came in and set the tone that said, "Hey, I'm focused on threats. I've been a pen tester my whole life. I'm more worried about that."

Todd Beebe: A pen tester will always find that Windows 2003 server that people forgot about, but still has domain admin credentials on it. Compliance and governance will forget about it because they're like, hey, we got 96% coverage on our patches. It's just a different mindset.

Todd Beebe: When I told them, hey, this is my mindset. If that's a mindset you're comfortable with, then I'm the right person. If not, I'll wait until the right company comes along. I'm not a governance, compliance guy. I'll say this. I'm a selective compliance guy. I care about certain patches that I know, as an attacker, I can target. I'm not downloading the Microsoft ... I can't remember what it is, but it has, like, 2000 different registry settings that you need to "harden the box." I'm like, none of those I'll go after. Maybe, five, you know what I mean? Let's focus on those, that are threat actor-related, not just so we can check a box and tell PCI or whoever that we're compliant. Yet, those companies still get attacked because they're spending a lot of time and energy on the compliance, governance side, which means they don't have the time to spend on the threat actor, whose changing their tactics because they figured out that you blocked macros. So, now, they're going to find something else to get in.

Jen Ellis: Yeah, yeah. I mean, I think compliance is a necessarily business reality, and is also good in organizations where it generates budget and air cover for security practices. But I think it's very smart that your organization takes an approach of going beyond that, and looking more strategically at the real-world attacker impact to the business. It sounds like a lot of that comes from you, so props for that. Kudos. We love it.

Todd Beebe: Thanks. Like I said, I'm not knocking it, but I think there should almost be two different teams. Risk and compliance team, that's really worried about risk as in fines, or employees that are doing things they really shouldn't, like hitting sites they shouldn't. It isn't really criminal activity.

Todd Beebe: It's like, do you want your police officers that are out investigating crimes also working, sitting side by side with the Legislature, to build new laws? I mean, you know, they can only be in one place at one time. Do you want them on the street, or do you want them in a discussion, you know what I mean? If you're a big enough city, you can afford both. If you're small enough, or you don't have the budget, guess what? That police officer now has to go help somebody else, and he might also volunteer on the fire department. You know what I mean? It gets down to, what's the priority of the organization. I think it's slowly starting to shift to a little bit more worried about the criminal activity.

Todd Beebe: Because I never heard of Home Depot, or Target, nobody ever told me that the year before they got compromised, they failed their Big Four compliance audit. You know what I mean? They always pass them, or they have a few ... Remember, my background is pen testing, but I worked for Ernst & Young. I was working for an audit company. Like I said, it seemed like there was never an audit finding that said, okay, that's it, we're shutting your doors. You guys can't do business until you fix this. Like I said, I'm not knocking it, but if nobody is focused on that criminal activity, something bad's going to happen.

Jen Ellis: No, I agree. I think it's really important that you cover both, for different reasons, and in different ways.

Todd Beebe: Yeah.

Jen Ellis: Hey, Tod OG, you've been kind of quiet because I've been yabbering. Is there anything you want to ask?

Tod Beardsley: Oh, no. I did not want to interrupt. I think that Other Todd has a particularly blessed position, where he can just choose not to deal with compliance.

Tod Beardsley: Also, I think it's so funny that security people also put compliance in this other boring bucket. Just to extend your firefighter/cop analogy, I don't know any firefighters that just sit around and complain about building codes. People have building codes for a reason. Yes, fighting the fires is the fun part.

Todd Beebe: It's two different ... You need two different people.

Tod Beardsley: Right.

Todd Beebe: One of the companies I first worked for, they said ... It was the Gallup organization. They talked about hiring the right person for the right role, so they said, "Hey, if you have a fish, if you teach them to swim, they'll swim even faster. If you try to teach them to fly, they'll struggle with it their whole time. And birds won't do well at swimming." I mean, when you see a bird, when they're flapping around, they ain't moving like a fish. You get them in the air, they'll fly forever.

Todd Beebe: It's finding the right fit. Again, I think when they decided ... Security, even though we've been in it forever, it's still trying to ... It was getting all the buckets of, hey, you've got all the responsibility associated with compliance, governance, criminal activity, civil activity, whatever. That's all just you guys, because that's not really IT, or whatever. That's not part of keeping systems up. Over time, it was like, they built more and more compliance rules. It's like, okay, now the attackers are getting better and better. Okay, how can we focus on both? So finding the right fit, and finding the right responsibilities is kind of the challenge. Like I said, luckily I got in a place where they said, okay, "Focus on what your strength is, Todd." And that's attackers.

Jen Ellis: I think it sounds great. Exactly to your point, the fire marshal is not the same as the firefighter. I think all of us in security have to be the firefighter sometimes.

Tod Beardsley: Yeah.

Jen Ellis: Yeah, I like it. There's lots of good lessons that you can learn from the world of firefighting.

Todd Beebe: Ironically, I'm a volunteer firefighter, so that's why I used that analogy.

Jen Ellis: I feel like now we have to thank you for your service twice over. Todd, it's been an absolute delight chatting with you, thank you so much for coming on. I would like to suggest that anytime that Tod OG is not available, we could just slot you right in, and no one will be any the wiser!

Tod Beardsley: Yeah, it makes it a lot easier. Oh, Tod from Texas? Yeah, he's on this week.

Todd Beebe: You can just call me T2, because where I work, my boss, he was here first and his name's Todd. He's T1, and I'm T2. Since I'm now the second, I can just be T2.

Jen Ellis: I'm assuming that this is a Terminator reference? I'm just going to go with that.

Tod Beardsley: Obviously.

Jen Ellis: Yes, thank you. I'm sorry for spelling it out, it's kind of my thing.

Jen Ellis: All right, yeah. Thank you so much. Hopefully, at some point in the future, you might come back on again. I feel like you have lots of great stories that we would love to get into even more.

Todd Beebe: Are you bringing up my age again? That's just not fair. You were only supposed to bring it up at the beginning, and let everybody forget about that.

Jen Ellis: All right, brilliant. Thank you so much. Now, we get the Rapid Rundown. Hooray! So, in the last episode, we finished up the Rapid Rundown segment, Tod, with you had brought up that DHS CISA and OMB had both introduced a request for comments on some information around vulnerability disclosure programs for federal agencies. And I think that we wrapped that episode by saying that we were going to get Rapid7's public policy lead, Harley Geiger, to come on and tell us more. So, guess who we have today?

Tod Beardsley: We did indeed.

Jen Ellis: This is a momentous occasion because this is the first time that Harley has joined us on Security Nation. So, yay, very exciting!

Harley Geiger: Hi, everybody.

Jen Ellis: So, Harley, what's the biz with these two VDP things? What's happening?

Harley Geiger: So, the DHS component is a bit more interesting. So what they've done, is what they call a binding operational directive, or a BOD. And it is a compulsory directive for federal civilian agencies. So not military, just civilian agencies. And it is not a bug bounty and it is not a single vulnerability disclosure policy for the entire executive branch. It's a directive for each individual agency to come up with a VDP for their internet-accessible systems or devices. But it's not just an intake, either. So, it's not just like a security at directive. It also has a requirement in there about their vuln handling procedures as well. And it's a phased-in approach, so it requires a few things.

Harley Geiger: So, it requires agencies to develop and publish a VDP within 180 days. And it has to include the scope of the VDP. So, what types of testing are authorized and what types of internet-accessible systems are covered as well. How you can submit, you can submit anonymously and they have a commitment there to not pursue legal action against testing and disclosures that are in scope. And then they're supposed to expand the scope of their VDP so that all systems and services are within scope within two years of the publication of the final rule. And I mentioned it had some vuln-handling procedures. So they're supposed to set timelines for assessment and resolution, and they have to be reasonable. So they can't just catch it and keep it, and they can only have a reasonable restriction against outside disclosure.

Harley Geiger: So, there may be some restriction once you submit a vulnerability to an agency on disclosing it to the outside. But it has to be a reasonable time. It can't be out of proportion to the severity or complexity of the vulnerability. And in case anybody's worried about this, the policy is that if you submit a vulnerability through this process, it should be for defensive purposes only. So it's not going to be submitted to the federal government's vulnerabilities equities process, which decides which ones the government keeps and which ones the government then discloses to the public. And then there's some reporting requirements to CISA and that's kind of interesting. But I think this is a really big deal, actually. I mean, if the federal government can pull this off, then they're proving leadership among industry sectors.

Harley Geiger: So, OMB did a kind of a complementary thing where they essentially just issued a guidance that basically supports what DHS is doing and offers additional content on how agencies should undertake this activity. And this is important not just because it shows DHS and OMB are working together on it, but it's also—OMB runs the management and budget of federal agencies. So they're the ones that in many cases have a real stick that they can take to agencies if agencies are out of compliance with something like a BOD. So showing them working together shows a great deal of high-level support for this policy. And both of these, the guidance as well as DHS's BOD, are open for comments until the 27th. If you do want to get involved, drop them a note and just even if it's just general in support, I think that it would be helpful. So if you can't tell, we're already pretty enthusiastic about what DHS and OMB have done.

Tod Beardsley: Hey, Harley, can I jump in here with one kind of nerd-out feature of both of these requests for comment?

Harley Geiger: Please, nerd away.

Tod Beardsley: They are both being hosted at GitHub, and so here's the coolest thing is that in order to comment on these, you just open either an issue or you can edit the text itself through a pull request. It is the coolest. I have never seen this from the government. I don't know how often they... If they've ever done this before, maybe they have. But I mean I feel like they really know their audience. The people who would have things to say about this kind of stuff probably already have a GitHub account and probably are already used to this format. But I think it's great. I think it's great to have it on GitHub.

Harley Geiger: That is nerding out. That was great.

Tod Beardsley: Yep.

Jen Ellis: Firstly, you're adorable. And secondly, I feel like we know whose fingerprints are on that, so props to the individual that I think probably was behind that. Very, very cool.

Tod Beardsley: Well I was surprised it's on both. It's on both the DHS and the OMB one. So that was especially... And it looks like the OMB one is getting a lot more traffic, too.

Jen Ellis: As Harley said, the two offices have worked very closely together on this and I think that's a really... It's been a fantastic collaboration and again, great work to them. We appreciate it. So I mean I think... Thank you, Harley, for taking us through that and I highly encourage people if they're interested in vulnerability disclosure, if this is a topic they care about, take a look at these things. It is totally okay to submit comments saying this is awesome, well done. You don't have to be critical to put in comments. You can put in comments in support. And actually, they appreciate that, it's good for them to know when they're doing something right as much as to know when they're doing something wrong. And silence is taken as indifference or a lack of knowledge more than anything. So yeah, I strongly encourage it.

Harley Geiger: Yeah. So even really obvious good stuff is useful to call out in comments because then it is easier for the agencies to defend it against detractors both within the federal government as well as outside the federal government.

Jen Ellis: Okay, so Harley, thank you for walking us through the VDP stuff. I don't want to make this into the policy edition or the policy hour.

Harley Geiger: I can't wait for that, though.

Jen Ellis: Right, but it does feel like there's a lot happening in cybersecurity policy right now. Just this week, there was an encryption hearing, apparently encryption back doors are a thing again and we're going to talk about it again. Supply chain seems to be a constant topic that we're seeing in headlines. Huawei is the villain du jour. I feel like just last week, there were two Senate proposals for privacy legislation. Is this right? Am I remembering correctly?

Harley Geiger: You are remembering correctly. These privacy proposals are both in draft form and they are from senators who have a good deal of control over the process. These are not the only bills out there, there's actually quite a number of cybersecurity and data privacy bills in the Senate and the House right now. But here we're talking specifically about Chairman Wicker and Ranking Member Cantwell and the Senate Commerce Committee and that's the committee that has jurisdiction over this issue in the Senate. And so their draft bills we need to care about quite a bit. And unfortunately it was not a bipartisan joints draft that came out from both of them. So it shows that there is still some daylight between the two. I'm sure they're working together to come up with a unified bill.

Harley Geiger: But as it is right now, the fact that they issued them separately is an indication that there's still more work to do. And they are taking comments on their drafts as well. And to be clear, why we care about privacy, one is the data security, so security requirements for personal information is a subset of privacy. And it's a routine feature of any sort of privacy legislation. But then also making sure that there are exceptions for cybersecurity activity in privacy rights so that you don't have an obligation to, for example, contact a phisher that you are going to share their phishing email with another company to warn them. Right? And so the good thing about both of these bills, so the draft legislation from Sen. Wicker as well as Sen. Cantwell, is that they include both of these issues.

Harley Geiger: So they have a data security requirement that's in there and exceptions for cybersecurity activity are also in there. And there is though, one issue in particular with Sen. Wicker's draft bill that I think could be pretty problematic for cybersecurity and cybersecurity companies. I'm hoping that this will get resolved as they are taking comments on it. And that issue is this very thorny one of preemption. So preempting the states, and this is one of the main reasons why companies want a federal privacy bill, is so that they don't have to deal with the large number of state bills that come out and you have to patchwork and it's too complex and yadda yadda. But if the preemption, if you preempt the states, if it's too broad, then you end up trumping bills or trumping laws that maybe have nothing to do with privacy.

Harley Geiger: And unfortunately, the way that Wicker's preemption provision is written, does that. I'll just give you an example. It says that states would not be able to maintain or impose or require any standard, any bill, any regulation related to privacy or data security and associated activities for any covered entity. And it's like, well it's all security standards and all security laws, even if they have nothing to do with privacy, you're going to say the states can't do anything about that? I mean so this concerns us a bit. It'd be better if the preemption were scoped to the actual requirements of the bill.

Jen Ellis: Right. Yeah. Right, right. Okay. So I mean it sounds like a thorny issue and there's probably some distance travel. So not super likely to see anything moving on this all that soon. Is that fair?

Harley Geiger: That's right. I think we're not looking at this moving in this year. However, I think that next year, if it's going to move, it's because it's an election year, it'll probably be within the first quarter would be my guess.

Jen Ellis: Okay. So is there any cybersecurity policy that you think might move before the end of the year?

Harley Geiger: I'm hearing that there is an IoT bill, I don't know if you all have discussed it before, but we've discussed it on the Rapid7 blog a few times. An IoT security bill that has a chance of moving. This is popularly known as the Warner Gardner IoT bill. It's the IoT Cybersecurity Improvement Act, and it does put into place some requirements for cybersecurity, for Internet of Things devices that the federal government procures. So as a part of the procurement process, they have to meet certain minimum standards for cybersecurity. And it, too, by the way, has a VDP component to it.

Harley Geiger: We're hearing that there is an effort now to sort of unify the House and Senate versions of this and to hopefully get it through the finish line before the end of the year, and good luck to them because there is not a whole lot of time. But still it's pretty exciting if they're able to actually get it moving. And I would just draw folks' attention to Jen's blog post on this at our Rapid7 blog, which goes into a good deal of detail about the differences between those two bills and what we like about it and what we don't.

Jen Ellis: Yeah, thank you for that. I appreciate it. And I will say that if you start on policy blogs on Rapid7 blog, it is a rabbit hole you can fall down because Harley has also penned many wonderful blogs on these topics that he's talked through today and other topics. But on those bills there is, I think, some distance traveled before they're really ready to be passed. So we'll hopefully see them come together and create something a little bit cleaner and more robust before it goes through. That's my hope. But it would be cool to see something go through and just hope that they do it in the right way. Harley, thank you so much for joining and taking us through all of this stuff. We really appreciate it. Appreciate you breaking it all down for us very much. And Tod, I appreciate you being as vigilant on GitHub as ever.

Tod Beardsley: I heart GitHub.

Jen Ellis: Thank you for co-hosting, as usual. Bri. Bri. Bri, thank you for putting up with us and getting us back down to the time that we were targeting. All right, until next episode. Thanks everyone. Oh, and happy holidays! I think this is it. We're out until the new year so the next time you speak to us, it'll be 2020. Oh my god.

Tod Beardsley: We'll be in the future.

Jen Ellis: Oh wow. Sci-fi ending.