Proactive Security Is the New Black: Lessons from the Trenches of Building a Security Product

Tod Beardsley:

Hello, podcast world!

Jen Ellis:

And look at us, we're sitting in a corridor together in San Francisco.

Tod Beardsley:

On day what, -1, I guess, of RSA.

Jen Ellis:

Of RSA. Which is why you can tell I'm super excited about everything. And today... Well, this is a first for us. We've never actually had the great privilege and opportunity of having-

Tod Beardsley:

The guest in the room, yeah.

Jen Ellis:

Sat next to us.

Alex Kreilein:

This isn't a room, this is a corridor.

Jen Ellis:

No, we lay out all the stops. We really go for it to make you feel special.

Alex Kreilein:

Awesome. Don't worry. I'll take it out of the Rapid7 party line.

Jen Ellis:

So, I would like to introduce you to Alex Kreilein, who's CISO of RapidDeploy. It sounds like it's just a thing we made up, like oh, Rapid7 at Rapid... Yeah, no. It's a real company.

Alex Kreilein:

We're probably real. We're probably real.

Jen Ellis:

Yeah. Like, why don't you tell us? Tell us. Prove it's a real company.

Alex Kreilein:

For sure. So my firm, RapidDeploy, was started about three, three and a half years ago. Really amazing and thoughtful co-founders. I'm super lucky to be able to work for these guys.

Jen Ellis:

Wait are you one of them? Did you just give yourself a big pat on the back?

Alex Kreilein:

No, no. God I wish, that'd be so dope. No, so these are two really wonderful guys who came together in Cape Town, South Africa, which is where our company was started. And what we build is basically the backend of 911 and emergency communication systems. So it's cool. It's a life and safety system. It's really hard, but we're doing it in a novel way, because the way that it's traditionally been done is literally the most old-school IT project you could imagine. And we do everything as a cloud-native SaaS solution in Azure Gov. We are running really cool and interesting technology stack services. We're trying to do everything through CI/CD, everything with the infrastructure as code. So, we're an actual modern novel SaaS company and a hardcore tech company, but it has this great mission where we get to save people's lives instead of optimizing shoe shipping, right?

Jen Ellis:

That, I mean... that legit sounds pretty damn cool.

Alex Kreilein:

It's awesome.

Jen Ellis:

But, how open are you to changing the company name?

Alex Kreilein:

I'll totally take that to our marketing officer. Yeah, yeah.

Jen Ellis:

You heard it here live.

Alex Kreilein:

Yeah. But I've been CISO for about a year and a half. And we're getting our information security program really up and rocking. And it's been hard, because one of the hard things is we've moved, as I'm sure a lot of companies that you guys work with as customers, we move really quickly. We've come this really far distance in a short period of time, and it's hard because security tends to not do that. Security moves really slowly, and they're like, captain handbrake. And so most of the people who I interviewed to work on my team, or most of the vendors who I looked to work with to integrate it into our stack, they are perfectly appropriate for 2014, and I wish them jobs in the past. But we're in the future, and it's hard to work in the future because we're the first to deploy a thing. We're the first to deploy a past service.

Jen Ellis:

I mean it's literally in your name. RapidDeploy. It's like a whole thing.

Alex Kreilein:

It is. It is. But it's so hard to be first, because there's no one who's there to help you, and there's no framework or rule book. And in security, we tend to have these very old constructs, and are people like, well they're extensible to anything. And I'm like, really? Is it extensible to a CI/CD pipeline based in Terraform with like deployments and Kubernetes? They're like, what are those words? Cool. Cool. Awesome.

Jen Ellis:

Oh my god, Tod's so happy right now. He loves when we have people on who took tech to him.

Alex Kreilein:

Yeah. Dope, man. It'll be fun.

Tod Beardsley:

I'm very excited.

Alex Kreilein:

Yeah. So, that's our background, and that's some stuff about us.

Jen Ellis:

That's cool. Awesome. So before that, what were you up to?

Alex Kreilein:

Yeah, before that I ran a small investment fund, where I've focused on investing in cybersecurity startups.

Jen Ellis:

Nice.

Alex Kreilein:

And I had my own in parallel with that called SecureSet, which was the country's first cybersecurity boot camp. And it was awesome. It was super important. I worked for a long time through the federal government, on the Hill, which was tough, and in the inter-agency, which was tougher for other reasons. And the No. 1 thing that I found was, I just couldn't hire great people. They're just like... It's not like there's some full-stack dude who's awesome, who's like also I rock in security engineering, and I can build everything from your infrastructure to your AppSec programs. So because they don't exist, we have to teach them. And now I'm learning that the hard way, as a CISO at a company of like, man, they really don't exist. I was super right. So yeah, that's some background on me.

Jen Ellis:

I'm gotta say, Tod, I kind of love that with this gig, we get to talk to really interesting people.

Tod Beardsley:

Yeah.

Alex Kreilein:

You think I'm interesting? You should talk to my fiancée. She's way cooler than me.

Jen Ellis:

When's she coming on? Is she available?

Alex Kreilein:

Yeah. God, I wish.

Jen Ellis:

Let's just end this interview now. When can she get here?

Alex Kreilein:

Yeah, yeah. Totally.

Tod Beardsley:

Cool. So, I guess we can... Well, before we jump into it. I would like to ask you a question Alex, and that is, what... Can you just tell me what hacker movie, hacker TV show, hacking scene, that speaks to you?

Alex Kreilein:

"Sneakers." It's 100% "Sneakers." And it's Robert Redford, coming down from the ceiling into that "controlled room." Only because I wish I was as cool and good-looking as Robert Redford. That's literally it. Otherwise, it'd be like some dope thing from "Hackers" or "Johnny Pneumonic," or something else. We're all nerds for this.

Jen Ellis:

“Johnny Pneumonic.” We've never had that as answer.

Alex Kreilein:

What?

Jen Ellis:

I like that as a reference point.

Alex Kreilein:

Oh yeah.

Jen Ellis:

That's great. Particularly as we are in a city with Keanu Reeves right now.

Alex Kreilein:

That's exactly right.

Jen Ellis:

He's coming on later, I believe. At any minute now, he's going to come out.

Alex Kreilein:

I just assume that anytime there's an opportunity to play with small firearms, he's close by, so.

Jen Ellis:

So, why don't you tell us about a project that you're working on to advance security? Since it's kind of the thing that we do on this podcast.

Alex Kreilein:

So, I ran a tiny investment fund, it was really small. But we were looking at companies that were trying to focus more on prevention. And RSA is a perfect conference to talk about how we've just given up on that. And it's really exhausting. And I think honestly not only it's a defeatist and shortsighted, but I think it's a total cop-out, and a complete excuse for just not doing your job. Every time I have an EDR alert that pops, that means that I didn't do something properly, that allowed someone to get to a place where they got on an endpoint and hit me in the first place, right? Every time I get a UEBA alarm, right? That means that I didn't have a first order control that worked. And if I had more of those, and I focused on that, then I would have less that would be needed in detection and response.

Alex Kreilein:

But people have given up because they've just been told that it's impossible to do, and that it's a losing battle. And that may be true, but to say that you've lost before you ever started is exhausting. And I think the seminal quote from Gen. Douglas MacArthur is, “Wars are lost on two words: too late.” And so, it's delightful that you want to focus all of your time and effort into this data science project of, I'm going to put agents everywhere and sensors everywhere. And I'm going to integrate everything to my SIEM, and it's going to be amazing. I'm like, cool. So what have we talked about for configuration management? They're like, oh, that's so boring. I'm like, okay, so you don't believe in configuration management, patch management, or vulnerability management. So you're just down with just saying there is no perimeter. And I get it. It's not always going to work. These aren't mutually exclusive, though. And this is the big piece. So I have a good EDR and UEBA program. It's not perfect, but it works really well. So now I'm taking the opportunity to "shift left."

Tod Beardsley:

And just to expand those acronyms, EDR is, Endpoint Detection Response. UEBA is?

Alex Kreilein:

User Entity Behavior Analytics. It's your product, you should know that.

Tod Beardsley:

No, I'm explaining it to the audience.

Alex Kreilein:

Yeah.

Jen Ellis:

Oh, sorry. Normally I do that shit. But because I was like, oh I totally get what you're talking about. It's cool.

Alex Kreilein:

No. I can see all the stars in your eyes.

Jen Ellis:

I am. I'm very like ahhh.

Alex Kreilein:

Yeah. So, I love the vendors I work with. I'm really proud to be able to work with some great ones. But the problem that I'm trying to solve is not something that is offered very easily and simply by any vendor solution. And that's, configuration patch and full vulnerability remediation management. So how are we doing that? We have both a benefit and a detraction, and that we're completely cloud-native. And this is not a requirement in order to try this project, but it's helpful. So, what we're doing is, I'm taking all of the NIST 800-53 controls, and I'm writing them in a system security plan that's written in Markdown language. All of my docs, my entire compliance program is written in code. Now, it's Markdown, so it's not real code, but it's code. And what that enables me to do is machine-read it. I can version-control it, I can rock out with it.

Jen Ellis:

This feels like something that somebody should have open-sourced by now. Somebody should have done it and just made it-

Alex Kreilein:

No, I'm 100% doing exactly that.

Jen Ellis:

That's awesome. I love it.

Alex Kreilein:

So, I'm going to write big awesome systems security plan that's going to be continuously updated, and I'm going to strip out certain values that we don't want everybody to see, and I'm going to give it away to the world.

Jen Ellis:

Oh my god, the fact that you're open-sourcing it. Okay, now Tod's got stars in his eyes.

Alex Kreilein:

Yeah, yeah. So, when we do this, when we do this important work, one of the first things that we find is, because we have everything in a machine-readable format, we can do some really interesting things. And one of those, and this is what I'm going to be working on over the next couple of months, we're going to take that NIST 800-53 baseline that we're writing to. And for us it's going to be a moderate-impact baseline, and it's whatever. But we're going to take all of that. And I know that there's 159 controls that I need to implement, and I think I can do about 140 of them through Terraform. And we're going to write the entire control framework in Terraform, and we're going to run everything through CI/CD.

Jen Ellis:

And for those who are not familiar, like me? What is Terraform?

Alex Kreilein:

So, Terraform is a declarative language. So, it's not like C sharp, which is like... I cannot understand that. I mean, I can, but it's not my jam.

Jen Ellis:

Isn't that a musical note?

Alex Kreilein:

Yeah, for sure. So, it is a musical note.

Jen Ellis:

Yeah. Killing it.

Alex Kreilein:

Yeah, so. Unlike F-, which is my score at writing C sharp. So, I'm-

Jen Ellis:

Nice. Oh, I like letter puns.

Alex Kreilein:

So what we can do in Terraform, is it extracts away a lot of the things that people normally get hung up on. Which is, having to declare out absolutely everything in the most granular way. And I can just say, for example, on this subscription ID, right. In this region, I want to do X, Y, and Z, right? So, as an example, you use a password policy as an example. I want to enforce that on my cloud estate, we are using passwords for accounts, and including everything from a database, up to a credential ... an account for a human being, or an identity, that use this convention for passwords, right? This name and string and all that length. Now, normally you would do this by hand. You would log in to an interface and you would click stuff on a GUI, and then you'd have to scale that across your n'th number of subscriptions. So if you have 300 subscriptions, which of course you wouldn't, but if you had a huge amount of subscriptions doing that across all of them by hand all the time, is really complex.

Jen Ellis:

Right. And boring.

Alex Kreilein:

And boring. And it's labor-intensive, and you're going to make mistakes, because we're humans. And what we're trying to do, is we're trying to-

Jen Ellis:

You and I here, really.

Alex Kreilein:

Yeah, yeah. Tod's not even around. But what we're trying to do is something really stupid, which is we're trying to make ourselves as good as machines. And we're asking ourselves to do things that machines are great at. Instead, we should ask ourselves, what are we really good at? Communication, context, creativity. That's what humans do really well. And then we're going to build a program around that, and we're going to declare things in easy to write and understand languages. So, once we have all of our 140 controls written in Terraform, which you can absolutely do, then we're going to ship them off to our DevOps team. We're going to say, look, this is your template, right?

Tod Beardsley:

Well it's like a spec.

Alex Kreilein:

It's a spec. Yeah. This is the strategy we want to implement. It's up to you guys to make this live, right? We're going to implement that through CI/CD, which is continuous integration, continuous deployment, or delivery, depending on who you ask. So we're going to inject those onto our cloud estate, and it will configure the cloud infrastructure. And then separately, we'll use other tools like Puppet, or Chef, or Ansible, or others to configure the operating system environments, and we'll be able to configure how Kubernetes gets run. We'll be able to configure how our infrastructure gets to operate, and we're going to set up guardrails for the one big purpose of allowing our developers to be as creative as is humanly possible.

Alex Kreilein:

I would never want to limit the creativity of the really brilliant people who we pay good money to come work for us. And so, every time that I want to implement a control, I get to do so as guardrails, instead of saying absolutely not, right. I get to say absolutely yes, but within these parameters. My job is to create a box of safety, so that they can work and do anything within that box, and then to enforce that they can't go outside.

Tod Beardsley:

So I'm not particularly familiar with Terraform. When you were describing it, I thought oh, this is something like Ansible. But what you're actually doing is like ... it's like a spec for your fleet of Ansible scripts, right? Or Puppet?

Alex Kreilein:

Yeah. So, it's a spec for your cloud infrastructure estate. Now you can do a lot of that stuff through Ansible, or Puppet, or other techniques. There are design trade-offs. We've selected Terraform. But also if you're on AWS, you could use cloud formation. If you're on Azure, you could use Azure policy, or the resource managers, or as your blueprints, which are actually written and declared statements. But the issue with that is, I still have to tap into each one of the environments and touch things and do all this stuff. So this removes it one level back so we can basically have a read-only subscription, which is our ideal end state, where absolutely everything goes through CI/CD. So we have check-ins and check-outs for everything. We have review of code.

Jen Ellis:

You're empowering your DevOps people to go do their thing, basically.

Alex Kreilein:

100%.

Jen Ellis:

So that you can basically make them more productive?

Alex Kreilein:

Yeah. I mean, if you didn't know anything about cybersecurity, how hard would it be for you to configure a system properly? How would you begin to do that?

Tod Beardsley:

That's what kills me, Alex. It's like this guessing game, right?

Alex Kreilein:

Oh yeah.

Tod Beardsley:

Okay, I'm going to tell the devs. It's like, okay, well we need some kind of password box that goes here, and just develop that. And then they do it, and then they give it to you, and like, this is garbage, because blah, blah, blah, blah. And then it's this whole back and forth. It's super-duper time-consuming, and it's all fraught emotionally.

Alex Kreilein:

Oh, and it causes-

Tod Beardsley:

I'm calling you dumb, you're calling me dumb.

Alex Kreilein:

That's right.

Tod Beardsley:

And that... To me what it sounds like, what this thing is like, it avoids that entire conversation.

Alex Kreilein:

Oh, 100%.

Tod Beardsley:

Because you are declaring it in a way that is consumable. It's easy to write, easy to consume.

Alex Kreilein:

Yup. That's right. And you can just read it, you can just sit down and you can actually read the damn thing, right? And so it's tough, because I never want to get into a place where I made my engineering team feel I was setting them up for failure. And normally we do this in security, because we're not in touch with an important human emotional element, which is, there's all this time and passion and creativity that goes into systems administration, and development, and infrastructure deployment, and even security and all this stuff. And we're all really passionate, but we're not really well aligned to communicate. If for no other reason than just to enable communication, this works really well. I wrote it down for you. I implemented it with you. We came to these decisions together, right? So, now let's go fight the really hard fight problems.

Jen Ellis:

I've never heard that security people aren't good at communication. Are you sure that's a thing?

Alex Kreilein:

Yeah. Yeah. Oh, have you not attended a security conference before? That's cool.

Jen Ellis:

This is my first one.

Alex Kreilein:

Oh, congratulations.

Jen Ellis:

Hurray. It's starting on a high note, with RSA.

Alex Kreilein:

Oh really? Yeah. I mean, it's tough to be a security person, because like you're oftentimes thought of as the “no police.” And I just want to break that mold altogether.

Tod Beardsley:

I mean, no.

Alex Kreilein:

Yeah, that's right. That's exactly, that's wrong. And focusing on prevention helps us do that by setting up clear guardrails and systems for people to succeed.

Tod Beardsley:

Well, and presumably you could check this thing in then, right? They can make a pull request, or whatever. Whatever the thing is. So you can have that kind of back and forth, debating around like, well how serious does your passwords have to be? Things like that.

Alex Kreilein:

Totally. But I mean, if I get a print out from a SCAP-certified vulnerability management product … if I got one of those, right, I could sit down with it, and I can read it, and I can say, okay look, here are all my vulnerabilities, but I can compensate for some of these. And I might be able to leave them in production a little while longer if I lock down the environments around them, and I properly configure the system, right? I don't need to do everything all the time is exactly as fast, because I have a strategy for risk management. Which is really what this is. It's saying that we buy ourselves an enormous amount of value if we try and control the environment upfront, as opposed to waiting for things to happen and then everybody has to scramble, and then everybody has their hair on fire, right?

Alex Kreilein:

And this way, too, I have clearly printed controls, so when I go to get audited, somebody says, “Well show me how you do configuration management for tokenization,” or something like that. Where I mean, that's obviously wrong. Show me how you would implement token management. Cool. Here's the script, right? You can sit down, and you can read it. Also, here is the PNG of what it looks like in Azure for US. Do you have any more questions? If not, then you can go outside now. So it's helpful. It's really helpful for everybody.

Jen Ellis:

It sounds pretty cool.

Tod Beardsley:

It sounds wonderful.

Alex Kreilein:

It's not done. I mean, at some point we're going to have to really power this thing through. But we're testing it. It's working super well. When I talk to people like CISOs at really big banks, or at other controlled environments, they're like, that's exactly what we want to do. Our one advantage, is that we're small and we can do it a little bit faster, but we know the strategies, right, and it's agnostic to your environment. It doesn't matter what your environment is-

Tod Beardsley:

That's the other part I like a lot about it. That you're not tied into Azure, you're not tied into AWS, or anything like that. It's portable.

Alex Kreilein:

Yeah, for sure. So, that sort of thing, right? We focus on prevention through that strategy. Our hope is that we can also tie this into full vulnerability lifecycle management as well. That is to say, if we get a vulnerability that pops, which of course we're going to get, how do we define out success, right? So, one way we can do that is by then segmenting infrastructure into risk groups. And we can say, here is the VNET, or here's my network, whatever we want to call it. Here's my network slice, that's just for my most critical infrastructure, so everything in there. We may just choose to only let VMs live for 30 days, right? I just don't want to accept any risk. Well, I can define that through, and I can say if the virtual machine is in this VNET, and it is less than 30 days, let it live. If it's more than 30 days, rehydrate that thing. And you can do that through automation.

Alex Kreilein:

And I think this is the big piece, which is, security people have been struggling and we're making it worse for ourselves by not focusing on automation. And all we talk about is automate this, and automate that. But all we do is PowerShell, and Bash, and CLI, and that's nice, but that's not automation—that's administration. And so, you've got to back it out, and you've got to focus instead on other things that are these reproducible, highly scalable automated processes. And you can apply them through Bash, and PowerShell and others. But, if you're so stuck that the only thing you want to do is rewrite, or reuse the same thing over and over again, you don't have a strategy, you have a formula for failure.

Jen Ellis:

I love how passionate you are about this.

Alex Kreilein:

I love this.

Tod Beardsley:

It's so cool.

Jen Ellis:

So you said that you partway through, and so far it's going pretty well.

Alex Kreilein:

I haven't broken the company!

Jen Ellis:

That is a plus.

Alex Kreilein:

Yeah, it's great.

Jen Ellis:

How did you get by, and how did you get the project off the ground?

Alex Kreilein:

So, I'm blessed to have some great co-founders who started our company, right? And as an employee for them, they wanted somebody who wanted a real strategy. And our guys are through and through, from our CEO and CTO, who started RapidDeploy, to our chief operating officer, even our chief financial officer. Everybody's like, well look, we're small but, we are really nimble and we're smart, so we have to find better ways of doing things. And then we hired an incredible DevOps engineer, in our Cape Town office, who came with us at this great strategy. And honestly, I didn't understand it at first, because, it’s not that it's foreign, it's not like hearing Portuguese for the first time, we you're like, what is that? Is that Klingon?

Jen Ellis:

They do sound incredibly similar.

Alex Kreilein:

Not really, but yeah. But it was like, I think I see this, but I had to step back and look at kind of the strategic advantage. And the more and more I started talking with some of our engineers who aren't from the security space, I listened to the expectation that they had, instead of what they were trying to tell me to do. Because the expectation that they had, was that this was going to work in their DevOps pipelines, in the same thought process that they had under the same conventions using the same tools. And so, if you just don't begin with saying, “I have your answer, don't talk and sit there quietly and listen to me.” If you take another approach of, maybe I should shut up, and just listen to what the infrastructure operators are trying to do. You can come together with some really great tools and techniques.

Jen Ellis:

Would that be your biggest tip to other people who are wanting to try and get by-in other projects is like, start by listening to what other people care about? And figuring out how to make what you're talking about relevant to them based on that?

Alex Kreilein:

Yeah, so like in product management, the first thing you do is you start with interviews, right? And you listen, and then based off of that feedback and analysis, in your own intuition, you then have to take a next step of saying, okay, well now I have to make this live. So you write epics, and you write user stories, and you write technical work items, and you do all that stuff. And you manage it like an agile sprint process.

Alex Kreilein:

But the thing that's important before you ever get to that in the first place, is to define out what acceptance criteria is. What does success look like? And for us, success looks like not hiring 75 security engineers, right? And also success for us looks like using highly automated and repeatable processes that can be audited against, that can be checked. And also success looks like a, what if Alex gets hit by a bus insurance package, which is what we call Git, right? And so, when we look at, well how can you be successful? We started down selecting, and then there was a finite number of options in the option basket. And then I personally do a total cost of ownership analysis, and I do a return on security investment analysis. And I want to see, if I'm going to put money behind this thing. And this is where I stop thinking like a technologist, and I started thinking more like a company operator.

Alex Kreilein:

If I'm going to put money behind this thing, and it's got to live for forever, or until the next thing.

Jen Ellis:

This is where your investor background comes in.

Alex Kreilein:

Yeah, totally. Then I need to think through, well, how much is it going to take to get there? What's my security investment return going to be? Which is different than an ROI. ROI is like, I give you $5, we make $50, our profit was $45, that was good. Security investment thinking is actually radically different, because it's, I give you $5, you punch me in the face. So, it's always a net loss. I'm not going to make money for my company if we're not thoughtful and careful. So the way to do that is to say, what do I think I'm going to get back from spending this money in security? I know I'm setting it on fire, right? But the question is, is it for a good purpose, or is it not?

Alex Kreilein:

So, Bruce Schneier wrote a great article a number of years ago about return on security investment, and he calls it ROSI. And there's an actual formula. And the nice part about formulas, is you can even use them for estimation, even if you don't have real data. So, I sat down and I was like, okay look, it's not going to take an army of people to own and operate and maintain that. It's going to take a lot of upfront work to build it. But then it's about modification and extensibility. It's not about complete maintenance over and over and... It's code, it's not hardware, right? It's software. At some point, all hardware will fail, and all software will work. And so the nice thing about it, when software works is, it continues to work, right? So if we can get it right the first time, and we can continuously improve it, it can live in our company for as long as those technologies are relevant.

Jen Ellis:

So, for anyone who's listening, who's interested in putting together their own project plan, and even if you don't have necessarily hard data on this, you're saying ... so, Schneier. And its return on security investment, ROSI.

Alex Kreilein:

Yeah.

Jen Ellis:

Go Google that, that'll help you figure out how to make a case for this, and talk about value.

Alex Kreilein:

Yeah, because every company has to operate within financial constraints. And so, this idea of I want this tool, or this is the best tool. That's nice, but somebody's got to pay the invoice, right? So, figure out how much is it going to cost to do it? Then figure out how much am I going to get back in terms of security improvement from my firm by doing this, and then resource plan it.

Alex Kreilein:

How many people am I going to need? What's the time, and what's the time to value, right? Because the cool thing about this, is my time to value is the amount of time it takes for me to build a control in Terraform. Plus, the amount of time it takes for me to upload it to Git, so that's a second. Plus the amount of time it takes for us to push it through CI/CD, and that's whatever our sprint plot process is. So my time to value is actually really short. Even if it's expensive, I'm immediately getting a return, right? And it saddens me that some of our security brethren and don't have a sophisticated thinking about it. But the thing is, you have to if you worked in IT. The IT guys have to do that exercise all the time. We don't, and it's to our failure.

Jen Ellis:

I agree. But I also think that there's no shame in not knowing how to do a thing.

Alex Kreilein:

Oh god, yeah.

Jen Ellis:

If you're willing to try and figure it out and try.

Alex Kreilein:

Oh my god. Yeah.

Jen Ellis:

And actually, if you've never thought in those terms, if you've never had somebody help you think in those terms, and trained you to think in those terms, it's a really daunting thing to think about how to get started.

Alex Kreilein:

Totally.

Jen Ellis:

And you don't want that to be the thing that holds people back from raising the concept of a project, and trying to talk it through, and get buy-in from it. And so, you don't necessarily need to have all those answers. You don't need to know the numbers and be pat on this stuff. But think about how you would define success as you were saying. You created commonality where everybody was bought in to what success looked like. And so, you could at least use that as an agreement point to get started, even without having some sort of value calculation.

Alex Kreilein:

Yeah. And you don't have to have hard numbers. It's perfectly acceptable to say, this is estimation. But it's informed by any process at all, right. Because I think the thing that's really hard is, somebody comes to you and they say, I need... First of all, when they say I need, that's always bad, right? It's like, I have this idea for a project, I think it's important, here's the value. This is how much it's going to cost. And here's what I think we're going to get out of spending that money. That's a way better approach than, I need $75,000 for this thing. Like okay, no. Because you've just set me up for a binary outcome. It's either yes, or it's no. As opposed to, I have this process, do you agree with the process?

Alex Kreilein:

Because, the thing that's really... And I learned this from other people. But what's really smart is, when you get people to agree that your process is correct, when you tell them what the outcome is, it's really hard for them to say no. Because they were just talking about how right you were, 30 seconds ago. But at some point, everything needs a green light from a budget officer. And so, while we're talking... The first part of this was cool talk about Terraform, and Puppet, and automation tools and stuff. That's not what's important. What's important is, arriving to a conclusion based on a set of thoughtful parameters, and appropriately getting the buy-in based on, didn't we agree that we wanted to do this together.

Jen Ellis:

People frequently confuse the means of doing something, with the reason for doing something.

Alex Kreilein:

Totally.

Jen Ellis:

And they are not the same thing. And I think particularly, with technologists who are passionate about technology, obviously, it is really easy to get so wrapped up and focused on that how piece, that you lose sight in the discussion. Even if you have it in your core, and it's the thing that's driving your behavior. You lose sight in your discussion on the why, and you always need to bring it back to the why, and make it relevant to the organization.

Alex Kreilein:

You know, an easy tactic for people just to get started on this, just like... Everybody's trying to sell you everything in security. So you're going to have somebody who's on the sales side, he's going to come to you and they're going to say-

Jen Ellis:

Rapid7 makes great software, by the way.

Alex Kreilein:

It's very true. They're going to come to you and they're going to say, hey, I want to get X. I want you to buy this thing. And the first question that people in security on the receiving end should ask is, okay, so what's the value of that? Define for me what that value is. And give them the liberty and the opportunity to help you, because it is literally their job to assist you in this transaction right now. It's also their job to get you to agree to the transaction. But, the implied dance that we're doing is, hey, you've got budget, I want your budget, how can I take your budget off your hands? Well, you tell me how that's valuable. And so you do the hard financial exercise.

Jen Ellis:

Cool.

Alex Kreilein:

Yep.

Jen Ellis:

So, it sounds like it's going well. Have there been road bumps? Have you had any challenges along the way?

Alex Kreilein:

Yeah, I mean just getting started is the first hard part. So... And that's true with most projects, right. Just getting started-

Jen Ellis:

I like that fact that, as you said that, you basically rubbed your face in exhaustion.

Alex Kreilein:

Oh yeah. Totally.

Jen Ellis:

You were like, oh god.

Alex Kreilein:

Oh my god. A couple of roadblocks. So roadblock one, was just figuring out what the limitations of Terraform were. Or rather with... Well two parts actually, what the limitations of the tool is, and what's my limitation, because I'm limited. Even if the tool is limitless, I only have so much time in the day. So much capacity. I'm not a real engineer. So, the idea that I can just implement this is wrong.

Alex Kreilein:

So, the first roadblock was staffing and planning, right? There's got to be a human being who's responsible for this thing. So you can have the best tool in the world, and it's fine, but unless you can deploy it by clicking your heels three times, and saying there's no place like home, you cannot implement it. It's just not that simple. So, staffing and resource planning, step one. I failed there completely. I did not understand what it took to get this project off the ground. I did not know that I needed two or three DevOps people. I thought that it was as easy as going on Terraform.docs and you can just wish it into existence. But so, I made a mistake in not understanding the totality of what it meant to go to market with it.

Alex Kreilein:

Step number two was not understanding how this was going to line up with the rest of our infrastructure process. So, it's cool that I can give you 140 different controls in Terraform. You want to know what's not cool, is for me to give you 140 controls in Terraform, while you've been already building your environment to not know about the 140 controls I was about to give you.

Jen Ellis:

I don't know, that sounds awesome.

Alex Kreilein:

Yeah, that's awesome.

Jen Ellis:

People don't love that?

Alex Kreilein:

Totally. I'm super happy that I'm separated by a massive ocean and multiple continents from my engineering team in Cape Town, South Africa. So, it's very helpful. So, yeah. I mean, I didn't at first have a good thought process for how we line up the work. And then I think the third thing... So, the first problem is human staff management. Second problem is a process problem of just collaboration and communication. And I think the third problem, is figuring out how to consistently and recursively update those controls, given the changes that we will want to make in our entirety of our stack, right. That's not easy.

Alex Kreilein:

So it's one thing for a thoughtful person on the DevOps side to... I think there's beef wellington coming. Yeah. Casper, is that you? So, it's one thing for them to outwardly want to make a change. It's another thing for your infrastructure to change, and that's going to happen, and you're not going to be able to control it all because there's drift. And you just have to compensate.

Jen Ellis:

Okay. So, you've had some good learnings along the way. So what advice would you give to other people who are looking to get projects off the ground, go and do their own thing, even if it's not the same kind of project? Even though it does sound like everybody should go and do this project now. But wait until you've open-sourced it, because it'd be much easier.

Alex Kreilein:

Yeah, 100%. You can pay me at bitcoin dot... Okay. So, couple thoughts for getting a project off the ground. I think the first one is to actually sit down and design it. Sounds really crazy but 90% of the time-

Jen Ellis:

No, I'm with you.

Alex Kreilein:

It's like oh, I'm just going to start implementing. Maybe one of the last things you should do, is open up a text editor, right. Or Atom, or whatever you're using to sublime. That's the last thing you should do. First thing you should do is, pencil and paper, not pen. Pencil and paper. So, document out some sort of architecture.

Jen Ellis:

You know, they have computers now, and you can delete, and overwrite-

Alex Kreilein:

You can, you can. I actually find, when I'm forced to sit down with old-school methods, I'm more thoughtful. Because things are not as easy.

Jen Ellis:

You go back in time, and then you-

Alex Kreilein:

Yeah.

Jen Ellis:

Right.

Alex Kreilein:

So, start with an architecture, that's thing one. Thing two is, then cost it. Cost it out. And figure out how much is this going to cost to implement. Because if you're looking at doing something that is $165,000 project, but your security budget is $170,000, there's no bueno. This is not going to work.

Jen Ellis:

I don't know. That sounds good. You're covered.

Alex Kreilein:

For sure.

Jen Ellis:

You've got 5K left over-

Alex Kreilein:

You got 5K to got to RSA.

Jen Ellis:

Right. That will get you the pass, and one night in a hotel.

Alex Kreilein:

That's right. That's right. So yeah, the output is, architected first, then come up with a planning budget around it. And then I think the third part is, just make sure that what you're doing is actually different from what's out on the marketplace. So, I get people a lot of the times who pitch me ideas about projects, and I'm like, you know that's already done, right. You can consume it as a service in the cloud, or here are three open-source repos that you can just use, or here are two great products that you could use.

Alex Kreilein:

And then the response is always, well I don't know about that open-source repo, but I'm definitely not going to use the product. I'm like, okay, well why not? Why are you not going to use the product? They're like, oh, because you can definitely do it on your own, or you can definitely do it in open-source. I'm like, cool. Well I could build an iPhone too, right? 100%. With enough time, and monkeys, and effort, right. I can do that. But the reason why the second step is so important, is to figure out how much it's going to cost you, is because when you then go out to compare what you want to build against other solutions, then you know, is it cheaper to build it, to buy it, or to ally with a third party. Which is some basic business school crap that I definitely got off a Pluralsite course somewhere, I'm sure.

Alex Kreilein:

But yeah, so that trade space is tough. And then when you're done with that analysis, and you know what you want to build. You know that it's markedly different from what's already in the market. And you've been able to find out that you can make a return on the security investment by doing it yourself. And all this stuff is not technology stuff. It's all business school stuff, right? Which is the hard part. But once you've done that, you've removed all of the friction from being able to take it to your CTO, or your CSO, or whoever. Because you can sit down with them and say, I did the homework, right? I have this really smart idea. I want your buy-in, and here's what's important for you to know. This is what it looks like, this is how much it's going to cost, this is why it's different from what's on the marketplace.

Alex Kreilein:

And you sum those things up in those three points, and 90% of the time you're going to have a really good conversation. You may not always get what you want, but nobody's going to say you were stupid.

Tod Beardsley:

You've made that CFO's job way, way easy.

Alex Kreilein:

Totally.

Tod Beardsley:

They'd love you for that, right.

Alex Kreilein:

Yeah, totally.

Tod Beardsley:

So, you've already... You've ended that first conversation with an ally. Regardless of how they come down on it... Maybe they have some suggestions or something like that. But at the end of it, you have an ally.

Alex Kreilein:

Yeah. And if you're not talking to your CFO, but you're talking to a security architect, right? You've now empowered them. You've completely given them the entire toolbox that they need to completely go out there and build something great. So, people talk about wanting to build good relationships in InfoSec, but that doesn't happen at a happy hour. That happens by putting the grit in, and actually helping someone succeed.

Jen Ellis:

Oh my God, we're canceling the happy hour.

Alex Kreilein:

Yeah.

Jen Ellis:

That's it.

Alex Kreilein:

Yeah. Happy hour over. It can also happen with a happy hour. These are not mutually exclusive things.

Jen Ellis:

Happy hour's back on.

Alex Kreilein:

Yeah. Thank God. Yeah. So it's... I think these are some of the strategies that security's not really caught up on. Because we tend not to think about it like a business, unless we're in business. And when you're in the trenches, you're like, I just got to get on. I got to build this PowerShell script, dah, dah, dah, dah. Okay, great. But then when you're done with that, you've got to back out, and you've got to think, what am I doing here? And this is by the way... This is not just a technique that works at startups, or a technique that works at big Fortune 500 companies, or just at government agencies. This is just how organizations work. And I think there are a lot of really brilliant people, who are limiting their ability to be truly brilliant by not understanding how to actually build. And this is how you do it.

Jen Ellis:

It's hard to be impactful if you're not thinking about impact.

Alex Kreilein:

 Yeah. Yeah, it's tough. And your impact is not just being the most amazing person on Splunk. Your impact is not just being the most amazing person on whatever tool. That's a great outcome, but the impact is to truly give change of an organization.

Jen Ellis:

Okay. Great advice. Thank you very much Alex.

Alex Kreilein:

Yeah, totally.

Jen Ellis:

That's awesome. Thank you for joining us, and experiencing-

Alex Kreilein:

This weird creepy hallway.

Jen Ellis:

Right. Experiencing the-

Alex Kreilein:

God, I wish you guys could see this.

Jen Ellis:

The thrills of the noise. Yup. Okay, cool.

Alex Kreilein:

You guys are going to let me out of this cage, right?

Jen Ellis:

No promises.

Alex Kreilein:

Cool, cool. Talk to my customer success manager.

Jen Ellis:

Yeah. When they visit to feed you.

Alex Kreilein:

Cool, cool.

Jen Ellis:

Okay, so a huge thank-you to Alex for a great interview. Thank you for coming in and talking to us. And now through the magic of technology, we'll fast-forward through time, and we'll go from one dodgy recording scenario, RSA, to another, the Coronavirus. Hey, Tod!

Tod Beardsley:

Hey.

Jen Ellis:

I'm imaging the Coronavirus might be something we talk about in the Rapid Rundown today.

Tod Beardsley:

Oh yeah, I think it's a law. Every podcast has to just take a moment to talk about COVID-19 or Coronavirus or SARS-CoV-2, I believe, is the more technical term.

Jen Ellis:

It sounds like its CVE.

Tod Beardsley:

It's very much like a CVE. Yeah, there's even a number in it. It's been kind of a crazy couple of weeks here. We recorded that interview back at RSA, that seemed like an age ago, when we could look at people in person and that's fine. And that's the thing. It's like we live in a different world now, and if I sound any different, it is totally your imagination because I'm where I usually record things, at home. Don't tell my boss, but I rarely go to the office. I work at home probably three or four days a week, anyway. So I'm pretty practiced at that. So if you have any questions about how to work at home, please feel free to reach out to me. But a lot of people do in our industry. We're knowledge workers, we tend to work at home or in weird places a lot, especially like pen testers, they're working all over the place. So if you are listening to this, you are probably already a security professional, and you might know people who are starting to struggle with this. For example, my wife is an attorney, and attorneys generally don't work at home. They love offices and paper and stuff like that. And so now she's at home, not 30 feet from me, and it hurts me to look at her work because she's like on a couch with a crummy Lenovo keyboard and just typing away. She's going to feel this in like three or four weeks, unless I intervene. So after I'm done recording this, I'm going to go intervene, and ask her what I can do to get her to a get desk and a real keyboard. That's one pro tip. Being on a couch sounds like fun and it is for a while until you feel it in your wrists and your shoulders and your back and your hips and all of that. So try to find a workspace that's actually a workspace and someplace you only do work in. That's a good psychological trick to figure out, like when you're at work and when you're at home.

Jen Ellis:

Well, a clear distinction of time I think is very important. I will say I have taken up temporary residence, and yes, in case they're listening, it is temporary, family, in my brother's attic. That's right, everybody. I have reached the pinnacle of my existence. I am living in my brother's attic. He told me he's excited about it.

Tod Beardsley:

You're like a Gothic monster. I mean-

Jen Ellis:

I feel great.

Tod Beardsley:

Infosec crone of London.

Jen Ellis: He did tell me that he was going to stop penning some sort of terrible poetry about this, about me in the attic. And then over dinner they started doing it. Yes, good times Corona has brought out for everybody. And as you say, I am currently sat on a sofa with a laptop.

Tod Beardsley: It's so bad for you.

Jen Ellis:

So, I apologize for the terrible recording experience.

Tod Beardsley:

Well yeah, so I mean, Coronavirus changes a lot of stuff, but not so much for people who already work at home. So, if you want to do a good deed, try to coach your friends and family members who also happened to do knowledge work and are now stuck at home on how to do that in a healthy and sane way.

Jen Ellis:

I will say that we have a couple of blogs on the Rapid7 website, or rather on the Rapid7 blog, and if you go onto Google and just search Rapid7 blog COVID-19, you'll find there's three or four blogs up, and they offer advice to both security teams looking to protect their remote workers and also people who are actually embracing remote working for themselves. So check those out. Obviously start bartering for everything with toilet paper as a currency.

Tod Beardsley:

Oh, what are you talking about? I'm not that rich.

Jen Ellis:

Yeah, Bitcoin you can afford but toilet paper, nah.

Tod Beardsley:

I also have some actual news, some infosec news that's not related to COVID-19.

Jen Ellis:

No, I don't believe it. It's not possible.

Tod Beardsley:

It is. It's very short. So, if you'll recall from our last podcast, I talked about the Emotet trojan and how it was scanning nearby Wi-Fi networks for a way to hop networks. And it was a mystery to me, because I'm not a malware guy, of what they were using for passwords. And I assumed it was like a short list of passwords that were likely, like Mirai did back in the day. But the update is, I now have the list of passwords.

Tod Beardsley:

I worked with Rapid7's own Eoin Miller, who is a incident response malware analysis kind of guy. We looked through the... He did all the heavy lifting. I say we, but it was him. But we looked through it, and the password lists that Emotet use is garbage. Most of them won't work. They're too short for WPA2. WPA2, by the way, requires at least an eight-character password. Most of the passwords that it uses is fewer than eight characters. So the only thing that would work on are like old, old-timey web password-protected wireless networks, which is a super broken technology, and most people aren't using WEP anymore. Hopefully if you are using WEP, try to take a look at updating your firmware to where WEP is no longer available.

Tod Beardsley:

But yeah, so the password list is super garbage. As far as I know it is unpublished. I'm going to be publishing those today, and we will have a link in the show notes of where you can download those password lists. It's not exciting at all. If you've ever seen a list of top 500 passwords, with like password and password1, and trustno1, and 696969, like those are the kinds of passwords we're talking about here. So they're not particularly sophisticated.

Jen Ellis:

Is trustno1 still a big password? It feels dated now, surely.

Tod Beardsley:

It is stupid and dated, and people find it, like people invent it. That's the thing. It hits a pattern that just is very appealing to humans because it's kind of clever and easy to remember and has a 1 instead of an O-N-E. But even that one is what, two, four, six... Oh, that is exactly eight characters. So that would work for a WPA to a wireless access point, but don't use that one. So yeah, we'll upload those. I'll just have a gist or something, and we'll link you to it.

Jen Ellis:

Awesome. All right. Well thank you for taking us through the Rundown. So with that I will say a huge thank-you to Alex, our special guest this episode, and I hope that he's well, whichever bunker he's currently in. A special thank-you to Tod as usual for edumacating me, and a huge thanks to Bri, our amazing producer who is probably delighted not to have to be in a studio with me right now. And also a special thank-you to my family for putting up with me being in their attic.