Knowing is only one-third the battle. Another third is responding. And the last third is responding quickly. It’s not enough to just have the first two thirds. We need to be faster, but how?
Check out this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Steve Zalewski. Our guest is Jason Elrod (@jasonelrod), CISO, MultiCare Health System.
Got feedback? Join the conversation on LinkedIn.
HUGE thanks to our sponsor, Eclypsium
Full transcript
David Spark
Knowing is only one third the battle, another third is responding, and the last third is responding quickly. It’s not enough to just have the first two thirds. We need to be faster, but how?
Voiceover
You’re listening to Defense in Depth.
David Spark
Welcome to Defense in Depth, my name is, David Spark, I am the producer of the CISO Series, that is the network for which Defense and Depth is on. Joining me for this very episode is Steve Zalewski. Steve, let’s hear the sound of your voice.
Steve Zalewski
Good afternoon, David.
David Spark
Good to hear you, Steve. Our sponsor for this very episode, and it has been a brand new and phenomenal sponsor of the CISO Series, is Eclypsium, and they protect devices and supply chains by identifying, verifying and fortifying firmware code throughout the enterprise. More about them later in the show, but, first, I want to talk about our topic that you brought up on LinkedIn, Steve. You asked the question of, “What can we do as a pragmatic first step to make our cybersecurity teams quicker and more responsive?” Now, being fast is not something that comes naturally I’ve noticed, and I think the issue is speed has to be a core mission of the entire organization, and you constantly have to look for ways to achieve that speed. What’s your take, Steve?
Steve Zalewski
This question actually came from a conversation I had with some of my peers up in Napa, when we were looking at what was top of mind for all of us? And one of the things that was frustrating me, which is you hear this consistent theme which is we’re letting the attackers win the speed race. And, to my point was, “Well, then why are we doing that?” With my peers. It caused me to think about that to say, “Look, as we run our organizations, what is a simple, pragmatic thing, that we can do?” It’s like getting better. To be able to drive your organization to go faster. And out of that came a whole set of responses and a whole set of ideas that we’re going to discuss, but that was the genesis of the question.
David Spark
I like it and, to help us with that discussion, is a friend of yours, who I have just met and am excited that he’s here with us, it is the CISO for MultiCare Health System, Jason Elrod. Jason, thank you so much for joining us.
Jason Elrod
Thank you, David, happy to be part of the discussion today. Looking forward to it.
Does it play nicely with others?
00:02:28:16
David Spark
Abhishek Singh of Araali Networks said, “What is needed is continuous authorization, where there is continuous evaluation, which goes beyond monitoring to actionable containment, should badness actually happen.” And Jonathan Waldrop of Insight Global said, quote, “Establish a means for organizations to share specific technical information in the aftermath of a breach or major cyber incident.” And, John Haden of Trend Micros said, quote, “Vendors, like us, need to implement a text, SMS, mobile phone messaging system. When organizations are under attack, there’s no time to
submit tickets, we need to get someone on the horn.” I know the feeling of when something bad has happened, and submitting a ticket is not a good feeling is it, Steve?
Steve Zalewski
[LAUGHS] Everybody loves those ticketing systems, right? What gets me is, it’s a great way to measure your efficiency, because you can track how many go in. It’s terrible for speed and effectiveness, because it doesn’t actually help you get to where you need to be.
David Spark
By the way, they always give you an option to mark as urgent; everyone marks as urgent. [LAUGHS]
Steve Zalewski
Yes. And the thing for me is well, that’s great, but, again, put in a ticket so some poor human, and I have to do eyes on glass with my team, to then determine what is reasonable actions to take. One of the conversations I talk about here which is, where’s the automation? Where’s the machine speed?
David Spark
And that’s what Abhishek brings up.
Steve Zalewski
Right. So, why aren’t we using machine speed on defense, to manage the machine speed of attack? And that’s why the attackers are winning the speed race.
David Spark
Jason, where does automation come in a speed to contain or respond, race?
Jason Elrod
Speed to contain, speed to respond, well, if we’re talking about communication I think having multiple layers there. When something really is an urgent or a critical and it’s risen to an actual incident level, not always rely on those tickets, it’s not a ticketing item. We have a lot of great collaboration tools at our disposal, so, when there is an incident you can rely on things like a slack channel or team channel, spun up directly for it. You can have that focused conversation around just this particular instance, just what is critical here, and I think that’s a great opportunity. Now, sometimes, if it really is an incident of magnitude, you may not have access to your regular corporate tools. So, I think having an understanding of, “Hey, what is that backup? Is that signal for us, is that texting?” Having that calling tree and, honestly, that network, “Who do I need to call?” Or, “Who do I need to call who knows who I need to call?” And having that available for you in your personal Rolodex is extremely important.
David Spark
Steve, when you were at Levis, do people on your team and not just in your immediate team, but, in the wider organization know the place to call? The red phone number?
Steve Zalewski
I will tell you the executives did, [LAUGHS] and those are the ones that got the immediate attention, even if it wasn’t necessarily a crisis. But here’s what I would say which was security awareness training, which was giving people multiple ways to try to tell us, and what we used to say was, “Call any of us,” it didn’t make any difference. If you know one of us through security awareness training, through friendships, through whatever we did through the company, our responsibility, every one of us, was, if the phone rang we answered, because if you were reaching out to us and it was me you happened to know, that was my point, it didn’t make any difference who. So, it gets back to the people in the process. One of the things I asked is do people process their technology, and one of the key themes here, at least as we’re starting out is, does somebody know who to call when they realized they made a mistake? Because generally what happens is, somebody realizes they made a mistake and now how quickly can we get in front of that mistake? Where there is no downside to the individual, because we want to encourage them to tell us, because we’re here as the safety net.
Can it be solved?
00:06:56:07
David Spark
Martez Reed of Morpheus Data said, quote, “The development of simple and robust test automation, with containers, public cloud and other advancements the process of patching should be fully automated and regularly test new patches as soon as they’re made available. The potential impact is what slows the process which is why the automated testing is critical to moving things forward.” And, Robert Gezelter said, quote “Incidents that do not happen do not need responses. One should start with an initial working goal of eliminating vulnerabilities during development.” They were arguing a very severe shift left case of, let’s try to avoid the need to respond quickly ever at all, or dramatically reduce it. How much has this shift left, specifically around vulnerabilities helped, and, the other concern is, are we solving vulnerabilities that don’t need to be fixed? Jason?
Jason Elrod
It’s a great question and I think, yes, you need to shift left. You need to get unapologetically good at the basics, because if you want speed, efficiency leads to speed. You can react instantly on things, but your accuracy’s not going to be there, so often you have to go back again and again and again. When you’re really good at what I would consider the basic cyber hygiene and concentrate on the constant never-ending improvement there, you’re going to have efficiencies which will lead to speed. One of my favorite quotes about this is Bruce Lee, in talks about, “I fear not the man who has practiced 10,000 kicks once, I fear the man who has practiced one kick 10,000 times.” And your program and practice needs to be like that. You need to be the practitioner who’s done it 10,000 times so it happens, it’s automatic, it’s efficient and it’s effective.
David Spark
This was the question I asked of the doctor who circumcised my son, “How many times have you actually done it?” [LAUGHS] And here’s the thing, he said, “No-one ever asked me that question.” “Seriously?” [LAUGHS] That’s pretty top of mind with me. [LAUGHS]
Steve Zalewski
Now I’m going to jump in on this too and I’m going to go, did you see that we actually pivoted, because what we did, my asking, which was how do we stop the attackers from winning the speed race? The first one was, how good are we at responding to attacks? Which was an incident happened, what are we doing and how do we drive speed into the incident response and containment? The second pivot is, app-dev. How good are we at using automation to be able to drive vulnerabilities out of code, so that it doesn’t get to containment? And how good are we at driving it? My point here is there really is two key areas in any security organization; one is incident response, and one is proactive baking security in. What I really like here is, we’re talking about both of those as areas where you need to increase the speed, both have definite advantages, but really, the key is for every CISO, you have to ask yourself, which one of those two is the higher priority for you and your organization to be able to invest your critical resources?
David Spark
This goes back to what I said at the very beginning is that this is all about you only achieve this if this is a company goal to go faster, because I’ve got to imagine that, Jason, tossing to you is, how do you even achieve it if it’s not already the company goal?
Jason Elrod
I think you express it the way Steve just did. I’ll put it in the firefighter mode, so, you can be really good at putting out fires, but how about we get really good about not ever starting them. I think when you start looking at that, I just need more and more resources, the demand against my capacity to put out fires, how about we reduce the amount of kerosene and matches in the same room? That starts talking about vulnerability, but not necessarily, we’re vulnerable to the fire starting, right? But, what I’m talking about here is exploitability management; remove the ability to exploit it. I don’t need to remove both things, I just need to remove one of them. Stop lighting fires so we don’t have to put out so many fires.
Sponsor – Eclypsium
00:11:25:01
Steve Prentice
Action speaks louder than words, perhaps no-one is more passionate about this idea than Scott Scheferman, principle strategist at, Eclypsium, a company dedicated to proactive firmware defense. In his words, “Without proper action, you are doing cyber defense wrong.”
Scott Scheferman
A lot of times we do a cyber theater and we talk amongst ourselves and we all nod our heads. If you don’t already know that you shouldn’t be using multi-factor over SMS, you’re doing it wrong and you haven’t taken that action as an organization fast enough. If you don’t already know that [UNSURE OF WORD] like TrickBot are going down to firmware, and you have a TrickBot infection, and then your forensics are not actually looking to see if the device itself is compromised at firmware level, before you reintroduce the device to the contested environment, you’re doing it wrong. We tend to focus on this low hanging fruit, patch cycles and dealing with requests from the board saying, “Hey, are we protected from the latest thing I just read about in the news?” And you get in this fire drill on your heels mode, and you’re telling yourself, “Well, it’s a matter of when not if,” and you’re building your whole legacy as a CISO as one that’s reactive, and one that anticipates a breach. And my call to arms, for CISOs is to say, the second you build your legacy around being on your heels, you will, in fact, find yourself on your heels reacting all the time. When you can treat these areas, you start to realize, I can actually get ahead of these things as an organization, as a sec-ops cadence. Being able to make decisions that matter and in time to matter, fast enough to matter.
Steve Prentice
To learn more about Scott and his team’s approach to proactive firmware security, go to, Eclypsium dot com. That’s, E-C-L-Y-P-S-I-U-M dot com.
What are the best practices?
00:13:06:20
David Spark
Gaurav Banga of Balbix said, quote “Measure mean time to patch or mean time to remediate rigorously.” So referencing what we just talked about in the last segment. “Set aggressive goals, automate discovery, prioritization, evaluation, and dispatch.” And then, Gaurav said, “Automate what you can.” And then, Steve LeChance of, Tenacity said, “Cloud issues come about because of human error happening as a result of the speed race.” So, he believes actually going too fast, we make mistakes. And, Jared Herman of, North Labs responded, quote, “Misconfiguration is like leaving your front door unlocked because you’re moving too fast.” So, I feel pro and con about this, Steve, they think speed often is the cause of problems to a degree, but I don’t think they’re fully on board with that feeling, but keep that on mind that that can happen.
Steve Zalewski
So, what they’re calling out is digital transformation and the need for the business to fail fast is not the same as security, which security can never fail. And so while we’re trying to increase our speed and our efficiency is to be able to manage the risks of the business trying to go peddle metal fast and thinking, if it doesn’t work, they don’t care; which is all the clouds they go into, the shortcuts they want to take with regional systems, where we still have to maintain a holistic defensive front. So now, going fast is, “Alright wait a minute, what can I protect in the company? What can’t I protect? Can I rely on mutual aid? Can I do conversations with automation on people processing technology where I have to get better at the insurance policies I’m taking and the cost and the effectiveness against the digital transformation?” When they start to bring that in I say, this again gets to ours, which is attackers are winning the speed race because they’re automating everything and just have to wait for us to make a mistake, to businesses forcing us to make more mistakes, but we can’t tell the business no. So how do we try to reinvent and simplify our infrastructures, so that we can bring maximum speed to the points of pressure that are most important to the company? I think that’s what we’re trying to talk about here. This is where the conversations are moving through the different domains and responsibilities of a CISO.
David Spark
Jason, I think this is a situation of, are you looking at speed as a benefit or a vulnerability and, if you’re not careful, it could actually be a vulnerability.
Jason Elrod
Good distinction. Actually I look at it from a standpoint of, automate, delegate and distribute the low hanging fruit in your organization. When asked about the speed question earlier, I didn’t say go faster, I said, get more efficient. Efficiencies equal speed. Organizationally, you need to enroll your entire organization into the security posture of the enterprise. You essentially crowd source the capacity you need, and you rely on parallel processing for speed. So, I teach and train the basics so my community can hand out the traffic tickets, while my swat team deals with the true adversaries. Essentially, I think we have a mutual friend who likes to use the term, “protect to enable”; shout out to Malcolm on that one. But I would say, organization but from the security’s program, I need to trust to enable security in my organization and to trust my employees and train everybody accordingly, in order to get that, so everybody’s watching, so I’m not just walking on a small subset of the organization, but it’s top of mind for everybody. Crowd source the capacity and use parallel processing for speed on that.
David Spark
So, the answer here is, the story shouldn’t be about speed so much as efficiency, and that efficiency will yield speed, like you said at the very beginning.
Steve Zalewski
But don’t forget, effectiveness, from a security organization is actually what speed wants, not efficiency, that’s what a CIO needs. And so that’s part of the conversation as well, to be able to realize that speed for effectiveness is, ultimately, where we’re moving the conversation, and human error is the randomizer that’s causing us to try to figure out how we maximize that efficiency, depending upon what it is we’re trying to protect.
Jason Elrod
Spot on. Yes, you’re right, it is being effective. So, how can you efficiently be effective? And that’s how you drive it. Great distinction.
What aspects haven’t been considered?
00:17:53:07
David Spark
Ryan Clarque of Levi Strauss, said, quote “I would call out the impact of alert slash, combat fatigue is real and impacts the people component more than most realize. Not taking care of our people can be our Achilles heel.” So this is teasing what we just talked about in the last segment, and Parker Brissette of Colorado Judicial Branch, said, quote “Lower the barrier for entry. Teach as many as possible and open up to new ideas. The attackers win with numbers, both financial and people. Most security teams I’ve come across could easily justify the need for twice as many hands as they currently have.” This is a call for, as we talked about, people processing technology of you’re going to be a lot more efficient and effective with a more people focused plan and allowing more people to participate in security. Jason, I’m assuming you’re on board?
Jason Elrod
100%.
David Spark
So what is your take on what both Ryan and Parker here say, in that how would you be aware of abusing your current staff and bringing in new staff all in the guise of, “We’re bringing you on to make us more efficient and more effective which will then yield speed”?
Jason Elrod
I mentioned it earlier, I think the focus needs to be on exploitability management instead of vulnerability management. There are hundreds and thousands of vulnerabilities out there, zero days, zero minus one days, that we haven’t even discovered yet. In essence, vulnerability management is a CISOphian task and that’s why the teams get burned out on it. From a security standpoint, I think you need to find the pass or passes, of Thermopylae in your organization. So where your efforts will be most effective. Where are my efforts force multiplied the most? Where can my 300 hold off a million? So, I should know my own house, right? Where all the doors and windows and which ones are watched and locked and which aren’t and why and focus on those. We spoke a little bit more about having that expanded crowd source capacity. Again, I want my community handing out the traffic tickets, so the swat teams, they are dealing with the true adversaries, but I also want to know, where are they most force multiplied and have them concentrate on those security choke points, where most likely, they’ll be able to be successful.
David Spark
Steve, when you work with a team how are you on a weekly, daily basis, looking for those points of increasing efficiency and effectiveness? How does that structure create itself? Where do you build that in?
Steve Zalewski
I’m going to say, there’s a difference between leadership and management. Now you are really starting to challenge around, how are you as the CISO thinking about this problem? Are you managing the problem, so as people are bringing it to you, you’re listening to them, you’re being sensitive to them, you’re trying to find some more resources? Or, are you doing leadership that says, “I’ve given you the mission, I’ve given you the resources, now, you have the responsibility and the authority to decide what goes on your plate and what comes off your plate and I will support your decision.” You can’t do it all, so are you thinking about what the most important stuff is? And the reason why I say it that way is, when I look at these and I think about it in my time at Levis as architect and deputy CISO and CISO was, I looked at it as I had three teams; I had an incident response team, heroic firefighters on deck every day doing what it takes to stop a fire and put it out, and they could run 24 and 36 hours and then they’re out of gas. But when they’re having to put out two to three fires a day, and those fires are primarily due to human error, because it’s phishing and everything else, they have to manage containment. So, speed of containment and automation is what they had to do. Then I looked at my app-sec, application security, what are we doing to bake security in? Therefore, how good are my people working with my developers? For them to understand the responsibility that they have and to be able to pull the stop on any project if they realize that the developers are running amok, and it doesn’t mean try to support them and help them along, it means these are the expectations I’m here to meet it. Then the third is, look at your business information security officers, the ones that are out there, that are working with your legal teams, your business teams, your lines of business, who are there to be able to not be your trusted security adviser, but are there to make sure that you understand the risks that you’re taking, that the security team cannot cover that bet for you, because the business, ultimately, decides. So, how good are we at making sure that we then raise the red flags, when the businesses are doing things either on purpose, or, on accident, that are putting the larger company at risk. That is leadership, to look at those three organizations and ask them, “How do we win the speed race for effectiveness, when they’re all very different and I have to support all my children?”
David Spark
Let me give an analogy. This is very similar to just production work, anything you’re doing video or audio production, any kind of production work you’re doing. Everyone has to be on the same page in terms of, we are generating this thing and we all have our part in this. In our early days of production I remember having to interview people who were so bad on camera, or so bad on the microphone, and they would say, as if they were the producer, “Ah, just fix it in post.” Not realizing what that phrase means. No, you just nail your lines so we don’t have to spend an extra six hours fixing your massive screw up, just do your part. It drove me nuts when people would say it and they would say it like a joke. I’d go, “No, this isn’t a joke, you’re just going to nail your lines and you’re going to stop being bad, just be good.” Jason, you’re laughing.
Jason Elrod
Yes, I agree. It’s all about throughput, right? Just because my part’s fast, doesn’t mean the next part’s going to be fast or efficient or could actually handle it. Sometimes, actually optimizing just one part of the holistic process breaks the rest of the process. You might think you’ve been doing something great over here, I’m much better at the beginning part of it, not realizing that increased pressure on the line’s going to break it somewhere else, and make it infinitely more difficult for us to pull everything through to a successful conclusion.
Steve Zalewski
And I love the passion, David, because now you see what most security people do which was, look, I’m here and you’re thinking I’m a cost, I’ve got to get by security and security is simply looking at you and going, “If you just think about it for a minute, you’re going to save all of us an awful lot of headaches.” The way I look at it is, “Hey, if you want to go play in traffic, go for it, but don’t come to me looking for an insurance policy for your wife and kids when you were out playing in traffic because you didn’t listen to me when I said, that’s a major highway, and you have to use the crosswalks.”
David Spark
But let me say the opposite of what I said and tell me what the equivalent of this is in your world. I refer to the person who was horrible on camera, awful on the mic and it drives me crazy. I’ve had the complete opposite, where you get someone who just speaks in soundbites, super easy to edit, makes your life a dream, the interview’s over in a just a few minutes, “Oh my god, thank you for showing up.” What is the equivalent of that in security?
Jason Elrod
Yeah, so I’ll say it’s somebody that comes to us and says, “Here’s my problem, here’s how I know it’ll be successful, but I don’t know the solution. Could you please help me.” Versus, “Hey, here’s a solution I want you to implement.”
Closing
00:26:06:14
David Spark
I like that. Alright, that brings us to the end of our show here, but, before we truly conclude, I always ask both guest and co-host, what their favorite quote is, why. And I will start with you, Jason, which quote was your favorite and why?
Jason Elrod
I’m going to have to go with, Parker Brissette, Colorado Judicial Branch, “Lowering the barrier for entry. Teach as many people as possible and open up to new ideas.” That’s crowd sourcing the capacity. That’s the parallel processing that you need in your organization to keep pace, to be efficient, to be effective. So, love the quote, completely align there.
David Spark
Alright, good choice. Steve, your favorite quote?
Steve Zalewski
I’ve actually got to go with two this week, and you hate it when I do that, hear me out. I am going with, Steve LaChance of Tenacity, who says, “The issues are human error,” and we talked about there’s a whole class of issues that to go faster have to do with you managing human error. And the other side of that though is, Jared Herman of North Labs, “Misconfigurations is leaving the front door unlocked,” configuration is code, app-sec, getting your application development environments to be as locked down as you can, where you can manage the configurations and the vulnerabilities. Why I picked those two is because I actually think those two balance the entire conversation we had today, in order to be able to go faster, there really are two or three different key domains in security that you have to think about what that means and how you lead your teams and your company to be able to manage those as quickly as you can.
David Spark
I like it. Well, thank you very much, Steve Zalewski and also, Jason Elrod, CISO of MultiCare Health Systems. Jason, I’ll let you have the very last word here, we’re going to be wrapping this up. One of the questions I ask all my guests is, are you hiring? So, make sure you have an answer for that. I want to first though, thank our sponsor, Eclypsium, thank you, Eclypsium. Remember, they are in the firmware security world, so identifying, verifying and fortifying firmware code throughout the enterprise. More at, E-C-L-Y-P-S-I-U-M dot com, check them out. Steve, any last thoughts.
Steve Zalewski
Thank you, listeners, we wouldn’t be able to do this without all your responses to LinkedIn and Twitter that David and I put out there. We ask hard questions and we really appreciate the transparency that you provide, that allow us to do, Defense in Depth.
David Spark
By the way, I completely echo what Steve said, that’s not a flippant response, truly, we live for your responses, or this show essentially gets it’s juice from your responses. So, thank you. Jason, any last thoughts and, are you hiring?
Jason Elrod
So, last thoughts. We’ve bounced quite a bit in the conversation today and one of the things it brought up and, Steve, you really drove it home for me, I can think of nothing more wasteful or dangerous in our organization than doing something efficiently, that you should’ve never done in the first place. So I think it’s really important to talk about effectiveness, because if you’re busy shooting yourself in the foot and you come to ask me “make that more efficient”, I can give you a bazooka, that’s way more efficient, go for it. But let’s talk about the fact that you’re shooting yourself in the foot, and maybe not do that. That idea of shifting left and really hitting that. That would be my parting thought on this.
David Spark
And, are you hiring?
Jason Elrod
And we are hiring right now, so identity and access management, PCI, come by, we’re ready.
David Spark
If someone wants to get in contact with you, where should they find you?
Jason Elrod
You can reach out to me on LinkedIn any time. I am on LinkedIn. As well, if you want to go ahead and reach out to me directly at MultiCare it’s, Jason dot Elrod, at MultiCare dot org.
David Spark
Excellent. Thank you very much and I’m going to echo what Steve said at the end of the show here, we appreciate your contributions and also listening to Defense in Dept
Voiceover
We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site: CISOSeries.com where you’ll also see plenty of ways to participate, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at David@Cisoseries.com. Thank you for listening to Defense in Depth.