Why have we conflated vulnerability discovery with vulnerability management? There are lots of tools that classify what’s out there, but they don’t help you take the next step.
Check out this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Mike Johnson, CISO, Rivian. Joining us is Yaron Levi, CISO, Dolby.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor, Intezer
Full Transcript
Intro
0:00.000
[David Spark] Why have we conflated vulnerability discovery with vulnerability management? There are lots of tools that classify what’s out there, but they don’t help us take that next step.
[Voiceover] You’re listening to Defense in Depth.
[David Spark] Welcome to Defense in Depth. My name is David Spark, I’m the producer of the CISO Series. And joining me as my co-host for this episode, you normally hear him on a different show, but there’s a reason I’ve brought him for this one. You’ll recognize the voice. It’s Mike Johnson, the CISO of Rivian.
Mike, say hello to the audience.
[Mike Johnson] Hello, audience. It’s been a while. Glad to be on the show again.
[David Spark] He is nice Defense in Depth people. You can treat him nicely as well.
[Mike Johnson] Yes, please. I’m very fragile, so please treat me nicely.
[David Spark] By the way, we can be found at CISOseries.com, where all our programming can be found. Please go check it out and find our other programming as well. Our sponsor for today’s episode is Intezer. Extend your security team with AI. That is exactly what they do. They don’t replace it. They extend it, make it more valuable than what you already have.
Take a look at what Intezer’s doing. We’ll be talking about it a little bit later in the show. All right, but first, Mike, let’s talk about today’s topic, which is a topic you brought up.
[Mike Johnson] Mm-hmm.
[David Spark] Cybersecurity teams don’t exist to discover vulnerabilities, right? Their job is to manage those vulnerabilities into an acceptable risk to the business. And Mike, so you posted about this on LinkedIn, trying to explain that discovering does not equal remediation, and the two are very much not equal.
In fact, most vendors will show you your problems for free. That’s the part they give away. Where are these tools understanding this and where are they still falling flat? Set us up here, Mike.
[Mike Johnson] This is an interesting topic that really shows the evolution of cybersecurity. When vulnerability, what we then called vulnerability management, started, it was really just scanning. It was go and find issues. And then like, great, we’ve got this big list of issues. And then teams realized, hey, we actually need somebody to fix these.
And then it was throw everything over the fence. I worked for a company who had a tool they called Ticket Cannon, that whenever a vulnerability was detected, ticket was automatically opened without prioritization.
[David Spark] Wow.
[Mike Johnson] And that made it difficult for other teams to then figure out what should they fix first, and that leads to a case of things not getting fixed.
[David Spark] By the way, appropriate name for that tool, Ticket Cannon. [Laughter]
[Mike Johnson] It was well named. The name was spot on.
[David Spark] Spot on, yeah.
[Mike Johnson] We then started moving into, well, okay, just fix all the crits first. Just fix the crits. This was our first step, this was our first foray into prioritization. But still that was like, great, we’ve now fixed every critical and we’re just going to ignore everything else. So, this is where we are today is we prioritize criticals, other teams fix those very quickly, but then you’ve got this long tail.
And so that’s where we really need to move beyond just identifying them. We need to be more focused on remediation. So, rather than starting from the beginning, the scan, let’s start at the end. Let’s start at the remediation and work our way backwards.
[David Spark] I like that philosophy. And I was thinking who would be the best person to join us in a conversation on this topic? And there’s one other person I know who’s as ludicrously passionate about this subject as you are, and that is our guest that I roped in for this. He is the CISO for Dolby, Yaron Levy.
Yaron, thank you so much for joining us.
[Yaron Levi] Oh, David, great to be here again. Mike, pleasure to be here with you now you’re on this side of the fence. So, yeah, it’s awesome to be here.
No one said it was going to be easy.
3:58.937
[David Spark] Eoin Keary of Edgescan said, “Vulnerability discovery is the easy part, relatively. Prioritization, tracking, adhering to SLA, risk acceptance, measurement, alerting, asset onboarding, coverage, cadence, frequency, scheduling, integration, and data flow of intel, not to mention accuracy and validation and exploitability verification.
It’s certainly more than an old scan.” I like how he added that all up. Luka Mar of TMX Group said, “Know what you own, know who supports it, document it, audit it regularly for changes, classify your data, know which servers/applications are critical, store or transmit sensitive data, are they internet facing.
From there, there should be a good understanding of who is responsible for what type of patching. Add them all together, and you’ll have a pretty good program. Of course, culture and senior leadership buy-in is helpful as well.”
All right. What I like about these two quotes here, Mike, is they kind of threw the whole kitchen sink in, saying, “Oh, it’s a lot more than discovery.” And if you didn’t realize that, I think these two quotes open up your eyes.
[Mike Johnson] This is a good way to introduce people that it’s hard. If you see this long list of all these things that you need to care about, and you’re kind of nodding along, like, “Yeah, that all makes sense. That all makes sense.” Oh, wow. This is a lot.
[David Spark] Like especially Eoin’s quote. I mean, that is a lot, but he’s right on all of it.
[Mike Johnson] He is. And I think though, when you have these big problems, this is the old parable of like how do you eat an elephant? And it’s one bite at a time. You have to start breaking this down. And Luka starts to go into that when they’re talking about the importance of asset management and how that factors in and how you can leverage asset management to make this something you can tackle.
And if you understand the priority of your assets, how important they are, that’s a natural feed into the prioritization problem of vulnerability management.
[David Spark] Yaron, one thing I see here is this is really a process management problem. And I think about we have processes in our business, but they weren’t perfect on day one. We keep tweaking them and tweaking them and tweaking them. And I got to think that’s the same thing here. It’s like you got to start with something and then just keep tweaking it.
Yes?
[Yaron Levi] Yes. And I think if you think about the problem in general, and I like both what Eoin and Luka said, but if you take everything they said and all the components, and there’s a lot here and it’s complicated, but most of what they’re talking about is no different than IT hygiene. And by IT, I don’t mean the IT department doing everything.
I mean information technology. I mean it’s operational discipline of how do you manage your infrastructure, how do you manage your applications, do you know who owns them, do you know what they are? What they’ve been used for? How critical they are? This is just brushing your teeth every morning. That’s exactly what you’re supposed to do.
And that’s why in my head, I also tend to distinguish between patch management and vulnerability management. I know a lot of people kind of mix them both.
[David Spark] Walk me through. How do you define the difference there?
[Yaron Levi] So, for me, the patching side is that brushing your teeth. This is that the operational discipline of managing what you have, making sure it’s up to date, it’s running, and so on and so forth. Now, the vulnerability management is the next step for that. It’s the next step for what you cannot patch or what is not patched or whatever you have as it is.
How is that exposed or what exposure does that give you or what to the organization that now you need to manage? And that’s where you look at the vulnerability. That’s where you look at the risk. So, that’s the fact that you have so many vulnerabilities. Okay, it’s one thing. But how you consider them in the context can actually exploit them, can actually do something with them.
So, that’s why I think we need to separate it. All the hygiene management, the process you alluded to, David, all of that, that’s part of the ongoing, again, I would call it IT hygiene. The step beyond that, whether that’s exposing us to anything, that’s where the security team comes in to help quantify that.
What else are we missing?
8:18.639
[David Spark] Christopher Langton of Vulnetix said, “A similar misconception – most penetration tests do not equal vulnerability discovery. Just as where vulnerability management aims to start from discovery and often end there, a pentest starts at reconnaissance to uncover defects, potential vulnerabilities, and stops there.
Vulnerability management also needs a verification stage, preferably continuous verification, to detect future regression. Review is also key because the risk acceptance/do nothing is a valid business decision but is not a mitigation. It is not the end of the vulnerability management process. You must always treat these as “except until” or specifically have a review date on everything with a rule it cannot be accepted/deferred at a review.
A real plan needs to be made.” And let me also throw in Maor Franco who said, “Risk management does not equal vulnerability management, does not equal vulnerability discovery.” Throwing more things that don’t equal each other here.
[Mike Johnson] [Laughter]
[David Spark] “For years, we’ve been preaching for a patch, patch, patch, hoping for a vulnerability-free utopia. We both know that will never be the case. Hence, your point on focusing on the relevant ones first. Not having a system that can tie in a vulnerability/exposure to an active threat, a vulnerability to possible impact, its risk of exploitation and reach, an assumed no-fix approach due to…without the appropriate guardrails will only last our vul whack-a-mole game and de-focus our attention.” So, I’ll just run out, there’s penetration tests, that whole environment that’s involved in that, and also risk management not being the equal of vulnerability discovery or management.
I will start with you, Yaron, on this.
[Yaron Levi] Yeah. So, the first question is like what is a vulnerability? Because the vulnerability is not necessarily always just a software defect. It can be configuration. It can be many different things, right? So, even if I think about a house, even if your house has a door and the door has a lock and you have windows and the windows are locked and everything else, I would still argue that your house has vulnerabilities because somebody can still cut through the sheetrock.
I mean, the lock can be compromised, etc., etc. So, maybe the distinction is there are always going to be weaknesses. It doesn’t matter even if you’re going to be fully patched, there are always going to be weaknesses. The question is whether those weaknesses can actually be exploited and what is the level of effort that need to exploit those.
Even if I had like a steel door in front of my house, somebody’s going to come with a bulldozer, that’s not going to help much, right? So, that’s essentially what we’re trying to quantify. We’re trying to say, “Okay, yes, software defects are some type of vulnerabilities.” Going back to the hygiene, we need to address those in a timely fashion, etc., etc.
But it doesn’t end there. It’s like what all those vulnerabilities, how do they eventually, the configuration we have, the systems we have, the weaknesses that we have or we don’t have, all of that, how does that contribute overall to whatever level of weakness that we have in the system that potentially can be exploited and how much of that can we mitigate, can we afford to mitigate, or even possible to mitigate?
So, using like penetration testing as an example, I agree, it’s not a vulnerability discovery, but it’s actually the way to test how further can you go once you identify a weakness. Can I actually get through that door? So, that’s another way to help us quantify and maybe prioritize in a much more meaningful way.
[David Spark] And correct me if I’m wrong, that also tells the story of what a vulnerability means. Just to say that we have a weakness here doesn’t mean anything until you have a pentest that says, “Oh, look what we can do with this weakness,” or excuse me, a criminal can do.
[Yaron Levi] That’s exactly right. I would argue by just having a server on the network, it’s a weakness, it’s a vulnerability, it’s a potential vulnerability. If the server was at the bottom of the ocean under like six feet of concrete, it’s not a vulnerability.
[David Spark] It’s also not useful for the business.
[Yaron Levi] Correct.
[David Spark] [Laughter] All right, Mike, I throw this to you, same thing about what equals vulnerability management, which I think is what we’re kind of leaning on more here. How do you compare it to risk management and to penetration testing?
[Mike Johnson] I think some of this ends up getting maybe semantics around logical expressions and what equals means and what equals doesn’t. Yaron, I liked what you were saying around penetration tests can discover vulnerabilities. They shouldn’t be your only mechanism for discovering vulnerabilities, but they can certainly uncover a vulnerability that you didn’t know about.
And so I do think it’s a useful tool for discovering vulnerabilities.
But I also like what you said about reachability mattering. If that is a vulnerability that cannot be exercised, it is still a vulnerability, but that then brings us back to prioritization. Should that be the first one that you go fix? No, you’ve got a bunch of others that you should go fix before it.
And this is also where some of our common scoring mechanisms like CVSS can actually be problematic, where something could have a CVSS score of 9.8 and ring the alarm bells, but it’s on a system that cannot be reached. It is a standalone system that has no way of anybody talking to it. Shouldn’t be the first thing that you fix.
So, I do think the reachability is a really useful tool in understanding and helping and assisting with prioritization.
Sponsor – Intezer
13:58.951
[David Spark] Before I go on any further, let me tell you about our sponsor, and that would be Intezer. And you’re going to want to listen to this because it has to do with your SOC and alert fatigue. So, alert triage and investigations are time consuming for security teams. I don’t need to be telling you this, but I’m just setting you up because it doesn’t have to be that way.
Smart security teams are using AI to automatically investigate alerts from the security tools 24/7 with an average triage time of just two minutes. So, how does this actually work? Well, Intezer is an innovative platform that integrates with your security tools to monitor alerts, collect evidence, and investigate every artifact.
When Intezer uncovers evidence of a serious threat, it escalates the findings to the SOC analyst. Intezer also reduces noise, correlates alerts, and automatically resolves over 90% of false positives. This means even low severity and informational alerts get investigated. You aren’t wasting time on false alarms, and you have actionable recommendations for serious threats.
Intezer extends your team by emulating an experienced SOC analyst with an AI framework backed by years of industry research. It’s designed to be a cost-effective, easy to set up platform that provides detailed, transparent results that SOC analysts can trust. No playbooks, no chatbots, no engineering to set up.
With Intezer supporting your SOC analyst, your team can eliminate alert fatigue, uncover hidden threats, and stay focused on what matters most. AI won’t replace your SOC team, but it can be a game changer. For more, go to Intezer’s site, it’s intezer.com.
What are the elements that make a great solution?
15:45.743
[David Spark] Charles Immordino said, “Successful vulnerability management is a bit like triage. I’m absorbing everything from red team findings, the SOC, AppSec, and more besides scan content.” So, scan content’s just one element. “From there, if it takes a determined plan and some tact to get reductions in place and things resolved, or if it was simply as find it and fix it, no one would have vulnerabilities.
A great deal of convincing admins they actually own assets because of the lack of asset management present. From the outside, it looks simple. From the inside, it’s very nuanced.” That’s a really good point. Very easy for someone else to tell you, “Oh, just fix your vulnerabilities.” They’re not living inside your environment.
[Mike Johnson] Mm-hmm.
[David Spark] But let me also mention Brad Phillips of Cox Communications. His quote, “A lot of big companies are pulling in specialized remediation managers that have the expertise to coordinate the efforts and work hand-in-hand with the vulnerability management program, a zero vuln footprint, and those so-called flatlines are never going to happen, especially in larger organizations.
The sooner leadership can realize that, then the program can move forward and be successful.” All right. I think the good point of it gets really, really messy because I think it’s just a lot of moving parts that also involve humans, right, Mike?
[Mike Johnson] I love the combination of these because what they’re really pointing out here is those who’ve never had to deal with vulnerability management think it’s easy. Like this is like just fix it. Just fix everything that you find. And they’re really pointing out that this doesn’t really work that way.
You can’t really just say, “Hey, we’re going to fix everything. We’re going to get to zero vulnerabilities.”
It’s back to Yaron’s point of the only secure or invulnerable or useless at the same time system is something that’s under the ocean. If you actually try and fix everything, when you get a notification that, hey, it needs fixing, you’re going to have a hard time with your business. And your business is going to have operational uptime challenges because, “Hey, we just took it down for a patch yesterday.
There’s a new patch out today. We got to do it again.” And that just doesn’t scale in the modern business. So, they’re really pointing out that on the outside, it seems easy, but the reality is it isn’t. The last thing I’ll say on this, there’s the saying of simple versus easy. Running a marathon is simple.
You just start running, but it’s not easy, and I really think that’s what they’re stressing here.
[David Spark] That’s a really good analogy. I want to mention something that you analogized earlier, Yaron, about the idea of the house of having the locks on the doors and the windows shut and stuff. Vulnerability management doesn’t necessarily need a scan. Like, you can have a good environment that still has plenty of vulnerabilities.
It doesn’t need necessarily a patch for there to be a vulnerability, right?
[Yaron Levi] Yeah, absolutely. I mean, start with the threat modeling and start with your system architecture. I mean, your system architecture can be flawed or actually is flawed, I mean, to begin with. It’s just a matter of how many flaws you have. I’ll tell you a little story, right? So, I don’t know if you guys have it, maybe you have it in your bathroom, but I have a bathroom scale.
And when I wake up in the morning, one of the things I do, I get on the bathroom scale, and the bathroom scale tells me every morning, you weigh 215 pounds. Okay, what do I do with it? It doesn’t tell me how to lose the weight, and that’s the challenge that we have with all of those programs, right?
I mean, we have all those scanners, and we have all those tools , but they basically tell us how much we weigh, but they don’t tell us how to fix those. And it’s like, as Mike said, everybody thinks, “Oh, it’s easy,” right? “Hey, lose 20 pounds.” We know it’s not easy to lose 20 pounds.
But same thing with the vulnerability measurement, just patch it. It’s not that easy to patch it because you impact the business. How are you going to impact the business is really important, I mean, to patch the system. So, it’s much, much more nuanced. And I think that’s where not having that real context.
And I think as Brad here mentioned, if you take all this information and you just need to bring it to a place that you can take all the things under consideration, what does it really mean? How exposed am I? How easy it is to explore it? What does it mean from a business context perspective and everything else?
Then you can use all that information to prioritize. But just simply to go and say, “Oh, we’re just going to fix everything,” it’s not practical.
And just one more note about that, think about it from another perspective, from the people that are needed actually to do the job. We’re asking people to do something that the best result they can get to is zero. Now, how demoralizing is that to do something and put all the efforts that the best thing you can ever get to is zero?
And it’s never ending, right? So, we have to think completely differently about this problem and how to address it. I don’t know if you guys saw it or remember, but Sunil, I think he was a guest on the show in the past.
[David Spark] Yeah, Sunil’s been on the show, and I saw him not too long ago too.
[Yaron Levi] He had a good talk at RSA, I think like 2020. And when he was comparing how we treat servers, and he compared that like pets and cattle.
[David Spark] Yes, I know about this.
[Yaron Levi] That was actually a useful idea, at least how to approach the problem in a different way, to think completely differently about how do we manage our infrastructure as an example, right? That’s what we need to do more of, as opposed to just keep scanning, keep patching, and just kind of endlessly running around and chasing our tail.
What are they looking for?
21:16.475
[David Spark] Dennis Spalding of Charles Schwab said, “Management equals prioritization in respect to your organization’s unique environment, configurations, and operational requirements.” Nice callback to our previous segment. “High does not always equal high for everyone. If you’re just taking the vulnerability discovery tool risk ratings as is, you could potentially be spending a lot of unproductive cycles on items that are not high risk in your particular environment and configuration, leaving vulnerabilities that could be more impactful in your environment longer than necessary.” So, I’ll start with you, Mike, what is it you need to take into account as you’re determining the importance of dealing with a vulnerability, a patch, a concern?
Like what are the variables you take into account?
[Mike Johnson] First of all, it is key that you are thinking about your own environment and not just as Dennis points out here, saying, “Hey, it’s a high. The tool told me it’s a high, therefore it’s a high for me.” You really need to look at, back to the asset management, how important is this thing to us?
What does it do? What is it connected to? What are the mitigations that may or may not be in place? Is it reachable? Is it something that can actually even be connected to by someone on the internet? Does it require you to bounce through something else? Does it require them to download an app and look at some sort of local code?
You have to take any number of variables into effect. And the thing is anything that I say right now is wrong for every listener right now because it’s different for them than it is for me.
[David Spark] Quick follow-up question. When you are a new CISO, you come in, and I’m sure this is an evolving thing, how long do you think it takes for you to get a basic understanding of your environment? Because I assume day one, you don’t understand your environment. How long do you start to feel comfortable with your environment, understanding it?
[Mike Johnson] It really depends on the business. Obviously, “depends” is not where I’m going to leave the answer. If it is an internet-facing application, single-product company, it’s using all sorts of modern capabilities, built on top of AWS, so on and so forth, you can understand that pretty quickly.
That’s an environment that you can ramp up on very quickly. If it is something that the environment has been around for a long time, look at Yaron’s world, Dolby’s been around for a long time. I’d love to hear Yaron’s answer to this question, but when he joined, it probably took him a while. It took me about three months to figure it out at Rivian.
And I won’t say that even after that three months, I really…
[David Spark] No, no, I’m sure there’s always things you discover. Like you uncover a rock, “Hey, I didn’t know this was here.” [Laughter] One of those things. Okay, Yaron, throw this to you. How long did it take you before you started to understand Dolby’s environment?
[Yaron Levi] Yeah, months. I mean, it’s definitely months. As Mike said, Dolby’s been around for 60 years, and it evolved a lot and changed in those 60 years. But also have a lot of culture that has been over 60 years, right? I mean, so you’re dealing with a lot of things, and months, I mean, to understand the business, to understand the complexity of the business, the different nuances, I’m still learning almost every day.
And I can tell you some things that I’m thinking about today, I’m like, “Man, I thought about it kind of wrong maybe when I started the first three months, and now I’m shifting and changing.”
[David Spark] Say you’re dealing with vulnerability problems in the first two months versus vulnerability problems six, eight months down the road. How are you handling them different between two months and eight months?
[Yaron Levi] I think the main thing you’re doing different is maybe have a better understanding of the context. That’s what you’re going to do different. But the basic hygiene is the same. I mean, do you have the process in place? Do people again, brush their teeth and eat their broccoli, right? I mean, do they follow processes that they have for their systems?
That’s probably the first thing that you have to look at. And if not, maybe that’s a place to start. I mean, put a policy in place, start a process, right? I mean, start to understand what the inventory is. But if it’s running pretty well, is yes, how do we better understand the context to perhaps make better decisions for prioritizing the risk for the business?
[Mike Johnson] What I would say is the process, and Yaron kind of triggered a thought here, the process is pretty similar from company to company. The context, the execution, is what varies so wildly. And that’s what takes a while to come up to speed when you’ve joined a new company is the context and how do you really operationalize the pretty well-understood concept of managing vulnerabilities.
Closing
26:10.209
[David Spark] Well, that is a good point to close this discussion and to ask you, Yaron, lots of really good points from the quotes that we’ve pulled here, I want to know from you which quote was your favorite and why?
[Yaron Levi] I think my favorite quote is from Luka Mar from the TMX Group who said know what you own, know who supports it, document it, audit it, regulate it for changes, classify it, etc., etc. Because everything that he talks about, for me, contribute to that context. And that context is essentially what kind of helps you to make those better decisions in prioritization.
[David Spark] Essentially what Luka was saying is you’re going to learn those over the first few months, essentially. All right, Mike, your favorite quote and why?
[Mike Johnson] I’m going to go with the other pairing, Eoin Keary, from that same segment. What I liked about that, and again, you summed it up well, David, of talking about vulnerability discovery is the easy part. Prioritization, tracking, SLA, risk acceptance, so on and so forth…
[David Spark] I’ve always known it’s kind of a long list, but he really put it together well here.
[Mike Johnson] Exactly. And that’s what I liked about it, is it really, it’s a good way of thinking about the whole problem, but also such a good introduction into this episode to really set up that this is a hard problem. It is something that has many components, and Eoin summed it up well.
[David Spark] Well, that brings us to the very end of our show. I want to thank our sponsor, and that’d be Intezer. Remember, extend your security team with AI, with Intezer, intezer.com. Thank you very much, Mike. Thank you very much, Yaron, and thank you to our audience. We greatly appreciate your contributions and for listening to Defense in Depth.
[Voiceover] We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site CISOseries.com where you’ll also see plenty of ways to participate, including recording a question or a comment for the show.
If you’re interested in sponsoring the podcast, contact David Spark directly at [email protected]. Thank you for listening to Defense in Depth.