HomePodcastDefense in DepthDefense in Depth: Proactive Vulnerability Management

Defense in Depth: Proactive Vulnerability Management

How do we turn the tide from reactive to proactive patch management? Does anyone feel good about where they are with their own patch management program? What would it take to get there?

Check out this post and this post for the discussion that is the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Steve Zalewski. Our sponsored guest is Sumedh Thakar (@sumedhthakar), CEO, Qualys.

Got feedback? Join the conversation on LinkedIn.

Thanks to this week’s podcast sponsor, Qualys

Qualys is a pioneer and leading provider of cloud-based security and compliance solutions.

Full transcript

David Spark

How do we turn the tide from reactive to proactive Patch Management? Does anyone feel good about where they are with their own Patch Management Program? What would it take to get there?

Voiceover

You’re listening to Defense in Depth.

David Spark

Welcome to Defense in Depth. My name is David Spark. I am the producer of the CISO series. We are now starting our fourth year of the CISO series; we just celebrated our third year. Steve Zalewski is here joining me as my co-host. Steve, make some noise, let people know what your voice sounds like.

Steve Zalewski

Hello, audience.

David Spark

All right. More on Steve in just a second. First, I want to mention our sponsor who has been a phenomenal sponsor this CISO series and that is Qualys, and guess what? Qualys is responsible for bringing our guest today. More on that too in just a moment. First Steve, let’s talk about the topic. Set us up on this discussion of Patch Management.

David Spark

You were looking for aggressive Patch Management solutions. Do you feel that anyone delivered on this sort of aggressive ideas around it? Do you think anyone is actually happy with their own Patch Management Program?

Steve Zalewski

Nobody of any size is happy with their Patch Management Program. No way; not yet. The question is: “Did anybody answer my question?” What I was asking for was maverick thinking, or “out of the box” thinking to look at the problem. There’s a couple that we’re going to talk about today that I think are touching on how we need to rethink Patch Management and what we can do to approach the problem as a proactive exercise.

David Spark

The whole element of being proactive I think is really interesting. We’re going to get to that in the very first segment. Let me introduce our guest, our sponsored guest for today, who is a return champion with a new title. He is now the CEO over at Qualys, Sumedh Thakar. Sumedh, thank you so much for joining us.

Sumedh Thakar

Thank you, David, for having me on the show. It’s a pleasure being here.

How did we get here?

00:02:14:11

David Spark

Erik Bloch of Sprinklr said, “Isn’t the nature of patching always reactive?” We hopefully deploy with a lockdown system, but then new updates to packages and software come out and we’re instantly behind the 8-ball, reacting or updating. We also have to take into consideration DevOps, QA and engineering before pushing out an update to make sure it doesn’t break anything.

David Spark

Tony M said, “Upstream efforts would be on software companies using secure coding practices and proper testing before the patches get deployed to customers.” Everything we do and at organization level is downstream or reactive. So, both Erik and Tony are arguing, the whole concept of proactivism at misnomer. What do you think Steve?

Steve Zalewski

As long as you look at this as an operations, security operations or IT operations exercise, it’s reactive, patches happen, which means now you’re behind the 8-ball to get them applied. I think that was why I was asking for some maverick thinking, which is, if that’s the only way that we can do it, we’re going to fail. Continue, for all the reasons that have been outlined. I think there’s the beginnings, at least in this part, as to what does proactive mean by looking at it, not as a security operations patch window?

David Spark

Let me throw this to Sumedh. What does proactive Patch Management mean to you? How do you respond to both Erik and Tony’s comments here saying like, “It’s just reactive by nature.”

Sumedh Thakar

Yes, I think as long as we are going to have software and we’re going to have humans courting software, we’re always going to have bugs and issues that will always need to have patches and need to be fixed. I don’t think that there is any solution that we can see right now where we just don’t have to have any patches that come up now. Where we can be proactive more is in reducing the amount of time from the point that a patch comes out to the point where the patch gets deployed to reduce the exposure window. And that’s where the process of patches, which basically today is a long process involving multiple teams. The question here is: can we get proactive in the sense that as soon as a patch comes out, can we just go ahead and apply the patch and reduce your exposure time and better our security?

How do we make this everyone’s concern?

00:04:36:09

David Spark

Hank Masters of Secureworks said, “I think this is mostly a cultural issue, and many times the people actually apply the patches on the operation side are not the same people on the vulnerability management side. So, they do not see eye-to-eye and their priorities are different.”

David Spark

Kevin Kentner of CrowdStrike said, “Perhaps we also need to solve this problem. How do we architect our system so we actually can patch, instead of being given extremely limited windows?

David Spark

So Sumedh, I’ll throw this to you first. They’re arguing that this whole issue comes down to humans because so many people get physically involved with conducting the patches, making sure they can’t work when the patches are applied, what they can do to test the patches before they go into production? How much are people involved in this process?

Sumedh Thakar

I think that’s a big part. It’s the people and it’s not just a number of people, it’s the silos within the groups, between security group and IT group. It’s different pool sets that are being used. So when you’re detecting it’s a different tool; when you’re patching, that’s a different tool. So I think a lot of that does come down to just the process itself, it’s not the application of the patch itself as this can be very quick. You know, this is an issue in general that comes out of the fear of when the patch should be applied. If you look at more modern operating systems; on the consumer side when you look at our phone and our laptops and things like that, we’re getting apps that are updating themselves and patching themselves really all the time. In many ways your phone apps are a lot more up to date than your enterprise apps that you’re using. I think that’s because there are no people involved, your phone just updates its app on its own. On the enterprise side we do spend a lot of time between the different groups and different teams and different tools trying to get this patching working.

David Spark

Going to come back to you on this and ask you about how we can get to a more automated state. First, let’s go to Steve. Where can this people prom be ameliorated? I also think there’s a lot of emotional issues that come in here as well.

Steve Zalewski

What you’re seeing… and I like this comment by Hank is, many organizations, from a security perspective, are in very different places. In some places security operations does the patching under IT operations. In other cases it’s under the CISO. It really depends upon the culture of the company as to where the function resides, which is why you see a lot of this back and forth. If it’s all one team, your team is overwhelmed. If it’s two different teams, then the teams are competing for priorities of those time windows for what they have to do for IT operations, updates vs security patch. As long as we have this conversation, it’s a lose-lose conversation. The transition I think we want to talk about here is not, how do I secure the business as, (the sheer number of these technical patches) as it is: how do I ENABLE the business? And, therefore, let’s look at the business risk so that I can reduce the workload on those two teams to be able to apply the necessary patches in an appropriate timeframe.

David Spark

Sumedh, you had referenced the fact that, sometimes our phones are more secure, given how they automatically update and patch themselves than enterprise systems which become a little more convoluted. I think that may have to do with the mobile phone updates are one user kind of a situation, unlike enterprise systems, which is more than one user. I know that Qualys is working or has deployed a more automated solution. Can you speak to that and what has been the problem with automation up until now?

Sumedh Thakar

Yes, you know, I think it’s not just about one user. I think it’s the concern about are we going to break things when we are applying patches quickly and in an automated way. I think that part, more and more, I would say, with the new software development processes with automation has been changing. In the past you would get a massive update of a software once every one year and that had way too many things and a higher chance to break. As software developers are moving more and more towards smaller, quicker, more targeted patches and updates, I think the risk of breaking things progressively reduces. So, one of the things we focused on with our zero touch patching is, essentially, bringing that concept to say that on certain devices, can we keep certain applications patched as soon as the patch from the vendor comes out? So you don’t have to go through the process of detecting the vulnerabilities through a scanner and then sending it to the IT team and then they are spending their time aligning their schedule. And then, finally, patching the system. Can the systems be patched, or software be patched? Of course there’s going to be concern around data center asset vs remote asset, but it’s all about how do we reduce the time that people are spending on this? So what we see with our customers is, a lot of them are looking to say, “I have way too many remote devices, laptops that are on home networks that I need to patch. I don’t want to patch them manually.” If those can always be updated and automatically up to date, then the fewer resources that I have, I can spend that more on the data center side. I think that’s kind of the transition that we are seeing.

David Spark

Here’s my core question around that: the ability of physical automating patching, isn’t the problem. It’s everything else that you mention. Why is now a good time, or what is it that you’re avoiding that was not avoided before?

Sumedh Thakar

I think it’s just a matter of how long the exposure can stay, and how willing are you, as an organization, to do that? Combined with the fact that more and more software is being updated in smaller chunks and more frequently, reducing the risk of breaking things. So, today, if you look at the exchange vulnerability that came out, I think we had like 40,000/50,000 systems that were on the Internet exploited within a matter of 48 hours, because the attackers were using automation to go out and compromise systems that were not patched. So the need for faster and automated patching I think is increasing. The ability to get systems patched as soon as possible is increasing. Right now that’s driving people to say, “Systems are increasing and the number of people I have, it’s hard to find security, cyber security people, and the amount of time it takes for them to exploit the system is reducing as well.” So we need this automation.

If you look at the problem this way:

00:11:36:20

David Spark

Mathew Biby, CISO over at Satcom Direct said, “Perhaps the development of a framework similar to attack (referring to the Mitre Attack Framework) that could be a template to provide organizations with a form of contextural awareness of vulnerabilities in a given environment.” And Yaniv Bar Dayan of Vulcan Cyber said, “Why don’t we ask ourselves, ‘What is the impact of exploration of a potential vulnerability on this particular business service or asset?’ And, according to the answer, apply controls REGARDLESS of the fact that a vulnerability exists, or not. Once we’ve applied these concepts we can apply capabilities like machine learning for vulnerability correlation and prioritization or automation and orchestration for remediation. Lastly, Jonathan R. at CISO over Lightspin said, “Imagine if we took the threat level approach to Sim Engineering responsive and preventive measures (referring to WAAF, EDR and SOAR, creation tuning) and apply it to Patch Config Operating System Management.

David Spark

So what do you think Steve? These are a few different sort of ways of looking at a new approach to Patch Management. What’s your thoughts on this?

Steve Zalewski

I would say, “We’ve got some maverick thinking here.” I am going to tip my hand on my favor for the end but, Yaniv Bar Dayan really has touched on the concept. What we’re now saying is, “Instead of securing the business, we want to protect the business.” We really want to enable the business. And so with Yaniv’s approach it’s, let’s take a look at the business holistically and realize that, the need to be able to patch an application or a system, or an operating system, we have to look at it holistically and understand the true risks of the company. We have to figure out if we have sufficient defense and depth for compensating controls. It isn’t just a game of hurry up and apply any patch that somebody’s identified, that could be exploited to the likelihood of exploit.

David Spark

You know, in one of the stats we’ve heard from a competitor in your space, and that is that 60 to 70 percent of vulnerability just really don’t need patching at all. You could avoid it all together, but if you run an automated solution it seems like, well, if you’re automating just go ahead and do it. Like, what’s it going to hurt? Although, as you said, “It’s a lower risk if it’s smaller patches over time.” Is there a need to prioritize the patches or, in automation, just let it go? What do you think?

Sumedh Thakar

Yes, I think you “hit the nail on the head” here with the prioritization. I don’t think it’s about saying that “It’s not required or we don’t need to patch these systems.” It’s about prioritizing the ones that should patch first and then the ones that should patch second basically. And today one of the challenges of taking this stance for patching is because there’s just too many vulnerabilities that need patching and the translation of the actual patch takes time. And so organizations really need to take a risk.

David Spark

May I ask you, how do you figure out that 30 to 40 percent of vulnerabilities need a patch? Where do you begin, and is there a way that Qualys is helping?

Sumedh Thakar

Yes, absolutely, I think that’s one of the things that now, as we’ve updated our vulnerability management, our solution is to include prioritization as part of it. This is important because you can prioritize vulnerabilities based on the criticality of the asset based on exploitability of the vulnerability, if something is actively included in an exploit kit, you want to prioritize that first. In fact, just today we announced a ransomware risk assessment tool from Qualys where we actually prioritize the vulnerabilities used by ransom reactors. About 100 of those as the ones that you should really focus on so that you can fix those first to make sure that you can reduce the risk of ransomware. So there are many different ways that we work with our customers to prioritize what they need to focus on first. Once that is done then you can leverage the automation for the low risk factors that need to be applied. I think we should always touch everything. It’s just a question of how do you segment that between the first priority, second priority, third priority. I think that’s where you need a combination of the prioritization and the automation of the tooling to be able to accomplish that.

David Spark

So, to go back to the beginning, it’s not easy; it’s not simple, it takes a lot of stuff too. I mean, you’re not looking at one source. Like, “Oh, here’s the ultimate source”, and we’re going to reference this later, like, the CVSS score like, well that will just tell us everything and just go by that, right? I mean, we lean too hard on that or, we have historically.

Sumedh Thakar

Right, and there are many different sources, there are multiple different places where we get intelligence from and, that is the challenge that enterprises face: how are they going to go and collect this intelligence from many different sources? That’s where having that completely embedded as part of the solution really makes sense. Like helping find the vulnerabilities prioritizing them then also helping fix them is truly more the need rather than just giving you a long list of CVEs that you should patch and that really doesn’t help customers.

David Spark

That’s not sustainable, Sumedh. If I have 10,000 potential patches, and that is not an unreasonable number for a middle sized company to have, that’s right across the entire stack from operating systems to middleware to applications, to firewalls etc., that is a reactor problem we cannot fix. It’s way past any type of automation and anything else. Don’t forget too, a patch has an impact to a system, can be a simple patch, can bring the application down. That’s why all the testing and all the complex systems we’ve built, which is why I’m pushing really hard to say, “We need to maverick thinking.” As long as we look at it as an operations Patch Management; just do the high stuff first and eventually get to the medium and low, I would argue that 75 percent of the company struggle just to do the immediate and the high, and all the rest are just left. Then everybody just kind of throws up their hands and say, “We ignore the rest.” That’s why I’m arguing, we have to do some maverick thinking and think about the business and realize we’re never going to patch all 10,000. You may patch 4,000; I may patch 50. To the business risk and how we measure that, it’s appropriate for both of us.

Sumedh Thakar

That’s it, you know, I completely agree with that. That’s really why we need to change our thinking in terms of different approach towards tooling, towards the process, so that we can really get to the point where we have more automation in terms of patching things and prioritizing things. The humans cannot go and really figure out between that 10,000 list; which one should I patch first that is going to reduce the risk to my business?

David Spark

Yes but, Sumedh, what’s good enough? You’re abdicating your responsibility to help me know what good enough is with, “Well, if you prioritize”, it’s left as an exercise to the reader. We can’t do that. We’ve got to step up and take a much more aggressive approach to talking about “good enough” as balanced risk for the business; to enable the business not to secure. That’s why I really was excited about this concept and am pushing so hard on you is, that’s where we have to revert, from reactive to proactive maverick. How do we have that conversation now?

Sumedh Thakar

Yes, I completely agree. Experts like Qualys and others need to be the one being able to tell the customer looking at their data, which ones are the ones that have the highest risk for them. Ultimately, to allay the fears of the customers. It’s like helping them figure out which assets and which vulnerabilities in their environment cause them the highest amount of risk. They often don’t have the capabilities to figure that out. There are many different sources out there that can be collated, like I gave the example of the ransomware focusing. As a customer I do want to reduce my ransomware risk and we are providing a way for them to focus on the vulnerabilities that have been known to be used by ransomware. So that’s a much easier thought process to sell, and that’s going to help me reduce my risk significantly, rather than me, as an enterprise, having to go figure out all the vulnerabilities that would need to be patched to reduce my ransomware risk.

David Spark

Good answer!

What aspects haven’t been considered?

00:20:29:11

David Spark

Steve Smith of Mindbody said something I referred to earlier. “The common failure I see in vulnerability management is basing everything off of a CVSS score. Your typical patching teams cannot keep up.” What Steve Zalewski was referring to just a moment ago. Jerich Beason, CISO, Epiq said, “Today we measure twice and cut once out of fear of causing an outage. But I long for the day when we can cut and then monitor because, arguably, the ramifications of missing a patch is significantly worse than a low likelihood outage we can quickly recover from.” I’m going to start with you Sumedh. I mean, this is some pretty cool maverick thinking. Let’s not be overly careful. What’s worse is not the outage, but what’s worse is the damage that a piece of malware could do to our business, yes?

Sumedh Thakar

I completely and wholeheartedly agree with Jerich. The way we’re thinking about this today is, like I mentioned earlier, as software developers are getting a lot more practice. They’re putting out smaller and more targeted and more frequent updates. That certainly helps reduce what we used to have in the past where once a year you get a massive update where lots of things could break. So I think, if you combine that to say that updates are getting more targeted, more frequent and smaller, and the amount of exposure that we are willing to accept is reducing now because of the speed at which attackers are going out and doing this, I think really, we need to be thinking about how we balance the potential of getting compromised severely with the vulnerability. How do we balance that with automated updated of the software, which can potentially cause an outage? So today we see this in IT already. Like, when people are going into AWS and things like that, there’s a lot of automation being built in. Spinning up new servers, spinning down servers; completely automated. IT is doing more and more automation with DevOps and I really feel that we need to take a look at this. That is the one thing that we cannot get away from patching. Today you are in the environment where DevOps people are building new images almost every single day. So we have the opportunity to be a lot more proactive with patching by actually being able to include those patches, roll them in. Of course we can reduce the risk by doing more automated testing, but I think we need to change our mindset in terms of thinking that if we can actually patch the system first and then handle any small risk, that can actually completely change the way we’re doing things today.

David Spark

Steve, I’m letting you have the last word on this topic.

Steve Zalewski

I agree with Jerich, which is, we should make it so that patching is almost benign. That whether we get it right or wrong, it is a healthy thing to do, not a critical thing to do. I look to the industry, and I look at what some of the SaaS, and in particular, platform as a service, but primarily, SaaS service vendors, have done. They have figured out how to do better vulnerability management of the infrastructure that’s patching in the other components behind the scenes where we don’t have to look at it. So there is progress there. The SDLC, Software Development Life Cycle, which is building the applications that are designed to be resilient in their patching of individual systems, can be done. Which also means, they’re resilient to be able to withstand some failure. There’s progress made there, but how do we bring that into every company that has their own applications and a lot of legacy infrastructure? We’re talking about greenfield to get it right going forward and learn and most of the companies aren’t there, and will never be. Always the legacy is the reason why everybody throws it up there, we can’t do anything. I think to your point, there’s some maverick opportunity to learn, but we have to let go. The consequence though is third party and imputed fourth party risk. More and more what I want to be able to do is make it somebody else’s problem to solve that hard issue. How do I manage that risk because, at the end of the day, I build a product and I have a supply chain of all of this third party stuff that I’ve consolidated together? Digital transformation, how do I measure that risk so that I know that all those vendors are applying appropriate vulnerability management and doing enough to meet my risk posture?

David Spark

I’m gonna sum it up this way: It is better… and agree or disagree with me both of you… it is better to deal with the risk of patches being actually placed, activated than it is to deal with the risk of not patching things and the unknown risk that is out there. So we’re shifting risk to what I assume is a considerably reduced level of risk. Sumedh, are you with me on that?

Sumedh Thakar

I do, and I think that there is a process here that can be prioritized as well. Like there’s a lot of assets that you can actually patch immediately, like laptops which we see everywhere out there today and then reduce the amount of effort in testing as an example. There’s no reason why all the Adobe and Chrome software on my laptop should not be automatic updated all the time; I don’t need to wait for people to push out patches because these are coming out almost every couple of weeks. They don’t tend to break things as much. Then I can focus resources more maybe on production systems to build a better process that I can get auto-patching in those environments as well.

David Spark

Sounds good. All right. Now we come to the end of our show where I ask both of you, what was your favorite quote and why? Steve, I think both of you have already teased what your favorites are. So I’ll start with you, Steve. What was your favorite quote and why?

Steve Zalewski

So, I actually have a tie today. That was somewhat based on the conversations as we did it today to realize there were two key pieces to maverick thinking. The first is what I alluded to earlier, Yaniv Bar Dayan from Vulcan Cyber, around looking at business vulnerability management and enabling the business, and being able to understand how to protect, as opposed to just patch.

Steve Zalewski

I want to add Kevin Kentner from CrowdStrike. His point, and it was what I was alluding to when I talked about SaaS and passes, we’ve got to re-architect our systems. We don’t have to re-architect Patch Management. The maverick thinking is we’ve got to re-architect our systems so that they’re resilient enough to be able to be patched or mitigated and measured. That way we get out of the 10,000 moles, whac-a-mole problem. I really have to give it to Kevin as well, those are my two.

David Spark

All right, Sumedh, Steve’s obviously breaking the rules and picking two. You know you can’t do that Steve. Sumedh, your favorite one is… and you can pick two if you go that far.

Sumedh Thakar

No, I think I’m going to pick Jerich Beason. I really think that’s the most maverick thinking, and my opinion is we really need to start just making bad Patch Management and patching things as part of the process. Now I will say that it’s true what he’s saying, we do need to be able to re-architect our systems in a way that we can patch easily, and more frequently. We need to also be able to reduce our risk by leveraging business prioritization to only auto-patch the vulnerability in systems that really are causing immediate risk. Just to reduce the amount of risk you may have from being able to patch proactively. Jerich is making a really good point. I think to achieve that we will need some of the things that others have mentioned here to come together.

Closing

00:28:24:10

David Spark

Good point. That brings us to the close of our show. I want to thank both of you, and Sumedh, I’ll let you have the very last word here. First, I want to thank our sponsor, which is your company Sumedh: Qualys. Thank you so much Qualys. If you don’t know how to spell Qualys. It’s spelt Q-U-A-L-Y-S, and if you were to throw a “.com” at the end of it, guess what? You’d arrive at their website. I bet you if you just type “Qualys” into Google, it would also bring you to Qualys as well. Two different ways to get there. So thank you for sponsoring this and many other episodes we’ve done. Steve, any last thoughts on this topic and/or any comment for our guests?

Steve Zalewski

I want to thank you Sumedh. Excellent conversation. I pushed hard at your. This was good for our audience to hear. To our our audience, thank you. My ask for maverick thinking and to look at a very difficult problem, once again, the audience has done a great job of demonstrating just how strong they are as practitioners and being willing to step up and help us look at this problem.

David Spark

Awesome. Now, Sumedh, any last thoughts? Also, one question I ask everybody, which is: are you hiring? And two: do you have any special offer or anything you want to point our audience to?

Sumedh Thakar

Yes, thank you guys. I actually like the conversation because in maverick thinking we need to be able to ask the tough questions and answer them. I think one of the things with Qualys, who have been in this space for a long time and who have really seen the viewpoint of the customers, we ourselves made the transformation recently to move from being a vendor that’s just giving you a list of CVEs to someone who is building in the prioritization; business prioritization, risk prioritization, as well as now providing the ability to do the patching in an effort to help organizations bridge that gap very quickly. The ransomware service we just launched for a ransom risk reduction is really about prioritizing and focusing on helping you. So today, when we’re asked, “Hey, what have you done to reduce our risk for ransom? There is nothing prescriptive out there other than high level documents.” What we did is focus on bring a much more curated list, something that we can focus on, help reduce the risk. We’re really excited about helping organizations, not only see that list, but actually be able to patch that. As this is a Cyber Security Awareness month, we’re actually giving this service for free for 60 days. Organizations can go and get themselves the assessment and actually also patch. Almost all these vulnerabilities are leveraged by ransom and our patch is available for five plus years. I think that is really what we’re focusing on. Your users can go to [UNSURE OF WORD]. com and they can look at what we’ve done from that ransom risk assessment and prevention perspective. That will really help you over the next few days to quickly assess your environment and get in a better shape to reduce the risk of ransomware.

David Spark

Awesome. Thank you very much. We will provide links on the blog post for this very episode. I want to thank Steve Zalewski, my co-host. I want to thank Sumedh Thakar, our guest. I want to thank our audiences always for your contributions, whether witting or unwitting, we appreciate them all the time. And we appreciate you listening to CISO series episode of Defense in Depth.

Voiceover

We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cybersecurity. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site: CISOSeries.com where you’ll also see plenty of ways to participate, including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at David@Cisoseries.com. Thank you for listening to Defense in Depth.

RELATED ARTICLES

Most Popular