Cybersecurity budgets are increasing, by a lot. What’s fueling the increase and where are those budgets being spent?
Check out this post for the discussions that are the basis of our conversation on this week’s episode co-hosted by me, David Spark (@dspark), the producer of CISO Series, and Geoff Belknap (@geoffbelknap), CISO, LinkedIn. We welcome our sponsored guest Nick Kakolowski, senior director of research at IANS Research.
Got feedback? Join the conversation on LinkedIn.
Huge thanks to our sponsor IANS Research
[David Spark] Cyber security budgets are increasing by a lot. What’s fueling the increase, and where are those budgets being spent?
[Voiceover] You’re listening to Defense in Depth.
[David Spark] Welcome to Defense in Depth. My name is David Spark. I am the producer of the CISO Series, where Defense in Depth lives. And here joining me live at this very moment is none other than Geoff Belknap, who many of you also know as the CISO of LinkedIn. Geoff, say hello to our audience.
[Geoff Belknap] Hey, David, and hey, audience. How are you? Oh, that’s great to hear. You’re looking good.
[David Spark] Our sponsor for today’s episode, who’s also responsible for bringing our fabulous guest, is IANS Research. They’re also responsible for today’s topic, or it was inspired by some research they did. We were quite fascinated, and we were like, “Let’s do a whole show on this.” So, they did a study on cyber security budgets, which have grown dramatically. I’m going to do a quick caveat. It just happens the time we’re recording this, we’re going through an economic downturn which may not be reflective of the time the study was created. But it’s probably inevitable that this budget increase will bounce back. So, we’re going to not focus on this very short moment in time of our economic downturn, and we’re going to sort of focus on the results of this research study and what we knew was going on before we were dealing with economic issues that are a more global problem. Or, well, more of a problem I know definitely to the US. Maybe not as global as I’m saying. So, I’m going to ask you, Geoff, what do you believe is fueling these increases in cyber security budgets? Because they’ve seen, and our guest will into it in great detail here… He’s got some fascinating information on it. But early information I heard is it’s going up by like 100% and more. What do you think is fueling these increases, and do we understand where the money is being spent? Is it being spent on new hires, more tooling? What do you think?
[Geoff Belknap] Yes, all of those things it’s being spent on. I don’t think my budget has gone up 100%, but I think in general we’re seeing a lot of budgets go up. I would like to attribute that to two things – one, that people understand security is a business differentiator. Security done well is part of maximizing upside for your business, not just minimizing downside. It’s not just insurance. It’s a way to compete more aggressively in your sector. And two, I think people are realizing if we just bring this to the macro-economic climate that exists at the moment that we’re recording this, if you were going to go through a tough time, and you’re a business that’s going shields up and ensuring your business succeeds in the long-term, you realize that security is going to be a big part of that. if you have a sudden issue that you didn’t plan for and you’re not prepared for from a security, privacy, or trust perspective, you may not survive a time that is not economically easy to operate in. If you are prepared for that and you’re invested in that, your business is going to be more durable and more resilient long-term.
[David Spark] Money is a key factor. But as we will see, not necessarily the grand solution to all problems. But the person who’s actually going to come with a lot of knowledge on this subject and some actually hard research on this very topic is our sponsored guest today, who is the senior director of research at IANS, Nick Kakolowski. Nick, thank you so much for joining us today.
[Nick Kakolowski] Thanks, David. Great to be here.
Why does this still happen?
[David Spark] Chan Vuppalamarthi of LinkedIn said, “Thanks to COVID,” he says, “seismic transformation of digitization of majority of customer needs has led to said/unsaid, quote/unquote, disclosed/undisclosed significant uptick in security events. From my learning, pretty much major organization and board have realized the important in our focusing on cyber safety and increasing their budgets.” And this is kind of pretty much what you said. The awareness is now everywhere. I would also say the comment I’ve made many times before, the big cyber news stories which used to break in the trades now break in the mainstream news first.
[Geoff Belknap] Yeah, certainly there is no hiding when you have this kind of issue anymore. This has really been the last few years where security is unquestionably mainstream. Nobody is wondering if security is something that investors care about, if boards should be thinking about. We’re here. It is a real thing we have to worry about. I think… I hate to thank COVID for anything, but I do think as people looked at how to operate their business and how to… Again, I’m going to use the word resilient. How to think about being resilient in a time of crises or a time of pandemic, you start to sort of process…it might have started how do you take care of your employees, and how do you make sur that you can keep operating your business. But the next thing you’re going to think about is what are all the other black swan sort of long-term events that you might have to worry about. You come to realize that security adds a lot of value here for your business, and now it’s time to make sure that you’re doing the right things.
[David Spark] I’m going to throw this to you, Nick. There was a time many said, “Well, the only way we’re going to increase budget is if we get hacked.” That is no longer the necessary factor here. What did you discover in your research?
[Nick Kakolowski] Yeah, anecdotally agree completely with everything Geoff said and would just add based on your question, David, only three percent of respondents… We had about 500 respondents on this. Told us that their budgets decreased. It’s a full 79% that said their budget increased. And even if it was just from general year to year financial changes, that budget increase was 12%. The average budget on the whole is at 17%. If they changed their budget because a competitor was hit by a breach or a similar major industry disruption, that increase was 20%. They ended up increase staff spend by 36%. If they repositioned in the market… There was MNA [Phonetic 00:06:21] in the business or they released a product that expanded their digital footprint, that budget increase is 27% on average and 53% for staff. Then if they’re hit by a breach themselves, we’re looking at like 30% budget increase on average and 64% increase on staff spend.
So, we’re just seeing there are so many external forces that are pushing businesses to recognize the importance of security that it’s pushing them to spend more. But we also see this small segment of companies, about six percent of our respondents, said they had a budget increase of 100% or more. That’s what we see. The CISO gets hired with one or two people on their staff. We actually had about ten percent of our respondents only have one or two staff members. Suddenly that org gets hit by a breach, that budget is going to skyrocket because that business has basically been neglecting security and staying behind the market. And now they’re going to play catch up and correct all at once.
[Geoff Belknap] I wonder, Nick, how much of this do you break out by size of the organization? How much of the people that are starting from scratch are small or medium sized enterprises versus large enterprises that are doubling down on what they’re doing?
[Nick Kakolowski] So, we have some of that in our data, but it’s hard to speak directly because the thing about security budgets is you don’t really know what’s just security spend versus what’s IT spend that security is responsible for and reports as part of their budget versus what is being used to support a business unit. What we see in some of those larger organizations is the spend is huge, but it’s all over the business. When you get into the smaller orgs, the security spend is more isolated to security, and it’s just smaller relative to the overall IT spend.
[David Spark] One quick question – how long have you done this budget study?
[Nick Kakolowski] This is our third year.
[David Spark] I’ve got to assume none of those years… And I know that’s only a few years. But historically from all the studies I’ve seen, cyber security budget has never gone down. It’s just continuously gone up. Geoff, from your anecdotal experience, yes?
[Geoff Belknap] I think everyone would like it to go down. We’d all like to imagine that it’d be cheaper to operate a security program over time. But I agree with you. I doubt it’s ever really going down relative to…
[David Spark] Or even really maintained. What Nick has been saying, the overwhelming majority is increasing massively.
What’s going on?
[David Spark] Kyle Thomas of WEX said, “Changes in regulatory requirements coupled with a shift to the Cloud has incrementally increased security budgets for tooling. Market changes related to a vendor shift to OPEX, operating expenditures services, is also driving an increase in security budgets. The interesting juxtaposition is that most organizations are struggling with the shift from the depreciation of CAPEX, capital expenditures, assets to OPEX services and are actively working to reduce the reoccurring expenses in their budgets through tool and/or vendor consolidation and procurement practices.” This is interesting. I don’t look at the expenses sheets, so I don’t know what is actually costing more. But I definitely feel that there is a significant shift from CAPEX to OPEX. Nick, did you have any insight on this?
[Nick Kakolowski] Yes. So, we look at our budgets. On the whole, you’re looking at about 90% of the budget on off prem software, 10% on outsourcing, so 29% on stuff that’s going to generally be characterized as OPEX, and 9% on on prem software, 60% on hardware. So, roughly 16% on traditional CAPEX spending.
[David Spark] By the way, audience who’s listening to this, I don’t think you’re going to need to write all this down. Is there going to be a way at the end of this people can actually see some of these results?
[Nick Kakolowski] Oh, yeah, we’ll have public versions of the report going out probably sometime in October.
[David Spark] All right. Well, this will come out after that. So, not to worry, audience. You will get a link to this. You don’t have to write all these numbers down. Geoff, I got to assume you’ve seen a significant shift from CAPEX to OPEX, yes?
[Geoff Belknap] Oh, yeah. Over the last 15 years, I’d say we’ve had a steady but rapidly increasing shift from CAPEX to OPEX. Just as Nick points out. I think Kyle is hitting to this as well, the move to the Cloud really shifts. You’re not investing in CAPEX. Which I think is fantastic because the problem we used to have when I started in security leadership was you had to buy a firewall or a really fancy advanced security device like a Mandiant box or something like that, or FireEye box. What you would find out is that your depreciation of that asset would be over five years, but the useful life of it would only be maybe a year or two years. So, you’re going back to your CFO asking for CAPEX budget again for something that in their mind should have lasted you five years.
But security and technology change so rapidly. So, now that things are moving to the Cloud, what you’re seeing is a lot of times my opinions… I’m curious if Nick has any data to support this. Is that your OPEX spend on the Cloud is often times more impactful and affective than your on prem spend for security tooling. And two, you get updates to that software for free usually, or you get regular updates to that where you’re not investing in the Cloud, and then 18 months later whatever you spend on the Cloud is no longer useful. You get free hardware refreshed the way you would never expect to get that on prem. So, I think in a lot of ways it’s very beneficial for us to move some of this spend OPEX.
[David Spark] Let me just also add to that, Geoff. I agree to go to OPEX, but then the other thing is getting rid of stuff, unloading it. There’s a major cost to that, and time, and people that once you go to OPEX you don’t have to deal with that at all.
[Geoff Belknap] Yeah, you do not have it dispose of assets when you’re not spending on assets. You’re spending on service. Now, there’s a lot of debate about whether that’s good spend or whether the actual cost is better relative to managing yourself on prem. I’m a fully converted believer. I believe the Cloud environment is always going to be a more impactful way to deploy your dollars for security.
[David Spark] So, any insight in sort of the long-term recognition of the value of OPEX, Nick?
[Nick Kakolowski] I would just say anecdotally… We can’t really [Inaudible 00:12:33] our data. But what we hear from our community is really that there’s a growing fatigue with tool sprawl. It’s not so much how you’re spending the money. It’s more like are you throwing a tool at everything because suddenly you don’t have to spend a big capital expense to get it. And so you’re just getting the technology and realizing that suddenly you don’t need another report with logs that you don’t have time or people to process. You need human resources and ways to actually make sense of that data and use it to build your program. And so the key to manage the tools in such a way that you’re not relying on them for things that they just really can’t deliver.
[David Spark] Jerich Beason, CISO over at Capital One, said, “The business is assuming more functions traditional IT orgs would have centralized, and they are developing more applications than ever before.” Yes. “As IT services become further decentralized, security teams get thinner, trying to support them. A big portion of the budget is labor, and unfortunately the AI ML pipedream,” that’s the artificial intelligence machine learning pipedream, “of compensating for headcount hasn’t materialized yet.” Yeah, that’s never going to happen I don’t think in any real way. I should also throw out Sean Cassidy of Asana said, “A lot of our new budget is going to training existing folks. We doubled our training budget last year, and we’re going to double it again for next year. We’re seeing a huge ROI on training.” What about you, Geoff? Because I got to assume you’ve got to have a lot of it if you’re taking on a lot of OPEX. There’s a lot of education on all these platforms, yes?
[Geoff Belknap] I think there is a lot of education. It depends on what your hiring strategy is. So, for me I have not as heavily invested in training existing skilled engineers so much as moving into building up engineers from scratch. What I mean by that is we have a couple of different programs at LinkedIn that I’m very excited about. One is called Reach, and one we manage called You’re Up [Phonetic 00:14:38] through a partnership there from the organization called You’re Up where in the Reach program maybe you’ve already got some technical skills, and you have a very entry level security skillset. We’re going to hire you on, and you’re going to become an apprentice. And you’re going to learn by doing. You’re going to be partnered very closely with some set of mentors.
There’s some formal engagement there to sort of get you up to be a full performing security engineer. And so we’ll invest a fair amount of training there. And if you’re a You’re Up apprentice, you’ll come in, and you’ll be given basic tech skills to be given an understanding of how to survive in a modern day office, and how to use email, and Slack, and things like that. But all of those things for us have been leaning into like, “Let’s bring more people into security.” I find that we already have a lot of training resources available to us, but a lot of my peers, like Sean and Jerich, are absolutely in the right direction of upscaling people and investing in their skills and talent, and making them more affective. I find that to be really impactful in a bunch of different ways.
[David Spark] So, Nick, I don’t know how much insight you had on where they were putting the spending and more importantly also was that spending affective. Because I think it was more future thinking. But what do you know from your research and also anything anecdotally?
[Nick Kakolowski] Yeah, staff is by far the largest expense in budget. So, it’s 39% this year. It was 30% last year. So, it’s really not moving. It’s absolutely critical. Anecdotally when we talk to our community, CISOs regularly tell us, “We need more people. We need to find ways to retain our people.” That’s really where they talk about training. They’re spending about four percent of their budget on training. It was the exact same number last year. It sounds small, but if they’re spending 60% on hardware and 40% on training, that tells you a lot. We also heard there’s a really nice advantage to training in that you can only spend so much compared to what you’re spending on say a big software solution. The relative chuck of your budget that you get value from is huge there because affective training on the margins is so much less expensive than a big tool or a huge hardware investment, and you relatively speaking get value out of that by retaining staff or getting staff from another department into security. The cost of hiring…the headache of hiring is so high that the overarching value of spending on training and building up your staff is just so great relative to other parts of your budget.
[David Spark] That is an excellent point. Because if you think about the cost of hiring… Geoff, the pain of trying to get new staff in is so high. If you realize it could have cost us just a little bit more to retain someone had we trained them… You would go in that direction. And the benefit they’re happy, you’re happier because they know more. They’re able to do more. It’s kind of a win/win all the way around, yes?
[Geoff Belknap] It absolutely is. I think everyone is figuring this out. That you no longer can spend nine months looking for the perfect purple squirrel that fits all the…ticks all the boxes and fits all the needs that you have from a talent perspective or from a skills perspective. What you can do is find someone who ticks 40% of the boxes or 60% of the boxes and send them to a class or invest in them, partner them with a mentor. I can’t underscore enough what Nick is saying here. The cost of sending somebody to some training is not a million dollars. We’re talking tens of thousands of dollars. And in the grand scheme of things, that is a drop in the bucket compared to what I am spending to retain and recruit every one of those people.
Can it be solved?
[David Spark] Jonathan Waldrop of Insight Global brings up a very cogent point, that money does not solve everything. Let me read his quote. “I don’t equate the success of a security program with the size of the budget, either as a percentage of IT or an overall budget. There are plenty of ways to throw money at a problem and not solve anything. That said, if you’re in the Cloud and you’re not investing in Cloud security platforms, you’ve got a gap to fill.” So, I’ll start with you, Geoff, on this. Nick also kind of mentioned this. Like buying a tool is kind of a simple way to just solve this stop gap measure. But you can… It’s not hard to spend money ineffectively. It’s pretty darn easy.
[Geoff Belknap] It is very easy. But just want to go back to the premise here. Security, like happiness… Money does not buy you security, but it sure helps. It makes it much easier to get to secure. That being said, you can absolutely throw money down a drain and get zero value for it if you’re not being thoughtful about how you’re deploying that resource. I think I’d love to see over time how this was reflected in Nick’s survey. But I think what you’ll find is if you throw money away enough times, your leadership will figure out that that is what’s going on, and they will find a different leader that knows how to deploy that cash a little bit more intelligently.
[David Spark] Nick, do you have any insight? You’ve only done it for a few years now. But any insight into how affective spending has been?
[Nick Kakolowski] Yeah, we can’t speak directly to spending being affective through our research studies because it’s such a subjective thing. But we can say… We were just talking about this yesterday. That security as a percentage of IT budget, it’s a great benchmark because it just gives you something to work off of. We see ten percent in our study repeatedly as the norm. We just had a session with a board member yesterday at one of our events talk about how they look at it as security spend being three percent relative to bottom line revenue. They asked, “How do you get to that number?” And he basically said, “We did it for a while and realized it’s what worked for our business.” That’s really the bottom line here is as a security leader it’s about going in and figuring out how does our solution solve a specific business pain point, how does it help a business unit deal with a strategy to get out to the market, and how does it reduce risk that has a real chance of happening. And then can we go through and justify across our spend that it’s affecting the organization’s core mission and core goals? Once you’re doing that, you’re going to have cohesion with business leaders as your allies when you go to the CFO looking for money, and your budget just becomes an exercise in figuring out what your priorities really are as an organization, not just as a security team.
[Geoff Belknap] I want to point something out real quick here that I think is important. Because I’m going to tease out a phrase Nick used here, which is core mission, core goals. It has been forever that we have likened security spend to a percentage of IT spend, which I think we really need to stop. I think the way your board member or whoever you talk to, Nick, thought about this, which is a portion of revenue or a portion of total spend or even if you had to break it into a portion of R and D, we have to stop thinking about security as a part of your IT spend. It’s not. It’s a part of how you operate your business. It’s not just a fancy part of IT. It is a critical part of how your business is going to succeed in the long-term. And refactoring your perspective to see it that way is going to ensure your long-term success.
[David Spark] Excellent point, and let’s close it right on that. that was awesome. Nick, thank you very much. This is now the point of the show where we ask which quote was your favorite. You don’t need to read it all, but you can kind of sum up which quote you like the best. Which was your favorite quote, and why?
[Nick Kakolowski] I liked Jerich’s quote. The business is assuming more functions that traditional IT orgs would have centralized.” Security really is becoming embedded in the organization, and the most successful teams are the ones that work hand in hand with business leaders to share ownership of risk. Once you’re doing that, you’re going to have an easier time getting the budge you need because the rest of the business is taking responsibility alongside you.
[David Spark] Geoff, your favorite quote, and why?
[Geoff Belknap] I like everything Nick said. I don’t know if that’s…
[David Spark] [Laughs] I agree.
[Geoff Belknap] I think I’m breaking the rules here, so I’m going to go with Sean Cassidy.
[David Spark] But let me pause just for a second. Audience, I want you to know that Nick had all of this in his head. He wasn’t reading anything. He knows all these numbers in his head. They’re all just locked in there.
[Geoff Belknap] I’m fairly sure Nick is a Mentat. If you’re not sure what that is, go watch “Dune.”
[David Spark] Nick brought the knowledge today.
[Geoff Belknap] He really did.
[David Spark] Hardcore. We appreciate it. I don’t think anyone has slammed this much knowledge in such a short period of time on one of our shows before. But hold on, we’re going to get back to that. Geoff, your favorite quote, and why.
[Geoff Belknap] Yeah. Well, we’re really going into depth here on Defense in Depth this time. My favorite one is from Sean Cassidy from Asana. Hey, Sean. “A lot of our new budget is going to training existing folks. We doubled our training budget last year, and we’re going to double it again for next year. Seeing a huge ROI on training.” I love this because it really brings the focus to… And I think Nick was talking about this earlier. It’s not just tooling. We don’t just want money to invest in tooling, or computers, or UB Keys [Phonetic 00:23:28], or whatever else we want to do. We want it for investing in people. We want to uplift the people that make the hard decisions, that do the hard work every day, that work harder than our computers in most cases. And more of those people seeing a growth path in front of them is great for our security program.
[David Spark] I will also quote Jesse Whaley, who I have quoted many times before. He’s the CISO over at Amtrak per the mention of the purple squirrel. He refers to the unicorn. We’ve heard those two terms used interchangeably. He said, “Don’t go look for a unicorn. Grow your own.” And you do it through training as well. All right. This brings us to the end of the show. Nick, I’m just going to echo what I said moments ago. Man, did you bring the knowledge today for this episode. Hold tight. I’ll let you have the very last word. I want to thank your company, IANS Research, for sponsoring today’s episode and being a phenomenal new sponsor of the CISO Series. I’m going to want information as to if you’re hiring and where we can find this report. Actually we’ll probably link to it on this post. But anything else about IANS. But first, Geoff, any last words?
[Geoff Belknap] Yeah, David, the most important thing I think to keep in mind is budgets are always going to go up. One of the things we didn’t get to fully talk about today. But hey, find me on LinkedIn. I’ll post something about it later. Which is we have to make that budget more impactful, more affective spend. I think we’re doing it, but I think that is the next thing to look at is not just the raw amount of the budget but how much impact that budget is having.
[David Spark] That’s a great topic for a future episode. Nick, final thoughts. Just give us the low down on what we can expect from IANS next and what people should know.
[Nick Kakolowski] Thanks, David. Thanks, Geoff. It was great to be here. We are working hard at publishing studies around CISO compensation, security budgets, CISO satisfaction with the role and the staffing salaries for folks who report directly to CISOs. So, it’s like your heads of GRC, heads of architecture and engineering, those kind of parts of your team. We have those coming out throughout the end of the year. Free versions for the public, deeper dive versions for IANS clients, and deeper dives versions for anybody who takes the survey, which is still open and available on the IANS website as we are constantly adding to that [Inaudible 00:25:32]
[David Spark] That is a great tease. Free if you participate. I like that.
[Geoff Belknap] Get out there and join the survey.
[David Spark] Join the survey. Excellent. Well, thank you very, very much, Nick. That was Nick Kakolowski. He is with IANS Research, the senior director of research over there. And thank to our audience as always. We greatly appreciate your contributions and for listening to Defense in Depth.
[Voiceover] We’ve reached the end of Defense in Depth. Make sure to subscribe so you don’t miss yet another hot topic in cyber security. This show thrives on your contributions. Please write a review, leave a comment on LinkedIn or on our site, CISOseries.com, where you’ll also see plenty of ways to participate including recording a question or a comment for the show. If you’re interested in sponsoring the podcast, contact David Spark directly at david@CISO.com. Thank you for listening to Defense in Depth.