Being part of a security operations center (SOC) means that when everything is running right, no one knows you’re responsible, but when things go wrong, everyone knows who to blame. There’s a constant pressure to stay vigilant. Sometimes these efforts to excel will lead a SOC into overwork – doing too much, buying too many shiny new tools, and churning through people.

We asked our community of experts for their suggestions on how to get everyone on board and committed to a positive and efficient security culture. Here are 25 recommendations from 51 of your colleagues and peers.

Got feedback? Join the conversation on LinkedIn.

Huge thanks to our sponsor, Vectra AI

Automation

1. Begin by reducing unnecessary efforts by the SOC – automate instead

Much of the work we used to do can be done more quickly and consistently through automation. Security experts almost universally agree that any opportunity to automate is a first step to improving the SOC.

So, what can be automated? “Anything that’s routine and carries a high level of confidence in decisions,” said Mark Eggleston, CISO, CSC Global.

This frees people up to do more important tasks. “Look to automate 95 percent of the work to free analysts to examine the five percent that requires real insight and talent,” said Edwin Covert, head of cyber risk engineering, Bowhead Specialty.

Dr. Andrew Aken, CIO and vCISO, DocDrew LLC, agreed: “Automation will dramatically increase the quality of work, and focus can turn to the most serious or unique incidents.”

“It should also improve the analyst’s work-life balance and job satisfaction while reducing OpEx,” said Billy Norwood, CISO, FFF Enterprises.

2. Don’t reinvent the wheel! Choose best automation practices already out there

As much as you may want to automate right away, you’ll need some help developing the process. “Take advantage of economies of scale via a managed security service provider (MSSP) before moving to automation,” suggested Bowhead’s Covert.

Some techniques already speak for themselves through SOCs that have deployed them. SOAR (security orchestration, automation, and response) is one of these. “It can automate routine tasks, enhance alert quality, reduce false positives, implement effective threat intelligence to proactively hunt for threats and contextualize alerts, and streamline incident response processes with playbooks and drills,” said Rama Balla, cloud security architect, Bendigo and Adelaide Bank.

3. Automation improves with better communication

“Don’t view the people in the SOC in isolation. They need to know who to reach out to, based on what they are seeing,” said Dennis Pickett, vp and CISO, Westat, who suggested that you “designate someone on the security team to be the lead for interactions with the network operations team.”

“Create executable runbooks to allow teams to respond to incidents consistently and communicate quickly,” suggested Russ Ayres, svp of cyber and deputy CISO, Equifax.

One suggestion, recommended Bozidar Spirovski, CISO, Blue Dot, is to “use retrieval-augmented generative models. They have value in being able to direct people during an incident to the right document in the corporate documentation. Recipients can be reminded of some component in the system that they need to understand.”

“This might include partnering with vendors to reduce alert fatigue and focus on the things that matter most,” added Joshua Barons, CISO, San Diego Zoo Wildlife Alliance.

Control

4. Manage only what you can measure

“Where you spend your time and resources should be driven entirely by metrics,” said Nicole Ford, svp and CISO, Nordstrom.

This is doubly important when the SOC is outsourced. “It’s a waste when something appears to be effective but is not,” suggested Westat’s Pickett, who notes that “for example, if you are using an outsourced SOC, your staff will have to duplicate some tests to ensure your SOC is staying vigilant and you are getting value for your money.”

5. Kick out the noise for better focus

Having access to so many detection and analysis tools in a SOC means they deliver lots of information, which can be just as bad as not having enough.

Is your SOC doing too much simply by osmosis? Start by turning down the volume of inbound information and take a stand against uncontrolled inbound data.

“Too many security tools generate excessive noise without relevant business context, which leads to a loss of critical signals and hampers a team’s response efficiency,” said Mathew Biby, CISO, Satcom Direct.

Instead, “focus efforts around the most critical data, which requires hyper scalable discovery and highly accurate classification as the foundation,” advised Lamont Orange, CISO, Cyera.

“Every vendor talks about reducing alert noise. If a SOC analyst gets 5,000 alerts per day and the vendor claims to reduce alert noise by 80 percent, the SOC analyst still gets 1,000 alerts per day. Stop talking about reducing alert noise and start talking about the quality of the attack signal we send to the SOC,” said Mark Wojtasiak, vp, product marketing, Vectra AI.

This quality-improving effort could include “changing the charter of the SOC team to look for things that are happening, not for things that might have happened,” said Steve Zalewski, co-host, Defense In Depth.

Here are some specific examples of excessive input and what you can do to reduce the noise:

• “Establish more reasonable standards of communication that keep everyone informed,” advised Howard Holton, CTO and industry analyst, GigaOm.
• “Deploy an IP block list for places who shouldn’t be talking to you,” said Stephen Cicirelli, CISO at large.
• “Determine what asset logs are ‘nice to have’ versus ‘must haves,’” added Edward Contreras, senior evp/CISO, Frost Bank.
• “Use AI to assist in log review and to minimize the noise from the team,” said Steve Gentry, founder/CEO, Cognate Cyber.

6. Stop thinking you must hoard all the data

“We’re so terrified that every log, event, and alert could be important, that we’re afraid to tune our detection systems more aggressively,” said Adrian Sanabria, host, Enterprise Security Weekly. “To catch an attacker, we only need to see some of what they’re doing – not all of it.”

This well-known concern of “you need to catch it all” will unnecessarily drain financial resources, noted Yabing Wang, vp and CISO, JustWorks, who noted that “the cost for storage or queries on the SIEM (security information and event management) tool is one thing, and it could be double the price when you need to store them in AWS or Datadog for infrastructure and operations purposes.”

Setup

7. Set up your foundation

One of the primary tenets of building a security program is to focus on the fundamentals. If your base security program is in place, and you can see the whole picture, then your SOC has the proper framework to begin operations.

“We’re using a combination of network monitoring, endpoint detection and response, and log analysis to get a complete picture of activity across our systems,” said Equifax’s Ayres.

“We also look for patterns of good behavior to help us baseline business activity at different calendar intervals,” added Frost Bank’s Contreras.

Dealing with wastefulness

8. You can’t always solve the problem by hiring more talent

It’s not always necessary to grow your team to maximize effectiveness.

“Start by effectively managing the resources you already have,” suggested Sivan Tehila, founder and CEO, Onyxia.

They’ll appreciate the challenge and the opportunity for career growth.

“Create a workplace that values the contribution of the professional. Talent is one of the most undervalued commodities in this discussion,” added David Emerson, CTO, SolCyber.

But it is also important to make sure you have the right people on the bus. “Too many organizations approach SOC talent as entry-level,” said John McClure, CISO, Sinclair Broadcast Group. “Sometimes the ‘right’ talent might need to have more experience.”

9. Zoom in on your biggest wastes of time and money: excessive tooling

Much of the problem comes from “shiny new solution” syndrome: the belief that something newer will fix your issues.

“People tend to chase technology because it is often easier than doing the hard work of tuning existing tooling to perfection,” stated Joshua Brown, vp and global CISO, H&R Block.

“Look for redundancies in the products and services you purchase,” said Jim Bowie, CISO, Tampa General Hospital, “such as when your company buys a product that does something similar to one already purchased.”

Another time waster is “ensuring proper integration of apps and cloud services,” offered Dr. Diane M Janosek, CEO, Janos LLC, “especially with the plethora of security overlays.”

10. Stop searching for the silver bullet solution

Vendor marketing would like us to believe that there is one single product or service that will take care of all that ails a SOC. Stop searching.

“It’s a massive waste,” said Matt Hand, director, security research, Prelude Security. “The security product ecosystem is complex for good reason. Find products that solve an explicit need rather than the one generalist that is mediocre.”

Cyera’s Orange blamed some of this silver bullet mindset on poor process decisions and poor execution. “Companies buy multiple point products as tactical solutions to poor security processes. This is where we begin to spin out of control,” he said.

Don’t get caught buying more than you need.

“The cybersecurity market is rife with incentives well beyond the bounds of practical application,” said SolCyber’s Emerson. “Organizations that need a lock on the front door end up with an air defense battery on the rooftop. Focus on modest defenses that can be performed comprehensively and without lapse.”

11. Limit how vendors use your time

It’s not just the tools – it’s also the people who sell them.

“Strive to reduce the number of tool vendors, and insist stubbornly any new tool must replace at least two others,” said Dutch Schwartz, vp of cloud services. SideChannel.

Stay away from vendors or their reps who won’t provide the benefit your SOC needs.

“Watch out for services like data loss prevention or cloud applications security brokers that are expensive and which generally cost more than the corresponding return,” added Michael Weiss, former CISO, Human Interest.

Insist on connecting with the right people. “Time is wasted at vendor meetings where a salesperson is sent when we specifically wanted a technician,” said Tampa General Hospital’s Bowie.

12. Watch out for THWADI (that’s how we’ve always done it) syndrome

Does your team prefer manual work because “that’s how we’ve always done it?”

“Many SOC teams today still rely on manual intervention to monitor, triage, or respond to threats,” said Tyler Martin, head of enterprise security, FanDuel.

Manual intervention as a first stage of defense might not be the best way anymore, especially with the growing capacities of AI.

“AI helps us search through current Internet data and delivers customized responses when new threats or industry incidents arise. This serves as an alternative to manually navigating through discussion groups, technical news websites, and newsfeeds,” said Cassio Goldschmidt, CISO, ServiceTitan.

13. Keep tabs on the paperwork

Regulators, compliance, and paperwork always cost time and money.

“Monitoring requirements from customers or regulators may be valid,” said Blue Dot’s Spirovsky, “but they may not correspond to how an organization operates.”

“Often the companies asking for paperwork like third party risk management (TPRM) questionnaires don’t care about the answers so much as being able to check the box that they asked,” quipped Weiss. “In many cases, the questions are already answered on the company’s security portal, but those asking don’t actually take the time to look.”

Damage control

14. Take burnout seriously

So many of the items listed above – lack of metrics, paperwork, and timewasters – can lead to team burnout, warned Nordstrom’s Ford. “Analysts should not spend time on tasks below their skill level due to poor role definitions or lack of specialized tools.”

“Excessive, unnecessary labor can quickly lead to burnout,” said Nick Espinosa, chief security fanatic, Security Fanatics.

SideChannel’s Schwartz noted that, “alert fatigue is tough to solve. Time and budget for training must be factored in and fanatically protected to train your SOC staff and cross-train others.”

If you don’t manage burnout, you’ll just exacerbate your team’s problems.

The SOC process is “a revolving door of people who burn out or who don’t want to work on-call 24/7. It takes time to find candidates and at least six months to train them. It also demands time of a senior person,” said Blue Dot’s Spirovsky.

15. Find out the truth about false positives

Security experts are unanimous on the frustrating wastefulness of false positives.

“Tracking down false positives and trying to tune rigid rules-based systems to fit the dynamic realities of blue teaming is a huge expense,” said Adam Koblentz, field CTO, Reveal Security.

False positives are “largely a manual exercise taking away from higher-level activities,” said Kush Sharma, director, municipal modernization and partnerships, Municipal Information Systems Association, Ontario.

As such, “they should be subject to a deep analysis,” said Cyrus Tibbs, CISO, PennyMac, as they are “ripe opportunities to identify preventive controls that should be implemented.”

“Tuning for false positives is like weeding a garden: it needs to be done consistently,” added Davi Ottenheimer, vp, digital trust and ethics, Inrupt. “If you wait too long to understand your environment’s nuances, you might already be dealing with an undetected breach.”

“Detection tuning is an often overlooked art,” said Omer Singer, vp strategy, Anvilogic, “but it can free up countless analyst hours.”

“Neglecting this step will almost certainly cause problems later,” said George Strassburger, CISO, SG Computers.

“Ultimately, false positives can kill,” warned GigaOm’s Holton. “Don’t allow project deployment to lose steam at 80 percent. That last 20 percent is about tuning the system to eliminate false positives.”

Recovery

16. Let the business dictate how you’ll improve your signal

Start by asking, “What does the business need?”

It’s one thing to know where the wasted time and money are going, but what does that have to do with the business’ goals? Start by reassessing your knowledge of your business and your threats.

“If you don’t know your business you can’t understand your signal,” said Hadas Cassorla, fractional CISO and CISO coach, Scale Security Group. “Think about what the bad actors want and how they might get it.”

“Define your business objectives,” said Tomer Gershoni, former CISO, ZoomInfo. “This allows determination of SOC characteristics such as the most critical systems you need to monitor, minimum viable level of audit logs, and the number of analysts needed.”

17. Fix your internal problems with your own tools

No need to shop around. You might already have the right tools on board.

Nordstrom’s Ford uses AI-powered cloud security tools to identify “toxic combinations of vulnerabilities, misconfigurations, or software flaws, which helps prioritize the fixes.”

But remember, even if you fix everything, not every tool can do what you want it to do. “You can’t make the wrong tool become the right tool,” added Sinclair Broadcast Group’s McClure.

18. Partner up with your vendors. They should be there to help you optimize their tools in your environment

When you do turn to your vendors for help, be specific about your needs, and about your current setup. Make sure they are receptive to these comments and willing to work with you (not just sell).

“Work with your SIEM provider and implementation partner to assess if your current deployment model is optimal or if another model would better suit your needs,” said Frost Bank’s Contreras.

“It’s also important to ensure that vendors improve the tools to the SOC’s needs based on feedback,” said Security Fanatics’ Espinosa. “If a vendor isn’t open to feedback, why are you working with them?”

Data intelligence

19. Swim with the current. Take advantage of standards and public data

We spend so much of our time looking inwards, it’s easy to forget there’s a world of valid data out there. Sometimes you just need to ask.

CISOs like to talk to VCs because they’ve got their eyes and ears out looking to invest in hot new solutions.

“Get tactics, techniques, and process (TTP) signal from public sources to help find signal associated with wider-scale attacks,” added Weiss.

“Look at innovative startups in the space to address pain points,” suggested Andrew Wilder, CISO, Community Veterinary Partners.

And there’s even more that’s just sitting there waiting to be read and digested.

“Get tactics, techniques, and process (TTP) signal from public sources to help find signal associated with wider-scale attacks,” added Weiss.

FanDuel’s Martin also notes that while “geolocation and impossible travel alerts are extremely noisy, they’re often good signals that should not be suppressed.”

20. Take a fresh and critical look at how you’re approaching the data

How can the data you collect, from public sources as well as from within your systems, truly be more useful to you as an analyst? Robb Reck chief trust and security officer, Pax8, advised team leaders to “change the SOC mindset from (responsive) to engineering (proactive).”

“Concentrate on identity (user/service) and address (vectors/vulnerabilities) that could have the most impact on your business,” said Satcom Direct’s Biby. “Not all vectors/vulnerabilities warrant immediate action.”

If the data you’re collecting isn’t providing any results, move away from it.

“We are focusing on data-driven questionnaires and moving away from generic ones,” said Tampa General Hospital’s Bowie.

Cyera’s Orange delivered the idea of informed data detection and response. “When an average person walks into an empty room, so what? But when that average person, who has not logged in for 102 days, walks into a room filled with sensitive data, and has MFA turned off? Now I care! Data security is characterized by rich levels of context.”

When it comes to close-focus data analysis, “you certainly want to automate,” said Westat’s Pickett. “It takes a person looking at logs and talking to users to determine if an activity alert was something benign or serious.”

Don’t limit yourself to software tools. Hardware implementations can greatly reduce the noise, suggested Weiss: “Minimize the attack surface by limiting access to only company-issued devices and focus on hardware authentication with something like a YubiKey.”

21. Integrate your threat intelligence with your SOC

Avoid the classic conundrum of being able to collect and interpret intelligence data, but then not having the facility to distribute it to the people who need to know.

“Integrate a threat intelligence platform that correlates data from multiple sources, providing contextualized, actionable insights, and minimizing manual threat analysis time,” suggested Stephen Kowski, field CTO, Slashnext.

Seek out better technology and techniques for collecting, sorting, and distributing the data.

“Use a logging standard that requires specific fields, log types, etc. based on function and system, and maintain close relationships between the threat intel team and the SOC/blue team. This helps ensure existing and emerging threats are recognized,” suggested H&R Block’s Brown.

“Every SOC should be running the standards of Next Generation SIEMs and UEBA (user and entity behavior analytics) tied into threat intelligence platforms, including automating playbooks in their SOAR platforms,” said Security Fanatics’ Espinosa.

Efficiency

22. Question your alert latency: “Can you hear me now?”

Alert latency in a SOC is a killer. The longer an alert must wait, the greater the potential for damage. The best solutions are a combination of the right people and the right automation.

“AI can also help with the latency between when an incident occurs and when corrective actions are taken,” said DocDrew’s Aken.

“Eliminate human middle layers from the process,” said ServiceTitan’s Goldschmidt. “For suspicious activity in an account, replace the manual investigation and reach out to the account owner for confirmation with a clear, human-readable explanation of the event sent automatically out-of-band to the asset owner.”

Blue Dot’s Spirovsky suggests a triage approach: “Blend three types of latency with three techniques: incident to signal requires automation; signal to investigation is a human factor element; investigation to action is an analysis paralysis moment that needs fast decisions, not a committee.”

23. Build AI into your efficiency process

AI is everywhere, but that doesn’t mean it’s all useful. Although, it’s possible to experiment with raw AI tools to devise an in-house tool, “don’t try to build your own AI solutions,” advised Sinclair Broadcast Group’s McClure.

Instead, let the vendors do the heavy lifting. “At our scale, we see the most benefit from successful AI enablement within our vendors’ tools. They are bringing the benefits of AI into our existing solutions,” said Quincy Castro, CISO, Redis.

Always keep AI in perspective – it has good and bad points. “AI will be used as a helper application for security operations, whether standardizing on a single AI engine or using the AI capabilities of many tools,” said, Jon Oltsik, former analyst of Enterprise Strategy Group..

“We utilize machine learning and generative AI for predictive threat analysis and automated incident response. This allows for rapid identification of zero-day threats and sophisticated phishing attempts, enabling our SOC team to focus on strategic tasks rather than routine threat analysis,” said Slashnext’s Kowski.

Goldschmidt also said he “uses AI to pose natural language questions about the data it has gathered about our environment.”

“We can solve the problem by dialing up the level of automation in SOC, particularly Tier-1 alert investigations, which is by far the most manual-intensive part of SOC,” added Edward Wu, CEO and founder, Dropzone AI.

“You can leverage AI to reduce or remove the need for Level 1 and Level 2 analysts,” suggested Defense in Depth’s Zalewski. “AI is a reasonable alternative to either reduce staff or move them into Level 3 roles, where value and job satisfaction are higher.”

24. Beware of letting AI run amok

Because of the newness and ever-changing behavior of AI, the potential for unanticipated problems remains high. Questionable practices, along with the uncontrolled expansion of AI means there’s a powerful need for guardrails.

“We are seeing a shameful trend of AI companies abusing the data they are provided ‘to improve services,’ and it conflicts massively with privacy laws and confidentiality. AI has a lot of potential, but security is always about essential hygiene. AI won’t make us secure,” said Blue Dot’s Spirovsky.

Inrupt’s Ottenheimer provides a dose of caution: “Any use of AI could exacerbate things by further reducing transparency controls and accountability for vendor-related breaches.”

“Maintain strict privacy standards regarding SOC data and avoid vendors with questionable practices,” Ottenheimer added.

With all these concerns about losing control when you let AI run loose, “make sure you have an AI governance policy in place before using AI,” advised Aamir Niazi, executive director/CISO, SMBC Security.

25. SOC efficiency is still mostly a human pursuit

Not everyone feels the need to improve their human-centered SOC. They are happy with what they’re doing because they’ve gone deep into the human element of IT.

“I am currently not exploring any AI-based SOC analysts, said Martin Choluj, vp security, ClickHouse. “My team focuses on creating high-quality alerts, with context, which are manually triaged by my team. We use a mix of data security lake, ETL (extract, transform, load), detection and alerting tools, plus SOAR. No AI magic, just old school security engineering practices focused on low noise, high-value alerts based on data.”

Their confidence is based on the team and the culture they’ve supported.

“We are at the high end of maturity, so we don’t struggle with inefficiencies, so much as always looking to improve. There are tools that can replace Tier 1 analysts, but much of that is already done in SOAR,” added Rick Doten, CISO, Carolina Complete Health.

“We don’t really have issues with unnecessary effort at our SOC. I suppose we attribute this to having sound processes and a highly tuned set of tools that don’t produce much noise. This, combined with a well-trained staff allows us to provide 24×7 SOC support to our 18,000 users,” added Mical Solomon, CISO, Port Authority of New York and New Jersey.

Conclusion

SOC efficiency is both a science and an art. Yes, we need proven technology and the best of new AI tools, but overall, we need to focus on building a strong team of experienced people who can combine practical how-tos with knowledge and insight. As Adam Arellano, former vp, enterprise security, PayPal, suggests, “block out intentional time at every level with every member of the SOC for process improvement. It’s not enough to just be good at putting out fires.” Got feedback? Do you have other pieces of wisdom about SOC efficiency to add? Join the discussion on LinkedIn.