Generative artificial intelligence (GenAI) powered by large language models (LLM) is among the most emergent technologies in the information security industry at the moment. The agenda for next month’s RSA Conference offers a crude but useful bellwether for this: a simple search for “AI” on the conference agenda page yielded 127 results, meaning there will be more than 100 AI-themed talks at this year’s conference, including (shameless plug incoming) a talk I plan to give on leveraging LLMs to synthesize intelligence reporting to develop tailored defense strategies.
We’ve written extensively about the promise and perils of GenAI and the benefits it may bring to the field of security operations (and specifically how we’re using GenAI at Red Canary). However, “security operations” is something of an umbrella term that encompasses numerous distinct functions and specialities. In this way, GenAI offers different benefits—or in some cases no benefits at all—to different parts of a SOC. The question we’ll try to answer in this article is this: How does AI benefit the various functions you might expect to find in a modern SOC?
There’s no uniform or universally accepted way to organize a security operations center (SOC), but we’ve broken down the modern SOC here. The following image summarizes the architecture of a modern SOC, but we’re going to focus on just the detection and response subsection for this article.
Note: In all of the following examples, we believe the benefits of GenAI tools can only be realized in partnership with—and under the supervision of—human experts. We are not advocating for a fantastical, AI-powered, autonomous SOC.
Gartner® SOC Model Guide
Learn moreThreat intelligence
Reduced to the most basic level, a threat intelligence function serves to inform decision making about which threats to prioritize. Intelligence teams develop threat models for organizations and study the broader threat landscape to figure out the threats, adversaries techniques, and risks that an organization ought to be concerned about. Since reading and writing are core competencies—and core responsibilities—for intelligence analysts, GenAI tools can help teams synthesize or summarize information from disparate sources and then assist with the presentation of that information.
GenAI for synthesizing and summarizing
There seems to be a ceaseless torrent of threats, vulnerabilities, exploits, and incidents that coincide with corresponding reports, mitigation guidance, proofs of concept, tool releases, and other noise. I’m confident there are more minutes of security content on the internet than there are minutes in the average human lifespan.
It’s all way too much…for a human. GenAI tools, however, are pretty good at summarizing long reporting into manageable briefs or, alternatively, synthesizing information from multiple reports into a single aggregation. And, importantly, they do it really fast. On the less advanced end of the spectrum, an analyst can simply ask their preferred GenAI tool to cherry pick the salient bits from a newly published malware analysis report. On the more advanced end, a threat intelligence team could programmatically and continually load RSS or private intel feeds into a bespoke GenAI tool, using it to highlight publicly available intel reporting that is relevant to their organization—or even ask questions of the information stored within it.
In either case, take a trust-but-verify approach to the output of your AI tools. If some detail of an AI-generated summary weighs heavily into a single assessment, double check the source material.
GenAI for writing intelligence reports
AI-generated writing is often formulaic and uninspiring. This is bad if you’re trying to write the great American novel but matters far less for an intelligence report that’s optimized for ease-of-ingestion (by humans and/or machines) and economy of language. It’s true that GenAI-written first drafts can leave a lot to be desired, but so do first drafts in general. However, if you give your favorite AI chatbot a template—or better yet if you ask it to create a template for you (something they’re quite good at)—it can churn out decent first drafts in a predictable, easy-to-consume format.
The extremely important caveat to consider is that you shouldn’t simply ask a GenAI tool to go and write an intelligence report. You should feed the GenAI tool extensive notes, including observations and defensible assessments, and then ask it to translate those notes into a formatted intelligence report. From there, it’s simply a matter of revising early drafts and turning them into polished and presentable final products. If you like writing or have an aptitude for it, then the GenAI tool may be more useful for structuring your reports or proofreading them.
It’s true that GenAI-written first drafts can leave a lot to be desired, but so do first drafts in general.
Further, intelligence reports cater to different kinds of audiences. Tactical intelligence is detailed and operational so that it informs practitioners, like a detection engineer or an incident handler, who need to know how to develop detection abilities for or respond to a threat, respectively. However, this kind of report is considerably less useful to a CISO, CIO, or CEO who hasn’t the time or need to pour through code snippets or YARA rules and probably doesn’t care about the subtle differences in tradecraft between the delivery affiliate and the ransomware-as-a-service operator. Here, again, you can take your own longform intel report, feed it into the GenAI tool, and ask it to draft a shorter version of the report that’s designed to speak to an executive audience.
Prompting will be critical here, as a GenAI tool won’t implicitly know how to speak to an executive audience. You need to give it detailed guidance so that it will focus less on pedantic technical details and more the overall risk a threat poses to an organization and the general level of coverage the organization has against the threat. An AI-generated executive summary should be considered an early draft and iterated upon accordingly.
GenAI for managing intelligence
As time goes on, your own library of intelligence reports may become unwieldy. A properly trained AI can serve as a research assistant to help identify pertinent information hidden within unmanageable stacks of threat profiles, request-for-information responses, and bulletins. Old reporting can become stale or even inaccurate over time, and while it’s difficult for any intel team to manage, GenAI tools have the potential to retroactively scour old reports, compare them to data contained in new ones, and surface contradictory information that may require your team’s attention. Additionally, a GenAI tool could serve as a tutor for intelligence analysts, creating lesson plans and exercises to help them become more knowledgeable about the contents of your intelligence library.
Threat research
A threat research function is similar to threat intelligence but simultaneously operates at a deeper, more technical level. Threat researchers ensure that intelligence priorities are sufficiently contextualized, in part by understanding the inherent coverage limitations for a threat given the tooling available to the security operations team. They look not just at what is prevalent or worrisome in the present, but extend their viewpoint into the future, striving to understand the implications of evolving adversary techniques, the variations within those techniques, and the ways that adversaries may attempt to evade security controls.
The benefits of GenAI tooling are probably most elusive for a threat research function, since it’s probably the least reactive of the core SOC functions. Every other function is reacting to a tangible thing happening in the wild, whether it’s a new threat, the challenge of developing a way to detect it, investigating the alert it generated, or responding to the incident caused by it. By contrast, threat research is more concerned with the unknown:
- How might the adversary evolve?
- What can’t we detect?
- What are the relevant data sources for a technique or threat and what does the data tell us in a human-readable fashion?
Threat researchers who are able to effectively answer these questions will be well positioned to train GenAI models, but human curiosity, patience, and expertise are more important traits for answering these questions than GenAI’s brute-force ability to crunch numbers and analyze data. However, GenAI tools may prove useful as a mechanism for analyzing software.
GenAI for software analysis
Understanding the inner workings of software—benign or malicious—is a core component of threat research, and AI could be helpful to researchers attempting to ascertain how an existing piece of software works, whether it’s something as big as the Windows operating system or as low-level as a section of assembly code from a decompiled binary.
Threat researchers dive deep into software for a variety of reasons that range from seeking to understand how a popular piece of software might enable adversaries to analyzing how an information stealer evades security controls and just about everything in between. GenAI tools have the potential to serve as an assistant here, clarifying the purpose of certain functions or explaining what a bit of code does. In fact, two such plugins for Ida and Ghidra already exist.
It’s unreasonable to expect complete programming fluency from threat researchers within or across languages, but GenAI tools can help bridge that gap by demystifying obscure coding techniques or translating unfamiliar code into understandable or analogous language.
Detection engineering
Your detection engineering function exists to develop means for detecting threats. Intelligence is a critical input to help detection engineers prioritize the threats they should be focused on detecting. Threat research helps them fortify and future-proof their detection analytics. Investigation and incident response teams offer important checks and feedback on the efficacy of their analytics and the breadth or shortcomings of their coverage.
Perhaps the most obvious (and alluring) GenAI use case for detection engineering is to prompt GenAI tools to come up with new detector ideas, which can then be flushed out by a human detection engineer. Additionally, we believe that GenAI tools may also be beneficial for helping manage a detector library and for describing the purpose of detectors.
GenAI for detector development
While we haven’t tested this at scale, we’ve found some limited success in asking GenAI tools to create detection logic. If you give the GenAI tool example detection analytics (so they can replicate the structure and purpose of a detector), you can then ask it to create detectors using natural language. The output is by no means production-ready, but it’s a great brainstorming tool for detection engineers as they begin the process of attempting to create detector logic for a new threat or technique. Explain what you want to detect in plain language, and then use the output of the GenAI tool as a starting point for iterating toward a better detector. We’ve had some success doing this for atomic tests as well.
GenAI for detector management
Like how a library of intelligence reports might become cumbersome, so too might a library of detection analytics. Over time, depending on how many detectors there are in production, security teams may run into significant overlap and redundancy in their detector library. This can lead to increased storage and processing costs, not to mention the lost time developing detection analytics that already exist. GenAI tools are great at pattern recognition, so training one against your detector library, asking it to look for detection analytics that are fundamentally similar to one another, and then having a human actually compare them can help reduce redundancy and cost. Depending on the structure of the detection analytics you write, you may be able to use an AI tool to glean interesting metadata bout your detection analytics too, such as:
- What data sources do we leverage most frequently in detectors?
- What are the most common commands we look for in our detectors?
- Which process shows up in most of our detectors?
GenAI for describing detectors
Similar to the software analysis use case in the threat research section above, a GenAI tool could be pretty effective at looking at a detection analytic and describing its purpose, especially if the tool is trained on your organization’s specific detector schema. The applications for this are numerous:
- You could allow a GenAI tool to take a first craft at describing new detectors in your documentation—or reviewing existing detectors and corresponding documentation.
- As is the case with intelligence profiles, you could create a sort of GenAI tutor for training staff on the contents of your detector library.
- You could potentially use a GenAI tool to ask questions of your library, such as, “Do we have detector coverage for obfuscated PowerShell?”
Investigation
The investigation team triages and examines potential threats uncovered by detection analytics or alerts to confirm their veracity or determine if they are false positives. When they find a threat (or when they’re triaging it for that matter), they use context from the intelligence team and elsewhere to inform incident handlers about what happened and how they can best respond. GenAI might provide the greatest potential benefits to an investigative team, since they are very much in the business of taking disparate information and data and attempting to tell a coherent story with it. Benefits might include helping with data analysis, alert correlation, enrichment, and—you guessed it—descriptions.
GenAI for analysis, correlation, and enrichment
If a security team is performing detection at scale, or across a great number of tools, then there’s a decent possibility that relevant signals for a threat may emanate from multiple tools. For example, you could have a suspicious login alert from an identity provider, a potential threat surfaced by a custom detector that looks for suspicious email forwarding rules, and any variety of alerts from a phishing protection or email security service. They may all be describing a small slice of a single broader incident, but connecting those dots may be prohibitively difficult when you’re inundated with an overwhelming number of incomprehensible alerts.
A well-designed AI tool can analyze the data from these various sources in real time and suggest correlated events and alerts for a human analyst to review. Further, it could attempt to help your investigative team enrich detection or alert timelines by drawing additional information from your intelligence or detector libraries that could then be reviewed by the analysts performing the investigation and writing up the detection or alert timeline. All of this could help your investigative team during the triage process as well, simply by providing them more information about the severity and potential consequences of a threat sooner.
GenAI for descriptions
As we’ve brought up numerous times, a GenAI tool will be relatively proficient at looking at the contents of a threat timeline or sequence of alerts and taking a first crack at explaining what might have happened. As always, a human will need to review the output, but learning on an AI assistant of sorts can expedite the process and may turn up important details a human may have overlooked.
As an example, I provided our GenAI tool, Red Canary Copilot, with a confirmed threat detection, and asked it to describe the threat to me. Here’s what it returned:
The current threat is identified as [REDACTED] and is classified as a high-severity Malicious Software (Dropper/Downloader). The affected endpoint is a workstation, and the threat has been detected by EDR vendors [REDACTED] and [REDACTED]. The threat is consistent with the Intelligence Profiles 'fin7' and 'netsupport_manager_rat'.
The timeline of events includes several indicators:
1. A child process (client32.exe) was spawned from powershell.exe without any malicious indicators detected.
2. A file named [REDACTED].msix was created by Google Chrome, which is identified as malicious based on open-source threat intelligence.
3. A PowerShell process executed with a command line that is not known as malicious.
4. The client32.exe binary, associated with NetSupport, was executed, which is a tool that adversaries often use for remote access.
5. An outbound network connection to a malicious IP address (X.X.XX.XXX) was made by client32.exe.
6. A process related to the [REDACTED] application executed without any malicious indicators detected.
7. A PowerShell script ([REDACTED].ps1) contained within the [REDACTED].msix installer package was executed, which is considered malicious.
8. The same PowerShell script was executed directly from the command line.
9. The content of the malicious PowerShell script was loaded, with a snippet of the script provided.
The threat occurred on a host with the hostname [REDACTED] and the username [REDACTED] or [REDACTED]. The threat is notable for the use of a malicious file download, the execution of a known remote access tool, and the execution of a malicious PowerShell script.
Not a perfect description, but a good start that could be refined and added to by an experienced analyst.
Incident handling and response
The incident handling or response function exists to understand the scope of alerts, detections, or incidents, further validate them, understand their severity, and ultimately contain or eradicate them. GenAI benefits for an incident handling team could include typical use cases like drafting incident reports but also unique ones like coming up with focused security recommendations based on an organization’s unique infrastructure.
GenAI for drafting incident reports
We won’t belabor the point since it’s already mentioned in numerous sections here, but, given access alerts, detection timelines, intelligence profiles, coverage assessments, and other information, a GenAI tool could likely take a competent first pass at writing the first draft of an incident report. Have it create a template, ask it to write the report, then your incident handling team can revise and rewrite until it’s in a good place.
GenAI for security recommendations
A GenAI tool can also easily draft recommended response or mitigation actions, either to help expedite the response process or to help mitigate the impact of future incidents. We wrote extensively about how this can work in our blog on improving security operations with GenAI.
What AI can’t do
AI tools are not effective stand-ins for human judgment and decision making. Think of GenAI tools as assistants, not replacements. Some areas of security operations where we think AI won’t provide much benefit include:
- Human management
- Work prioritization
- Customer advisory (whoever your customer is)
- Risk and impact analysis
- Tool selection
- Identification of emerging threats or potential threat vectors
- Remediation
One major concern for our collective embrace of AI tools is that we may become too comfortable with them and too reliant on them, eventually becoming careless. This is precisely why it’s extremely important to always review and verify the output of any GenAI tool that you’re using for any important task.
Think of GenAI tools as assistants, not replacements.
Conclusion
We are not delusional AI utopianists. We understand that GenAI tools require constant supervision and that their output should be distrusted by default. We don’t believe that GenAI tools are going to drive humans out of the SOC. Much to the contrary, we’re hopeful that AI will drive the drudgery out of our humans. When we assess the ways that we might implement an AI model, use cases that diminish boring, repetitive tasks are every bit as important to us as use cases in which an AI model would thrive. And we encourage others to view AI that way as well.
As we wrote earlier this year, “Qualitatively, GenAI agents help our experts both get to the fun part—making a decision informed by security-relevant data—faster, and spend a higher percentage of their day doing the fun part.”
Related Articles
How Red Canary works to create an accessible coding environment for all