Skip Navigation
Get a Demo
 
 
 
 
 
 
 
 
 
Resources Blog Product updates

Incorporating AI agents into SOC workflows

With the right guardrails, AI agents quantifiably improve speed without compromising accuracy

Jimmy Astle

AI agents are transforming how security operations centers (SOC) operate by tackling one of the most critical challenges: context gathering. Context gathering is a fundamental part of detection and response; most of the time it is also tedious and repetitive, making it well-suited for handling by agents. Since we’ve implemented agents in our standard workflows, Red Canary analysts are stopping more threats faster for our customers, all while spending less time on monotonous tasks and more time on the fun, intellectually stimulating parts of detection and response.

This blog builds on an earlier blog focused on AI-enhanced investigations. I will explore why AI agents hold so much potential for SOCs, how they automate busy work, and how they have become a viable alternative to traditional workflow automation.

AI agents 101

To illustrate the security potential of agents, consider the process of analyzing whether an employee login is suspicious. Analysts must gather a variety of contextual information to determine whether that login is benign or malicious, e.g., associated IP addresses, recent risky logins by the user, VPN and ISP usage patterns, device characteristics, login location patterns, and other relevant data. Gathering this information is crucial for informed decision making, but does the act of gathering that information require creative problem solving? In most cases, no. Because it doesn’t require critical or analytical thinking—the core competencies of our analysts—we want to move context gathering off their plates. AI agents give us an opportunity to offload that activity to software.

Login assessments are one of our most productive use cases for using AI agents, but before we dive into the details, it’s worth defining a few things up front…

What is an AI agent?

An AI agent is an intelligent system that dynamically interacts with data, other systems, and users to execute tasks based on real-time inputs and contextual understanding.

Unlike traditional automation, which relies on rigid, predefined workflows, AI agents use state-of-the-art models, like large language models (LLM), to adapt their behavior depending on tasking.

How are agentic workflows different from traditional automated workflows?

What sets AI agents apart from traditional workflow automation is their ability to dynamically adapt to new data and investigation contexts. Where traditional automations follow static, predefined rules, AI agents analyze and synthesize information across diverse sources, ensuring a more complete, nuanced understanding of potential threatening activities.

As an example, let’s say you want to determine whether a login attempt from a foreign country is suspicious. To get traditional automation to do this, you’d need to hard-code specific rules, such as:

  1. Define a list of trusted countries
  2. Flag logins from untrusted locations
  3. Check login time against typical user activity

 

The problem? These rules are too rigid. If the user is legitimately traveling or using a VPN, the automation can’t easily adjust without additional hard-coded logic, making it prone to false positives or negatives.

With AI agents, all you’d have to do is instruct the agent to assess whether the login is suspicious, providing relevant data needed to make such assessments, like IP addresses, geolocation data, recent user activity and baselining, and device information. The agent can then dynamically interpret this data, considering patterns and context across multiple sources such as whether the login matches a known travel pattern for the user or if it’s consistent with past behavior. No complex rule sets required; the agent’s inherent adaptability ensures it can handle edge cases more effectively.

This adaptability introduces efficiencies but also elevates the quality of outcomes in ways traditional static workflows simply can’t. By allowing AI agents to manage context gathering and initial assessments, analysts can focus on the complex decision-making that truly requires human expertise.

What types of AI agents are out there, and what are their pros and cons?

A common misconception is that all AI agents are autonomous, and thus using AI agents means introducing a ton of risk into your workflows. Neither is the case. At a high-level, you can think about two types of agents: autonomous and non-autonomous.

  • Autonomous agents: These agents operate with minimal human oversight, dynamically adapting to context and making decisions independently. While they excel in situations that require creativity, dynamic reasoning, and unpredictable scenarios, their scalability and reliability require rigorous testing and fine tuning due to the inherent randomness of LLMs. These agents tend to be really good at brainstorming new ideas and are most powerful in the hands of expert users who can validate and refine their output.
  • Non-autonomous agents: These agents work within predefined, tightly controlled workflows, requiring human intervention or approval for critical actions. They are ideal for scenarios where precision, reliability, and oversight are critical, reducing the operational risks associated with LLM randomness.

A SWOT analysis of autonomous and non-autonomous AI agents

The sweet spot for the SOC:
Non-autonomous agents with humans in the loop

As we experimented with both types of models, we discovered that while fully autonomous AI agents are incredibly powerful, they come with significant challenges in terms of scalability, testing, and reliability. The randomness inherent in how LLMs operate presents a fundamental question: “How can we depend on an agent that produces highly variable, probabilistic results?”

In short, we can’t. At Red Canary we measure our time to detect, triage, investigate, and respond down to the second and will only accept minor variability to our benchmarks. Additionally, across almost 70,000 threats detected last year, customers only flagged 291 as having issues–i.e., we delivered a 99.6% threat accuracy rate to our customers. Adding fully autonomous agents to workflows that are already highly performant threatens the reliability of our service, and thus they’re not a fit for production environments.

Applying non-autonomous agents to our workflows

Our approach for integrating agents started with first principles: identifying the key questions security analysts ask when investigating events and determining where AI agents could add value. This “SOP-first” approach—grounding automation in standard operating procedures (SOP)—is similar to how you would approach traditional workflow automation. This excellent blog outlines the SOP-first approach for non-security workflows. The execution is where AI agents truly set themselves apart though. Unlike traditional automation, which relies on rigid, predefined steps, AI agents leverage adaptable tools for data retrieval, enrichment, and analysis to dynamically interpret context and take action. This flexibility allows agents to handle the all too common unstructured inputs from security tooling, evolve alongside our SOC’s processes, and unlock efficiencies that static security event automation simply cannot achieve.

Let’s walk through an example

Consider this real-world threat where our anomaly detection technology flagged a login to Salesforce from a new internet service provider (ISP) that had never been used by this user before. This triggered Red Canary’s agentic flow investigations, which automatically kicked off an investigation based on pre-defined SOPs and decision-making processes.

Alert for login from an unusual ISP

In a traditional workflow, an analyst would have had to manually gather and correlate various pieces of information to assess the nature of the login, such as:

Was the ISP rare or unusual for this user and organization?

This requires checking the ISP against both internal baselines (user’s historical ISPs) and external data (known risky ISPs), which can take 5-10 minutes depending on the available tools and data sources.

Was a VPN involved in the login attempt?

Verifying VPN usage involves cross-referencing the IP address with known VPN IP ranges, typically taking 3-5 minutes if the VPN data is readily available.

Was the geolocation consistent with the user’s historical activity?

The analyst would need to compare the login’s geolocation with past logins, checking for patterns or anomalies. This step can take 5-8 minutes, especially when working with unstructured data from multiple sources.

Did the user’s device or operating system change compared to previous logins?

Checking for consistency in device type and OS often involves retrieving device history from logs and matching it with the current session data, taking about 5-7 minutes.

Were there any recent changes in MFA settings or other security configurations?

An analyst would review the user’s recent activity, including MFA enrollment or disable actions, which can take 5-10 minutes depending on the complexity of the environment.

Screenshot of suspicious logon alert from Salesforce.com

In total, performing all these steps manually can take 25-40 minutes per investigation, especially when dealing with unstructured data from multiple sources like identity platforms, SIEM logs, and external threat intelligence feeds. This doesn’t account for delays caused by fragmented data or the need to switch between different tools, which can further increase the time required.

How our flow investigations streamlined this investigation

In real time, this anomaly triggered an automated agentic flow investigation, which:

  • identified the ISP as rare for both the user and the entire organization
  • checked for active VPN usage for this user and the entire organization
  • validated that the login was successful
  • assessed whether the login location was consistent with previous logins
  • correlated device and user agent consistency

 

Comment from Red Canary analyst providing context for the alert

The AI agent’s analysis concluded that the login attempt was potentially threatening due to the combination of a rare ISP and active VPN usage, two features that significantly increase the risk level. The agent flagged the event as suspicious activity, automatically added context to explain its reasoning, and published an event for further review by a Red Canary analyst.

Instead of requiring an analyst to manually gather and correlate all this information, the AI agent performs these tasks automatically and presents its findings in an easy-to-consume concise format. The Red Canary analyst then reviews and validates the agent’s findings and takes appropriate action by publishing the threat to our customer as a true positive.

The entire process was completed in a little over 3 minutes. This represents a significant reduction in time compared to the 25-40 minutes it would have taken using the manual workflows.

Agentic workflows deliver efficiency and reliability

This real-world example illustrates how agentic workflows can significantly improve both efficiency and accuracy in security investigations. By automating time consuming, repetitive tasks like data gathering, enrichment, and correlation, our agentic flow investigations reduced the investigation time from 25-40 minutes to just over 3 minutes, without sacrificing the reliability that Red Canary customers expect.

In addition to reducing investigation time, agentic workflows ensure consistency across every investigation. Unlike manual processes that can vary between analysts based on experience, workload, and fatigue, AI agents adhere to the SOPs ensuring that every investigation meets the same high standards of accuracy and completeness. This consistency not only speeds up operations but also enhances the overall trust in the Red Canary process as customers receive a reliable service regardless of the complexity or volume of alerts.

Humans as guardrails

While agentic workflows deliver significant speed and consistency benefits, they are far from a plug-and-play solution. Without proper guardrails, they can introduce variability and operational risks. Think of these agents as promising new hires, equipped with the right foundational knowledge but inexperienced in the nuances of real-world SOC operations. Without guidance, even the best new hire will make mistakes. This is where SOPs come in. SOPs act as a training manual, providing clear, detailed instructions to keep agents focused on executing tasks consistently, without straying into unnecessary or incorrect actions.

We’ve applied this methodology by breaking down our SOC workflows into specific components, enabling AI agents to operate effectively across various critical domains, including:

  • Cloud platforms: AWS, Azure, GCP
  • Identity management systems: Okta, Entra, Google Workforce
  • Security information and event management (SIEM) systems: Microsoft Sentinel

 

These data sources power the agents’ toolkits, which retrieve data from our data lake, perform levels of data cleaning and enrichment, and conduct feature extraction and engineering that feed final agent decision support.

When implementing LLMs, you are inherently limited by the amount of information you can pass to the model at any given time, as well as the short-term and long-term memory available to agents. To scale effectively, Red Canary has focused on caching and limited agent memory tailored to our investigation workflows. Since our investigations center on security event analysis, we rely on compensating controls in our XDR platform to handle higher-level correlation and association.

Integrating AI agents into these focus areas, we have achieved faster triage times without compromising accuracy or scope.

In workflows where Red Canary Copilot flow investigations are enabled, we have observed a 60 percent reduction in mean time to notify customers of a threat, with no degradation in alert accuracy or quality across our supported identity and cloud integrations.

Striking the right balance of automation and human input

The success of AI agents at Red Canary stems from our ability to integrate them into workflows already shaped by years of human expertise. In every process, analysts are not just operators but contributors to the learning process, labeling data and offering real-time feedback on decisions. This iterative feedback loop ensures that agents don’t just automate tasks but improve over time while maintaining the high standards our customers expect.

By strategically taming agents through human-in-the-loop feedback, robust testing, and well-defined workflows, we’ve achieved a balance that leverages the power of AI while mitigating the risks of variability. This approach enhances our operational speed but also ensures that the decisions made by AI agents are accurate, reliable, and aligned with the needs of the SOC.

Getting started with AI agents in your SOC

Implementing agentic workflows might sound complex, but with the right tools and frameworks, you can quickly begin experimenting and driving efficiency in your workflows.

Here are a few frameworks and tools, listed in order of increasing complexity to set up and onboard:

  • OpenAI Function calling: While not a full agentic framework, this is the fundamental building block for AI agents. It’s an ideal starting point for understanding how to use AI models to interact with tools and automate tasks
  • CrewAI: A developer-friendly framework that simplifies the design and orchestration of AI agents. CrewAI makes it easy to create and deploy agentic workflows tailored to your SOC’s needs
  • LangFlow: An open-source, visual framework that enables you to build, test, and customize AI agent workflows. LangFlow provides an intuitive way to bring LLM-powered automations into your SOC
  • LangGraph: A framework designed to create multi-step workflows with LLMs, giving you fine-grained control over task orchestration and agent collaboration
  • AutoGen: Microsoft’s open-source framework for developing multi-agent systems that can collaborate to solve tasks efficiently. AutoGen is ideal for dynamic, complex workflows requiring coordination across multiple agents

 

For organizations already invested in the Microsoft security stack, turnkey solutions like Microsoft Security Copilot can accelerate the adoption of agentic workflows. Security Copilot provides pre-built workflows, integrates seamlessly with Microsoft tools, and simplifies agent deployment. Security Copilot includes a Red Canary plugin that retrieves Red Canary security event information, enabling you to automate workflows and trigger custom Azure Logic Apps. These apps can power Copilot-driven agentic workflows tailored to the unique needs of your SOC.

To learn more, check out this blog for a deep dive into our flow investigations, and this primer on laying the foundation for GenAI in your SOC.

 

The unusual suspects: Effectively identifying threats via unusual behaviors

 

Red Canary: At the heart of your security operations

 

Safeguard your identities with Red Canary + CrowdStrike Falcon® Identity Protection

 

Red Canary coverage of Google Cloud Platform enters general availability

Subscribe to our blog

 
 
Back to Top