AI-powered threats

AI-powered threats represent an evolution in tooling, not a revolution in attack techniques. Adversaries are leveraging AI in two major ways that align with how the industry overall is attempting to integrate AI:

1. Force multiplier: Adversaries are using AI in existing development workflows for planning, creating, and distributing malware and facilitating attacks.
2. Automation: Adversaries are attempting to automate workflows to leverage AI for attack techniques.

Command-line interface (CLI) tools, Model Context Protocol (MCP) servers, and large language models (LLMs) in general are attractive to adversaries because they offer the same advantages they provide to legitimate users: automation, flexibility, and broad access to systems and data.

Research from Google’s Threat Intelligence Group (GTIG) analyzing government-backed threat actor use of Gemini found that Iranian, Chinese, North Korean, and Russian APT groups are using AI to support reconnaissance, vulnerability research, payload development, and post-compromise activities. In September 2025, Anthropic detected and disrupted what they assessed as the first largely AI-orchestrated cyber espionage campaign, where a Chinese state-sponsored group used Claude Code to execute approximately 80-90 percent of tactical operations autonomously, with human operators serving primarily in strategic supervisory roles.

Beyond leveraging AI for coding, adversaries are also heavily relying on AI to execute fraud, not only through business email compromises and spear phishing but also to mimic individuals on phone or video conversations. For consumers, deepfake technologies are becoming rapidly harder to spot. Deepfakes may be used to directly fraud financial officers in companies to deliver fake invoices to adversaries or even to trick IT administrators into giving adversaries access to the environment.

AI-powered threats don’t require revolutionary new security approaches. The same principles that protect against “traditional” tradecraft also work with AI—least privilege, comprehensive monitoring, and defense in depth. Defending against threats that use AI, in other words, isn’t hopeless. It’s about getting the fundamentals right. However, the brief history of information security has proven that getting the fundamentals right is expensive and complicated.

AI tradecraft in 2025

Throughout 2025, adversaries integrated AI into their operational workflows, using tools like Gemini, ChatGPT, and Claude to augment capabilities across the full attack lifecycle. This was made evident in a report from Google’s Threat Intelligence Group (GTIG), which analyzed prompts from adversaries who attempted to use Gemini, revealing consistent patterns of AI adoption for productivity gains rather than developing entirely novel capabilities. Below is a quick summary of activity covered in the GTIG report.

Anthropic detected and disrupted what they assessed as the first AI-orchestrated cyber espionage campaign at scale. A Chinese state-sponsored group, designated GTG-1002, developed an autonomous attack framework using Claude Code and MCP tools to conduct operations without direct human involvement in execution. The framework broke down complex multi-stage attacks into discrete technical tasks—vulnerability scanning, credential validation, data extraction, lateral movement—that Claude executed based on carefully crafted prompts from human operators.

These developments demonstrate how AI provides adversaries with speed, scale, and automation rather than fundamentally new capabilities. For skilled actors, AI tools offer a helpful framework, similar to how Metasploit or Cobalt Strike streamlines operations. For less skilled actors, AI provides a learning and productivity tool enabling faster development and incorporation of existing techniques, effectively lowering the barrier of entry for adversaries to conduct different types of attacks.

What does this mean for defenders?

From a detection standpoint, there will be minimal changes to how threats present themselves. Adversaries will continue to use the same techniques—AI simply lowers the barrier of entry for adversaries and allows them to operate faster.

To that point, a defender’s ability to differentiate AI-powered threats from threats that don’t leverage AI is limited. Red Canary has seen phishing campaigns that seem to be luring victims into LLMs, and we’ve almost certainly detected numerous threats that leveraged AI at some point in their development. JustAskJacky, the second most prevalent threat Red Canary detected in 2025, is a functioning AI chatbot that answers users’ questions but executes encoded commands in the background.

Further, we’ve conducted proof-of-concept research positing various ways that adversaries might leverage AI in the future, including by abusing “agent mode” features to trick users into granting credential and account access to malicious AI agents. While we anticipate this will become more of a problem as users become increasingly conditioned to granting account access to AI tools, we don’t think this is fundamentally different from traditional phishing.

Ultimately, detecting these threats is business as usual, and we don’t see that changing any time soon.

While the use of AI within your organization is not inherently malicious, it does introduce new risks as users become more comfortable delegating their access and responsibilities to AI agents. As innovative Agentic AI tools emerge, they increasingly rely on users’ permissions to perform tasks, making these tools targets for exploitation. Solutions like OpenAI’s Atlas browser and ChatGPT’s “agent mode” exemplify how autonomous AI agents can introduce new, unmanaged vectors for prompt injection and data exfiltration. As adoption of these technologies grows, organizations must proactively assess and secure the ways AI agents interact with sensitive data and systems.

Protecting environments from AI-powered threats relies upon the same fundamentals as any existing threat. The only difference is that defenders should be relying on automation, AI or otherwise, in their environments to match the speed at which the adversaries are operating. AI-powered threats simply increase the speed and adaptability of adversaries.

Specific prevention, mitigation, and response techniques for AI-powered threats follow the same paradigms of other, non AI threats. Defense in depth, zero trust and continuous monitoring will always provide the best security from any threats. For a detailed breakdown of defending your own AI infrastructure you can reference our section on Threats to AI Infrastructure.

Traditional detection strategies remain effective for identifying AI tool abuse. The same log sources for Cloud Accounts or endpoint telemetry apply to AI powered threats. However, AI-powered threats put a larger emphasis on the detection and remediation workflows. As adversaries increase their engagement speed, defenders need to keep up. Detection is only so good insofar as it allows for action before impact. This applies to both AI-powered exploitation and deepfake fraud. Robust standard operating procedures (SOPs) are the best defense against any threats, AI or otherwise.

For a more robust discussion of log sources related to AI, refer to our Threats to AI Infrastructure section.