Detection validation: going atomic on false negatives

Over the last several years, threat actors and malware developers have evolved their methodologies, learning from their mistakes and pushing the boundaries of security controls. This leaves analysts, engineers, and others charged with keeping the kingdom safe, in a neverending spiral of updating their defenses, hunting tactics, and detection and prevention methods. As such, many organizations often turn to vendors and open source repositories to help maintain their defenses. While these can be great assets to an organization, there is an often overlooked and unfortunate truth to these addons: false negatives.

Trust but verify

While there is no way to truly eradicate false negatives from an environment, you can dramatically limit them through validation. By ensuring that detections and hunts are validated to find the intended activity, organizations can have more confidence in their defenses and close gaps in detections, where false negatives can creep in.

Depending on your available toolsets, this exercise can be difficult to get started. Fortunately, with the widespread adoption of the MITRE ATT&CK framework, organizations can more easily translate their defensive strategy into associated ATT&CK technique IDs and ultimately implement validation. Starting with defensive strategy, analysts and engineers can first confirm that the controls in place are able to successfully detect the most critical risks and threats to the enterprise.

Validation is where Atomic Red Team thrives

Built by Red Canary and open sourced to the community, Atomic Red Team leverages the MITRE ATT&CK framework (with new sub-techniques!) to organize tests that can validate that a detection or mitigation control mechanism works as intended. This empowers security teams to match tactics, techniques, and procedures (TTP) from their defensive strategy to the ATT&CK framework and execute the corresponding tests from Atomic Red Team, without the need to carry out a full red team engagement. It is worth mentioning at this point: red team engagements have tremendous benefits and cannot be replaced in whole by Atomic Red Team tests.

Atomic Red Team organizes all tests by their associated ATT&CK technique ID, such as T1027.001, and any given technique can include multiple tests, or multiple variations of the same test. By installing the available Invoke-AtomicTest PowerShell module, analysts and engineers can easily execute tests on Windows, Linux and, macOS. You can watch the video below to learn about how we at Cyborg Security have used Atomic Red Team tests to empower our tier-1 analysts.

Why stop at testing? Let’s emulate!

Testing is an excellent start for validating security controls. Atomic Red Team will ensure, at a basic level, that a security control can detect or prevent a specific ATT&CK technique.

This is awesome, but what happens when a specific threat employs a technique, such as T1059.001 (Command and Scripting Interpreter: PowerShell) and mixes in multiple layers of T1027 (Obfuscation of files or information)? Will your security controls still detect or prevent the threat? The short answer is: maybe. Malware operators continuously update and change their methods of executing PowerShell commands from programs and with varying obfuscation methods. Enter emulation.

Cyborg Security’s analysts realized early on that Atomic Red Team could be leveraged to carry out complex attack scenarios and chain together new and existing tests to emulate specific threats—a process we call cyber threat emulation (CTE). One such example is a chained set of tests that emulate the Emotet malware. This Atomic Red Team CTE performs a sort of “testception,” where multiple tests are created under their standard technique IDs (such as T1566.001, T1059.001, or T1027) in order to emulate a phishing email that contains a malicious document that downloads and executes a binary.

Furthermore, the tests allow for substituted input variables, such as different email text or document text. Cyborg Security analysts created this CTE as an initial working proof of concept to integrate threat-specific tests into the existing Atomic Red Team framework, enabling more realistic emulation, greater community contribution, and increased ease of use.

Keep your analysts happy

Cyborg Security set out to make an analyst’s job as easy as possible, which in turn helps secure organizations from ever-evolving threats. We worked tirelessly, not only to provide contextualized threat hunt and detection packages with unprecedented contextualization, but to satisfy another common complaint from analysts: how do I verify that the detected attack is actually an attack?

This is a very common struggle, especially with the ongoing skills shortage. The issue is compounded when responding to sophisticated or novel attacks. Many analysts spend hours a day reading documentation, references, or doing independent threat research in order to validate an attack. As a result, every analyst approaches threat analysis in their own way. One of the hidden benefits of approaching Atomic Red Team tests from an emulation standpoint is that analysts are able to observe real-world, threat-specific attack scenarios, on demand, and from the perspective of their own tools.

“The perspective of your own tools” is an important distinction, as it positively affects the analyst’s ability to respond to the threat in the future. Industry-leading organizations and specialists provide trainings for this; however, that training is often carried out using a toolset under ideal conditions, and as any analyst can tell you, ideal conditions are seldom the norm. CTE is not only beneficial for analysts to understand what the threat looks like in their own environment but can also be used as a training tool to perform fun and engaging detection exercises, leading to more rigorous and repeatable investigations.

One of the hidden benefits of approaching Atomic Red Team tests from an emulation standpoint is that analysts are able to observe real-world, threat-specific attack scenarios, on demand, and from the perspective of their own tools.

Where do we go from here?

CTE is difficult and time consuming, requiring hours of analysis and research for every individual threat. Cyborg Security has partnered with Atomic Red Team to bring threat-specific emulations to the community with the ease of running a single command. Cyborg Security’s Emotet CTE is freely available to the community via our GitHub page (feedback is always welcomed and encouraged!). This GitHub repository is a forked version of Red Canary’s Atomic Red Team repository and is currently separate, but a merge is in the works. By following the simple instructions on the main page, you can download Cyborg Security’s Atomic Red Team CTE alongside Red Canary’s to start testing and developing new tests.