Research in Action: How to Test Your Defenses With Atomic Red Team

In the weeks since we launched the Atomic Red Team testing framework, we’ve been blown away (no pun intended) by the security community’s response. Yesterday we had a hands-on training session, and it was even more exciting to hear directly from teams that are beginning to use the framework to improve their detections. We had so many great questions from attendees, we didn’t have a chance to get to them all. Below are the questions and answers.

If you missed the session, we encourage you to check out the on-demand version. It lays the groundwork for how to use the framework and includes three hands-on labs: regsvr32, chain reactions, and measuring progress.

Training Session Q & A

Is there a big difference with trying to detect this activity via Sysmon vs an EDR solution?

The Atomic Red Team tests are intended to be product neutral. While we used Sysmon in our example, we want these tests to cover multiple products and don’t want to endorse or promote one product over another. Check out this GitHub if you would like to learn more about using Sysmon to collect events.

The focus of the project is to test your full security stack. If you’re evaluating a product and want to determine what it can detect or prevent, that is where Atomic Red Team comes in. Want to download payloads and see if a sandbox solution detonates it properly? Check out some of our PowerShell commands. Want to confirm your authentication logging is working? Check out our password spray example. The best part is, you can chain these together to simulate a lot of activity at once.

Another attendee asked if Sysmon could be considered “EDR Lite.” It certainly can be. The Sysmon DFIR GitHub has additional resources on getting it tuned properly for specific event collection. You can see how others are using Sysmon and determine if it fits your organization’s needs.

Are there any additional events in the Security Event logs to correlate with the Sysmon events?

Yes, Event 4688 is referenced here. It’s actually very helpful for detecting new processes spawned, so thanks to the analyst who asked that question.

Are you releasing the .xlsx you used to generate detections and events to test?

Roberto Rodriguez created the spreadsheet we used, and we want to once again give him mad props for creating such a helpful resource. It’s a great place to start tracking detections over time. You can find the sheet here. Also, be sure to check out Roberto’s blog post that explains how to use the heatmap.

How does the heatmap allow you to differentiate between: a) threat blocked; b) threat detected; or c) threat information available and gathered for hunting?

The heatmap focuses on Threat Hunting, which is a great starting point. Over time, we will look to add additional items to measure, such as whether a technique was blocked or detected, and which product within your security stack caught it. This is important for every organization to measure and see product effectiveness over time. Maybe a current product can be tuned to where it will prevent a specific technique earlier on versus post-execution detection on the endpoint. We will definitely be working to provide these types of metrics as we progress, so stay tuned.

Couple questions on the Regsvr32 Lab. What is the difference between unregister & not register? Is scrobj.dll the only type of dll you can run with Regsvr32?

Regsvr32.exe is a command-line tool that registers .dll files as command components in the registry. So, if you call register, or don’t use the /u, you will end up “tattooing” the registry. The method that I use when I call /u is to give you a path of execution, without the implications of leaving artifacts in the registry. Also, you may not have the required permissions to actually register the object. So, in a nutshell, /u, in my opinion, the most attractive switch for an attacker to use.

As for the different types of dll’s you can run, Regsvr32 can actually call any DLL that exports the right functions. A great example of this can be seen in this blog post from Verint’s Cyber Research team. Executing a COM scriptlet is just one of the capabilities. Other examples are documented on the MITRE page for Regsvr32.

So the EVTX is SysMon.evtx?

Upon installation of Sysmon, you can view the event logs within Event Viewer. Follow the path: Applications and Services Logs > Microsoft > Windows > Sysmon > Operational. You can find some additional information and see how to setup Sysmon with the repository we mentioned.

Are you looking to use some of the techniques to show the value of disrupting chains through deception techniques and/or log detection capabilities?

One area of research I am particularly interested in is: “How can I trust the telemetry from a compromised host?”  Your question is along those lines. Are there ways for attacker to disrupt, or degrade our telemetry collection? Yes. What are these? We need to explore this further. There are different ways to collect telemetry. For example, are you using EventLogs or a filter driver? How is telemetry packed/shipped? These are all questions we would like to explore, but they currently fall outside of the charter and goals of our Atomic Red Team Tests.

Is this chaining technique effective over the long term?

We think this is the true value of our testing framework. We want to provide you the basic atomic building blocks. It will be up to you, like MineCraft, to use them in creative ways. What we want to provide is a catalog of actual execution scripts that teams can use. Is this the best and most effective way to test? Are there better ways? Probably. Our aim is to provide some tests that would work for both small and large teams.

Thanks again to everyone who attended the training session. Atomic Red Team is designed to be a collaborative effort, so please continue to provide feedback and any questions you may have. Email us at research@redcanary.com. We look forward to making security better together.

Ready to start testing your defenses? Watch the on-demand Atomic Red Team Training Session.