Skip Navigation
Get a Demo
 
 
 
 
 
 
 
 
 
Resources Blog Threat detection

Crude OilRig: Drilling into MITRE’s Managed Service Evaluations

Here’s how Red Canary detected and thwarted simulated OilRig activity in MITRE’s inaugural Managed Services Evaluation.

Brian Donohue
Originally published . Last modified .

MITRE Engenuity conducted its first ever ATT&CK Evaluations for Managed Services this summer. ATT&CK Evals, as they’re commonly called, have been around for a few years, having conducted three enterprise evaluations against various vendors’ endpoint detection and response (EDR) products and one industrial control system (ICS) evaluation testing ICS security vendors.

In past enterprise evaluations, each vendor relied on its own EDR sensor to gather telemetry and generate alerts. The managed services test evaluated 15 managed detection and response (MDR) vendors and managed security service providers (MSSP) on their products’ ability to operate a detection service on either their own proprietary tooling or by relying in part or entirely on third-party tooling.

We were proud to take part in this evaluation alongside some of the best security products in the business. There’s certainly some nuance in the ways that different vendors presented what they observed and detected during the evaluation, but the good news for the whole industry is this: everyone seems to have successfully detected a simulation of this adversary.

In this blog, we’re going to empty our notebook and walk you through what we detected, why, and how we responded to MITRE Engenuity’s emulation activity. We’ll try to cover everything, but there was a lot going on all at once.

 

Background on the test

We’re going to spend most of this blog explaining how our intel analysts, detection engineers, threat researchers, and incident handlers reacted to the emulation, and showing you what this looked like from the perspective of the customer, in this case MITRE. However, before we get to the fun stuff, we need to take some time to set the stage.

What did MITRE do?

Generally speaking, the Managed Services ATT&CK eval was a closed book test for which we instrumented a specially configured environment (more on this in a moment) for the MITRE Engenuity team. The MITRE folks then ran a 10-step adversary emulation, testing our ability to observe, detect, and respond to the threats they were emulating. This test was fundamentally different from MITRE’s previous Enterprise and ICS Evaluations because MITRE did not announce the adversary they would emulate nor the techniques they would leverage in advance.

MITRE Engenuity was basically a Red Canary customer for one week. They ran a red team engagement against Red Canary in an environment that was intentionally configured to allow the emulation to run unencumbered. The point, after all, was to test detection, not prevention.

In this case, the MITRE Engenuity Team chose to emulate a threat actor commonly known as OilRig. OilRig is a so-called advanced persistent threat (APT), thought to be working in the interest (or at the behest) of the Iranian government. The group is well known for a series of attacks targeting public, private, and civil society organizations in Saudi Arabia, but OilRig remains active and has been accused of targeting a wide array of organizations in Gulf States and more broadly around the Middle East.

In other words, our friends at MITRE pretended to be our foes at OilRig.

Due to the test’s requirements, all prevention technology and features provided by our EDR partners were turned off. Red Canary’s Automate feature was enabled and set to report the remediation actions it would have taken based on threats published during the evaluation.

So… how’d we do?!

This blog post makes no effort to qualitatively assess Red Canary or anyone else’s performance in MITRE’s Managed Services Evaluation. Instead, we want to show you everything we observed and detected, independent from MITRE’s assessment of our performance. Our goal is to tell the whole story of MITRE’s emulation like we would for any other adversary, as if we’re writing a post-incident report. In order to do that, though, we need to provide some quantitative information about our performance in the test.

The evaluation consisted of 10 steps, each of which included multiple substeps. In all, we detected 82.9 percent of the substeps. We could nitpick endlessly about what counted as a detection, why certain substeps might not warrant detecting at all, and whether all of the substeps should’ve been weighted equally, but that’s really beside the point. In the spirit of transparency, we’re going to highlight the things we missed right alongside the things we detected. While most of the misses were discovery techniques that might not warrant a detection under normal circumstances, we admittedly and undeniably missed one important substep.

If you’re interested in learning how to interpret the results of the test, you’ll want to read our executive briefing.

 

Dissecting MITRE’s Managed Service Evaluations

We’ll start with a high-level overview of everything that happened—mapped to MITRE’s official 10-step evaluation—before examining everything we observed and detected in depth.

Timeline of emulated OilRig attack

TL:DR

Click on each step to navigate to a more detailed description of what we saw for each event.

Step 1: A user downloaded a malicious document on a workstation called theblock (10.1.0.5), establishing command and control (C2) and persistence.

Step 2: The adversary performed various discovery commands for situational awareness and to enable follow-on targeting on theblock.

Step 3: The adversary conducted credential theft on theblock and exfiltrated those credentials to an adversary-controlled C2.

Step 4: The adversary downloaded a web shell to theblock and copied the webshell to an Exchange server, waterfalls (10.1.0.6), leveraging stolen credentials from step 3.

Step 5: The adversary leveraged the webshell on waterfalls to execute various discovery commands for situational awareness.

Step 6: The adversary accessed the web shell implanted on waterfalls to drop and execute credential theft tooling and exfiltrate stolen credentials back to adversary-controlled C2.

Step 7: The adversary moved laterally via port forwarding on theblock to access waterfalls via RDP.

Step 8: The adversary performed a pass-the-hash attack to gain a shell in the context of another user, a SQL administrator, on the waterfalls Exchange server, then used this access to move laterally to a SQL server, endofroad1.

Step 9: The adversary discovered and collected SQL server database backups and exfiltrated them directly over the Exchange Web Services API.

Step 10: The adversary removed any file indicators on endofroad1.

Test concludes! 

The long version

NOTE: In the following sections, we attempt to describe MITRE’s emulation activity in chronological order from the time we observed each behavior. This analysis mostly aligns with MITRE’s 10 steps (summarized above).

Step 1


Initial Access

As is the case with most bad things on the internet, it all started with phishing…

Spearphishing Link

On July 13 at 12:10 UTC, a user with the account name “Gosta” logged into a workstation called theblock.boom.box, clicked a link, and downloaded a ZIP file named marketing_material.zip from shirinfarhad[.]com.

In an actual customer environment, this attack likely ends right here. Microsoft Defender for Endpoint (MDE) recognized that this document was malicious and generated an alert. The following screenshot shows part of the threat timeline that our detection engineers built as they analyzed this event:

Screenshot showing detection of marketing_material.zip from shirinfarhad[.]com.

Side note: Preventive controls and missed detections

If MDE didn’t block the document outright, then Red Canary Automate would have triggered a playbook to notify the customer, ban the hash associated with the malicious doc, and isolate the endpoint. However, as we noted earlier, the point of this exercise was to test detection capabilities across an entire campaign, and therefore we disabled many preventive and response controls that would’ve otherwise acted on this threat.

MITRE did not give us credit for detecting the Phishing technique in this step. When we communicate information about threats to our customers, we only communicate information about which we are confident. In this case, we hypothesized it was likely a phishing email, but we didn’t have direct evidence so we didn’t speculate in the alert we sent to MITRE. In a real intrusion, we’d work with the customer to gather information necessary to determine without question the initial access vector and respond accordingly.

Execution, Discovery, Defense Evasion 

Malicious File

Within minutes of the initial login (12:11 UTC), Gosta had unzipped the file and opened a Microsoft Word document titled GGMS Overview.doc.

Visual Basic, Obfuscated Files or Information

Gosta closed the document at 12:12 UTC, which initiated a VBA macro. We extracted, decoded, and analyzed the macro, which initiated a barrage of activity that we will detail in the following paragraphs.

Side note: Missing Discovery and Defense Evasion activities

MITRE did not give us credit for detecting mostly discovery-related substeps (and one defense evasion substep) emanating from the macro. We certainly missed some substeps and didn’t effectively communicate about others that we observed in the macro code (which was included in our detection timeline but omitted from this blog due to its length). In the real world, however, making an effort to detect discovery actions at scale will only mire your security team in noise. Catching and acting on the malicious macro is the most important thing.

Ingress Tool Transfer, Masquerading

Importantly, the macro retrieved a remote access trojan (RAT) called b.doc that was actually a binary named SystemFailureReporter.exe.

Side note: Early attribution clues emerge

At this point, analysts on our Intelligence Team assessed that MITRE was plausibly emulating OilRig for the following reasons:

  • OilRig has reportedly leveraged a binary called SystemFailureReporter.exe in past campaigns
  • The ZIP file is downloaded from shirinfarhad[.]com, an allusion to a 12th century Persian poem that’s popular in Iran
  • We knew that MITRE had emulated well-known APT groups in previous evaluations

While we suspected MITRE might be emulating OilRig, it was not yet confirmed at this point.

Persistence

Scheduled Task

The macro also created a configuration file called update.xml and a scheduled task that would launch the SystemFailureReporter.exe binary every five minutes once the user Gosta had been idle for 10 minutes. SystemFailureReporter.exe first ran at 14:05 UTC in the user context of Gosta, who was listed in the enterprise admins security group.

Side note: Automated response playbooks

At 12:48 UTC, roughly 30 minutes after Gosta downloaded a suspicious document and executed the malicious VBA macros stored in it, Red Canary Automate began triggering notification playbooks based on indicators associated with the malicious VBA macro described above. Due to the requirements of the evaluation, these playbooks were intentionally configured only to inform the customer (in this case MITRE) that we had detected a threat. Under normal conditions, these playbooks also would have isolated the endpoint theblock.boom.box and banned any relevant hashes, including SystemFailureReporter.exe, long before the critical follow-on activity we’re about to describe. In all likelihood, this would have seriously hindered or entirely stymied the adversary.

Command and Control

Web Protocols, Symmetric Cryptography

At 14:05 UTC, SystemFailureReporter.exe establishes a connection to the command and control server at 192.168.0.4 over the HTTP protocol on port 443.

Side note: Missing substep!

We failed to report to MITRE that SystemFailureReporter.exe also made an XOR-encrypted connection to that same IP address.

 

Step 2


Discovery

Too many techniques to list

SystemFailureReporter.exe initiated a sequence of discovery actions beginning at 16:05 UTC and ending at 16:10 UTC. The activity included the following commands:

 

CommandFunction
Command:

whoami

Function :

This command displays the domain and user currently logged on to this host.

Command:

hostname

Function :

This command displays the name of this host.

Command:

ipconfig /all

Function :

This command displays the full configuration for all network adapters on this host.

Command:

net user /domain

Function :

This command displays the user account information for this host’s primary domain.

Command:

net group /domain

Function :

This command displays the group information for this host’s primary domain.

Command:

net group "domain admins" /domain

Function :

This command enumerates all members of the Domain Admins group; members of this group have admin rights over the entire domain.

Command:

net group "Exchange Trusted Subsystem" /domain

Function :

This command enumerates all members of the Exchange Trusted Subsystem group; members of this group have full control over the domain root.

Command:

net accounts /domain

Function :

This command checks the policy settings for this host’s primary domain.

Command:

net user

Function :

This command shows the current user’s account information.

Command:

net localgroup administrators

Function :

This command enumerates all members of the local administrators group; members of this group have administrative rights for this endpoint.

Command:

netstat -an

Function :

This command displays all the active TCP and UDP connections on which the host is listening.

Command:

tasklist

Function :

This command displays all of the currently running processes.

Command:

sc query

Function :

This command displays the information for currently running services.

Command:

net user gosta /domain

Function :

This command displays the info for the user Gosta from the domain controller on the current domain.

Command:

net group "SQL Admins" /domain

Function :

This command displays the info for the SQL Admins group from the domain controller on the current domain.

Command:

nslookup WATERFALLS

Function :

This command queries DNS to obtain the network info for the host WATERFALLS.

The last command, nslookup WATERFALLS, is a harbinger of what’s to come, but before we get into the next phase of this attack, we need a brief sidebar.

Side note: Detecting discovery actions

We observed the execution of SystemFailureReporter.exe less than two hours (14:05 UTC) after Gosta opened the malicious document that initiated this incident. Another two hours passed before we started to observe the aggressive discovery actions detailed above (16:05 UTC).

If GGMS Overview.doc had slipped past the initial preventive controls and managed to execute its hidden VBA macro, Red Canary would have detected the above discovery actions and initiated response controls to isolate the affected endpoint right then—less than four hours after Gosta logged in and well before any credential theft, lateral movement, and exfiltration activity.

 

 

Step 3


Command and Control

Ingress Tool Transfer

At 18:03 UTC, SystemFailureReporter.exe ran via a scheduled task and wrote another binary to disk, b.exe.

Screenshot illustrating SystemFailureReporter.exe detection
Credential Access

Windows Credential Manager

Upon analysis, our intelligence and research teams determined that the b.exe binary was emulating a known malware strain called VALUEVAULT, all but confirming our suspicion that MITRE was in fact emulating OilRig.

Side note: Identifying OilRig!

FireEye first reported on VALUEVAULT while investigating social media phishing campaigns in 2019, and the malware is now widely associated with OilRig. VALUEVAULT is known to extract credentials from the Windows Vault and create a SQLITE database file in the AppData\Roaming directory named fsociety.dat to store the stolen credentials.

This is precisely what happened in the evaluation: b.exe dumped the windows credentials stored on theblock.boom.box into a SQLite3 database called fsociety.dat.

Screenshot of detection of b.exe

At this point, our Intelligence Team was almost certain that MITRE was emulating OilRig. In a real-world scenario, there would remain some doubt about this attribution because any number of adversaries could have leveraged VALUEVAULT, written credentials to “fsociety.dat, and mimicked other indicators. However, our assessment was made easier by knowing that MITRE would be emulating a well known adversary.

Exfiltration

Exfiltration Over C2 Channel

At this point, the adversary exfiltrated fsociety.dat from theblock.boom.box to their C2 server (192.168.0.4) via an HTTP post request.

Side note: We missed this!

This is probably our most glaring omission. We simply did not observe the adversary passing the fsociety.dat file back to their C2 infrastructure, which was an important part of the intrusion. Many C2 frameworks offer adversaries the ability manipulate packet structure and effectively hide the contents of their communications during exfiltration. We were able to identify requests that likely contained the exfiltration, but complete visibility into network telemetry is a known short-coming of most EDR products. As we do with any misses during testing, we’re considering how we could improve our detection coverage to catch similar behavior in the future. Further, we support the ingestion of Network Security telemetry and alerts from products like firewalls, network detection and response, and DNS records, which would have proven helpful in fully observing this substep.

 

Step 4


Command and Control, Lateral Movement

Ingress Tool Transfer, Lateral Tool Transfer

At 19:33 UTC, SystemFailureReporter.exe wrote contact.aspx to disk. Shortly thereafter (19:43 UTC), it copied that contact.aspx to another endpoint (technically an Exchange server) waterfalls.boom.box, which is the same endpoint upon which the adversary performed a DNS lookup earlier as part of their discovery.

Screenshot of OilRig detection

Persistence

Web Shell

This web shell provided the adversary the ability to write, download, and delete files from disk, as well as execute binaries that are present on the infected endpoint. The previous image illustrated telemetry from theblock.boom.box copying an Active Server Page Extended (.aspx) file to waterfalls.boom.box. The following image shows the telemetry associated with that action from the perspective of waterfalls.boom.box:
Screenshot of OilRig detection

Defense Evasion

Hidden Files and Directories, File Deletion

Meanwhile, back on theblock.boom.box at 19:48 UTC, the adversary deleted contact.aspx from the endpoint and leveraged the attrib +h command to hide the .aspx file from the typical folder view and from ordinary directory listings on waterfalls.boom.box.

Screenshot of OilRig detection

Step 5


Persistence

Web Shell

On July 14 at 17:30 UTC, the adversary then leveraged the web shell to launch the Command Processor (cmd.exe), which ran as SYSTEM (this has important implications when we get to the Mimikatz activity later). It proceeded to run multiple commands consistent with post-exploitation, which we will highlight in the coming sections. The following graphic shows w3wp.exe spawning the command shell:

Screenshot depicting OilRig detection

Discovery

Multiple Discovery techniques

In the space of a few minutes, the adversary used the newly spawned command shell to gather information about system network configuration, enumerate a list of active network connections, and see who had system access on the host via the ipconfig, netstat, and whoami commands. The following image shows telemetry associated with the ipconfig and netstat commands:
Screenshot depicting OilRig detection

Step 6


Command and Control, Credential Access

Ingress Tool Transfer

Leveraging the web shell, the adversary then wrote a binary named m64.exe to disk at 18:33 UTC on waterfalls.boom.box. This binary was in fact a renamed version of the credential-stealing tool Mimikatz. As we noted earlier, the command shell that spawned Mimikatz ran with system-level privileges, which granted Mimikatz the privilege level required to perform the actions described in the next section.

LSASS Memory

Immediately thereafter, the adversary leveraged Mimikatz to open a handle into the Local Security Authority Subsystem Service (lsass.exe) and used the sekurlsa::logonPasswords command to extract credentials from it, writing them to a file called 01.txt, which potentially contained hashes for the following users:

  • gosta
  • tous
  • Mariam
  • vendor_domain_admin
  • evals_domain_admin

Screenshot of LSASS memory dump in OilRig detection

Exfiltration Over C2 Channel

The web shell then ran its HandlePOSTFileDownloadServer() function to download files off disk, very likely in an effort to exfiltrate credentials from 01.txt.

Defense Evasion

File Deletion

At 18:37, the adversary ran commands intended to delete m64.exe and 01.txt from waterfalls.boom.box.
Screenshot of Mimikatz in OilRig detection

Side note: Detecting Mimikatz

We’ve developed detection analytics for different aspects of Mimikatz, at least three of which are on display in the above image. One option involves looking for processes that obtain a handle into lsass.exe. Alternatively, you can look for module loads of dynamic link libraries like samlib.dll or vaultcli.dll, which you can see in the following image:

Screenshot of Mimikatz in OilRig Detection

A third option is to detect specific command-line parameters that are associated with Mimikatz. In this case, those commands were: privilege::debug and sekurlsa::logonpasswords.

 

Step 7


Command and Control

Ingress Tool Transfer

Back on theblock.boom.box on July 15 at 12:06 UTC, SystemFailureReporter.exe wrote a copy of plink.exe, the PuTTY command-line interface used for SSH tunneling.

Screenshot of OilRig detection

Protocol Tunneling

At 12:11 UTC, the adversary leveraged plink.exe to create an SSH tunnel and make RDP connections to their adversary-controlled machine, attempting to gain direct RDP access on waterfalls.boom.box.

Screenshot of OilRig detection

Lateral Movement, Defense Evasion

Remote Desktop Protocol, Domain Accounts

At 12:13 UTC, the adversary leveraged the stolen Gosta credentials to open an RDP session on waterfalls.boom.box using the established SSH tunnel via plink.exe.

Screenshot of OilRig detection

Step 8


Persistence, Command and Control

Web Shell, Ingress Tool Transfer

Starting at 12:15 UTC, the adversary leveraged the web shell to write several files to disk on waterfalls.boom.box:

  • ps.exe, a renamed version of the Sysinternals tool
  • mom64.exe, another renamed version of Mimikatz
  • nt.dat, an email-based C2 tool

The following images show the adversary writing each of these tools to disk.

PsExec

Screenshot of PsExec in OilRig detection

Nt.dat

Screenshot of nt.dat in OilRig detection

Mimikatz

Screenshot of Mimikatz in OilRig detection

Side note: Detecting renamed instances of PsExec

As we’ve written about extensively, you can often detect renamed tools by looking for the execution of binaries where the binary’s presented filename is different from the internal filename listed in its binary metadata. In this case, the file was called ps.exe, but the file’s internal binary name was “PsExec.”

Defense Evasion, Execution, Lateral Movement

Pass the Hash

The adversary then abused Mimikatzʼs pass-the-hash functionality and a previously extracted NTLM hash to switch the user’s context to the user “tous” at 12:18 UTC. Importantly, “Tous” is a member of the SQL Admins group, which the adversary discovered in a previous step.

Screenshot of Pass the Hash technique in OilRig detection

Windows Command Shell

Seconds later, mom64.exe (aka Mimikatz) opened an interactive command shell, as shown in the following image:

Screenshot of cmd.exe in OilRig detection

Lateral Tool Transfer, SMB/Windows Admin Shares, Service Execution

At 12:20 UTC, the adversary—now operating in the context of the user “tous”—leveraged PsExec to execute commands on a new host, a SQL server called endofroad1.boom.box, before moving nt.dat to the new host.

Screenshot of OilRig detection

Step 9


Discovery

File and Directory Discovery

Next, the adversary performed discovery of backup files on the SQL server endofroad1.boom.box.

Side note: We missed this!

We failed to include these actions in our timeline or report them to the team at MITRE. As with all misses, we’re considering how we could better detect similar behaviors in the future.

 

 

Step 10


Collection, Defense Evasion

Local Data Staging, Match Legitimate Name or Location

At 12:23 UTC, the adversary used PsExec to spawn a command shell and create a folder named “Vmware” and a file named vmware.exe on endofroad1.boom.box.

An analysis of vmware.exe revealed that the binary was actually a renamed instance of the email C2 tool from earlier, nt.dat. The adversary moved that nt.dat file into the Vmware folder and renamed it.

Screenshot of OilRig detection

Collection, Exfiltration, Defense Evasion

Data From Local System, Data Transfer Size Limits, Exfiltration Over Unencrypted Non-C2 Protocol, Obfuscated Files or Information

At 12:26 UTC, the adversary leveraged nt.dat (masquerading as vmware.exe) to exfiltrate a steganographic .bmp file containing the contents of a SQL database backup. Using credentials for the user Gosta, the adversary sent an email via Microsoft Exchangeʼs API and then removed their custom tooling from disk. In the process, the adversary resized some of the data to evade detection.

Screenshot of OilRig detection

Defense Evasion

Indicator Removal

Last but certainly not least, the adversary cleaned up their tracks by removing vmware.exe and the Vmware folder from endofroad1.boom.box at 12:26 UTC. At 12:31 the adversary removed the following files from waterfalls.boom.box:

  • mom64.exe
  • nt.dat
  • ps.exe
  • b.exe
  • fsociety.dat
  • plink.exe
  • update.xml

Screenshot of OilRig detection

Side note: We missed a pair of removals

We got partial credit on this step for reporting half of the removed files to MITRE. We failed to explicitly report the deletion of the VMware directory and the deletion of mom64.exe, nt.dat, and ps.exe.

 

Conclusion

The MITRE ATT&CK Evaluations for Managed Services is a great resource for evaluating different MDR providers and determining how they would help protect your organization in a real-world intrusion. All the participants should be commended for being transparent enough to take part in the evaluation, and the majority will likely be proud of their performance.

One of Red Canary’s core values is authenticity, which is why we were so excited to hear there would be an ATT&CK Evaluation for Managed Services. We welcome readers of the evaluation to dig into the data we provided and ask any questions about how this would translate to their security program.

When evaluating the test results, please remember that a stack ranking of technique coverage is not the best way to interpret the results. Instead, we recommended focusing on the security outcomes that each vendor was able to achieve. The best resource available to discern this information (other than this document, of course) is the archive of reports, emails, and other content that MITRE made available for each participant.

 

A defender’s guide to identity attacks

 

Single sign-on, double trouble: Credential theft using AWS access tokens

 

The three keys to threat hunting

 

The dark cloud around GCP service accounts

Subscribe to our blog

 
 
Back to Top