Diary of a Detection Engineer: Blown to BITSAdmin

In the hush of a lingering late-summer night, detection engineer Han Shan was working his way through the queue when a peculiar event caught his eye. The event—he would later learn—was a likely precursor to a known ransomware payload that had recently surfaced in the wild. Had it not been for this individual’s keen eye and the swift actions of others working in the Red Canary CIRT that evening, this story would have had a much different outcome. The following Diary of a Detection Engineer is as told by Han himself.

How it started

I received an event for the Background Intelligence Transfer Service administrative tool (bitsadmin.exe, aka BITSAdmin) downloading a file. BITSAdmin is a command-line tool built into Microsoft Windows that allows for queuing file transfer jobs. It’s often used for administrative tasks within an enterprise, but as with many tools, it can be used for nefarious purposes as well. So, while suspicious, it did not immediately stand out as a threat. However—on my initial lookover—there were a few things that emerged as “not normal.”

1. The IP address that BITSAdmin was downloading the file from did not seem consistent with normal admin activity.
2. The IP address was registered in Russia, which is of interest because we don’t typically see Russia-linked entities as business partners within most customer environments.
3. It was downloading a file called 1.html and saving it locally to the device as 1.exe in the ProgramData directory.

All of these observations together were suspicious enough for me to start digging deeper.

The ultimate payload

1.exe did not seem to successfully execute after being downloaded, so my next step was to look at the process tree to see what triggered this activity. The parent process of BITSAdmin was cmd.exe. On investigating that cmd instance, I saw that—in addition to spawning BITSAdmin to download the suspicious binary—it was also attempting to create several administrator accounts with passwords associated with large Russian business entities and interests, all bundled into a one-line cmd.exe /c command.

At this point, that was enough information for me to decide we needed to notify the customer about this threat. That being said, I still didn’t see where all this activity originated from, so I continued up the process tree.

Additional detections

The process that spawned cmd.exe was a process called beremote.exe, which is a Windows agent for Veritas backup software. That left me scratching my head because this activity definitely did not seem consistent with what I would expect from backup software. This looked like someone was exploiting the software to execute suspicious commands.

As per our standard procedures, I notified another detection engineer who was on shift to look over my drafted threat for any errors before notifying the customer. As he was reviewing it, I asked him his thoughts about the Veritas software. We agreed that this was definitely not expected behavior. While we weren’t seeing any additional activity after the creation of the admin accounts and execution of 1.exe, we wanted to get this threat published and sent to the customer so that our automation tools could isolate the device, ban the specified processes, and block the highlighted network connections. Additionally, we made the decision to escalate this to the Red Canary threat hunting team due to the fact that this looked like exploitation of a publicly exposed Veritas server.

Internal & external intelligence

As I finished writing up the threat and sent the initial notification to the customer, a third detection engineer hopped into our conversation with an open source report from Mandiant discussing a threat actor who was recently exploiting multiple Veritas CVEs. The three of us quickly reviewed the article and many of the techniques in the paper (BITSAdmin use, primarily) overlapped with what we were seeing. While we’ve yet to confirm the overlap, the threat actor detailed in the Mandiant report is a known ransomware operator. I passed this information on to the threat hunters who were, at that point, reviewing my published threat.

Customer communications

The timeline from detection to eradication was less than two hours total.

Our threat hunting team was able to contact the customer after-hours, and I hopped onto a video call with them shortly after confirming the threat. As I explained what we were seeing, the two threat hunters on the call walked the customer through some queries in their SOAR platform to verify that the initial endpoint was isolated and the appropriate processes were killed and/or network connections blocked.

Next, threat hunters helped the customer query for any other devices that may have been exploited, and we looked for BITSAdmin activity initiating network connections or the presence of 1.exe across their environment. I shared the information we found about the known Veritas vulnerabilities, and together we walked the customer through how to identify all Veritas servers in their environment, which ones were vulnerable to the exploit (based on which version of Veritas they were running), and which ones were publicly exposed.

After drafting that list, the customer had actionable steps they could take to mitigate future threats. They knew which servers needed to be updated and which ones needed to be hardened.

In conclusion

We reviewed several OSINT reports and performed our own cursory analysis to learn more about the binary’s expected behavior. 1.exe subsequently showed it installed remote management and monitoring (RMM) software, AnyDesk to be specific.

These additional details were passed along to the threat hunters and the customer. Though it is not commonly used in their environment, the customer decided to proactively set up an alert for any attempts to download AnyDesk as an added safety measure (despite the fact that there was no evidence of lateral movement).

Further analysis conducted by the Red Canary CIRT revealed the adversary used a fixed password in AnyDesk identical to the previously identified administrator account creation above. This proves persistence was undoubtedly thwarted that fateful night.