Breaking down a breach with Red Canary’s incident handling team

Editor’s Note: This story by Brian Nordli was originally published on Built In Colorado. It is republished here with permission.

The worm burrowed its way into the company’s network sometime in 2019. Perhaps it entered through an email, but, most likely, it came in via a USB port. One way or another, the worm had eluded the company’s anti-virus software and endpoint detection system. Then it started spreading across the company’s global network.

A few months after the infiltration, Chuck Frey, a member of Red Canary’s Cyber Incident Response Team, was starting his on-call shift when he came across an email from the client. Four out of five times, these messages are false alarms, he said, but the fact that the client had reached out before most people were awake concerned him. He opened it.

The client wrote that he had come across LNK files in his company’s network attached storage (NAS) — a telltale sign of a rogue AutoIT worm. An AutoIT worm can spread across a company’s network and exploit network vulnerabilities. If it went unchecked, a hacker could use that program to get a hold of the company’s intellectual property and hold it for ransom.

The client wanted to know if the LNK files were a bad sign, which Frey confirmed.

No system is ever entirely safe

No matter how much a company invests in cybersecurity software, it can never be entirely safe from hackers, who are continually coming up with ways to break into a company’s network.

The most common cyberthreats come from what Red Canary incident handler Taylor Chapman calls the malware family of 2018–19. These include a particularly nasty one-two punch of the trojans Emotet and TrickBot followed up with the ransomware Ryuk, which can infiltrate a network through email and wreak havoc. Once inside, hackers can gain access to a firm’s data, encrypt it and demand a ransom to unlock it. Those cases can have a devastating impact on the company.

“Companies are getting ransomed and the adversaries have taken it upon themselves to kick it up a notch and say, ‘If you don’t pay us, we’ll release your stolen data publicly,’” Chapman said. “If you’re a company on the NASDAQ or DOW, your stock is going to plummet.”

To combat these threats, Red Canary offers companies managed detection and response capabilities coupled with an on-call incident handling team. In most situations, the published detections alert the company to a breach and then the incident handling team works with the client to mitigate the threat. But sometimes, threats sneak under the radar and past security software. In those cases, a client may catch it when they notice unusual files or activity on their computer. It puts extra pressure on the handler to figure out what’s going on so they can mitigate the threat.

“These are the worst kinds of incidents,” Frey said. “You’re caught off-guard, and you still have to validate that what they’re saying is true.”

Frey’s first step: Figure out the scope of the issue.

Tracking anomalies to understand the scope of the threat

Every cyber investigation starts with finding an anomaly in the network that can act as a starting point, much like a detective’s warm lead. The investigator starts following the trail in the endpoint detection telemetry, seeing what activity spun out from that account or application, tracking when and where anomalies popped up.

Over time, the handler can start connecting dots and establish a fuller picture of the incident. Often, a flurry of activity around the same timeframe is a sign that the network has been hacked.

“If you start with one thing that’s interesting and rabbit hole your way down, you get timestamps and potentially affected endpoints and users,” Chapman said. “Then you can start saying: ‘What else happened around that time? What else did this thing interact with?’”

Frey started with the LNK files the customer brought to his attention in the NAS. From there, he worked backward, narrowing in on files shared from an individual user account rather than a team account, where others might have access. Using the endpoint software, he followed a trail of similar anomalies from that account and charted the timestamps in an Excel spreadsheet.

“If you start with one thing that’s interesting and rabbit hole your way down, you get timestamps and potentially affected endpoints and users.”

Through that process, Frey confirmed that the files were being generated programmatically — that is to say, the worm attached itself to the user account and was spreading itself across the network.

That was the good news. The bad news was that the customer had a distributed network that spanned North America, South America, and EMEA. Red Canary’s software could only generate insights into the network in North America, and the company hadn’t set up proper remote endpoints for its international networks, Frey said. There was no way he could confirm just how far the infiltration had reached.

Devise a plan to fully eliminate the threat all at once

When a cyberthreat is found, your gut instinct might be to immediately root it out of the system. However, that’s not always the best idea, Chapman said.

If a handler moves too fast to eliminate a hacker, they run the risk of tipping them off. They could then burrow deeper into the network and cause future problems. Chapman has seen hackers thought to be removed from a network hiding malware in internet-connected vending machines and thermostats.

Understanding what the hacker is trying to do and responding to that is crucial to eliminating the threat, Chapman said.

The incident handler can only serve as an advisor to the company. It’s up to the company to make the call on how to proceed.

“Yes, we want to get answers quickly, but we need to do it the right way,” Chapman said. “There’s a phrase in snowboarding, ‘Buy it nice or buy it twice.’ That’s totally the case here.”

In Frey’s AutoIT case, he worked closely with the customer’s security operations center to figure out a solution over a series of Zoom sessions. The company was at risk of exfiltration, so Frey knew they had to move quickly.

The first step Frey recommended they take was to ban the hashes executing the rogue AutoIT worm in its firewall and endpoint system. This would prevent the LNK files from spreading. From there, they also blocked all connections the worm had to the company’s command and control domains.

“Let’s not delay, there’s no change management,” Frey thought to himself. “We did get approvals, but we got leadership involved quickly to empower the workers to do their jobs.”

Since they didn’t have a full picture of the international risk the company faced, Frey and his counterparts created a script for the remote teams to install on the unmanaged machines to prevent any of the rogue LNK hashes from spreading.

Iterate and improve cybersecurity defenses

In an ideal world, this story would have a satisfying conclusion, where the perpetrators were caught and faced consequences. But that isn’t how cybersecurity works anymore. There are typically two types of hackers that Red Canary deals with — amateurs, who operate somewhat cautiously yet make loads of mistakes and move too fast; and experts, who operate more brazenly yet skillfully, artfully, and almost entirely hidden. Neither care if they get caught, Chapman said. And since most hackers are overseas, they can act as sloppily as they want and keep on returning. All Red Canary can do is strengthen its own detection software and help companies continue to invest and scale their security efforts.

“Most of our time and energy is on helping the customer improve their security posture,” Frey said. “Whether there’s an incident or not, that’s what we do all of the time.” Frey never could identify the source of the malicious AutoIT worm, nor could the company fully remove it from its system. But through his remediation report, he was able to offer some solutions. “Most of our time and energy is on helping the customer improve their security posture.”

He noted where Red Canary’s detection software fell short, and that the company would update it so that it would notify the cyber incident response team when AutoIT made a connection and send another alert when it creates an abnormally high number of LNK files.

On the client’s end, he recommended installing antivirus on its NAS devices, along with Carbon Black sensors on all of its devices so as to pass remote telemetry data to Red Canary. He also worked with the client to make sure it had better remote visibility into its network. Red Canary also banned the hashes and set systems up to implicitly distrust the certificate authority that had spread the LNK files and caused the issue, according to Frey’s write-up.

Those efforts helped stymie the worm and should tip the company off immediately if anything like it pops up again in the future.

If there are any takeaways from this incident, Frey said, it’s that companies need to continue training their employees on security protocol. They need to warn them not to click on weird links or insert foreign sticks into the USB drive. They should limit administrative rights and always make sure they’re deploying hardware security best practices as companies like Microsoft and Cisco outline them.