How to Baseline & Inventory in Minutes With Cb Response + Surveyor

Years ago, as Red Canary began to scale security operations atop the Carbon Black (Cb) Response platform, we immediately started to identify some common use cases:

Incident response and investigations
Root cause analysis
Inventory

Cb Response was built for the express purpose of supercharging the incident response process. Instead of painstakingly collecting terabytes of data that need to be loaded, carved, and analyzed using expensive systems, simply collect most of what you want up front so that it’s available the moment it’s needed. The same principle applies to root cause analysis. With all of the data recorded, you can easily wind back the tape to determine what happened and where a threat originated. The use case that we weren’t able to meet using built-in functionality was inventory.

The Importance of Inventory

InfoSec Jeopardy
A: Inventory
Q: What is the failed process undermining your expensive and outwardly mature information security program?
— Keith (@kwm) January 9, 2017

Inventory is the cornerstone of any successful security program. If you don’t know what you have, you can’t possibly monitor, defend, or respond. And inventory is more than simple asset identification and tracking. Let’s look at Red Canary’s use case as an example to expand. For our purposes, inventory means that we understand:

What endpoints we’re monitoring in our customers’ environments
The user accounts active on those endpoints
The processes that users are executing on each endpoint
How those processes are being leveraged (i.e., how are they being spawned, and what command line arguments)

Anyone who has used Cb Response (or other leading Endpoint Detection and Response platforms) recognizes the power it offers security teams. It’s easy to find binaries and processes that meet a specific criteria and then investigate them at a low level. For all of the strengths of EDR, one shortcoming is an inability to answer simple questions related to inventory and scope. It’s great to know that we have thousands of processes that meet some criteria, but many times we really need to start by answering some far simpler questions like:

How many endpoints are syncing data with Dropbox?
Who’s using PSExec and how is it being used?
What remote access tools are being used within my enterprise?

And as mentioned above, we don’t need every occurrence. We just need to know whether Dropbox adoption is closer to 9% or 90%. And do we have a critical mass of Microsoft Remote Desktop users with one or two stragglers, or are entire teams using unmanaged solutions for remote access?

Using Cb Response to Find the Answers

Even though the functionality didn’t exist in Cb Response, the data did! As soon as we identified the need to perform rapid inventory and scoping tasks via Cb Response, we turned to their REST API and Python client library. This allowed us to automate data extraction from the platform.

Next, we needed a simple way to define what we wanted to ask of the data. Here there were two primary types of inquiries: repeatable and ad hoc. Ad hoc queries were a simple task, given that any Cb Response query can be passed via the API. So, we allow these to be defined via a simple command line option.

For example, to find likely Dropbox users we’d do one of the following:
query process_name:"dropbox.exe"
query domain:"client.dropbox.com"

The more interesting questions are those that we ask time and again. Continuing with the Dropbox-based example, we expand out to identify the most popular sharing services. This is done by using a JSON-formatted file to roll up one or more identifying criteria into a logical group.

You can see from the following excerpt that we may opt to use multiple criteria, either a process name or a network connection associated with a given domain, to identify a Dropbox client:

Looking at another example, we can also define several process names, any one of which indicate the presence of the thing we seek:

Creating the Surveyor Tool

As we continued iterating, our Surveyor tool began to take its form. Surveyor was built to be a simple, flexible, and performant summary data extractor that allows Cb Response users to quickly understand macro-level information about their environment. To this day, the Red Canary Security Operations and Technical Account Management teams use Surveyor constantly to perform everything from initial baselining of a new Cb Response environment to incident scoping and even simple software inventory.

For all of our readers who are Cb Response customers, I would highly recommend that you try Surveyor. You will see that it’s easy to use and can help you uncover some very interesting data in a short amount of time.

Putting It All Together

So, let’s see it in action. Once you’ve installed it and defined your criteria, it’s time to turn Surveyor loose. If you’re looking for something specific, you can use the query option or a single definition file:

./surveyor.py --deffile definitions/file-sharing-and-backup.json

And in just a few seconds we now have a list of all systems, users, and processes that meet the defined criteria. Specifically, Surveyor outputs a deduplicated list based on the unique set of endpoint + username + process + command line. And it naturally outputs everything in the undisputed king of file formats: CSV.

These abbreviated results in this format mean that we can very quickly use command-line utilities or any spreadsheet software to filter or pivot and reveal the critical answer to our question: How many Dropbox users do we have after all?

Compare this to the result set that we have to parse when using the Cb Response UI.

The important thing to keep in mind is that the UI results, 711 of them in this case, aren’t a limitation of the Cb Response platform. They’re simply the default interface provided by the product. The API exists so that an alternate interface, such as Surveyor, can be implemented easily and in little time.

Moving Beyond the Basics

Out of the box we provide several useful definition files including:

File sharing and backup
Active Directory command-line interfaces
File transfer
Remote administration
Scripting utilities
More . . .

We’ve evolved these built-in definitions based on what we like to understand, and also what most customers or users are interested in learning about their environment.

You can even execute a single Surveyor run that will process them all:
./surveyor.py --defdir definitions

And as previously mentioned, the most productive use of Surveyor is to look for things that matter to your organization.

Use it to answer important questions right away: What users and processes talked to this evil domain?
./surveyor.py --query domain:tehevil.com

Enumerate interactive Windows users:
./surveyor.py --query “os_type:1 process_name:explorer.exe”

Take stock of licensed software utilization:
./surveyor.py --deffile msoffice.json

For several years now, Surveyor has been the workhorse of our own baselining, scoping, and inventory processes. It is used by some of the largest Carbon Black incident response partners, and by hundreds of our own customers and community members. Try it today, it’s free (as in beer)! If you have war stories, use cases, questions, or any feedback at all send it my way via Github or @kwm on Twitter.