Why a SIEM Won’t Solve All Your Problems: 5 Common Issues and How to Avoid Them

Today’s organizations suffer from a gap in detection capabilities. Research such as the Mandiant M-Trends report show that the median time to detect an adversary is 99 days. Even if you interpret this with a grain of salt, there’s no doubt that the ability to catch an adversary is far from where it should be. Many organizations look to implement a solution that addresses the gap in detection capabilities and turn to a SIEM. Unfortunately, they often find that it creates more issues than it solves.

As the author of the SANS class SIEM with Tactical Analytics, you might assume I’m a SIEM hugging advocate (and I am). I believe that a SIEM is one of the most capable detection solutions available.

However, “most capable” does not necessarily mean “best.” While I would love to say a SIEM is the answer to all things, that is simply not the case. Let me clarify this seemingly contradictory position…

Organizations often make the following assumptions when purchasing or implementing a SIEM:

• Blazing fast search and response times
• Ability to share data with any team or individual as granularly as possible
• Built-in threat intelligence that highlights adversary activities
• High fidelity alerts
• Big data capabilities and analysis
• Automated machine learning or behavioral analysis
• Automated data correlation

The perceived weaknesses generally include initial and recurring capital expenses, and costs for staff to maintain the solution.

For many modern SIEM deployments, most of the above perceptions are false. First off, we all need to keep in mind the reality that even a fully functional space station…er…SIEM…is still part of a larger security program. Other components must first exist for a SIEM to be successful.

In this article, I’ll break down:

• 5 common issues that organizations face when deploying a SIEM
• Use case of an organization that resolved one of these issues
• Recommendations and success requirements for organizations looking to buy a SIEM

Let’s dig in!

5 Common SIEM Issues

Issue #1: A SIEM is only as good as the data you feed it

Simply having a SIEM deployed means nothing to your actual security posture. Data that gets loaded to the SIEM is what gives it value. The problem is, there are a lot of bad data sources, and most data cannot easily be classified in a binary “good/bad” fashion. For example, Windows logs are almost universally collected, and they simultaneously represent one of the best and one of the worst data sources for security operations to use.

For starters, a Windows system does not even log all the events that typically matter. By default, process and command line logging, PowerShell logs, and Windows driver framework logs are not enabled. However, enabling these without tuning can overload a SIEM with large volumes of useless data. Even still, the Windows logs that are enabled by default are useful but also contain a large volume of noise. Log collection, parsing, and filtering is an art requiring time and patience as well as continuous re-evaluation for validity.

This doesn’t even consider how data is fed into a SIEM. If a log comes via syslog, a common log transport protocol, the administration must enforce parsing. This likely means the log’s end state in the SIEM includes the loss of some data and context from the original event. Going back to Windows as an example, a single Windows 10 system has over 800 fields found within the native binary XML log structure. Yet most SIEMs parse, reduce, and convert that source set down to only around 200 fields.

The result is like riding a horse when you could be driving a car. When I say mileage varies based on what you collect and how you collect it, I mean it.

Issue #2: A SIEM requires use case implementation

Can your organization detect an adversary that used a VBScript to invoke PowerShell, which in turn pulled down base64 obfuscated code, then established a backdoor, finally finishing up by scanning your internal network? Every single step of this process is easy to detect with the right logs and application of the logs.

A SIEM can’t automate information security domain expertise and application of your logs to your specific needs. Once the data is in your SIEM, you must be the one to tell the SIEM what to do with it. Basically, it is like buying a toy. The batteries are not included. You must go get them and plug them in. Cool toy…no batteries.

Ever asked a vendor for alert rules or techniques to catch the bad guys only to be told: “every organization is unique”? While there is some truth to that, there also are a bunch of techniques that can be used across any organization based on common attack methodologies. Don’t believe me? Check out these examples:

• MITRE ATT&CK: This open source framework is a model of attacker methodologies and adversary behaviors. Defenders can use it to identify the spectrum of techniques that an attacker may exhibit, then look across their processes and controls to identify gaps in detection and prevention coverage.
• Sigma: This is a SIEM neutral rule language that allows rules to be written to support any environment and any SIEM. In order for this concept to work, a conversion process takes the SIEM agnostically written rule and translates it specific to a given SIEM.

At the end of the day, SIEM + use case application = happy marriage. SIEM without use case application means you will be filing for a divorce at a later date. The SIEM vow of love, trust, and awesome detection capabilities will have been broken.

Issue #3: More data does not mean better detection

The era of Big Data is upon us. Security operations teams are scrambling to collect all the data they can get their hands on. This is…crazy!

One of the most common failings I have seen is a SIEM overstuffed with useless data. The result is what I call the “coffee break SIEM.” This occurs when even a simple search takes seconds or longer to complete. If an analyst constantly must wait for a search to finish, then he or she is being trained over time to choose what is worth searching for— whatever is returned the fastest.

A SIEM should augment analysis, not hinder it. Put simply: less is more. The more data you have, the worse the SIEM performs and the more rabbit holes your analysts can go down…slowly.

Issue #4: Lack of context

A SIEM should be built by analysts, for analysts. This means that when an analyst is reviewing alerts or logs, the SIEM should provide context and information in a meaningful way. Fortunately, log enrichment, or the art of adding context to a log, is something a SIEM is extremely successful at. Unfortunately, most SIEM implementations prioritize data collection over log enrichment.

As an example, assume an analyst is looking at a potentially suspicious domain being accessed called “covertc2.com” (imaginative, I know). A DNS log contains the domain, the source and destination IP header information, resulting IP address(es), and the DNS record type. Is this enough information to help the analyst decide if the domain is malicious, suspicious, or benign? Clearly, making such a determination based on this limited data set would be speculative at best.

Now, consider what you could deduce if the SIEM added context such as the following:

• The domain is not in the top 1 million sites accessed
• The domain was registered on 2016-11-01
• The IP is related to a business in Illinois
• The IP belongs to Total Server Solutions L.L.C.
• The domain does not appear in threat intelligence databases
• The domain does not appear to be randomly generated

This level of context does not give a 100% certainty of good or evil, but it certainly provides an analyst with the context to make a more informed decision about the domain observation itself.

Issue #5: Too much maintenance

Oddly enough, the one thing organizations readily accept as a weakness for a SIEM is that it comes with a labor commitment—sometimes a massive one. As a result, they hire dedicated staff or reserve the current team’s time to support data collection needs and the tender love and care of the SIEM itself. So instead of their SIEM taking care of them, they ended up taking care of the SIEM. All the while wasting tons of labor that could have been spent elsewhere.

If you spend 80% of your time performing maintenance tasks like deploying agents, parsing logs, or performing upgrades, then you are likely not getting the maximum value from your SIEM. Automation is critical to a successful SIEM implementation. Your environment is constantly changing, and automation is necessary to keep up with it so you can better spend your time on implementing tactical capabilities.

Use Case: 6 Weeks vs. 14 Months

Let’s look at an example of Issue #5 in action. Once I was working with an organization that had been rolling out a top 5 magic quadrant SIEM solution. This organization had 1.5 full-time employees (FTEs) assigned to the project for fourteen months. Since it had been over a year, they paused the project and took a hard look at what capabilities they currently gained from the SIEM. The findings were not good. After fourteen months they still had little to no detection capabilities and were in data collection mode.

At this point, I had the pleasure of stepping in and helping. In less than a month a new solution was rolled out. Automation was used to deploy log agents, network extraction, and configuration of data sources. Default, or canned alerts, were modified or disabled and new alerts were implemented based on enriched log data.

By the end of six weeks, less than 0.5 FTE delivered a solution that exceeded the capabilities of 1.5 FTEs over fourteen months. Clearly a much better return on the client’s investment.

Looking to Buy a SIEM? 6 Requirements for Success

A SIEM can be incredibly successful, but it requires all of the following:

1. Trained staff with knowledge in information security
2. Trained staff with knowledge of SIEM product (less important than above data point)
3. Ability to enrich data
4. Supports automation
5. Collects and uses data that matters
6. Applies use cases for detection purposes

Machine learning and behavioral analysis can also be helpful. However, do not focus on machine learning or behavioral analytics until you first have mastered the above requirements. An organization that masters the above list yet does not have automated detection capabilities will have significantly better detection capabilities than an organization that has not mastered the list and focused on a tool to automatically catch anomalies.

Key Takeaways

As a general rule of thumb, SIEM usage is most effective for mature, well-performing security teams. This is by no means a hard and fast rule; I’ve seen a number of security teams that are far from “master defenders” properly implement SIEM and see positive results. They were able to use their SIEMs for context and reporting to implement security controls such as firewall rules. However, it is far more common for security teams to jump into a SIEM and run into the issues I outlined above.

The key takeaway: a SIEM should not be your starting point, especially if you are just starting your security department and controls. In the crawl-walk-run world, it’s somewhere around a slow run. First, start with the basics. Things like endpoint security solutions, network segmentation, and firewalls go a long way for prevention but are necessary for detection capabilities. After doing this, the next stop is NOT to rush out and buy a SIEM. Only when you meet the requirements above are you ready to purchase, buy, and maintain a SIEM.

I hope these insights will help your team be successful, avoid common SIEM issues, and kick adversary butt.

About the Author

Justin Henderson is a SANS Instructor and course author of SEC555: SIEM with Tactical Analytics. As a security architect and researcher with over decade of experience, he is passionate about using his knowledge to mentor others.