Whether or Not SIEM Died, the Problems Remain

Last updated at Mon, 28 Oct 2019 17:10:23 GMT

This post is the second in a series examining the roles of search and analytics in the incident-detection-to-response lifecycle. To read the previous, click here.

Various security vendors have made very public declarations claiming everything from “SIEM is dead.” to asking if it has merely “lost its magic”. Whatever your stance on SIEM, what's important to recognize is that while technologies may fail to solve a problem, this doesn't make the problem any less serious or prevalent.

The debate over SIEM's demise is a distraction

SIEM's supposed magical life is unlikely to suddenly end for another decade because of the time it takes for the momentum for the industry's largest investment to fade. Despite this impressive market life expectancy, former SIEM vendors are clamoring to pile on and replace the dead SIEM 1.0 with their SIEM 2.0, or even SIEM 3.0. But the only reason these statements have made such an impact is the wide-ranging expectations of those engaged. Teams properly equipped to take the blank slate that often qualifies as SIEM technology and build an effective incident handling process on top of it realize its value. The other 95% of the security teams who invest the majority of their annual budget in simply deploying a SIEM continue turning it into shelfware to dust off whenever a high-risk audit appears in the calendar.

Unlike Jumanji, where a single toy automagically made jungles and creatures appear, SIEM solutions generate magic not through the technology, but through the people customizing and using it. I'll skip the obvious Harry Potter reference here to avoid insulting any SIEM engineers who don't like fantasy films, but if you cannot afford to hire a team of these wizards, you are doomed to carry on with the same low ROI as the other organizations out there that receive hundreds of thousands of alerts per day and day-long efforts to construct all of the complex queries necessary to answer enough questions to close an incident investigation.

These important problems are not close to disintegrating

If you ask the wrong people, SIEM solved security in 2006. Realistically, though, it solved a lot of the leading challenges  security and IT faced at the time by putting the historical data in one place. Centralization of logs made it possible for IT, networking, and security teams to address their biggest problem: accessing the data from their most critical servers and networking devices in a single place to troubleshoot and identify any issues quickly. Never before could all teams who need to investigate outages, software errors, and security incidents and even those who needed to answer every auditors' questions all go to a single place to do so within just a couple of days of digging through the data.

However, despite having done more to address these problems than any preceding products, the biggest reason SIEM's magic death is being so heavily discussed is its failure to consistently solve the most challenging of them: all cyber compromised detection needs in a “single pane of glass”. Again, the wizards out there have to manage, with the help of data scientists joining the team, to build some very impressive SIEM-based detection cores for their incident response armies, but the vast majority of organizations bounce from an IPS management console to a SIEM solution for some more details before obtaining more details from an endpoint forensics solution and pushing them through an unstructured data search solution to answer the final questions necessary to respond appropriately to the incident in question.

SIEM is rapidly losing trust because of its inability to adapt and aforementioned dependence on internal experts

There is absolutely a wealth of valuable data in even the least-used SIEM solution deployed today, but these largely outdated software solutions were designed to operate in an environment significantly different from today's. When organizations ran all third-party software and internally developed software on physical servers they either housed inside their offices or hosted in a rented space, there was an illusion of control because of the option to pull the physical power plug from the systems. However, thanks to companies like Google and Amazon, no successful company today still operates in that version of reality. Ten-year old solutions should not be expected to efficiently adapt to the modern need for elastic cloud computing and constant access to the corporate network via mobile devices.

Just as we learned from Days of Thunder, even the most effective driver (or incident responder) cannot perform with technology meant for a different environment. When Cole Trickle first moved from open-wheel vehicles to stock cars, he had to swallow his pride and let Harry teach him how to use the phenomenal car he'd built for him. Now, I'm no stock car racing expert [I'm not even a passing fan], but I do know I'm not insulting SIEM solutions with this analogy. You cannot just make some minor modifications to a stock car and compete in an open-wheel race, just as you cannot simply start piping data from your cloud and mobile management systems into your pre-existing SIEM server(s). You can invest in more hardware, hire a larger team, and even build a complex ETL pipeline to get your data into more modern data stores leveraging Hadoop [buzzword!], but the truth is, as you fall victim to this sunk cost fallacy, you'll spend insanely large sums of money trying to realize the value you could get by other means for a fraction of the cost.

Modern flexible solutions built for the security problems are needed to address these challenges

If you want to get back to addressing the problems SIEM was meant to solve, but do so in the modern landscape, you need to switch to search and analytics solutions built solely to address these very problems of one place to search all data, detect and investigate attacks in this very environment in which we live today. You could even reduce the time your team regularly spends maintaining the hardware and custom software enough to focus more on the day-to-day questions they need to answer.

Are you building in-house software? That's a facetious question. Everyone is. If you want to deploy a solution now designed to adapt as technology evolves, you need it to be immensely flexible to the machine data it ingests. If the goal is simple log management and search, you don't need an overly complex solution bolted onto an existing SIEM. You need flexible machine data search.

If your goal is simplifying how your team automates the alerts from your many complex detection technologies so you can spend more and more time hunting for new threats, you need behavioral analysis designed for the continually growing data generated by the people and systems in your organization. If you need to analyze incidents across your endpoints and managed cloud environments, you need an analytics solution designed solely for today's incident response teams.

If the problems described here are similar to yours, Rapid7 has a number of Incident Detection & Response solutions that you can read about here. They are continuing to adapt as technology and the attackers do.