Let’s start with the non-obvious truth— the problem with stopping data breaches is not about detection. Modern security systems detect a lot. They detect too much; according to a study by IT security firm Bricata, the average Security Operations Center (SOC) receives over 10,000 alerts each day from an array of monitoring and detection products. So clearly, not enough detection is not the issue.
The fact is that the problem lies elsewhere—it’s not detection; it’s correlation.
Detection is easy, but what comes next?
Detection, at its simplest form, is an alert. A user performs an activity, and a log is generated. Modern cloud security systems are built to detect. An average SOC can generate between 5,000 – 40,000 alerts a day (even more for large networks).
The problem with detection, however, is that looking at each alert tells you almost nothing. Nearly every log can either be legitimate or illegitimate, depending on the context it is created.
Consider the following activities; looking at each one, could you tell which is a legitimate activity?
- A system administrator logging from an unusual location. Is it because they are working on something urgent from their vacation, or because hackers have stolen their credentials from Eastern Europe?
- A user is accessing the network outside of business hours. Is that because they have an emergency at work, or are these hackers trying to go unnoticed?
- A DevOps engineer invoking an API call they have never used before for the first time. Are they rolling out a new product version, or is it a hacker attempting lateral expansion within your network?
- A database administrator is accessing a cloud-based storage bucket and exporting all the data from it. Is it part of their job, or has someone just stolen your entire user database?
The fact is that looking at a single alert – on its own – cannot reveal the intent behind it.
Attackers fly under the radar
In our own experience analyzing data breaches, we learned that in most cases, the malicious activities were indeed identified in time, but they flew under the radar.
Why this happens:
- Log overload: security managers are flooded with so many alerts that they don’t have time to analyze many of them. As a result, important events get lost in the noise.
- Low-risk alerts: many activities that make up a data breach are not high-risk, high-impact, but rather mundane actions with a low-risk assigned to them. As a result, they are frequently overlooked.
- Lack of context: looking at each activity independently of other activities does not reveal its intent.
- Stretching over time: data breach incidents can take weeks and sometimes months to unfold. Logs come in at a high volume daily, making it impossible to remember another alert several weeks ago and associate individual activities.
As a result of these realities of day-to-day security management, any means of manually analyzing alerts and putting them in context to identify malicious activity is doomed to fail.
Detection is important, but correlation is critical
The key, therefore, is not in more detection but correlation.
Correlation is the process of taking independent, seemingly unrelated events and correlating them across threat surfaces, resources, and time frames.
Think back to the list of example activities we listed above. On its own, each event was meaningless; we could not discern the intent behind it.
But consider the following chain of events:
- A user connects from a remote location at an unusual time outside of business hours.
- A few days later, the same user invokes for the first time an API call to list all privileges of the user.
- Over a few weeks, the user performs a series of connections to multiple storage buckets holding sensitive information.
- The user downloads data from a storage bucket to a location outside of the network.
Looking at these events in a linked chain of events looks different from just analyzing each event individually. Therefore correlation is so important— it allows you to identify a data breach in its entirety, not just the individual events that are part of it.
Correlation is such a critical component of cybersecurity and can make the difference between stopping a breach in time or reading about it in the news.