Kris Raney
Distinguished Engineer

Bro IDS (post 2 in a series)

August 23, 2018 by Kris Raney

In my previous post in this series, I laid out my plan to enable Threat Hunting in a scalable way for a cloud environment by integrating Bro IDS with CloudLens, hosted on Kubernetes, with Elasticsearch and Kibana as the user interface.

Before I get into the details, let me give a brief overview of Bro IDS and why I selected it for this project.

Comparison to other IDS

Bro calls itself an Intrusion Detection System, however it’s different than commonly known IDSes. Most IDSes look for signatures of known exploits in traffic. A typical IDS doesn’t really have an understanding of the traffic, it’s basically just watching for a sequence of bytes in a network packet that matches a sequence of bytes from a known exploit. When it sees something, it alerts.

Bro can do that, but it does a lot more. Bro has an ability to parse network packets to understand what’s going on inside. It uses this awareness to log metadata information about what happens. Over time by collecting these logs, you build up an information base about what has been happening in your network. It’s this information base that fuels the Threat Hunting process. That’s where you do your hunting; you dig around in this information looking for suspicious activities.

Other options

Of course, you can build up an information source from other sources besides Bro. For example, VPC flow logs for your cloud infrastructure, Netflow information from your datacenter, and output from other analysis tools. The advantage of generating data from Bro is, all of that info kind of looks the same. It’s normalized. And also, each network flow gets an ID, and info related to that flow carries the same ID. So it’s easier to correlate one type of event to another when they’re triggered by the same behavior on the network.

Feedback loop

The other advantage of Bro is that the same tool you are using to fuel your threat hunting can also do alerting. So using scripts very similar to the ones you write to collect metadata, you can set up alerts. That makes it easier to close the feedback loop, alerting on activity you’ve decided is suspicious, so you catch it early next time.

Threat Hunting Feedback Loop

That’s the goal with threat hunting, to create this full cycle with a feedback loop. You find anomolies through research, identify which anomalies are truly threats, then use that knowledge to detect those threats automatically. Those detections become more information that becomes part of your data. Bro helps at all steps.

The challenge

So at a high level, Bro seems great. But I found a couple of challenges in applying it to cloud infrastructure.

  • It doesn’t have a native way to capture packets in the cloud
  • It has its own concept of clustering, which kind of overlaps and conflicts with the Kubernetes-based clustering that I want.

That’s where this project comes in.

What’s Next

Having settled on Bro to handle the threat analysis in my solution, in the next post of the series I’ll talk about how CloudLens feeds it the network data it needs to do its analysis. I’ll follow that up later with how Kubernetes provides a scalable, robust hosting environment, and how Elasticsearch with Kibana makes the data easier to search and interact with. Finally I’ll get into the details of how it all fits together and how you can replicate it in your environment.