Time
1 hour 24 minutes
Difficulty
Beginner
CEU/CPE
1

Video Description

The segments makes introductions and provides some useful background information about Bricata, Zeek and it's open-source project and community. We also cover some of Zeek's common uses and discuss how it scales in large deployments through clustering.

Video Transcription

00:01
Hi.
00:02
Welcome to an intro to Bro Scripted.
00:05
My name is Adam Pumphrey. I'm the director of Threat research. Apricot
00:09
Apricot. I get to do a lot of very interesting work with bro and the network data it produces.
00:14
I'm excited to get to share some of what I've learned with you today.
00:18
I come from a network defense background and learn to work with bro out of necessity.
00:22
I wasn't a programmer, researcher, data scientists. My task was the defendant Enterprise Network, Environment and I looked at bro is one of many tools in my tool set.
00:32
Having had to learn, bro from the ground up with no real programming knowledge,
00:36
I'd like to try and help others like me grasp the foundational concepts to get up to speed quickly. With bro,
00:42
Kolkata is a sponsor. An avid supporter of the Bro Project,
00:46
the Takata Platform Employees bro as a best of breed traffic analysis engine and exposes its powerful scripting language to the user through an intuitive script editor and policy management interface.
00:57
The majority of this training is applicable to any bro deployment.
01:00
But in later segments are focus will turn operating a bro deployment via the Bra cada Central Management Council.
01:08
While awareness of bro in academia research in network defense communities is strong,
01:14
many who are new to Bro aren't coming from a background loaded with computer science and programming training
01:19
their analysts and network defenders with a job to do. And this talk aims to help bridge that gap and make some of the more powerful features of bro more readily available to those that have a specific operational goal in mind.
01:32
The main goal of this course is to provide you with an introduction to Bro, the application and the programming language
01:38
while the logs broke produces natively can be extremely useful in many ways and network defense
01:42
bruise full value is realized through it scripting interface.
01:48
First, it helps to have a solid understanding of what Bro does.
01:51
After describing how and where bro runs, how it receives packets and scales through its cluster mode,
01:57
I'll describe Bro's primary output. The log file.
02:00
I'll discuss the naming convention of roll logs, how their structure is determined and review the contents of several of rose, more prominent log files.
02:09
Next, I'll describe some common bro log management task and new script examples to illustrate how they could be accomplished.
02:15
This will serve as an initial introduction to the scripting language in the Bro. In that event,
02:22
to build on this foundation will then take a closer look at Bro's events.
02:25
I'll describe the event lifecycle event handlers and several built in events that are key to understanding how bro works with traffic data and how you can interface with that data using bro script.
02:37
Finally, I'll discuss some vital concepts of the scripting language in more detail.
02:40
I'll describe Rose various types and provide several examples of how and why they're used in normal bro operations.
02:47
This bro primer will leave you with a salad understanding of Bro's core concepts
02:53
and a Familiarization with the language, and hopefully plant the seed for some ideas that you might have for how you can use bro in your environment.
03:02
Rose Open Source Project is run out of the National Center for Supercomputing Applications in C S. A.
03:08
In its original birthplace, the International Computer Science Institute, I. C s I.
03:14
The project team leads Bro's Open towards Development, is primarily responsible for maintaining its code base and documentation
03:21
through its mailing list. IRC channel in various meet ups and conferences. Bro's community continues to grow.
03:27
There's no shortage of places to look for more information or help,
03:30
including a fully stocked documentation website and a YouTube channel containing many presentation from previous broken conferences.
03:38
Since its origin, Bro has been operationally relied upon in scientific environments and for securing university networks, research labs, supercomputing centers and open science communities. In recent years, the operational adoption of Bro spread into the private and public sectors
03:53
and his seeing growing attention from larger cyber security organizations around the world.
04:00
When Vern Paxson originally wrote Bro in 1995 he was working on a research project in which he needed a better way to monitor communications.
04:09
The strategy senator on this sort of log everything approach that produced well structured data for later use. An analysis
04:15
the next year in 1996
04:17
bro's transition in the operational use in development at the Lawrence Berkeley National Lab
04:23
and has since been adopted for operational use in many organizations around the world.
04:28
As you can see, Bro has been around for a long time and has a history of growth and enhancement.
04:32
A little known fact, bro, actually predates Snort, which was originally authored by Marty Roche in 1998.
04:40
In 2003 the National Science Foundation began funding bro related research at the International Computer Science Institute,
04:46
who ran the project exclusively until partnered with the National Center for Supercomputing Applications in 2010.
04:55
As a result, the software has been active inactive development for over 23 years
05:00
in the tool Second, automate and Enhance announces task that would otherwise be extremely difficult or impossible.
05:05
The software's continue to enjoy rapid enhancements over the past 10 years, including the addition of multi threading and broke control for cluster management in 2009.
05:15
In more recent years, we've seen the addition of the files and Intel frameworks, new protocol analyzers and major improvements to broke court
05:29
broke or is an open source application written C plus plus. It runs on many operating systems, and the website includes instructions for installing from source on Lenox, UNIX and Mac OSX.
05:40
You have to resolve a few dependencies before a piling.
05:44
Depending on your environment, it takes about 30 minutes to complete
05:47
rose. Primary purpose is to ingest network traffic in the form of Raul packet capture directly from a monitoring interface or frumpy cap files
05:56
as those packets or Red Bro dissect them in terms of them in the higher level events,
06:00
ultimately producing a comprehensive set of laws that describe the traffic and ingested
06:05
bro script. Another major component to Bro
06:09
is a powerful domain specific language built specifically for working with network traffic data.
06:15
As Bro is processing,
06:16
2009 marked the addition of Broken Troll. 1.5 is shown by the previous slide.
06:23
This is when the notion of broke clustering was initially introduced.
06:28
Now, Bro clusters are used almost exclusively in operational deployments as they provide inbuilt support for large traffic volumes through load balancing in ST sharing between bro worker processes.
06:42
Why should you use bro versus one of the many commercial products or combinations of net flow? Another open source software like Sakata, Snort and Argus.
06:50
All of these tools provide great features and functionality and overlap in some way with what broken do. So why Bro?
06:58
Bro Natively produces for boast network transaction logs that provide you with unparalleled visibility in the network communications
07:04
without being affected by policies of good and bad or authorized or unauthorized
07:11
and it's based configuration, Bro, is just about collecting the data.
07:15
For those that are responsible for network defensive forensics, the contacts and value
07:18
Rose law dating provides is immediately apparent.
07:23
Need to describe a host interaction or playback a record on how who's got compromised. Just look at the logs.
07:30
Bro's programming language also provides a way for you to explore a variety of analysis methods.
07:34
It allows you to create a new technique for finding suspicious network traffic,
07:39
then test refine it, increase its accuracy and add functionality.
07:44
Working to this process allows you to solve hard monitoring problems with reliable, manageable solutions.
07:49
This could make bro feel like a Swiss Army knife of sorts and invaluable tool in incident response and network forensics.
07:58
Rose frameworks like Files and Intel enhance core Bro's functionality
08:03
files allows Rhoda extracted, analyzed files transmitted across the network.
08:07
The Intel framework allows users to supply several types of intelligence indicators and hits or log separately for monitoring and triage.
08:16
Both great capabilities.
08:18
Rose Protocol analyzers its ability to accept input that adds meditated to network communications
08:24
and its programming constructs for describing network host behaviors make advanced analysis techniques like profiling and anomaly detection
08:33
accessible and available as tools you can use in the day to day effort to defend the network.
08:41
This training has put together with the intent to be relevant to bro running on any platform.
08:46
The focus of these earlier segments is to help you get more comfortable with broken and scripting language.
08:52
Later, our focus will shift to applying these foundational concepts to operating and distributed Takada deployment that includes Custom Bro Scripts and Brick Autumn provided Bro modules.
09:03
There are differences among pro implementations and requirements and complexity, also very with scale.
09:09
Brucato works to eliminate much of this concern by integrating bro into ah, highly customized solution stack that runs on a single appliance.
09:16
But doing so, Brucato eliminates hours of research, design and deployment effort that let you simply focus on the important tasks like defending the network.
09:26
As a commercial solution provider, Takada offers full support for its products. That Takada Network censor is purpose built from the ground up to provide a comprehensive set of capabilities under heavy load in normal network conditions,
09:39
Brucato works with customers to ensure their installation meets this goal.
09:43
In contrast, open source roll your own Bro Deployments. Takata focuses the user's attention on the development and management bro content,
09:52
not the administration of Bro processes.
09:56
Users are presented with a simple to use interface. A custom script editor that does real time code validation
10:03
and the ability to sexually manage policies in ST content,
10:05
all performed automatically and silently by the Central Management Council.
10:13
It's helpful to think of rose two main components. The event engine in the script. Interpreter. The event engine is responsible for turning packets into higher level events, and the script interpreter is responsible for executing script commands and working with the data contained in those events.
10:28
The script interpreter has the ability to consume external inputs that just started threat intelligence.
10:33
This is done via that the threat Intel framework or other information that describes sub nets i p addresses, applications and service is which is all handled by Bro's Input framework.
10:43
The input framework Reed's data files in the memory
10:46
typically stores those in a table and then makes that information via the table available for other processing
10:54
as packets. Under the event engine, low level analyzers begin and dissect them layer by layer
10:58
when the engine is finished, evaluating a Layer four header, Bro checks to see if the pack it belongs to a new session or one it already knows about
11:07
events air then triggered with their specified arguments.
11:09
The script interpreter consumes these events by way of event handlers,
11:15
event handlers attached to an event Q and A priority, and do some work with the data containing an event as it passes through the Q
11:20
bro, scripts
11:22
can at this point perform a variety of task on the data, including heuristic analysis, enrichment, statistics, gathering and pattern matching.
11:31
Finally, the last step, Bro, produces output
11:35
natively. This includes asking Logan a compact tab delimited format.
11:39
Bro creates separate long stream for each of its protocol analyzers, along with some of the special purpose logs
11:45
that could be used for monitoring health and behavior in the environment.
11:48
In addition,
11:50
broken be set up to extract files to disk or potentially elsewhere with a little bit of customization.
11:56
This is a powerful feature and not something that many other monitoring tools can offer.
12:01
Bro's aware of hundreds of file types and capable of extracting and hashing files and transit, which allows operators to identify file born threats in near real time
12:09
by enabling automated submission to malware analysis, sandboxes, malware databases and registries like virus total or four offline analysis.
12:22
The manager performs administrative tasks across the cluster, coordinating rolls and managing policy assignment.
12:28
The manager note is often used for collecting information from multiple worker processes and doing some correlation or longer term statistical analysis of that data.
12:37
For this reason, it's good to be aware that Bro processes can subscribe to events that are generated by other processes
12:43
proxy nodes past state information between other cluster notes.
12:48
Most importantly, the worker notes, who would otherwise be aware unaware of traffic. The other workers had analyst
12:54
This state sharing is critical to bro scalability. In particular, one captured network traffic is being load balanced across worker processes.
13:03
Worker process is handled, the bulk of the traffic analysis and or where the event engine is converting package. In the high level events,
13:09
workers are bound with specific monitoring interface, which could be physical virtual in the form of a bonded interface or buttered interfaces or colonel bypass received queuing systems like 1/2 packet RPF ring,
13:22
bro analyzers trigger log events as connections and application layer protocol sessions air created, used in terminated.
13:30
The longer node subscribes to those log events and handles all of the writing of the log output. The isolation of the logging function to a single dedicated node frees up the manager node for other analysis, task and help Single system bro deployment Scaled accommodates has sustained traffic rates nearing 10 gigabits per second.
13:48
Ricotta operates a slightly modified instance of bro, where the logger nodes right there output the Fife oh cues not directly to disk
13:54
from their break. Otto's event processing engine captures and riches and delivers the log event, the export process
14:01
or directly to the CMC, depending on the configuration.
14:05
To recap in this segment, I introduced myself and Bo Kata and laid out the purpose of the course in particular What we hope you get out of it.
14:13
I provided some useful background information about pros, origin, its history and the continued success of the open source project and community.
14:22
If you're interested in signing up for the mailing list or IRC channel, provide a link to more information about that. In the resource is section of the course
14:31
in this section. I also described Bro, the software. What it does and provided a few examples of why it's used in operational deployments.
14:39
Rosa powerful but complex application, and I also mentioned some some of the ways Brucato eases that pain and enables the user to focus on more important tasks than managing bro processes
14:50
in this segment. We also reviewed Bro's packet processing flow
14:54
and described how broke clustering enable scalability
14:56
through the delegation of specific functional task to cluster notes.
15:01
In the next segment, I'll continue the discussion of the law. Bro's Primary Output.
15:07
I'll describe how Bro Logs restructured and managed, based on definitions specified in bro script.

Up Next

Intro to Zeek Scripting with Bricata

The goal of this course is to provide you with an introduction to Zeek (formerly Bro) the application and the programming language. While the logs Zeek produces natively can be extremely useful, its full value is realized through its scripting interface.

Instructed By

Instructor Profile Image
Bricata
Instructor
Instructor Profile Image
Adam Pumphrey
CEO and Principal Consultant at Nimbus LLC
Instructor