Time
1 hour 24 minutes
Difficulty
Beginner
CEU/CPE
1

Video Description

In this segment we take a closer look at Zeek logs. We review some common naming and usage conventions and discuss how log structure is determined. We also take a close look at a few of Zeek's most popular native logs.

Video Transcription

00:00
In this next section, I'll discuss pros. Primary output Below
00:08
Rose Long Zehr, one of its most commonly sought after attributes
00:11
they provide for both descriptions of network communications that required collection of other tools to accomplish without
00:18
Bro's primary. Long file for recording network communications is the con law
00:22
entries in the con log or similar and concept to a net flow record.
00:26
However, condo records describe the originator and responders sides of communication on a single line.
00:32
This bidirectional flow format enables analysis of both sides of the communication
00:37
without having emerge and compare separate flow records
00:41
ago. Willbros Protocol Log files. The Con Law contains the You would Field, which is a unique identifier assigned to a connection. When Bro begins tracking,
00:51
the U. N can be used to quickly associate the Con Bog entry with other entries in Bro's other log files, like the Notice Law,
00:58
the HDP log
01:00
and other protocol related gloves. In the weird law,
01:03
TCP connections air reused in HDP. In situations like this, you will often see a single con log entry corresponding to multiple entries in other log files.
01:15
Most of Bro's protocol analyzers produce their own Longstreet.
01:19
There's several special purpose logs like DPD, the dynamic protocol detection log
01:26
and the notice in Weird Log that contain entries related to suspicious or unexpected activity in the environment.
01:33
Bro Natively supports two different log output formats. Jason
01:37
and its default tab delimited format.
01:40
Well, there are other output available through the use of plug ins on customizes, uncustomary ized instances of bro. We use the tab delimited format to write to local long files.
01:53
Most of Bro's protocol analyzers create their own log streets.
01:59
There they write log entries that describe client and server sides of the communications That bro is monitored.
02:05
The module that declares the low extreme typically defines a module named space of the same name.
02:10
So, for example, the http module creates and writes love entries to the http log stream
02:19
the contents of the log stream a determined by that modules Info record
02:23
Info record
02:25
isn't a built in type, but it's common practice for bro users and broke programmers to define a record type with the name info for use with the logging framework.
02:37
Deals that are meant to be included in the output are declared in the Info record with the log attributes
02:44
this is easy to overlook,
02:46
and it's a really good thing to double check and verify when you're adding a new field to an existing Longstreet
02:53
in the upcoming slides will cover these info records in more detail. As we review a few sample log entries
03:02
for the remainder of this section, I'm gonna be referring to live log files generated by Bro. These files reside on the linens virtual machine, where I've got open source bro installed.
03:12
As you can see from this directory listing, I have a bunch of different log files available. But for the purposes of this discussion, we're gonna focus on three logs that have been written out in a Jason format,
03:22
namely, Con de NS and http.
03:25
Jason has become a widely adopted data format, and many external tools are available to read and analyze it.
03:30
It's a great format if you wanna work with Bro Date and other programming languages. It's also useful for demonstrations and training because field names and their values are all in one place.
03:40
The downside is that Jason is considerably larger than its tab delimited counterpart because there's so much repeated information.
03:49
For instance, the Jason version of the con log is over three times the size of the tab delimited version.
03:55
I wanted that mention that when I'm doing here is duplicate e each of these log streams and under heavy load this control up a lot of extra storage
04:03
writing logs and both formats is useful for testing.
04:06
You can have major storage implications, so I wouldn't recommend doing this in production in case it's helpful. I've provided a link to the in the resource is section to the script I'm using. That creates a Jason version of these certain long files.
04:19
The first of Bro's native laws I'm going to discuss is the con log. To do that, I'll grab the most recent entry in the conduct Jason Log file and pipe it to the tool function in Pythons. Jason Library
04:30
This tool function read his single Jason object from Standard in and prints it out, and this more readable multi line format
04:38
similar concept to a network flow record. Entries in the con law can describe TCP, UDP or ICMP communications.
04:46
This is one of grows more powerful concepts.
04:47
All I P based network communications are tracked as a connection.
04:53
It's also important to note that bro determines the orientation of a connection based on what it views as the originator and the responders. In the context of that communication,
05:01
this is an important distinction from other network monitoring tools that refer to a source and destination.
05:06
Bro uses an originator, a ridge and responder rest.
05:14
The contents of the con log are defined by the Khan Info record, which is specified by Bro's Con Analyzer.
05:19
As you can see in this con log entry, Bro's Content for Record describes this flow by directionally that is, it contains information about the request, originator and responder
05:30
like a flow record. The con entry contains the standard five couple fields originator I P. Import Responder I P Import and Transport Layer Protocol, TCP, UDP or ICMP.
05:44
The five top ally P import information appear here in the I. D fields and the protocol in the proto field.
05:49
An important thing to note about the I. D. Fields is that the Field I D and the Con Info record is actually in itself a record
05:59
and Jason and ask a tab delimited formats.
06:01
The's some records in their fields can be identified by the period in the field name.
06:06
As you can continue to look around in the con log noticed the packets bites, duration and history fields.
06:14
Each of these have their own significance in value and are worth learning more about.
06:17
For example, you could use the duration field to look for abnormally high values that might be indicative of unauthorized or subversive
06:26
persistent connections in the environment.
06:29
The bites transferred fields could be used to detect abnormal transfers based on the size, rate and direction of data flow between to host involved in a communication.
06:39
These are just too simple examples of the many analysis techniques available using the field and values found in the condo.
06:46
The next low extreme will review was Bro's native D. N s Slow.
06:48
The D. N S Law contains records of Deanna's protocol usage in the environment, including D. N s Over TCP and UDP.
06:56
Deena's log is also buy a directional in nature. The client's query and the servers response are all included in a single line.
07:03
This is really helpful when reviewing and analyzing Deanna's traffic,
07:08
and it's even more robust in some of the logging you received from certain DNA servers.
07:13
Two of the more notable fields in the D. N s log are the query and answers field.
07:18
These fields are useful in many ways and incident response from looking for activity to malicious domains,
07:24
spotting domain generation algorithms and even identifying the NS tunneling.
07:30
This law also includes other Deena's Protocol headers like
07:33
authoritative Answer truncation, Riker Shin desired Rikers in Allowed
07:40
and Z the count bit.
07:42
These fields support a variety of analysis techniques. For example, one can monitor Deanna server responses
07:47
for the R A flag to be set to true
07:50
and then identify use that to identify and track recursive result vers in use in the environment.
07:58
As with many things and bro, the D. N s law isn't the only place you can find the NS related information.
08:03
So now some malware uses UDP communications over Port 53. That isn't actually D N s.
08:09
And you can find evidence of those things and other RFC noncompliance findings in bro's weird law.
08:16
For another example, let's take a look at the http low
08:20
like the D. N s log entries in the http log represent the client and service side of a request
08:26
all in a single line.
08:28
Http is a TCP protocol, so they're actually isn't an underlying TCP communication that Bro is tracking separately.
08:35
In fact, TCP socket reuses allowed and very common in HD d. P.
08:41
So it's a normal thing to see many
08:43
HDP log entries that relate to a single con log entry.
08:50
The method host in your eye fields could be used to reconstruct how the world was requested.
08:54
User Agent Field contains information that describes the browser operating system and plug ins used by the client.
09:01
If the request was a result of a redirection, that refer field will contain the euro. The referring Web resource
09:07
http log is a great date. It's worse for ongoing analysis techniques like hunting for suspicious user agents
09:15
or abnormal method usage.
09:16
It can also be the source of additional context and Intel and incident response situations.
09:22
For example, if a compromise occurred as a result of a website redirection
09:26
in the case of Mount Malbert, izing
09:28
and analysts can use the http log to trace the full Web session by following the sequence of referring and requested you or else
09:37
this can lead to the discovery of several other malicious domains and you or else
09:41
in this segment, we took a closer look at Bro's locks. I describe some of the comic conventions related to bro logs like the Log file Name the module Name in the Info record name.
09:52
The Info record used to create the log stream determines the field that will contain each record. Instance written to a log file. Is that modules? Info tight? In other words, bro will fail if you try to write a con record instance to the D. N s Low.
10:05
After I covered log naming convention and structure, we took an in depth look at Bro's Con de NS and http logs. In the next segment, I'll describe several log management tasks and show you a few live script examples to illustrate how those task can be accomplished.

Up Next

Intro to Zeek Scripting with Bricata

The goal of this course is to provide you with an introduction to Zeek (formerly Bro) the application and the programming language. While the logs Zeek produces natively can be extremely useful, its full value is realized through its scripting interface.

Instructed By

Instructor Profile Image
Bricata
Instructor
Instructor Profile Image
Adam Pumphrey
CEO and Principal Consultant at Nimbus LLC
Instructor