Web Server Logs Review Part 1

00:00

Hello, everyone. And welcome back to the course, identifying Web attacks through logs.

00:05

In the last video we talked about http and TCP I p

00:10

In this video, we'll start talking about Web server logs.

00:13

The learning objectives of this video are to understand the importance of logs,

00:17

understand the Web Server, Log Information

00:20

Review, Apache and G. I, X and I. I s log structures and perform some initial log analysis.

00:27

This is, of course, about logs, But

00:29

what are logs by our logs so important?

00:32

One of the definitions of a log is a piece of wood, but that definition doesn't really make sense in our context.

00:39

In our course, we will use the second definition that says that a log is a full written record of an event.

00:45

The Web server will be in charge of generating this lock.

00:49

Now we know the definition of a log, but we still need to know why they're so important.

00:55

Suppose your sock analyst and you receive a call from a user saying that their computers acting weird.

01:00

If you don't have the logs, you need to go to the user's computer and check it manually. And it might be too late.

01:07

But if you have the logs, you will be able to tell the user that everything is okay because you've checked the logs. And they say that anti malware has removed the malicious software.

01:18

In some cases, you could have the Web access log itself. Tell the user that they visited a malicious website and have been infected.

01:25

The more logs you have, the more information you'll have.

01:27

That's why logs are so important.

01:32

Also, the logs will help you to understand your infrastructure and applications so it can monitor them.

01:38

It will also help you in things like trouble. Trouble shooting

01:44

logs are so important that oh wasps top 10 projects included lack of logging as a vulnerability on the 2017 version.

01:53

If you don't know what a wasp is, don't worry. We'll talk about it later.

01:57

Logs can help you a lot

02:00

if you work or are planning to work as a sock analyst,

02:02

it's important to know what you need to protect because you can't protect what you don't know you have.

02:09

Since the log is a record, we need to store it.

02:13

There are two basic ways to do this

02:15

locally and remotely.

02:16

locally speaking, you'd save all the logs in the same place they were generated.

02:22

Remote storage is used when you need to send the log to another place. Maybe because you can't store it locally or you just want another copy.

02:29

Remote logging is useful because you can have other logs in the same place.

02:34

If you have many servers, it will help you to have all the logs in the same place.

02:39

Now let's start talking about Web applications or Web server logs.

02:46

Most of the time, we can split the information by answering a couple of questions.

02:51

Who did the action?

02:52

When was the action performed? And what was the action?

02:57

Most of the attacks will be detected here.

02:59

This is a really simple Web server lock

03:01

we can easily find in the log. What happened?

03:05

The I P address in user name is the Who

03:07

Date, and time is the When, and the request is the what

03:15

now?

03:15

One more example.

03:17

It's important to know that empty fields aren't allowed.

03:21

It's normal. It's your hyphen.

03:23

The hyphen means no information for that field.

03:25

Let's analyze our log. Example.

03:28

First we have the I P address

03:31

the next field is related. Toe RFC 1413

03:35

But this field isn't really common on the logs.

03:38

It depends on the Web server and Web application.

03:42

The next field is the user ID, followed by date and time.

03:47

The date and time format depends on the configuration of the Web server logging

03:52

The next field. Contains main information http method used by the client.

03:57

The file requested by the client and the http version

04:00

After we have the status code, the sizing bites of the answer,

04:03

the refer and the user agent.

04:08

Just to note this was an Apache Web server log example.

04:13

Okay,

04:13

now that I know the log fields

04:15

well, what can I do with them?

04:16

We always need to answer the three questions. Who, when and what?

04:21

The log will help us with this.

04:26

Let's check what information each field can tell us.

04:30

Source. I P and User I d would say who

04:31

date and time when, and all the others will explain what happened.

04:35

Just to clarify here is the info for each field

04:41

logs air used to rebuild the users behavior or actions

04:44

This log would say to us, the user with i p 10.3 point 89.4 was on the log in Web page and access Another Web page app Any dot html.

04:55

The user sent a get using Mozilla Firefox in June, close to 7 p.m. And the Web server answered with a 200.

05:01

So no errors.

05:02

RFC 1413 would say the user who made the request is asking for help identifying who did the action and when it's used. It really looks like user I d. Information.

05:14

As we said before,

05:15

it's hard to see a Web server using this field.

05:20

Here are some more examples.

05:24

Take some time and analyze these Web server logs.

05:29

First. We have this source I p address

05:31

after we have the date and time,

05:34

followed by the user request

05:36

and the user agent.

05:39

You can see in this last line that almost all the log fields are important and useful during the analysis.

05:45

Now

05:46

the only thing missing is the answer to the questions.

05:48

Who, when and what

05:51

Here we have the who

05:55

we only have the information about the clients i p address.

05:59

The date is the answer of the when question

06:01

all the other information is the answer to the what question?

06:06

This slide shows a review of the important log fields in their descriptions.

06:11

It's important to understand the log fields because they will help you to analyze Web server logs,

06:15

spend some time and take notes. If you want

06:17

a good place to get more information is on the Apache Web site.