Time
6 hours 3 minutes
Difficulty
Intermediate
CEU/CPE
6

Video Transcription

00:00
bloom. Welcome back. The Splunk Enterprise Certified Administrator course on Cyber A. In this video, we're gonna be doing a lab where we talk about rerouting, basically indexed fields or default fields using props and transforms. So
00:16
what we have here to start with is example of some logs that we brought in in a previous lab. These logs here where they came from, those two Apache log files, we can ignore these ones. They came from a swap file that I had. I didn't mean to dio, but
00:34
basically the problem here is we can see that these logs have a clearly different log format from these ones. So technically, we don't want these logs to have the same source type they should have. They should each have their own. So in this lab, we're going to show you how you congest thes and then break them out.
00:53
Two different source type.
00:55
So we're gonna do that with props and transforms, and
01:00
we will just get started. We'll jump into that command line over here of the search head, which is the device we configure these inputs on on. The first thing that we need to do is, um
01:14
Well, first we're gonna have to stop Splunk,
01:21
and then we're gonna have to clean the fish buckets so that we can re ingest those two files
01:26
on. Then we will need to configure our props and transforms so that they'll do the source type, reroute for us, and then we need to start *** back up and make sure that I worked. So
01:41
we'll start by running our
01:45
queen event Data Index, The fish bucket.
01:49
So this is the command for cleaning the fish book it which will remove all those
01:56
markers so that it'll re ingest the monitored files. So that's good. Now we need Teoh. Go to our inputs dot com and we can see here that this is the stanza that setting the source type to sample one right now
02:12
we can leave that. But you can also just remove this entirely because we're going to do the reroute. Sometimes it's nice toe have this set. So if the reroute doesn't work, you can see Oh, data came in with this source type, so it didn't work. So we'll leave it for now and we will add up splitting into that,
02:31
we will add,
02:35
uh, transforms
02:38
and we're gonna apply This transforms to that sample one source type
02:45
and we're gonna have three things we need to set Will be the deskee, the format and the Reg. Exe
02:52
So,
02:53
deskee for this is going to be, um, the field that we're changing and in this case, where changing the source type
03:02
so we'll have to all need to go to the props dot com configuration actually transforms because there's a special syntax you need to use when you're re keying metadata field. So I want to make sure that I don't mess it up.
03:19
Just so happens that the last time I visited this I was on the same exact link.
03:23
So these air, though, values that you use to reference those keys. So this will be our deskee
03:34
in this example.
03:36
And then the format will be
03:38
source type colon colon, and then whatever the new value we want to make, it is. So let's look at the two logs that we had. I can't look in Splunk anymore.
03:50
Um, I think I have them open right here,
03:53
so or this is the This is the Apache access log.
03:59
So we're gonna use that
04:02
as the name. We'll just call it
04:05
Apache
04:06
Access and the Reg. Exe to match. This would be this one.
04:13
So we just going to say if it starts with three numbers
04:16
and it's this source type,
04:18
then
04:21
set the source type field to Apache access.
04:26
Then we need another one
04:29
for the other data. So we're going to copy this
04:34
paste
04:36
on changing for the new rejects. The other logs, we can see this here in the background, start with a square bracket.
04:46
So we will just specify that, and then we'll call this Apache error.
04:57
Okay, so that is all we need in terms of transforms. We just need a register, match the field. We want to change the value we want to put in there and the data to apply it to.
05:06
Actually, I'm sorry. That's wrong.
05:11
This needs to be called
05:14
Apache Error.
05:16
Were you out
05:17
and I'm gonna change this Teoh cam. Okay, so it's a little more readable
05:24
in the props is where we'll specify
05:28
the source type that we want to apply this to here. It's just a nor veterinary name. That was my mistake.
05:33
Apache access for your out.
05:38
So we're gonna need Teoh use those to exact names,
05:42
so I'm going to copy them out
05:44
so that
05:46
we don't have any errors.
05:49
Okay, so now that that's made, we make our props.
06:00
Um,
06:00
that was gonna be an example for something else, but we will do that later. So this is going to apply to sample one is the source type we're gonna do transforms because we're calling Some transforms from transforms dot com. So this is what denotes that. It's attribute for that.
06:20
So then we just given an arbitrary name. We'll call it
06:25
Apache Source type three rounds
06:32
and we'll set this equal. Teoh these two values separated by a comma.
06:39
And so this will call those to transform stanzas and should apply.
06:44
So then this will happen when we start up Splunk and in just that data and should reroute everything properly so we don't have to worry about using a generic source site that doesn't properly define our data.
06:58
So let's run our start command,
07:03
get back to Splunk
07:06
on the way. We're going to check to see if this works.
07:11
IHS search for those new values.
07:15
We'll do Apache star
07:21
and for Law. So
07:26
so far, we have our Apache access
07:30
as well as our Apache error, so you can see that actually did work. And we can also check to make sure that all of them got rerouted properly, which they did. Otherwise we'd see a value there.
07:42
So that covers, basically rerouting that data that way. Now, there's one other thing I wanted to him on their head on, which is, um
07:54
how did not index data altogether?
07:58
So the way we're going to do this is we're gonna re do our props and transforms to discard one of these source types.
08:07
Um, except we're not gonna specified by source type of unspecified my rejects. But so we're gonna discard one of these
08:15
and
08:16
demonstrated how that's done. So
08:20
let's get back into our logs. Will need to stop Splunk again.
08:26
Hopefully, this will be a pretty quick demonstration. We could just switch some of the fields we already have. Well, that's loading. I'll find, um
08:33
this is gonna be the key
08:35
that we need to chain. No, not that one.
08:45
It's gonna be que
08:50
You must specify. No que So that's the setting will need. We will go back and we will clean
08:58
that event at again, so that will be able to read index that data again.
09:01
We will go back to our transforms, and we'll call this
09:09
so we'll keep this one. And we'll keep those debt that looked at a named Apache access.
09:15
And we'll call this,
09:18
um, Apache error.
09:20
Um,
09:24
no, no. You just about that. And we're going to say if it matches this rejects than the desk, he will be,
09:33
um,
09:37
que
09:37
and format will be
09:43
no que And so this will tell it not to index the data.
09:46
And we just need to go back to our transforms
09:50
and
09:54
change it. So it reflects the new name.
10:00
Make sure
10:03
or a props meant change this so that
10:09
our value is appropriate.
10:16
Um, and you know what? Actually,
10:20
I'm gonna switch this to sample to just to be sure that
10:26
where it
10:26
let's go back to our inputs.
10:31
Call this sample to so all those configurations should be what we need.
10:39
And let's just run a restart.
10:43
Then we'll check it out and hopefully Ah, we'll see that this stata was logged
10:48
and this data was discarded.
11:05
Onley concern I could possibly see is we might see some of this old data still come up.
11:11
But if that is the case,
11:13
we'll see double the amount of events for our access logs. So this should dumped on 14 and this should remain the same.
11:24
So you can see that jumped to 14. And we still have Onley just this one log. So that means that data was successfully discarded and was not indexed. So that is how you basically filter out data. You could do it much more specifically, like you could look for events that are just junk,
11:43
like maybe this is just all hyphens, each of these spaces, and it just doesn't provide any data.
11:48
In those cases, you could write a rejects to drop that specifically, or, if it's like Windows, you could find specific of Ben codes that aren't good, etcetera, etcetera. But basically anything that matches a Rogic's pattern, you can choose to drop just to save on licensing.
12:03
But that is how you would change an index field. If you're using a different one, just make sure to get the proper value.
12:11
Uh, and make sure that you place this either on a heavy Florida or in indexer wherever parsing is first taking place because that's where this will be applied. But that's how you change your desk, your keys dynamically for your default fields, or choose not to
12:31
log
12:33
a K Index data.
12:35
So that wraps up this video, and I look forward to seeing you in the next one.

Up Next

Splunk Enterprise Certified Administrator

The course is designed around the guidelines provided in Splunk’s Test Blueprint for the Certified Administrator certification, Splunk Docs, the Splunk Data and System Admin courses, and the experience of a Splunk Professional Services Consultant.

Instructed By

Instructor Profile Image
Anthony Fecondo
Splunk Professional Service Consultant
Instructor