5.3 Alerts
Video Activity
Join over 3 million cybersecurity professionals advancing their career
Sign up with
Required fields are marked with an *
or
Already have an account? Sign In »
Video Transcription
00:00
>> In this video, we'll use
00:00
our Splunk search skills to create alerts.
00:00
Splunk alerts are based on saved searches.
00:00
When the conditions you've specified are met,
00:00
you can set an alert to trigger.
00:00
In this example,
00:00
we've got a search that looks on an index called
00:00
firewall for events where
00:00
threat name equals SQL injection.
00:00
Then organizes the results into a table with source IP,
00:00
destination IP and message.
00:00
We can save this search as an alert to run every hour
00:00
and send an email if an event
00:00
matches that search in the time frame.
00:00
In this other example,
00:00
we could have the search
00:00
continuously run and trigger an alert when
00:00
there have been more than five events that have
00:00
triggered for this in the last 10 minutes.
00:00
These saved searches for alerts
00:00
can be scheduled to run every hour,
00:00
every day, or on certain days.
00:00
You can also schedule them using cron expressions.
00:00
If you're unfamiliar with cron expressions,
00:00
I recommend using a site like
00:00
Crontab.guru to help you schedule it.
00:00
This will give you a lot of flexibility for
00:00
scheduling and help you to stagger your alerts.
00:00
It's a good idea to schedule alerts at
00:00
different times to spread out
00:00
the impact of running a search.
00:00
You can also do searches in real-time,
00:00
where they're continuously running.
00:00
Typically, it's better to do
00:00
even a really frequent scheduled search
00:00
rather than real-time searches.
00:00
You can set up various actions for an alert,
00:00
including multiple actions in
00:00
response to a triggered alert.
00:00
You can, for example,
00:00
send an email notification when an alert triggers.
00:00
You could also use webhooks to update a web resource,
00:00
write the results to a CSV file,
00:00
add alerts to a list of recently triggered alerts.
00:00
There are also a lot of actions that come with apps.
00:00
For example, a popular ticketing tool called
00:00
ServiceNow has an app that when you set it up,
00:00
it allows you to easily open a ticket.
00:00
Phantom, Splunk's new purchase,
00:00
is an automation and orchestration tool.
00:00
It can take events and kick-off
00:00
more complex actions in response to alerts.
00:00
To receive phone calls, texts notifications,
00:00
and push notifications,
00:00
you could use Splunk's other
00:00
recent acquisition, VictorOps.
00:00
There are also other good tools for
00:00
this that integrate with Splunk like PagerDuty.
00:00
With that, I think it's time to
00:00
go in and create a practice alert.
00:00
I've got two machines that
00:00
I'll be working with in this video.
00:00
I have my Splunk search head
00:00
hosted on this Linux machine,
00:00
and I have a Windows machine with
00:00
a universal forwarder installed.
00:00
The Windows machine will serve to
00:00
generate new events for us to alert on.
00:00
I'm looking at the search and
00:00
reporting app on my search head here.
00:00
I've searched for Windows event logs that have
00:00
an ID of 4625,
00:00
which is a code for an account failing to log on.
00:00
I've further specified that I want events where the field
00:00
for failure reason equals
00:00
unknown username or bad password,
00:00
and then I've run statistics
00:00
on the returned events where I'm getting a count of
00:00
events by computer name
00:00
and have included other values in the display.
00:00
Right now, I've got one event.
00:00
One event might be pretty reasonable in this environment,
00:00
but maybe I want alerted if there have been
00:00
more than three of these in the last 60 minutes.
00:00
I'm going to add a pipe here and say,
00:00
where count is greater than three,
00:00
and changes from verbose mode to
00:00
smart mode because I don't need the full event data.
00:00
I just want what's going to show up in these dots.
00:00
Nope, I might have left to go too long.
00:00
We're going to do last four hours.
00:00
Oh, I didn't leave it wrong.
00:00
If you remember, I said there's only one event there,
00:00
so something you can do
00:00
to check and make sure your search is working,
00:00
is lower the count you need.
00:00
I'm getting results when I lower the count.
00:00
If I want to change that back up to three,
00:00
I can save this as an alert.
00:00
I'm just going to go up here to Save As
00:00
and I'm going to call this authentication failures.
00:00
For description, I'll just say this is in progress.
00:00
I want to see how this alert is doing.
00:00
I could set this to run every week,
00:00
every hour, every day.
00:00
I'm going to put it on a Cron schedule so I
00:00
don't have to wait too long to get my results.
00:00
To do that, I'm just going to say,
00:00
looks like it's 54 right now.
00:00
I'm going to give myself a few minutes for this
00:00
to run and say I want this to
00:00
happen every hour at the 59th minute.
00:00
This is going to check the last 60 minutes.
00:00
The results of this alert will expire in 24 hours.
00:00
But that doesn't mean the data will disappear.
00:00
If I want to, I can go back and run this search
00:00
again and specify this time frame of today,
00:00
during these hours and then I can still get
00:00
the results but I would need to
00:00
rerun the search to have those.
00:00
I want this alert to trigger when
00:00
the number of results is greater than zero.
00:00
I've already specified in
00:00
the search that I want the count to be higher than three.
00:00
I only want one alert.
00:00
Then for alert actions, I'm going to go ahead and say,
00:00
add to trigger alerts with a severity of medium.
00:00
I'm also going to want to send an email.
00:00
I'll fill out an email to send it to here,
00:00
and leave that priority normal.
00:00
If you see this name with the dollar signs around it,
00:00
this is called a token and it's going to
00:00
fill in the name of this alert in here.
00:00
If I want to, I could add extra information.
00:00
Like, if this
00:00
triggers call Bob or whatever I want in there.
00:00
There's going to be a link to the alert and
00:00
the results in the body of the email.
00:00
But I'm also going to want to put the results in line.
00:00
It will show this that we had in the search back
00:00
here in the body of the email.
00:00
Also, say attach a PDF with the results and with that,
00:00
I'm going to save it.
00:00
I can view the alert if I want.
00:00
Perfect. I've got my two actions here.
00:00
It's currently enabled and now I'm going to hop over
00:00
to my Windows machine and try and
00:00
trigger some authentication failures.
00:00
I have a user called Hermione.
00:00
That didn't work. I don't want it to work.
00:00
Just type it away. That's a good password.
00:00
But that's not her password.
00:00
I'm just going to keep trying to generate
00:00
these events so we can get a higher count here.
00:00
If I want to take a look at my alert,
00:00
I can go to settings,
00:00
search as reports and alerts.
00:00
I think I called this authentication failures.
00:00
This tells me the next scheduled time,
00:00
which should be in a minute here.
00:00
There's currently no alerts for it
00:00
and in a minute we should see that count go up.
00:00
Great. I've got an alert here now,
00:00
if I want to take a look at,
00:00
I can go to view recent.
00:00
Here's this alert.
00:00
This is when the results were created.
00:00
It ran really quickly and the status is complete.
00:00
If I click on this here,
00:00
I can see the results.
00:00
Perfect. We've got results
00:00
here matching the search.
00:00
This will run every hour at,
00:00
what did I have it set for?
00:00
It tells me the next time here.
00:00
It's going to run again in just less than an hour.
00:00
The other thing I can do is go under activity and look at
00:00
triggered alerts and it'll also show up there as well.
00:00
It gives it a severity here.
00:00
I can view the results from here too if I want to.
00:00
With that, we've successfully created
00:00
an alert and it's time for a quiz.
00:00
When an alert triggers, Splunk can blank.
00:00
There's a lot of options here, I'm going to help you out.
00:00
Splunk could send an email,
00:00
it could write a script at the alert to triggered events.
00:00
Save the results to the machine.
00:00
Other app actions, do HTTP POST requests.
00:00
With that, we've completed this video.
00:00
In the next video,
00:00
we'll be looking at reports and dashboards.
Up Next
Similar Content
