Time
5 hours 19 minutes
Difficulty
Intermediate
CEU/CPE
5

Video Transcription

00:00
Lesson 4.5 metrics for detecting incidents
00:04
in this lesson, we're going to talk about the differences between above the line and below the line security metrics and identify incident detection metrics that could be used to measure the health and effectiveness of the I. R team.
00:18
Now I have to tell you, Metrics is one of those things that I get a chance to work with with a lot of clients and, believe it or not, actually like metrics quite a bit. A lot of people don't use them. They hate them and most the time they don't have any metrics, really, but
00:35
normally doesn't ever end well for organizations that don't have good metrics. I have found that having good metrics while
00:41
yes, it's work to get them, and it's worked to put them together and report on them.
00:46
It will end up paying dividends with executive leadership to get extra money, to get personnel, to get just that reputation of being a
00:56
a professional organization, and you really somebody who knows what's going on within the organization. So I highly recommend taking the time to develop metrics for your cert in your I R program.
01:10
I want to walk through with you. Some metrics that I like to use when we talk about detection. So remember, prevention is ideal, but detection is a must. So if detection is a must, we have to know how well we do detection.
01:23
So I want to talk about these seven. That are some of my favorites. The first is mean time to detection. So from the point something happened to when you detected it, sometimes you don't know this until the incident is over and been fully investigated.
01:40
But that's a really good metric to see. How long on average is it taking your team
01:44
to detect something going on that shouldn't be going on?
01:48
Number two is the ratio of events to alerts. This goes along the line that I was speaking of earlier. About how much are you paying per alert
01:57
and how many false positives are you getting? And here we're talking about how many events are we looking out and how many of those events become alerts.
02:06
Also, the average time to triage. So you've detected something. It's gone to an analyst. It's being triaged. Now it's being investigated. How long did it take to do that triage? Is it a matter of seconds or is it taking you
02:22
quite a while to triage an incident? And if it is, why is that the case? Is it because you don't have good records?
02:28
You haven't identified your critical assets. You don't really have a process that's repeatable that all of your sock analysts used to do some of this triage.
02:37
Who knows what the reason is. But again, having metrics will point some of these things out to you and then the cost per alert. I've already talked about this one. The false positive rate is another good one to look at is tracking. This helps you put the necessary importance on tuning your SIM and your
02:55
components that are creating all these alerts
02:58
and making sure you're focused on the right things? Remember what you measure you will pay attention to, so make sure to measure what matters, and it is important to to consider what you're metrics are. It's
03:12
I've seen organisations measuring the wrong things and then because everyone wants to please the boss or the metrics or tied to performance or two bonuses.
03:20
Now all of a sudden people are off working on something that really doesn't help.
03:23
So be very cognizant of the metrics you use and that you're measuring what's truly important to the organization. And what you're measuring is helping you get across the finish line for whatever your goals are in your strategy.
03:38
What's the vulnerability? Scanning coverage? Are you only hitting 60% of the boxes in your environment? If so, that's not an acceptable number. And why isn't that? Why is that happening? Why aren't you able to get to the devices? You should be able Teoh. And then finally, your malware detection rate.
03:54
How frequently are you catching malicious software in your organization?
03:59
Is that a problem with your tools? That a problem with reporting up to you that maybe the local box caught it. But for some reason, the alert didn't get from it to your SIM tool or your sock or your I R team. So it helps you look for those kinds of problems.
04:14
And then finally, the box I have along the top remember your audience so above the line. If you were to look at all the metrics out there and there's a lot more than this and I'll share some more with you. But there's above the line metrics and below the line. And what I mean by that is above the line metrics or things that really have to do with
04:33
the health of your I R program
04:35
and those key performance indicators, or KP eyes that are for executive leadership. So above the line executives below the line. That's what the certain manager, maybe the cyst. So certainly the i R team. That's what they care about. And some of these are above and some of these air below.
04:56
You know, talk
04:57
about how you should define that and what you should be communicating up as you go throughout your program. As you communicate risk to executives, it's important to
05:06
remember those above the line metrics as KP eyes that I've mentioned already. What they care about is the health of the organization, but more importantly, how cyber risk is impacting the rest of the organization. So be thinking of that when you're developing these metrics and KP eyes. If you're reporting up,
05:28
maybe vulnerability scanning percentage that you have
05:32
coverage in might be inappropriate above the line metric. If you're having a historic problem with business units, not giving cybersecurity access to their devices. But if that's not a problem and it's really more of an I T issue that S E C M agents are broken or something like that, then
05:51
that's probably a below the line metric,
05:54
but above the line. Meantime, to detection, that's probably above the line. That's showing how healthy the organization is. And it's also showing how much risk there is because it takes you, let's say, 17 days to detect something in your environment. On average, that's usually not acceptable either. And really, when you have this conversation with executives,
06:14
you can frame it in a few different ways. One thing I like to show them, though, is here's your cybersecurity program. Here's how much money you're spending on cybersecurity Now, I can tell you, based on our metrics, it takes us 17 days to detect something. Once we've detected, it takes us another 10 days to fully respond to it and investigate it because we only have part time people,
06:32
and then it may take us another five days to fully remediated it,
06:36
and that's if it's a simple incident.
06:40
So that's where we're at today. Now There's a couple different things that you can decide on as the executive leadership,
06:46
and you can fund cybersecurity the same way. So here is, let's say, the Silver plan for cybersecurity And here's the gold plan. And here's the platinum plan under the silver. Let's say that's where we're at today. This is how long it's taking us to respond. Detect, Recover.
07:03
If that's not okay with you. Well, let's go to the gold plan. And if you were to invest another $1,000,000 a year, let's say we could add these specific tools, maybe another person, and we can cut that down Teoh. Four days to detect two days to respond and one day to recover, for instance.
07:21
Now, if you want even more than that, well, let's talk about the platinum plan that would cost X amount of dollars. This is what it would get you, and these are the numbers and where we would be at on average.
07:32
So that's a completely different conversation than just showing a bunch of numbers to executives. And it's been very effective for me and others that I have advised to do this kind of tactic, and it's not something that you do all the time. But there are certain times when this is an appropriate conversation to have,
07:50
and having metrics like this can really help.
07:54
Now I will go through metrics for response as well. These air just for detection. And if you have questions, feel free to reach out to me. If you want to talk anything more about metrics, let's go to the quiz question for this lesson. What do Above the line metrics refer to
08:11
a metrics that are designed for senior leadership to measure the health of a function
08:16
in business impacts,
08:18
be metrics that reach a certain threshold or see metrics that indicate a program is failing.
08:28
The answer to this, of course, is a This is the
08:31
above the line metrics that were geared towards senior leadership, the board of directors, executives that kind of a level
08:39
so in summary. In this lesson, we talked about the differences between above the line and below the line metrics and the incident detection metrics. I gave you some examples of things that I like to use that can be used to measure the health and effectiveness oven. I, our team

Up Next

Incident Response Lifecycle

This intermediate-level course will provide the student with in-depth instruction on the lifecycle of cybersecurity incident response.

Instructed By

Instructor Profile Image
Josh Moulin
Instructor