# Gathering Audit Evidence

Video Activity

This lessons discusses gathering audit evidence and focuses on two types: Direct: Proves existence of a fact Indirect: more circumstantial and based on inference This lesson also discusses statistical sampling techniques: - Random sampling - Cell sampling - Fixed interval sampling Participants also learn about what types of evidence of typical for ...

Join over 3 million cybersecurity professionals advancing their career
or

Time
13 hours 35 minutes
Difficulty
Intermediate
CEU/CPE
9
Video Description

Video Transcription
00:04
Okay, so now we're talk about how we gather the audit evidence.
00:09
We need to understand what's considered tangible evidence and what constitutes reliable evidence. So you think about the basic function of evidence. I need to prove something or just prove something.
00:21
Which is to also say that if we don't have evidence,
00:25
that means that we we don't have proof
00:28
just like it would apply to a criminal investigation of any type or
00:32
trying to find ah, fact related to some audit objective,
00:37
some kind of concept here.
00:39
And the important thing to point out is an oddity starts with zero points
00:44
and then builds up to their final score. As each point gets proven
00:48
than that accumulates towards the final report.
00:52
We need to understand the different types of evidence.
00:55
Direct evidence doesn't actually require any explanation, right?
01:00
It proves its existence. It's self explanatory, if you will,
01:04
then we have the concept of in fear in CE,
01:07
which is related to direct evidence. This means that you can draw a conclusion based on the evidence given, and we have indirect evidence.
01:15
This means that that we have a, uh,
01:19
hypothesis or a guess or a theory about what
01:23
the evidence actually means,
01:26
so that might include inference that might include presumption.
01:30
This is still
01:30
based on circumstances and whatever facts could be gathered,
01:36
but it's not
01:37
as strong as direct evidence. Another word or another phrase, rather for indirect evidence, is circumstantial evidence.
01:45
So it's reasonable. A bit of proof, but not as hard of the
01:51
doesn't prove things as as conclusively as direct evidence. Would we have to think about statistical sampling? What does this really mean
01:59
in the context of doing an audit?
02:00
It's basically a
02:01
mathematical technique
02:05
that
02:06
the auditor employees in order to get enough data to satisfying a requirement for something that's being tested or investigated.
02:15
Typically, these statistics are presented as percentages. That's
02:20
that's the norm. But there could be other, another types of units.
02:25
We have the idea of random sampling,
02:29
which is self explanatory, or you've got a large body of data and you're just picking some samples
02:36
from various different areas without any rhyme or reason. It's that says random is as you could make it,
02:43
and we have the concept of cell sampling.
02:46
This is where a sample is done at some
02:49
interval, a predefined interval
02:51
Maybe you're looking for certain types of data of once a minute or
02:55
or once an hour once a day, whatever the case might be.
03:01
And then we have fixed interval sampling,
03:05
and this is very similar to sell. Sampling were
03:07
doing a
03:10
incremental interval
03:14
in order to get a similar distribution of samples throughout a body of data.
03:21
There's also non statistical sampling,
03:23
so this means that the auditors judgment or their their experience an opinion comes into play.
03:30
That's why this is also called judgmental Sandwich.
03:32
So the auditor picks the sample size
03:36
and the methodology for extracting
03:38
the data as well as how many items are actually going to be looked at. Now we have to think about
03:46
the the evidence types that would be typically used in a information systems audit.
03:53
03:55
which includes things like transaction logs, together, systems, logs,
04:01
financial transaction information receipts
04:05
could be a lot of different things, but it's some kind of documented hard evidence that some event took place
04:13
that we have data extraction.
04:15
So if you've got large data sets, whether it's a long file
04:20
or a data coming out of a database or some kind of application,
04:25
you might use certain tools to pull data out that meets certain criteria. Basically, you create some kind of filter,
04:32
and the filter would apply to all the data and only matched those things that the auditor is interested in.
04:40
And we have oddity claims.
04:42
So the client or the audit EADS says that this is so
04:46
and they put that down in a written statement.
04:48
And this has its own value as evidence, of course,
04:55
but it may not be considered as strong of a type of evidence as direct evidence would be.
05:01
Could also look at other documented evidence, such as plans, policies and procedures.
05:08
These are all things that we discussed earlier, so we should know The difference is between
05:12
a policy, a standard procedure or guideline.
05:15
But having these in documented form
05:18
helps as well, because if these are,
05:20
uh,
05:21
items they're actually in use within the organization than they provide valuable evidence,
05:28
we have to think about doing testing,
05:30
compliance testing or suggest substantive testing.
05:33
This gives direct evidence to say that we looked at something we did some operation or we observed someone else doing this operation. We saw the input. We saw the behavior, and then we saw the output.
05:46
The last item is the auditor observing someone in the performance of their duties.
05:51
Or maybe they perform
05:55
a particular process of the auditor, watches them do it
05:58
perhaps more than once to make sure that it works the same every time.
06:02
Well, again, we see the mention of our of our cat computer assisted audit tools.
06:08
This could help in a lot of different ways. You might have tools that can help with understanding a system configuration that conduce certain vulnerability. Scanning,
06:17
doing network scanning, running sniffers, intrusion detection systems.
06:23
They even apply to systems that allow you to trace the functions of software by putting a
06:30
a tracer on certain operations to see how it goes through the operations of the software performs.
06:36
It was also tools toe analyze the configuration of applications.
06:42
This kind of goes into the area of application pen testing, fuzzing that kind of thing.
06:47
We also can use tools, too.
06:50
Thio get an inventory of all of our software licenses
06:54
given environment, that's an important thing to think about.
06:57
07:00
So this is just a sampling of some of the capabilities of computer assisted audit tools.
07:04
Obviously, there's more features available, depending on which
07:09
vendors tools chosen.
07:11
Then we have to think about the possibility of using cat tools for continuous
07:15
auditing.
07:16
This is analogous to continuous monitoring, where you're looking at your security controls
07:24
on a continuous basis to find problems as soon as possible and be able to take action as soon as possible
07:30
so we can start with online event monitors.
07:32
These look at transaction logs
07:35
tools that
07:38
event management tools such as Ark site Things of this nature
07:42
can let you get alerts at a moment's notice to let you know that somebody tried to log in as administrator
07:47
or someone has changed a network setting on one of the production servers.
07:54
These air ah, great tools for alerting you to different events in the environment
07:59
and that typically would fall into the same group as intrusion detection systems, intrusion prevention systems.
08:07
Maybe you have a dashboard that shows you the events as they occur.
08:11
You can also think about
08:13
ah,
08:15
embedding audit hooks into software,
08:18
and this is something that the software developers need to do
08:22
so they can basically look for certain events. And when an event happens in the software that can generate an alert that can be sent to an auditor,
08:31
this is a way of
08:33
putting some monitoring within the functionality of the software itself to ease the auditing process
08:41
that once a Net and alert gets generated. Then that transaction could be looked at more detail because it might
08:46
be considered suspicious or unusual, and that therefore
08:50
the event gets generally a lark. It's generated, so that event could be investigated more fully
08:56
that we could think about continuous and inter minted simulation or C. I S
09:01
was another audit tool.
09:03
This means that you can set up certain criteria
09:05
for events or transactions toe happen. And once those criteria are met, an alert gets generated so that the auditor can get some information for further investigation.
09:16
We could also do snapshot audits,
09:18
so this looks at
09:20
a series of data capture events, sort of like taking snapshots of some something that's moving. We'll look at the data from this moment, then we take another snap shot. Look at the date at this moment and so long,
09:33
and then this shows a sequence that a, uh,
09:37
transaction goes through in order to go from its initiation to completion.
09:43
We also have the embedded audit.
09:46
This is another
09:48
way of interfacing with an application so the auditor can create some dummy transactions,
09:54
and then they put those into the stream of live data transactions in order to see what the output looks like.
10:03
So if the dummy transaction gets processed in the expected way, it should
10:07
appear to be correct when it's completed, and then that could be compared against transactions that are performed with live data toe. Look for any differences in the way that the transactions were a process,
10:20
and we have system control audit review
10:22
with embedded audit files also known a scarf or a M.
10:28
So this selectively
10:31
picks autumn modules within some application software
10:35
and, uh,
10:37
samples transactions as needed, depending on the objectives of the auditor.
10:41
All right, so we'll summarize some of the cat methods here.
10:43
We can see we've got online event monitors, reads logs, generates alarms,
10:48
very low complexity
10:50
the audit
10:52
hooks in our programs.
10:54
This will flag those transactions again, both low complexity.
10:58
Then we've got our continuous and intermittent simulation
11:01
so defined criteria when the transaction meets those criteria, it gets alerted a za medium transat medium complexity because there's a little bit more involved in setting up a tool like this
11:15
that we have. Snapshots
11:16
capture data through very stages of its processing
11:20
again a medium complexity
11:22
solution
11:24
than the invented autumn module E a. M.
11:28
Producing the dummy transactions and processing them alongside live data and comparing the results that eyes a high complexity because it's a little bit more involved to do that.
11:37
There's more planning, of course, and more analysis
11:41
as a result.
11:43
And then we have the last one, which is System System Control Audit Review file with embedded audit modules
11:50
and This way program various modules to do different audit functions.
11:56
And, of course, that's the most complex. So we have a high complexity for this one as well.
12:01
So a lot of good choices, depending on your resource is and your requirements you can pick one that suits your needs the best.
12:09
All right, so let's move on to Elektronik. Discovery
12:13
we're talking about here is
12:16
the the difference between what
12:18
the oddity and the auditor expected to be discovered during the process of doing the on it.
12:24
So the scope is very important to set
12:28
so that the discovery process
12:30
is appropriate and it's
12:31
a level of effort
12:33
doesn't go beyond what's required and and doesn't go doesn't stay less than what's required.
12:41
So it's important to think about
12:43
the limitations on the scope.
12:46
So, for instance, the scope might. If it's too large, it could produce a burden on your production systems.
12:52
It might include things like recovered deleted data.
12:56
It could address email records.
13:01
It could address things that were
13:03
saved on a backup
13:05
tape or a backup solution,
13:09
so the scope could be very far reaching. And that makes sense that you wouldn't.
13:13
I want a limit
13:16
the scope to a individual system if the data has been moved or are kinda log or backed up somewhere else. So the scope has to adjust accordingly
13:24
in order to capture of the information that's required.
13:28
You also have to think about this idea of a claim of privilege
13:33
formulas and business secrets might fall into this category, so there could be some exceptions, depending on the situation that's being investigated are audited now. Once we've got some evidence, we have to decide how to grade this evidence
13:48
13:50
This is a, ah logical relationship between the Thea item that's being investigated and the evidence that's gathered.
14:01
We have to consider the objectivity of the evidence.
14:05
If if the evidence is objectively true, then we don't need thio spend much time doing analysis or or exercising judgment
14:13
14:16
been proven that it's objectively
14:20
accurate.
14:22
If more judgment is required, then the evidence becomes less objective.
14:26
So that makes sense.
14:28
We have to think about who's providing the evidence. How competent are they?
14:33
Where does the source of this information
14:35
The source could be an individual? What is their expertise? What is their experience?
14:43
We'd like to get information directly from a client,
14:46
not not through a secondhand source or 1/3 hand source
14:50
that potentially taints the information and dilute its value.
14:54
And then we also have to consider the
14:58
independence of the evidence,
15:00
just like an auditor is expected to be independent.
15:03
The provider of independence Sorry, the provider of evidence.
15:09
I should not have anything to gain or lose by by providing some information.
15:13
If that's the case, if they have something to gain or something to lose than that evidence is not considered as independent as previously might have been considered. So we look at the way to grade the evidence. We can start with
15:26
material evidence.
15:28
We can see that it's considered
15:31
unrelated to being poor evidence. It's it's indirect. It's considered good evidence, low relationship,
15:39
and then the best evidence is direct.
15:41
So there's no explanation or judgment required as we talked about a minute ago.
15:48
Thinking about the objectivity of evidence
15:50
it's subjective
15:52
requires some facts to explain the meaning.
15:56
That's that's just
15:58
pretty standard. But if it's in a best evidence category, it doesn't require an explanation.
16:03
When we think about evidence sources
16:07
I'm related to 1/3 party with no involvement
16:10
constitutes poor evidence.
16:11
If we want to have good evidence in this case, it would be in direct involvement by a second party. So a little closer to the source
16:18
and then direct involvement by the first party would constitute best evidence.
16:23
So the closer you get to the source or the source directly the better quality of the evidence.
16:30
Then think about the competency of the provider of the evidence.
16:33
It's poor evidence. The person's probably biased. Maybe they're non biased in the case of good evidence,
16:38
and they might be non biased and independent
16:41
in the case of best evidence, so nothing to gain nothing to lose. In that case,
16:48
if we're analyzing the evidence,
16:51
poor evidence is analyzed by nav INS novice experienced evidence by
16:56
provides good evidence by an experienced
16:59
analysts
17:00
and that expert analysts provides our best evidence.
17:03
And then, lastly, we have the trustworthiness
17:06
of low, medium and high. Obviously, best evidence is is the ultimate goal in every case.
17:11
But that may not always be possible, depending on what
17:15
the situation is.
17:15
Now we think about the life cycle of evidence,
17:19
so it does go through a life cycle, and it's important to consider these individual
17:25
phases.
17:26
First, we start off with identification.
17:27
We know that some evidence has been
17:32
discovered or identified, and it's
17:34
lending its support to the objectives of the audit.
17:38
That evidence gets collected
17:41
according to the procedures that were agreed upon
17:45
and according to the
17:47
goals of securing the information respecting confidentiality requirements and so on.
17:53
Then we have to preserve this evidence,
17:56
keeping it in its original state.
18:00
In the case of like a forensic investigation, that's a little bit more complicated.
18:06
We have to consider things like chain of custody, proper gathering techniques, proper documentation and so on.
18:15
Once that's done, the evidence could be analyzed.
18:18
This could be done with scientific tests, observation
18:21
some 10 substantiative
18:23
tests, qualitative and semi qualitative quantitative methods could also be used.
18:30
Then we think about analyzing the evidence after it's been preserved.
18:36
So you have to return
18:38
the evidence back to where it was removed from after the analysis has been done.
18:44
So, like the idea that you get some evidence out of the evidence locker, you do some experiments or tests with, and then you have to return it
18:51
back to where it waas.
18:52
Now that you've got more information, you think about presenting the evidence
18:57
so that this should support the auditor's report, support the auditor's opinion,
19:03
and depending on what type of evidence it is, it might have to be returned to the owner when all this work is finally completed.
Up Next