another area of concern
when, as I was giving some examples in the last section
of appealing to consensus appealing to authority
confirmation bias, a lot of these things are
to this idea of analysis of competing hypotheses
trying to figure out which of these conclusions is actually the correct one. Which of these possible outcomes might be the correct one on the C I. A dot gov website, they provide eight high level steps
to hopefully deal with this problem of competing hypotheses.
1st 1 is to find all the hypotheses that are being considered
and then do Cem Cem brainstorming or some kind of focus group type analysis.
You could do some simple voting, even let's see a show of hands for for the first hypothesis and so on and so on.
You might have to go through several rounds of that process to get it down to the last two or the last three, depending on how many you have to pick from
in order to organize the approach,
there should also be some notes taken as to what are the arguments and evidence for each of these hypotheses that appears to be
the most probable correct one.
You could then prepare Matrix. As it says here, I can have my, uh, hypothesis across about this is across the top and evidence down the side.
And then I could populate that with all the different supporting arguments and supporting evidence that
that has appears to be related to
these final few choices. As the process continues, there will be some expectation that that there will be some manipulation or fine tuning
the evidence or arguments, because after some additional consideration, someone might decide Well, this
doesn't really belong here. It's not really proving the point that it purports to prove.
So as you start to refine the Matrix, you can start to add things and remove things in order to make it more accurate and Maur
correct from a logical point of view
or even a diagnostic value
moving on to the last four steps, we can see that you would have some kind of conclusion that could be formed based on this refined matrix,
and at that point there should be some effort to try to disprove
the hypothesis. Trying to take the opposite side of the argument toe, look for holes to look for errors and thinking, logical fallacies, biases
that we discussed in the previous section. You should all be useful tactics at this point to use
to do some refer the refining further filtering of those things, which may have been incorrectly
added to this matrix to begin with,
then you want to look at the evidence in a even closer manner interruptus to find out. Is there any pattern of a few select pieces of evidence causing the conclusion to be reached?
Maybe that's not enough data to work with.
Maybe there's Maur announces that could be done or
even a repeat of earlier steps
in order to discover if there was a problem with the workflow or with the thinking that got to this point in the process.
Any information that leads to a conclusion that could be interpreted in more than one way should also be very suspect.
Certainly there are going to be different conclusions reached by different people because they each have their own opinions and so on. But
if the conclusions are reached from a point of logic,
then they should be consistent.
now the conclusions could be reported and we could rank them by probability or likelihood,
as we saw with the um
in the last module, there are expectations of
near certainties somewhere in the middle of something that's probable, maybe a 50 50 chance, something that's impossible or not very probable.
And you could use a scale like this in order to
expand on the value of this exercise.
And then the last point is looking for other
pieces of information that could be researched further
and could provide some help for future activity. So let's go ahead and look at the C I A website and see what they have out here for you to work with. It's actually a pretty detailed document on their website,
showing 14 chapters worth of information
on how to sort through these different competing hypotheses
in a methodical way in a way that's repeatable to some extent.
So we see the steps that I was talking about earlier,
looking at all the hypotheses to be considered
doing some brainstorming, moving on to making arguments
with some sort of significant evidence, being prepared, create the matrix
to organize this information
and even give some examples like Will Iraq retaliate for US bombing of his intelligence headquarters.
You could certainly look at this kind of analysis and populate your matrix with the different pieces of information to come to some sort of preliminary conclusion.
Then you do the refinement
and trying to be ruthless about getting rid of anything that does not support
the conclusion. Anything that's appear appears to be superfluous.
Then you might come up with your tentative conclusions and take the opposite tactic of trying to disprove as a way to strengthen the assumptions that have been reached up to this point.
Then you look at whether the conclusions are reached by certain critical piece of evidence versus an overall
agreement of the evidence that's been presented.
This helps to get rid of anomalies. Let's say
now you think about probability
by signing different waiting's to, uh, the things that are most likely that the true answer something that might be in the middle of something might be at the other end, where it's not very likely.
And then finally, you come up with your milestones for future observations and research.
So it's pretty good information to think about,
and it's definitely worth spending some of your time on this