In what ways can probabilistic reasoning be bad?
Almost all the real reasoning we do is based on imperfect knowledge. It is rare that we are dealing with situations in which we can have only given statements accepted as true as is the case in formal logic. Indeed even when trying to reason in purely formal logic people often bring in their imperfect knowledge of reality. We reach conclusions from our reasoning in a number of ways. Sometimes we have anecdotal evidence, sometimes we hear about surveys, sometimes we listen to people we trust. All these routes are used in everyday life in how we make decisions about what we believe and in what we choose to do. They also figure strongly in science, although science has a more earnest debate and can often rely on experimental evidence that is beyond dispute.
We weigh up the evidence in front of us and see where the balance of evidence lies. Sometimes it is a marginal issue. Sometimes there is a clear winner. In this process however there are a number of ways that we can combine the various bits of evidence in mistaken ways.
One way that people make mistakes can be called the ‘gamblers mistake’ since many gamblers make it. It is to assume that the outcome of an event is dependent on previous events when it isn't. For example if the roulette wheel comes up black three times then the mistaken gambler thinks that there is a higher chance that it will come up red than black on the next spin (actually of course it remains a 50/50 chance). This invents a dependency and hence causal relationship between events where there is none. I suppose this might be termed a superstitious mentality. Other errors of this sort can be attributed to taking short cuts in working out the maths of combining probabilities. This can be seen another manifestation of insufficient search.
Another aspect of bad thinking people commonly make when weighing up the evidence is to put too much weight on evidence that confirms existing beliefs. Sometimes this is shown in the choice of tests made. Tests are made so as to provide evidence of existing beliefs where tests that might provide evidence of alternatives are not chosen. A good way of avoiding this mistake is to stay remote from the issue that is being considered. There may be good reasons why you don't want to consider the alternatives thoroughly and equally - change is difficult and has a cost. However, for the moral ideal of discovering what the truth is; what is going to be the best final result; what is the best solution for everyone, you need to ignore the changes that you must yourself make: The cost to you of changing your thinking and habits on a given matter is negligible by comparison to the total good for all by sticking to a pure detached search for truth.