Arguments from samples

Whenever we sample or survey some (but not all) members of a group and then draw a conclusion about the group as a whole, we are reasoning non-deductively. We’ve had an example of this already:

​A nationwide poll of a random sample of thousands of home-owners revealed that 70% of them are opposed to increases in social welfare payments. Therefore, roughly 70% of the adult population of New Zealand opposes such increases.

In that case, remember, there was an obvious problem with the argument: all of people surveyed were home-owners, but the conclusion drawn is about the adult population, not just about home-owners. The sample was not representative.

Suppose we do a better survey: instead of only asking home-owners, we draw our sample randomly from the adult population of New Zealand. And suppose the results come out like this: 55% of the sample of thousands of adult New Zealanders oppose increases in social welfare payments. We conclude that 55% of all adult New Zealanders oppose such increases.

This is a stronger argument than before: now the sample from which we are generalising is a sample that is representative (so far as we can tell) of the wider population we are generalising to. Note, though, that the argument is still non-deductive. Unless you poll every single member of the wider population (in which case you are no longer arguing from a sample) the conclusion that what is true of the sample is also true of the wider population is not guaranteed.

The other thing we need to take into consideration, as well as the representativeness of the sample, is the size of the sample. In the example above, if we had only surveyed 10 randomly-selected New Zealand adults, rather than thousands, then we should definitely not generalise from the results of the survey to a conclusion about adult New Zealanders in general – the sample size is much too small.