Trust in Numbers

Posted on


I guess every field has to work out how to deal with the uncertainties that it (or its system of knowledge production) faces.

Lisa Jardine is absolutely correct that historians are too accepting of the completeness of their archives. You can never absolutely know if an archive is complete, randomly sampled, or biased. But this doesn’t mean that we should be similarly complacent, when we are in a position to determine what’s in a digitally searched archive. Or that we should make that problem worse by allowing things to be non-transparent. The digital has made the necessity of such reflection clearer to us, which is positive. But it has simultaneously made such reflection more difficult.

There are several types of uncertainty that confront the historian:

  1. things in the archive that you know aren’t there, so you don’t know exactly what they say
  2. things in the archive that you think are there but aren’t
  3. things in the archive that you don’t know exist and aren’t there

The second and third will lead to false claims to representativeness or completeness because in both cases you’re not searching something which is required for true representativeness or completeness. I am particularly concerned with the second category, as being a problem that particularly faces digital historians who, like us, are working with digital archives made by someone else and of unknown quality: searches when you don’t know what you’re searching – as in sources.

Every analysis is somehow based on a notion of the representativeness of a body of texts. If we can’t argue that a given text should follow wider patterns, why is it useful to study it? Similarly, why analyse all available sources, when actually they won’t tell you about the document you’re interested in?

Claims of representativeness in non-digital history depend on the reader trusting the historian, who presumably has read lots of sources. In digital archives, it depends on comprehensiveness – statistics (which we don’t know much about), your trust has to be in the researcher to be critical and able (ie put this in context).