Sunday, July 24, 2016

Prophesy and Proof in the ancient world and today:

Today is the Patronal Festival for churches dedicated in the name of St James (such as St James' Whitehaven) and some of the readings at Church this morning which referenced him triggered a line of thought about how people fall into very different intellectual traps about the nature of proof both now and two thousand years ago - and yet they have a remarkable similarity.

The Gospel reading was, of course, the story of how the mother of James and his brother John, perhaps the ultimate cringe-worthily embarrassing pushy parent of all time, asked Jesus if when he came into his Kingdom her sons could be seated next to him at his right and left.

Jesus told her that she did not understand what they were asking, and asked the brothers

"Can you drink the cup that I drink of?" (Matthew, 20, 22)

I remember thinking "Oops!" on one of the very first occasions that I heard the reading continue with their replies "We can." Even before Jesus said "You shall indeed share my cup" it was obvious how this was going to end.

As it happens it was my turn to read this morning the lesson from the Acts of the Apostles which records how James did indeed meet his end as a martyr - he was killed on the orders of Herod.

Stepping outside my own faith and opinions, I can see that what I was doing there illustrates the strength and the weakness of the human intellect. Our brains evolved as pattern-spotting machines, and they are superb at it. The only problem is that our ability to weave patterns which try to make sense of reality is that we sometimes impose patterns which are not really there.

We talk of "tempting fate" and the idea that someone who blithely answers yes when asked "Can you drink the cup that I drink of" thus seals their fate, following their Lord into martyrdom is so poetically powerful that few people could resist spotting it.

But there is nothing in Christian theology which requires you to assume this and I have never heard an evangelist use this as an argument for the truth of the religion - and they are wise not to.

In the event neither brother had quite such a cruel death as that of Jesus by crucifixion: James died by the sword and John in his own bed, but after being severely tortured. Both are, however, regarded as martyrs. That they would meet such a fate in an era when killing was a common means of settling religious differences was, of course, hardly surprising.

I look at the last sentence I wrote above and realise that it identifies me clearly as someone who grew up in the 20th century. Between the fall of the Berlin Wall and that of the twin towers the idea that anyone had the right to kill someone else for believing something different, be it in politics or religion, seemed to be in the inexorable decline that such a sick delusion deserves.

If only we could be so confident of that today. I have let the sentence stand, however, because even in the world of Al Qaeda and DA'ESH, of atrocities like those perpetrated by Anders Breivik in Norway or by the killers behind this month's massacres in Nice and Munich, such killings were much more common and more systematic two thousand years ago than they are today.

There is another  reference to prophecy in the passage from the Acts of the Apostles which I read in church this morning, At the end of Chapter 11 there is a reference to the prediction by a prophet called Agabus of a world wide famine "which in fact occurred in the reign of Claudius."

Of course, two thousand years ago, anyone who predicted a world-wide famine would be likely to be shown to be right within a couple of decades: in a pre-technological society famines afflicting a large proportion of the planet were a depressingly frequent event. And if they were advocating measures to be taken to reduce the vulnerability of society to such events they might simply be a very wise person who would not require any supernatural support to reach the conclusion that such measure were a good idea.

There may be people reading this who think "Ha - we would not be so gullible today."

But we are: we feed silly assumptions into statistical programmes in computers and then when the computers tell us it has found a relationship based on those assumptions, we believe them.

I was taught econometrics at university by Professor Angus Deaton, who was recently awarded the Nobel Prize for economics. One of the warnings he impressed on his students was the risk of "data mining" e.g. throwing large number of statistical tests at a set of data until it comes up with relationships on which you can base an economic theory.

But as one of his fellow Nobel Laureates wrote,

"Data mining" may use sophisticated statistical tools and powerful modern computers but it is no more rational or scientific than reading the entrails of sacrificial animals, praying to a volcano or throwing knuckle-bones.

The tools of analysis which have been developed by statisticians over the decades are designed to flag that you have found something significant of your result, or a more extreme one, is less likely to come about by chance than a certain "confidence level," most often a probability level of 5% or 1%.

For a single test of a pre-existing theory, that is reasonable. If the data confirms your theory and you subject it to repeated tests with different data and it continues to support the theory, you are building up increasingly strong evidence. That is how these tools are supposed to work.

But if you throw two hundred different tests at the same set of data, then by the laws of probability, it is likely to throw up two relationships which are significant at the 1% confidence level, and ten that are significant at the 5% confidence level, even if none of them are real.

If you do that without a clear idea of what you are looking for, and then construct a theory based on the relationships you found, and you would be better advised to pay an Augur to check the entrails of a sacrificial animal, because it would be a lot cheaper and quicker.

That is what Ron Coase meant and why Angus Deaton warned against "data mining."

And yet you can find multi-million pound consultancies in operation today selling big businesses computer software designed for "data mining."  If you ask them about this, they will say that if the "data mining" software they sell is used responsibility and as it is intended it will produce genuine insights.

Technically they are right. In practice it's simply a very expensive and sophisticated way of pretending to find evidence for the answer you wanted in the first place. You are basically just using computers and data feeds the way the ancient augurs used their sacrificial victims or the unscrupulous opinion pollsters used their respondents as demonstrated by Sir Humphrey Appleby in "Yes Prime Minister" ...

No comments: