Bayesian vs Frequentist

Heard of the ongoing debate in Statistics but never been sure what the fuss was all about? Here’s a quick run through to get you up to speed.

What is the debate?

Statistics refers to the tools we use to analyse data. Usually, after looking at the data at hand, we describe a statistical model to represent that data.

For example, we have a coin and we flip it 100 times, getting 55 heads and 45 tails. We could write down our ‘model’ to be of a weighted coin, slightly bias towards heads.

Here is the problem: On the one hand, we have some data suggesting that this coin is biased towards heads. On the other hand, we know that 100 flips isn’t much, it could get closer to 50:50 if we keep flipping, and coins aren’t normally weighted (outside of boring maths textbooks). When deciding what our coin ‘model’ is, what should we care about most — the data or our prior knowledge of coins?

Frequentist Idea

The Frequentist statistician puts the data first. “This coin looks to be unbiased, but we need to keeping counting the flips to be sure”, they say. 100 flips, 1000 flips, 10000 flips and on. They’d get a great idea of how to model the coin’s weight by championing the data.

Bayesian Idea

The Bayesian statistician puts a model down first. “Coins are rarely weighted, so it’s a good assumption to say this coin is fair. Let’s see if the data disproves this theory”, they respond. Given the result after 100 flips, they’d check to see how likely the results were given their assumption of a fair coin. If the results that occured are extremely unlikely to have been produced by a fair coin, than they would have to change their model accordingly.

The difference

The frequentist builds their model on the data provided. The bayesian explores how likely their model is to be true given the data provided, and adjusts it accordingly to try to be more accurate.

They often achieve the same results, but by looking at the problem from opposite ends!

Why does it matter?

To a mathematician however, using statistics to its’ further applications, one school of thought has other strengths to the other. Different methods are suited to working on different problems.