Saturday, August 5, 2017

Controversy Over the P-value Value

Rearranging Deck Chairs on the Titanic

There is much to agree with in Steven Novella’s article from this past week on the P-value. The latest news regarding the beleaguered statistical parameter used in hypothesis testing is the call to reduce its associated threshold for statistical significance by an order of magnitude from its venerable value of 0.05 to 0.005. This is a modest proposal compared the outright banning the use of P-values in recent years. But in any case, while a move to 0.005 would likely help to reduce problems, what is more desperately needed is the underlying training and peer review to ensure proper statistical testing, period, regardless of the value selected threshold for the P-value. This is because the problems discussed by Novella are dwarfed by hypothesis testing fallacies, such as false dichotomies, that routinely appear in the literature. Those problems, unfortunately, are routinely ignored.

1 comment:

  1. Cornelius, this may come as a surprise to you but I agree with you. The problem isn't with the p value itself. It is with the uninformed use of it. Its validity (or strength) is dependent on assumptions of data distribution, independence, etc. that aren't always met.