It's time to talk about abandoning statistical significance

Fans of The Galactic Traveler's Guide know that the answer to life, to the Universe and to everything, is 42. The joke, of course, is that the truth can not be revealed by a single number.

And yet, it is the work often attributed to P Values: Measure the amazing result, given the assumptions made about the experience, including the lack of effect. If a P the value falls above or below an arbitrary threshold demarcating the 'statistical significance' (such as 0.05) decides whether assumptions are accepted, articles are published and products marketed. But using P Values ​​as the sole arbiter of what to accept as truth may also mean that some analyzes are biased, that some false positives are over-typed, and that some real effects are overlooked.

The change is in the air. In a commentary published in this week's issue, three statisticians call on scientists to abandon any statistical significance. Authors do not call P The values ​​themselves must be dropped as a statistical tool – they rather want their use removed as an arbitrary significance threshold. More than 800 researchers have added their names as signatories. A series of related articles are being published this week by the American Statistical Association (R. L. Wasserstein). et al. A m. Stat. https://doi.org/10.1080/00031305.2019.1583913; 2019). "The tool has become the tyrant," laments an article.

Statistical significance is so deeply integrated into scientific practice and evaluation that it would be painful to discover it. Critics will argue against the idea that arbitrary gatekeepers are better than fuzzy, and that the most useful argument is to know what results should count for (or against) the evidence of the ## 147 ## # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # 39; effect. There are reasonable points of view on all sides; Nature does not try to change the way it looks at statistical analysis in the evaluation of articles for the moment, but we encourage readers to share their views (see go.nature.com/correspondence).

If researchers eliminate statistical significance, what should they do instead? They can start by learning about statistical misconceptions. The most important will be the courage to consider uncertainty from several angles in each study. Logic, basic knowledge and experimental design should be considered in parallel P values ​​and similar measures to reach a conclusion and decide on its certainty.

When developing the methods to be used, researchers must also focus as much as possible on concrete problems. People who are fighting to death for abstract theories on how best to use statistics often agree on the results when they are presented with concrete scenarios.

Researchers must look to analyze the data in several ways to see if different analyzes converge to the same answer. Projects involving outsourced analyzes of a dataset to various teams suggest that this approach can work to validate results and offer new insights.

In short, be skeptical, choose a good question and try to answer it in different ways. It takes a lot of numbers to get close to the truth.