Despite public promises, major scientific journals still allow abusive statistics and refuse to correct them / Boing Boing

One of the most prevalent forms of statistics in scientific studies is to analyze data retroactively to determine "interesting" trends; While such schemas may provide useful insights for future investigations, it is sufficient to select data that appears significant in a study that fails to prove the investigator's initial hypothesis may generate false – but apparently plausible – conclusions .

To combat this practice, many newspapers have subscribed to the Consolidated Standards for Reporting Trialals (CONSORT), a set of principles requiring researchers to publicly record, in advance, the data they will collect and use. how are they going to analyze them? note when they change methodology later, so that readers can deal with the consequences of these changes more carefully. CONSORT-compliant journals also promise to promptly accept, revise and publish correction letters if it is determined that the articles they have published violate the CONSORT standards.

Ninja physician Ben Goldacre (previously) and his colleagues reviewed every article published in five major CONSORT journals (New England Journal of Medicine, The Lancet, Journal of the American Medical Association, British Medical Journal, and Annals of Internal Medicine) for six weeks and, when they detected statistical practices contrary to the CONSORT principles, they informed the newspapers in writing, recorded and published their responses, and compiled the results, producing a ranking chart of the most productive newspapers. their commitment to good statistical practice. They also analyzed the reasons cited by journals (and researchers) for not publishing corrections, and highlighted significant gaps in the understanding of good statistical practices by journal editors and researchers.

The results are very bad. While the vast majority (76%) of the documents adhered to the CONSORT standards, non-compliant documents were very unlikely to be corrected, and when they were, it took a very long time (median: 99 days).

Two newspapers – JAMA and NEMJ – have flatly refused to publish one correction (On the other hand, the Lancet published 75% of the letters of corrections).

All underlying data, correspondence and other documents have been published on an excellent website. Goldacre and its partners presented their findings in a document on Biomedcentral.

Before performing a clinical trial, all results that will be measured (for example, blood pressure after one year of treatment) must be specified in advance in a test protocol and in a registry. ;clinical tests.

Indeed, if researchers measure many things, some of them will probably give a positive result by chance (a false positive). A predefined result is much less likely to give a false positive result.

Once the test is complete, the test report should then report any predefined results. When the reported results differ from those predefined, this must be stated in the report, along with an explanation of the timing and reason for the change. This ensures a faithful picture of the results of the test.

However, in reality, the predefined results are often not reported, while the unspecified results are, without being declared new. This is an extremely common problem that distorts the evidence we use to make real clinical decisions.

COMPARE: A Prospective Cohort Study Correcting and Monitoring in Real Time 58 Misstated Assays [Ben Goldacre, Henry Drysdale, Aaron Dale, Ioan Milosevic, Eirion Slade, Philip Hartley, Cicely Marston, Anna Powell-Smith, Carl Heneghan and Kamal R. Mahtani/Biomedcentral]

Compare tests

I am very happy to say that TWO NEW PAPERS are coming out TODAY from my group.

They represent one of the toughest, most interesting and exciting projects I have ever participated in, during my 44 years of nerdy life course.

Ladies and gentlemen: @COMPare_Trials is out ..

– Ben Goldacre (@bengoldacre) February 14, 2019

(via Neil Gaiman)

<! –

->