“Inconsistent” enforcement of data sharing rules by journals is making it impossible for researchers to interrogate published scientific findings, researchers have claimed.

Highlighting the “weak” policing of mandates requiring authors to share raw data from their studies, a paper by academics in Canada and Sweden claims the lack of publicly available results has led to a “transparency crisis in research” that limits the reliability of published findings.

In the paper published in its authors argue that, “despite wide-spread data-sharing policies, compliance remains low, enforcement weak, and monitoring almost non-existent” at most journals, “with many datasets remaining unavailable or inaccessible”.

Referring to a 2023 Science study analysing 7,750 biomedical papers between 2015 and 2020, only 9 per cent of papers pledged to make all their data publicly available and only 3 per cent followed through with this commitment.

����Vlog

Noting how the existence of a data availability statement often made little difference to whether authors shared their results or not, the paper also cites a 2022 study which found only 7 per cent of authors behind papers pledging to share their data on request actually did so when they were approached.

The full extent of the “data transparency crisis” was difficult to estimate given how little information was provided by journals on how they enforced or monitored data sharing practices, explained David Moher from the University of Toronto, who co-authored the PLOS Medicine study with Saul Martin Rodriguez and Rodrigo Fernandez-Gonzalo, both from the Karolinska Institute in Stockholm.

����Vlog

“Journals have very little public-facing information about their monitoring and enforcement of data sharing of what they publish,” Moher told ����Vlog, adding that “this compromises the research integrity of the publications and journals”.

However, it was difficult to enforce or monitor data sharing practices when the language of data transparency statements is often open to interpretation, he continued.

“Implementing monitoring is likely most beneficial when it is accompanied by unambiguous language,” he said, suggesting that guidelines on randomised controlled trials should reference the internationally recognised and should refer to checklists rather than use narrative statements about openness.

“Real enforcement for sharing data would provide the research ecosystem, including patients and the public, with some assurances that data is being shared,” Moher insisted.

Journals should not allow researchers to pledge to provide data upon request, given how seldom authors responded promptly to these enquiries, he added.

����Vlog

“Data sharing is not access to data upon request,” said Moher, adding that “prior to accepting an article for publication, the journal needs to monitor and enforce access to the underlying data supporting the results”.

Scientists should also seek to explain the “real-world consequences” of the failure to provide access to raw data rather than relying on “abstract problems of reproducibility”, added Rodriguez.

In the paper, the authors explain how flawed meta-analyses of observational studies, caused by missing datasets for published papers, suggested that menopausal hormone therapy reduced heart disease – a finding that influenced the care of tens of millions of women.

����Vlog

“Missing data can lead to incomplete evidence syntheses, duplication of research efforts, wasted public resources, and healthcare decisions being made on incomplete evidence,” said Rodriguez.

“Perhaps the challenge is not only to explain the consequences of inaccessible data, but also to encourage policies that prevent these situations from occurring again,” he said, adding that “the goal is not to punish researchers, but to ensure that important scientific and medical decisions are based on evidence that can actually be scrutinised and verified.”

However, isolated actions by individual journals to address this problem or “voluntary compliance” with codes are “unlikely to be enough”, continued Rodriguez, noting recent mandatory data and code sharing requirements adopted by The BMJ and PLOS Medicine.

“Meaningful change will probably require stronger commitments from the major publishing groups that collectively oversee a large proportion of the scientific literature,” he said.

����Vlog

jack.grove@timeshighereducation.com