Data-sharing and re-analysis for main studies assessed by the European Medicines Agency—a cross-sectional study on European Public Assessment Reports | BMC Medicine | Full Text

Abstract:  Background

Transparency and reproducibility are expected to be normative practices in clinical trials used for decision-making on marketing authorisations for new medicines. This registered report introduces a cross-sectional study aiming to assess inferential reproducibility for main trials assessed by the European Medicines Agency.

Methods

Two researchers independently identified all studies on new medicines, biosimilars and orphan medicines given approval by the European Commission between January 2017 and December 2019, categorised as ‘main studies’ in the European Public Assessment Reports (EPARs). Sixty-two of these studies were randomly sampled. One researcher retrieved the individual patient data (IPD) for these studies and prepared a dossier for each study, containing the IPD, the protocol and information on the conduct of the study. A second researcher who had no access to study reports used the dossier to run an independent re-analysis of each trial. All results of these re-analyses were reported in terms of each study’s conclusions, p-values, effect sizes and changes from the initial protocol. A team of two researchers not involved in the re-analysis compared results of the re-analyses with published results of the trial.

Results

Two hundred ninety-two main studies in 173 EPARs were identified. Among the 62 studies randomly sampled, we received IPD for 10 trials. The median number of days between data request and data receipt was 253 [interquartile range 182–469]. For these ten trials, we identified 23 distinct primary outcomes for which the conclusions were reproduced in all re-analyses. Therefore, 10/62 trials (16% [95% confidence interval 8% to 28%]) were reproduced, as the 52 studies without available data were considered non-reproducible. There was no change from the original study protocol regarding the primary outcome in any of these ten studies. Spin was observed in the report of one study.

Conclusions

Despite their results supporting decisions that affect millions of people’s health across the European Union, most main studies used in EPARs lack transparency and their results are not reproducible for external researchers. Re-analyses of the few trials with available data showed very good inferential reproducibility.

Reporting and transparent research practices in sports medicine and orthopaedic clinical trials: a meta-research study | BMJ Open

Abstract:  Objectives Transparent reporting of clinical trials is essential to assess the risk of bias and translate research findings into clinical practice. While existing studies have shown that deficiencies are common, detailed empirical and field-specific data are scarce. Therefore, this study aimed to examine current clinical trial reporting and transparent research practices in sports medicine and orthopaedics.

Setting Exploratory meta-research study on reporting quality and transparent research practices in orthopaedics and sports medicine clinical trials.

Participants The sample included clinical trials published in the top 25% of sports medicine and orthopaedics journals over 9 months.

Primary and secondary outcome measures Two independent reviewers assessed pre-registration, open data and criteria related to scientific rigour, like randomisation, blinding, and sample size calculations, as well as the study sample, and data analysis.

Results The sample included 163 clinical trials from 27 journals. While the majority of trials mentioned rigour criteria, essential details were often missing. Sixty per cent (95% confidence interval (CI) 53% to 68%) of trials reported sample size calculations, but only 32% (95% CI 25% to 39%) justified the expected effect size. Few trials indicated the blinding status of all main stakeholders (4%; 95% CI 1% to 7%). Only 18% (95% CI 12% to 24%) included information on randomisation type, method and concealed allocation. Most trials reported participants’ sex/gender (95%; 95% CI 92% to 98%) and information on inclusion and exclusion criteria (78%; 95% CI 72% to 84%). Only 20% (95% CI 14% to 26%) of trials were pre-registered. No trials deposited data in open repositories.

Conclusions These results will aid the sports medicine and orthopaedics community in developing tailored interventions to improve reporting. While authors typically mention blinding, randomisation and other factors, essential details are often missing. Greater acceptance of open science practices, like pre-registration and open data, is needed. As these practices have been widely encouraged, we discuss systemic interventions that may improve clinical trial reporting.

Clinical trial results for $3.2 billion Covid drug are missing in action

“The results of most clinical trials of the Covid drug molnupiravir (Lagrevio) have not been made public and remain completely unknown, a new study has found.

 

 

 

The drug is currently being administered to Covid patients in the United States, the UK, and India. The World Health Organisation has issued a “conditional recommendation” for its use in some patient groups. Global sales so far stand at $3.2 billion….”

Evaluation of publication bias for 12 clinical trials of molnupiravir to treat SARS-CoV-2 infection in 13,694 patients | Research Square

Abstract:  Introduction:

During the COVID-19 pandemic, Merck Sharp and Dohme (MSD) acquired the global licensing rights for molnupiravir. MSD allowed Indian manufacturers to produce the drug under voluntary license. Indian companies conducted local clinical trials to evaluate the efficacy and safety of molnupiravir.

Methods

Searches of the Clinical Trials Registry-India (CTRI) were conducted to find registered trials of molnupiravir in India. Subsequent investigations were performed to assess which clinical trials had been presented or published.

Results

According to the CTRI, 12 randomised trials of molnupiravir were conducted in India, in 13,694 patients, starting in late May 2021. By July 2022, none of the 12 trials has been published, one was presented at a medical conference, and two were announced in press releases suggesting failure of treatment. Results from three trials were shared with the World Health Organisation. One of these three trials had many unexplained results, with effects of treatment significantly different from the MSD MOVE-OUT trial in a similar population.

Discussion

The lack of results runs counter to established practices and leaves a situation where approximately 90% of the global data on molnupiravir has not been published in any form. Access to patient-level databases is required to investigate risks of bias or medical fraud.

EU Clinical Trials Register – Update

“Following the issuing of the Joint Letter by the European Commission, EMA and HMA, National Competent Authorities and European Medicines Agency have sent reminders to sponsors who were not compliant with the European Commission guideline on results posting. Thanks to these reminders, the percentage of posted results substantially increased. However, for some trials the reminders were not successful: detailed lists of these trials can be found here. …”

Clinical Trial Registry Errors Undermine Transparency | The Scientist Magazine®

“Confusion about terminology on the world’s largest clinical trials registry may be delaying the release of drug trial results and undermining rules designed to promote transparency, an investigation by The Scientist has found. 

Key study dates and other information are entered into the ClinicalTrials.gov database by trial researchers or sponsors, and are used by US science and regulatory agencies to determine legal deadlines by which results must be reported. The rules are supposed to ensure timely public access to findings about a potential therapy’s harms and benefits, as well as provide the scientific community with an up-to-date picture of the status of clinical research.

But neither the agencies nor staff overseeing the database routinely monitor individual trial records for veracity, instead relying on the person in charge of a given record to correctly declare information such as when a study ends and how many people were enrolled. …”

Adoption of World Health Organization Best Practices in Clinical Trial Transparency Among European Medical Research Funder Policies | Global Health | JAMA Network Open | JAMA Network

Abstract:  Importance  Research funders can reduce research waste and publication bias by requiring their grantees to register and report clinical trials.

Objective  To determine the extent to which 21 major European research funders’ efforts to reduce research waste and publication bias in clinical trials meet World Health Organization (WHO) best practice benchmarks and to investigate areas for improvement.

Design, Setting, and Participants  This cross-sectional study was based on 2 to 3 independent assessments of each funder’s publicly available documentation and validation of results with funders during 2021. Included funders were the 21 largest nonmultilateral public and philanthropic medical research funders in Europe, with a combined budget of more than US $22 billion.

Exposures  Scoring of funders using an 11-item assessment tool based on WHO best practice benchmarks, grouped into 4 broad categories: trial registries, academic publication, monitoring, and sanctions. Funder references to reporting standards were captured.

Main Outcomes and Measures  The primary outcome was funder adoption or nonadoption of 11 policy and monitoring measures to reduce research waste and publication bias as set out by WHO best practices. The secondary outcomes were whether and how funder policies referred to reporting standards. Outcomes were preregistered after a pilot phase that used the same outcome measures.

Results  Among 21 of the largest nonmultilateral public and philanthropic funders in Europe, some best practices were more widely adopted than others, with 14 funders (66.7%) mandating prospective trial registration and 6 funders (28.6%) requiring that trial results be made public on trial registries within 12 months of trial completion. Less than half of funders actively monitored whether trials were registered (9 funders [42.9%]) or whether results were made public (8 funders [38.1%]). Funders implemented a mean of 4 of 11 best practices in clinical trial transparency (36.4%) set out by WHO. The extent to which funders adopted WHO best practice items varied widely, ranging from 0 practices for the French Centre National de la Recherche Scientifique and the ministries of health of Germany and Italy to 10 practices (90.9%) for the UK National Institute of Health Research. Overall, 9 funders referred to reporting standards in their policies.

Conclusions and Relevance  This study found that many European medical research funder policy and monitoring measures fell short of WHO best practices. These findings suggest that funders worldwide may need to identify and address gaps in policies and processes.

Medical research funders across Europe tighten rules on clinical trial reporting

“Eight of the 21 largest public and philanthropic medical research funders in Europe are stepping up their efforts to improve clinical reporting, following an assessment that found widespread gaps in existing research waste safeguards.

 

 

 

At present, many academic clinical trials in Europe fail to make their results public, wasting taxpayers’ money and leaving large gaps in the medical evidence base.

 

 

 

The public institutions that hand out money to medical researchers can prevent such waste by putting into place eleven safeguards recommended by the World Health Organisation. …”

Finding My Way from clinical trial to open access dissemination: comparison of uptake, adherence, and psychosocial outcomes of an online program for cancer-related distress | SpringerLink

Abstract:  Few digital psycho-oncology programs have been adopted into routine practice; how these programs are used after trial completion remains unexplored. To address this, the present study transitioned our evidence-based 6-module CBT-based program, Finding My Way, into open access (OA) after completion of the RCT, and compared uptake, usage, and psychosocial outcomes to the earlier RCT.

Methods

Recruitment was passive, via promotion through (1) media and social media releases, (2) public lectures, (3) radio interviews and podcasts, and (4) clinician-initiated referral. Measures included number of enrolled users, number of modules completed, and pre- and optional post-measures of distress and quality of life (QOL).

Results

Uptake was lower in OA (n?=?120; 63% of RCT). Usage was markedly lower: 1.5 modules were completed on average (vs 3.7 in RCT), and only 13% completed a ‘therapeutic dose’ of 4?+?modules (vs. 50% in RCT). Research attrition was high; n?=?13 completed post-measures. OA users were more sociodemographically and clinically diverse than RCT users, had higher baseline distress (OA Mpre?=?36.7, SD?=?26.5; RCT Mpre?=?26.5, SD?=?21.7), and reported larger pre-post reductions than their RCT counterparts (OA Mpost?=?23.9, SD?=?20.7; RCT Mpost?=?21.2, SD?=?21.2). Moderate improvements in mental QOL occurred during OA (Mpre?=?37.3, SD?=?12.6; Mpost?=?44.5, SD?=?12.1), broadly replicating RCT findings.

Conclusion

Findings that OA users were more medically and sociodemographically diverse and distressed at baseline than their RCT counterparts, and — despite having lower usage of the program — achieved larger changes from baseline to post-program, will help to shape future intervention design, tailoring, and dissemination.

WHA clinical trial resolution: draft text now public

“The World Health Organization today published the draft text of the clinical trial resolution being debated at the ongoing World Health Assembly.

 

 

The resolution’s overall aim is to improve the coordination, design, conduct and reporting of clinical trials worldwide. It was partly spurred by the realisation that hundreds – maybe thousands – of Covid clinical trials have ended up as costly research waste….

 

Promoting, as appropriate, measures to facilitate the timely reporting of both positive and negative interpretable clinical trial results in alignment with the WHO joint statement on public disclosure of results from clinical trials and the WHO joint statement on transparency and data integrity, including through registering the results on a publicly available clinical trial registry within the [global trial registry network] ICTRP, and encouraging timely publication of the trial results preferably in an open-access publication.

Exploring measures during public health emergencies of international concern to encourage researchers to rapidly and responsibly share interpretable results of clinical trials, including negative results, with national regulatory bodies or other appropriate authorities, including WHO for clinical guideline development and emergency use listing (EUL), to support rapid regulatory decision-making and emergency adaptation of clinical and public health guidelines as appropriate, including through pre-print publication. …”

 

Why did clinical trial registries fail to prevent Covid research chaos?

“There is a long-standing global ethical obligation to register all trials before they start, shored up by regulatory requirements in some jurisdictions. Data from 18 registries worldwide feed into the WHO-managed International Clinical Trials Registry Platform (ICTRP), providing a continuously updated overview of who is researching what, when, where and how – at least in theory.

 

 

If the registry infrastructure had worked and been used as intended, much of the COVID-19 research chaos would have been avoided.

 

 

For example, researchers considering launching a hydroxychloroquine trial could have searched ICTRP and discovered that the drug was already being investigated by numerous other trials. Those researchers could accordingly have focused on investigating other treatment options instead, or aligned their outcome measures with existing trials. …

The global registry infrastructure has long been inadequately supported by legislators and regulators, and is woefully underfunded.

 

 

 

This persistent neglect of the world’s only comprehensive directory of medical research led to costly research waste on an incredible scale during the pandemic.

 

 

The WHO recommends that member states should by law require every interventional trial to be registered and reported. In addition, WHO recommends that all trial results should be made public specifically on a registry within 12 months, and that registry data should be kept up to date.

 

 

 

By enforcing these three simple rules, regulators would ensure that there is a comprehensive, up-to-date global database of all trials and their results.

 

In reality, existing laws in the EU and the US only cover a small minority of trials and are not being effectively enforced, while many other jurisdictions have no relevant laws at all. …”

 

 

Why preprints are good for patients | Nature Medicine

“Rapid communication of clinical trial results has likely saved lives during the COVID-19 pandemic and should become the new norm….

But during health emergencies, there are many tensions, one of which is the mismatch between the urgent need for information and evidence and the much longer time frames of scientific peer review and publication. The COVID-19 pandemic is the first global health emergency of the new information age, with data and results widely shared via social media. This has resulted in very real difficulties in distinguishing important information from noise, and real news from fake news. How should the research and medical community best manage this new reality?…

Some may argue that the speed advantage of preprints does not outweigh the risks of poor-quality, misleading or even fraudulent research being published and acted upon. I would counter that clinicians should not rely solely on peer review to assess the validity and meaningfulness of research findings. This is because dubious, perhaps fraudulent data can still get through peer review, as was seen with early COVID papers published and then retracted from two of the most prestigious medical journals. In addition, even valid data can be misleading. There has been an avalanche of observational data that passed peer review and was then used to justify treatments, most notably with hydroxychloroquine, but the susceptibility of observational methodology to moderate biases means that such data should not be the basis of patient care.

I take two lessons from our experience running the largest COVID-19 clinical trial over the last two years. The first is that that the preprint system has come of age, demonstrating huge value in rapidly communicating important research findings. Almost daily I am alerted through social media alerts from trusted sources and colleagues of important new findings published as preprints. A degree of immediate peer review is also available by means of the preprint comments section and from colleagues via social media. The full peer-reviewed manuscripts usually appear many weeks or even months later. I cannot envisage a future without such rapid dissemination of new evidence.

 

Given this new reality, the second lesson is that we must ensure that the medical community and policy makers are sufficiently skilled in critical thinking and scientific methods that they can make sensible decisions, regardless of whether an article is peer reviewed or not.”

‘The EMA is withholding too much information”, 1 May 2022

“Transparency is a requirement for better and safer patient care. There is no valid reason to hide information about clinical trials, their methodology or their results, or evaluation data obtained on drugs after their market introduction, particularly data on adverse effects.

The creation of the European Medicines Agency (EMA) in 1995 constituted a step forward, compared with the practices of France’s drug regulatory agency at the time. For example, the EMA’s online publication of information on drug evaluations, such as European Public Assessment Reports, was a major advance in transparency as to the data in its possession….

It is one thing for pharmaceutical companies to consider that data showing the limitations of their drugs are commercially sensitive. But it is quite another – and utterly unacceptable – for the EMA to actually orchestrate the concealment of these data by pharmaceutical companies.

Transparency is not a fad or an end in itself. In the pharmaceutical field, it is a requirement for better and safer patient care. There is no valid reason to hide information about clinical trials, their methodology or their results, or evaluation data obtained on drugs after their market introduction, particularly data on adverse effects.

 

Perhaps there are certain individuals within the EMA who are dissatisfied with this situation? Or who are simply resigned to the power relations at play? Or who feel that the way the EMA operates is a necessary compromise, given the varying legislation? If so, these individuals are not speaking up and their opinions are not reflected in the EMA’s practices. Whatever the case may be, Prescrire’s negative assessment of the level of transparency at the EMA is intended as a wake-up call for policy makers and for legal bodies (such as the Ombudsman) who are in a position to improve the EMA’s operational practices….”