Methodology over metrics: current scientific standards are a disservice to patients and society – Journal of Clinical Epidemiology

Abstract:  Covid-19 research made it painfully clear that the scandal of poor medical research, as denounced by Altman in 1994, persists today. The overall quality of medical research remains poor, despite longstanding criticisms. The problems are well known, but the research community fails to properly address them. We suggest that most problems stem from an underlying paradox: although methodology is undeniably the backbone of high-quality and responsible research, science consistently undervalues methodology. The focus remains more on the destination (research claims and metrics) than on the journey. Notwithstanding, research should serve society more than the reputation of those involved. While we notice that many initiatives are being established to improve components of the research cycle, these initiatives are too disjointed. The overall system is monolithic and slow to adapt. We assert that top-down action is needed from journals, universities, funders and governments to break the cycle and put methodology first. These actions should involve the widespread adoption of registered reports, balanced research funding between innovative, incremental and methodological research projects, full recognition and demystification of peer review, improved methodological review of reports, adherence to reporting guidelines, and investment in methodological education and research. Currently, the scientific enterprise is doing a major disservice to patients and society.

 

Open access: The universal norm? | Research Information

“Such a situation, while personally troubling, highlights the importance of open access medical research. I have been blessed to easily be able to get access to much high-quality, open medical research on this condition. Patients are most empowered when they have as much information as possible about their illnesses and I feel fortunate that open access in the medical disciplines has come this far. But, of course, there is still a road to travel. I will probably become a case study since there is not much information on this condition in a non-transplant patient. But this case study might not end up being open access research. As one of my friends put it, there’s “nothing like the warm glow of knowing your health problems have contributed to a paywalled PDF” …

But what I hope is that we find less damaging economic models than the article and book processing charges that have come to dominate. So-called “diamond” open access models, such as those that the team and I have pioneered at the Open Library of Humanities, offer equitable routes to open publishing and appear popular with academic libraries and academics. But such models are endangered by the continued dominance of massive commercial players and their transformative agreements, which threaten to consume entire library budgets in one fell swoop. 

On the one hand, I continue to feel personally opposed to the Big Business model of academic publishing in which large corporations extract massive profits and restrict the free and open flow of research. On the other hand, I am concerned by some of the austerity logics that come out of the open access movement and that devalue all publisher labour. As I wrote at the close of Open Access and the Humanities, almost a decade ago now: “Publishers perform necessary labour that must be compensated and any new system of dissemination, such as open access, will require an entity to perform this labour, even if that labour takes a different form at different levels of compensation”. Striking the right balance here through equitable economic models is the terrain on which the future battle of open access will be fought.”

Open science and data sharing in trauma research: Developing a trauma-informed protocol for archiving sensitive qualitative data. – PsycNET

Abstract:  Objective: The open science movement seeks to make research more transparent, and to that end, researchers are increasingly expected or required to archive their data in national repositories. In qualitative trauma research, data sharing could compromise participants’ safety, privacy, and confidentiality because narrative data can be more difficult to de-identify fully. There is little guidance in the traumatology literature regarding how to discuss data-sharing requirements with participants during the informed consent process. Within a larger research project in which we interviewed assault survivors, we developed and evaluated a protocol for informed consent for qualitative data sharing and engaging participants in data de-identification. Method: We conducted qualitative interviews with N = 32 adult sexual assault survivors regarding (a) how to conduct informed consent for data sharing, (b) whether participants should have input on sharing their data, and (c) whether they wanted to redact information from their transcripts prior to archiving. Results: No potential participants declined participation after learning about the archiving mandate. Survivors indicated that they wanted input on archiving because the interview is their story of trauma and abuse and it would be disempowering not to have control over how this information was shared and disseminated. Survivors also wanted input on this process to help guard their privacy, confidentiality, and safety. None of the participants elected to redact substantive data prior to archiving. Conclusions: Engaging participants in the archiving process is a feasible practice that is important and empowering for trauma survivors. (PsycInfo Database Record (c) 2022 APA, all rights reserved)

Helen Salisbury: Unintended consequences of open access to medical notes | The BMJ

“Patients now have access to their own medical notes, which is surely progress. More patients are now taking advantage of the ability to view all of the coded information in their GP record online—and this November they’ll have access to all of the free text written in the consultation from that date onwards….

But there are unintended consequences. One is the increased demand for explanations….

As junior doctors, we learn that everything we write in a patient’s notes may potentially be read by them and that we should be polite and objective, backing up opinions with evidence. I may note mismatches between symptoms and signs when my patient who says that she’s fine has nevertheless objectively lost weight, or when the child with dreadful tummy ache clambers energetically onto my couch to be examined. In the past only a handful of patients ever asked to read their notes, but many GPs will have experienced protracted conversations with patients who were unhappy with the contents. In the future, when patients have routine access to everything we write, I fear that I may have to spend more time explaining my record of the consultation….

More fundamentally, if there’s a high likelihood that all notes will be viewed online, will GPs stop noting their “soft concerns” that are so vital in both child and adult safeguarding?…”

How to disseminate clinical research – Charlesworth – – Anaesthesia – Wiley Online Library

“Gold Open Access under a Creative Commons licence is arguably a major way in which you can increase the reach of your work because most clinicians or patients cannot access paywalled content. If you submit to a subscription publisher, you will have the option of paying the article processing charge (APC) so that your work becomes Gold Open Access. This is expensive for most individuals and your funder or institution may pay the APC on your behalf – but you must ask. You can also ask the journal if other forms of Open Access are appropriate or possible (Table?1). Next, you can approach your department, institution or university to see what promotion it can offer. Sometimes, the journal may have declined to issue a press release, but others still might believe your work to be newsworthy. Authors could even approach newspapers, radio stations, broadcasters and journalists independently. Finally, you may wish to approach blog and podcast producers, conference organisers and social media influencers. The more methods used to communicate key messages from your work, the higher the reach of your paper….”

Why preprints are good for patients | Nature Medicine

“Rapid communication of clinical trial results has likely saved lives during the COVID-19 pandemic and should become the new norm….

But during health emergencies, there are many tensions, one of which is the mismatch between the urgent need for information and evidence and the much longer time frames of scientific peer review and publication. The COVID-19 pandemic is the first global health emergency of the new information age, with data and results widely shared via social media. This has resulted in very real difficulties in distinguishing important information from noise, and real news from fake news. How should the research and medical community best manage this new reality?…

Some may argue that the speed advantage of preprints does not outweigh the risks of poor-quality, misleading or even fraudulent research being published and acted upon. I would counter that clinicians should not rely solely on peer review to assess the validity and meaningfulness of research findings. This is because dubious, perhaps fraudulent data can still get through peer review, as was seen with early COVID papers published and then retracted from two of the most prestigious medical journals. In addition, even valid data can be misleading. There has been an avalanche of observational data that passed peer review and was then used to justify treatments, most notably with hydroxychloroquine, but the susceptibility of observational methodology to moderate biases means that such data should not be the basis of patient care.

I take two lessons from our experience running the largest COVID-19 clinical trial over the last two years. The first is that that the preprint system has come of age, demonstrating huge value in rapidly communicating important research findings. Almost daily I am alerted through social media alerts from trusted sources and colleagues of important new findings published as preprints. A degree of immediate peer review is also available by means of the preprint comments section and from colleagues via social media. The full peer-reviewed manuscripts usually appear many weeks or even months later. I cannot envisage a future without such rapid dissemination of new evidence.

 

Given this new reality, the second lesson is that we must ensure that the medical community and policy makers are sufficiently skilled in critical thinking and scientific methods that they can make sensible decisions, regardless of whether an article is peer reviewed or not.”

Patient involvement in clinical research: why, when, and how – PMC

Abstract:  The development of a patient-centered approach to medicine is gradually allowing more patients to be involved in their own medical decisions. However, this change is not happening at the same rate in clinical research, where research generally continues to be carried out on patients, but not with Patients. This work describes the why, when, and how of more active patient participation in the research process. Specific measures are proposed to improve patient involvement in 1) setting priorities, 2) study leadership and design, 3) improved access to clinical trials, 4) preparation and oversight of the information provided to participants, 5) post-study evaluation of the patient experience, and 6) the dissemination and application of results. In order to achieve these aims, the relative emphases on the ethical principles underlying research need to be changed. The current model based on the principle of beneficence must be left behind, and one that upholds the ethical principles of autonomy and non maleficence should be embraced. There is a need to improve the level of information that patients and society as a whole have on research objectives and processes; the goal is to promote the gradual emergence of the expert patient.

From the body of the paper: “According to some surveys, ?95% of patients and members of IRBs believe that patients should be informed of the results of the research study. Nevertheless, this is a fairly uncommon practice, and the usual situation is that after participating in a study, patients are not notified of the results.”

Usability and Accessibility of Publicly Available Patient Sa… : Journal of Patient Safety

Abstract:  Objectives 

The aims of the study were to identify publicly available patient safety report databases and to determine whether these databases support safety analyst and data scientist use to identify patterns and trends.

Methods 

An Internet search was conducted to identify publicly available patient safety databases that contained patient safety reports. Each database was analyzed to identify features that enable patient safety analyst and data scientist use of these databases.

Results 

Seven databases (6 hosted by federal agencies, 1 hosted by a nonprofit organization) containing more than 28.3 million safety reports were identified. Some, but not all, databases contained features to support patient safety analyst use: 57.1% provided the ability to sort/compare/filter data, 42.9% provided data visualization, and 85.7% enabled free-text search. None of the databases provided regular updates or monitoring and only one database suggested solutions to patient safety reports. Analysis of features to support data scientist use showed that only 42.9% provided an application programing interface, most (85.7%) provided batch downloading, all provided documentation about the database, and 71.4% provided a data dictionary. All databases provided open access. Only 28.6% provided a data diagram.

Conclusions 

Patient safety databases should be improved to support patient safety analyst use by, at a minimum, allowing for data to be sorted/compared/filtered, providing data visualization, and enabling free-text search. Databases should also enable data scientist use by, at a minimum, providing an application programing interface, batch downloading, and a data dictionary.

What do participants think of our research practices? An examination of behavioural psychology participants’ preferences | Royal Society Open Science

Abstract:  What research practices should be considered acceptable? Historically, scientists have set the standards for what constitutes acceptable research practices. However, there is value in considering non-scientists’ perspectives, including research participants’. 1873 participants from MTurk and university subject pools were surveyed after their participation in one of eight minimal-risk studies. We asked participants how they would feel if (mostly) common research practices were applied to their data: p-hacking/cherry-picking results, selective reporting of studies, Hypothesizing After Results are Known (HARKing), committing fraud, conducting direct replications, sharing data, sharing methods, and open access publishing. An overwhelming majority of psychology research participants think questionable research practices (e.g. p-hacking, HARKing) are unacceptable (68.3–81.3%), and were supportive of practices to increase transparency and replicability (71.4–80.1%). A surprising number of participants expressed positive or neutral views toward scientific fraud (18.7%), raising concerns about data quality. We grapple with this concern and interpret our results in light of the limitations of our study. Despite the ambiguity in our results, we argue that there is evidence (from our study and others’) that researchers may be violating participants’ expectations and should be transparent with participants about how their data will be used.

 

 

Full article: Putting plain language summaries into perspective

“Thanks to the open science movement, and especially open access publishing, it is becoming easier for readers outside of large research institutions to access research articles for free. The proportion of research articles that are openly available has been increasing year over year1. This increase in open access has removed one important barrier to accessing research information. However, another key barrier to access is understanding. After all, what is the point of research information being openly available if only a tiny proportion of the people who have access to it can understand the technical language it’s written in? …”

 

European parliamentarians urge action on missing clinical trial results

“A cross-party group of members of the European parliament has sent an open letter to regulators urging them to not drop the ball on over 3,400 clinical trial results that are still missing on the EudraCT trial registry, in violation of long-standing transparency rules.

 

 

Under European rules, institutions running investigative drug trials must make their results public within 12 months of trial completion. While the rules are set at the European level, responsibility for encouraging and enforcing compliance lies with the national medicines regulators in each country….”

The importance of transparency and openness in research data to drive patient benefit—Examples from the United Kingdom

“Key points

• Openness in research is discussed in many guises and brings many benefits and there is a need to join up, share good practice and talk a common language to make maximum progress.

• Importance of open publication as a key step to increasing trust and reducing waste in research.

• There is a need to be careful about the language used and it is crucial that the right safeguards are in place to protect people’s personal data. Personal data should not be ‘open’, and discussing it in this way risks its availability and associated use.

• Need to start with trust and involvement of patients and the public to ensure maximum benefit can flow from data.”

UK regulatory overhaul aims to make publication of clinical trial results a legal requirement

“The UK’s medicines regulator is planning to make the registration of clinical trial results and publication of their results a legal requirement as part of a wider overhaul of UK clinical trial regulation.

 

 

Regulator MHRA proposes to enshrine three transparency requirements in law:

 

 

Trials must be registered

Trial results must be made public within 12 months of trial end

Trial participants must be informed of trial results…”

COVID-19 and the Research Community: It is time for open access in clinical care | eLife

Abstract:  A healthcare center widely sharing its internal guidelines on how to treat COVID-19 patients “just wasn’t done.” As the pandemic raged at a Boston hospital, the next generation of clinical leaders pushed for change.

From the body of the paper: “Around mid-March, our hospital had put together its first internal guidelines to treat COVID-19 patients. We were about to upload them as a PDF to our hospital’s internal server for our staff to download and print them, when the next generation of clinical leaders spoke up. As pulmonary fellow-in-training Dr C Lee Cohen pointed out, how could paper printouts possibly keep up with the rapidly evolving data on COVID-19? Just as importantly, she and others suggested that we had a responsibility to share our findings with the world, not just with our staff. In anticipation, Cohen built the website COVIDprotocols.org to host our guidelines: this platform was web and mobile-based, searchable and could be continually updated. Evolving recommendations, accessible on our smartphones? That idea was an instant hit. Sharing them outside our hospital was not. What if we were wrong and misled professionals around the globe? Making internal guidelines widely public just wasn’t done. I hesitated. As the individual who had put together the interdisciplinary team working on our COVID-19 protocols, I felt ultimately responsible for any negative fallout. But the pandemic was too massive, and the global confusion too overwhelming; it wasn’t perfect, but it just had to be done. Despite our doubts, on March 20, 2020, we launched the website Cohen had built, and released our first set of guidelines to the public.

Evolving recommendations, accessible on our smartphones? That idea was an instant hit. Sharing them outside our hospital was not. What if we were wrong and misled professionals around the globe? Making internal guidelines widely public just wasn’t done. I hesitated. …”

 

Home – NIH ODSS Search Workshop

“The goal of the Workshop is to explore current capabilities, gaps and opportunities for global data search across the data ecosystem. Workshop will explore selected science drivers across these main themes:

Using search to build cohorts: finding data across different platforms/repositories using patient attributes in order to create a cohort of patients for clinical analysis
Using search to find relevant data & repositories: finding data & repositories in order to access and analyze the data further, including its use for creating computational models.
Using search for (complex) information retrieval: answering specific questions without the additional burden of data download or analysis…”