Why preprints are good for patients | Nature Medicine

“Rapid communication of clinical trial results has likely saved lives during the COVID-19 pandemic and should become the new norm….

But during health emergencies, there are many tensions, one of which is the mismatch between the urgent need for information and evidence and the much longer time frames of scientific peer review and publication. The COVID-19 pandemic is the first global health emergency of the new information age, with data and results widely shared via social media. This has resulted in very real difficulties in distinguishing important information from noise, and real news from fake news. How should the research and medical community best manage this new reality?…

Some may argue that the speed advantage of preprints does not outweigh the risks of poor-quality, misleading or even fraudulent research being published and acted upon. I would counter that clinicians should not rely solely on peer review to assess the validity and meaningfulness of research findings. This is because dubious, perhaps fraudulent data can still get through peer review, as was seen with early COVID papers published and then retracted from two of the most prestigious medical journals. In addition, even valid data can be misleading. There has been an avalanche of observational data that passed peer review and was then used to justify treatments, most notably with hydroxychloroquine, but the susceptibility of observational methodology to moderate biases means that such data should not be the basis of patient care.

I take two lessons from our experience running the largest COVID-19 clinical trial over the last two years. The first is that that the preprint system has come of age, demonstrating huge value in rapidly communicating important research findings. Almost daily I am alerted through social media alerts from trusted sources and colleagues of important new findings published as preprints. A degree of immediate peer review is also available by means of the preprint comments section and from colleagues via social media. The full peer-reviewed manuscripts usually appear many weeks or even months later. I cannot envisage a future without such rapid dissemination of new evidence.

 

Given this new reality, the second lesson is that we must ensure that the medical community and policy makers are sufficiently skilled in critical thinking and scientific methods that they can make sensible decisions, regardless of whether an article is peer reviewed or not.”

Widespread use of National Academies consensus reports by the American public | PNAS

Significance

Advocates for open access argue that people need scientific information, although they lack evidence for this. Using Google’s recently developed deep learning natural language processing model, which offers unrivalled comprehension of subtle differences in meaning, 1.6 million people downloading National Academies reports were classified, not just into broad categories such as researchers and teachers but also precisely delineated small groups such as hospital chaplains, veterans, and science fiction authors. The results reveal adults motivated to seek out the most credible sources, engage with challenging material, use it to improve the services they provide, and learn more about the world they live in. The picture contrasts starkly with the dominant narrative of a misinformed and manipulated public targeted by social media.

Abstract

In seeking to understand how to protect the public information sphere from corruption, researchers understandably focus on dysfunction. However, parts of the public information ecosystem function very well, and understanding this as well will help in protecting and developing existing strengths. Here, we address this gap, focusing on public engagement with high-quality science-based information, consensus reports of the National Academies of Science, Engineering, and Medicine (NASEM). Attending to public use is important to justify public investment in producing and making freely available high-quality, scientifically based reports. We deploy Bidirectional Encoder Representations from Transformers (BERT), a high-performing, supervised machine learning model, to classify 1.6 million comments left by US downloaders of National Academies reports responding to a prompt asking how they intended to use the report. The results provide detailed, nationwide evidence of how the public uses open access scientifically based information. We find half of reported use to be academic—research, teaching, or studying. The other half reveals adults across the country seeking the highest-quality information to improve how they do their job, to help family members, to satisfy their curiosity, and to learn. Our results establish the existence of demand for high-quality information by the public and that such knowledge is widely deployed to improve provision of services. Knowing the importance of such information, policy makers can be encouraged to protect it.

Patient involvement in clinical research: why, when, and how – PMC

Abstract:  The development of a patient-centered approach to medicine is gradually allowing more patients to be involved in their own medical decisions. However, this change is not happening at the same rate in clinical research, where research generally continues to be carried out on patients, but not with Patients. This work describes the why, when, and how of more active patient participation in the research process. Specific measures are proposed to improve patient involvement in 1) setting priorities, 2) study leadership and design, 3) improved access to clinical trials, 4) preparation and oversight of the information provided to participants, 5) post-study evaluation of the patient experience, and 6) the dissemination and application of results. In order to achieve these aims, the relative emphases on the ethical principles underlying research need to be changed. The current model based on the principle of beneficence must be left behind, and one that upholds the ethical principles of autonomy and non maleficence should be embraced. There is a need to improve the level of information that patients and society as a whole have on research objectives and processes; the goal is to promote the gradual emergence of the expert patient.

From the body of the paper: “According to some surveys, ?95% of patients and members of IRBs believe that patients should be informed of the results of the research study. Nevertheless, this is a fairly uncommon practice, and the usual situation is that after participating in a study, patients are not notified of the results.”

Who Uses Open Access Research? Evidence from the use of US National Academies Reports   | Impact of Social Sciences

“A fundamental principle of open access is that publication technology enables the widest possible audience for research findings. However, the extent to which open research is used outside of academia is often underexplored. Drawing on a dataset covering over a million user comments about their use of US National Academies consensus study reports, Ameet Doshi, Diana Hicks, Matteo Zullo and Omar I. Asensio find widespread use of open research in the public sphere….

Our classification project reveals that the impact of these reports extend far beyond the research community (see Results, Fig 1). We find that half of all report downloads are used for non-academic purposes, including to improve the provision of services by medical professionals, local and regional planners, public health workers, and veterans’ advocates, to name just a few of the 64 total categories of report use.  Heavy use is made of Academies reports on STEM education and how people learn by teachers, school administrators and teachers’ coaches.  Other notable reports with their prominent users included Dying in America (chaplains), Nutrient Requirements for Beef Cattle (farmers), and Best Care at Lower Costs (clinicians and hospital administrators)….

Open access repositories require significant resources, both technological and human, to sustain and innovate. The National Academies Press, for example, has developed an engaging user interface to incentivize browsing and ease of access to NASEM publications. The PubMed Central server, developed and managed by the US National Institutes of Health (NIH), requires millions of dollars per year to operate. Our research indicates there is an identifiable payoff to society for these taxpayer investments into people, technology and design to support OA publishing….

Librarians and open access advocates have long presupposed that open access to high-quality scientific knowledge could and should be viewed as a public good. Our empirical research suggests that the initial utopian aspirations regarding the public use and societal impact of OA may indeed rest on sound footing.”

 

Dismantling the ivory tower’s knowledge boundaries

“The major shift to open access during the pandemic began with the Free Read initiative, which launched the petition “

Unlock Coronavirus Research” for scientists in early February of 2020 and to which highly reputable medical publishers quickly responded. Before the pandemic, up to 75 percent of scholarly publications were behind a paywall. By comparison, a preliminary study of over 5,600 articles on PubMed suggests that more than 95 percent of scholarly articles related to COVID-19 are now freely available. This increase in accessibility resulted from the rapid adaptation by biomedical journals and publishers, including Elsevier, Springer Nature, Cell Press, New England Journal of Medicine, and The Lancet. These journals and publishers granted open access to research on COVID-19 research, often making it 

immediately accessible on the platform PubMed Central and similar public repositories. Free and open access to COVID-19 research quickly became the new normal for biomedicine, with available findings directly impacting the development of treatment protocols and vaccines. Yet the pandemic became more than a health crisis. Understanding the social, psychological, and economic implications of the pandemic were imperative to its continued management.

Social science research, which delivers insights into human behaviors, relationships, and institutions, was instrumental to policymaking and healthcare solution development during the COVID-19 pandemic. The importance of social science research to pandemic management was demonstrated by the 

shift in the topic of COVID-19 papers, from the initial focus on disease modeling, hospital mortality, diagnostics, and testing to an increasing focus on topics such as business closure, remote work, geographic mobility and migration, inequality, managerial decision-making, as well as accelerating innovation. Once the basic science on the virus were established, research on creating societal and economic resilience played an even larger role for beating the COVID-19 pandemic. One clear area that demonstrated the importance of social science research in informing COVID-19 management was the rollout of vaccines. Psychological, marketing, and information systems research played a central role in vaccine uptake across communities. A recent report by the National Institutes of Health called for the use of evidence-based strategies, such as 

behavioral nudges and strategic social norms, to increase vaccine uptake….”

 

 

Societal impact of university research in the written press: media attention in the context of SIUR and the open science agenda among social scientists in Flanders, Belgium | SpringerLink

Abstract:  Transferring scientific knowledge to non-academic audiences is an essential aspect of the open science agenda, which calls for scholars to pursue a popularization of their research. Accordingly, purposefully introducing scientific insights to the public at large is almost univocally deemed commendable. Indeed, in today’s models of research evaluation, the objects and activities considered are being extended beyond peer-reviewed journal articles to include non-scholarly popular communication. Although altmetrics offer one instrumental way to count some interactions with lay audiences, their reliance on social media makes them susceptible to manipulation, and mostly reflect circulation among niche audiences. In comparison, attention from non-scholarly media like newspapers and magazines seems a more relevant pathway to effectuate societal impact, due to its recognition in qualitative assessment tools and its broad, societal reach. Based on a case study of social scientists’ attention by newspapers and magazines in Flanders (northern Dutch-speaking region of Belgium) in 2019, this paper highlights that frequent participation in the public debate is reserved for high-status researchers only. Results show highly skewed media appearance patterns in both career position and gender, as eight male professors accounted for almost half of all 2019 media attention for social scientists. Because media attention is highly subject-dependent moreover, certain disciplines and fields offer easier pathways to popularization in media than others. Both the open science agenda and research assessment models value presence of researchers in popular media, adding written press attention to existing evaluation assessments however would disproportionately disadvantage early career researchers and exacerbate existing inequalities in academia.

 

 

Usability and Accessibility of Publicly Available Patient Sa… : Journal of Patient Safety

Abstract:  Objectives 

The aims of the study were to identify publicly available patient safety report databases and to determine whether these databases support safety analyst and data scientist use to identify patterns and trends.

Methods 

An Internet search was conducted to identify publicly available patient safety databases that contained patient safety reports. Each database was analyzed to identify features that enable patient safety analyst and data scientist use of these databases.

Results 

Seven databases (6 hosted by federal agencies, 1 hosted by a nonprofit organization) containing more than 28.3 million safety reports were identified. Some, but not all, databases contained features to support patient safety analyst use: 57.1% provided the ability to sort/compare/filter data, 42.9% provided data visualization, and 85.7% enabled free-text search. None of the databases provided regular updates or monitoring and only one database suggested solutions to patient safety reports. Analysis of features to support data scientist use showed that only 42.9% provided an application programing interface, most (85.7%) provided batch downloading, all provided documentation about the database, and 71.4% provided a data dictionary. All databases provided open access. Only 28.6% provided a data diagram.

Conclusions 

Patient safety databases should be improved to support patient safety analyst use by, at a minimum, allowing for data to be sorted/compared/filtered, providing data visualization, and enabling free-text search. Databases should also enable data scientist use by, at a minimum, providing an application programing interface, batch downloading, and a data dictionary.

What do participants think of our research practices? An examination of behavioural psychology participants’ preferences | Royal Society Open Science

Abstract:  What research practices should be considered acceptable? Historically, scientists have set the standards for what constitutes acceptable research practices. However, there is value in considering non-scientists’ perspectives, including research participants’. 1873 participants from MTurk and university subject pools were surveyed after their participation in one of eight minimal-risk studies. We asked participants how they would feel if (mostly) common research practices were applied to their data: p-hacking/cherry-picking results, selective reporting of studies, Hypothesizing After Results are Known (HARKing), committing fraud, conducting direct replications, sharing data, sharing methods, and open access publishing. An overwhelming majority of psychology research participants think questionable research practices (e.g. p-hacking, HARKing) are unacceptable (68.3–81.3%), and were supportive of practices to increase transparency and replicability (71.4–80.1%). A surprising number of participants expressed positive or neutral views toward scientific fraud (18.7%), raising concerns about data quality. We grapple with this concern and interpret our results in light of the limitations of our study. Despite the ambiguity in our results, we argue that there is evidence (from our study and others’) that researchers may be violating participants’ expectations and should be transparent with participants about how their data will be used.

 

 

Opening up Science—to Skeptics – By Rohan Arcot & Hunter Gehlbach – Behavioral Scientist

“The research community is no stranger to skepticism. Its own members have been questioning the integrity of many scientific findings with particular intensity of late. In response, we have seen a swell of open science norms and practices, which provide greater transparency about key procedural details of the research process, mitigating many research skeptics’ misgivings. These open practices greatly facilitate how science is communicated—but only between scientists. 

Given the present historical moment’s critical need for science, we wondered: What if scientists allowed skeptics in the general public to look under the hood at how their studies were conducted? Could opening up the basic ideas of open science beyond scholars help combat the epidemic of science skepticism?  …”

Full article: Putting plain language summaries into perspective

“Thanks to the open science movement, and especially open access publishing, it is becoming easier for readers outside of large research institutions to access research articles for free. The proportion of research articles that are openly available has been increasing year over year1. This increase in open access has removed one important barrier to accessing research information. However, another key barrier to access is understanding. After all, what is the point of research information being openly available if only a tiny proportion of the people who have access to it can understand the technical language it’s written in? …”

 

Aligning the Research Library to Organizational Strategy – Ithaka S+R

“Open access has matured significantly in recent years. The UK and EU countries have committed largely to a “gold” version of open access, driven largely by transformative agreements with the major incumbent publishing houses.[14] The US policy environment has been far more mixed, with a great deal of “green” open access incentivized by major scientific funders, although some individual universities pursued transformative agreements. Both Canadian and US libraries have benefitted from the expansion of free and open access in strengthening their position at the negotiating table with major publishers.[15]

Progress on open access has radically expanded public access to the research literature. It has also brought with it a number of second-order effects. Some of them are connected to the serious problems in research integrity and the growing crisis of trust in science.[16] Others can be seen in the impacts on the scholarly publishing marketplace and the platforms that support discovery and access.[17]

While open access has made scientific materials more widely available, it has not directly addressed the challenges in translating scholarship for public consumption. Looking ahead, it is likely that scholarly communication will experience further changes as a result of computers increasingly supplanting human readership. The form of the scientific output may decreasingly look like the traditional journal article as over time standardized data, methods, protocols, and other scientific artifacts become vital for computational consumption….”

European parliamentarians urge action on missing clinical trial results

“A cross-party group of members of the European parliament has sent an open letter to regulators urging them to not drop the ball on over 3,400 clinical trial results that are still missing on the EudraCT trial registry, in violation of long-standing transparency rules.

 

 

Under European rules, institutions running investigative drug trials must make their results public within 12 months of trial completion. While the rules are set at the European level, responsibility for encouraging and enforcing compliance lies with the national medicines regulators in each country….”

The importance of transparency and openness in research data to drive patient benefit—Examples from the United Kingdom

“Key points

• Openness in research is discussed in many guises and brings many benefits and there is a need to join up, share good practice and talk a common language to make maximum progress.

• Importance of open publication as a key step to increasing trust and reducing waste in research.

• There is a need to be careful about the language used and it is crucial that the right safeguards are in place to protect people’s personal data. Personal data should not be ‘open’, and discussing it in this way risks its availability and associated use.

• Need to start with trust and involvement of patients and the public to ensure maximum benefit can flow from data.”

UK regulatory overhaul aims to make publication of clinical trial results a legal requirement

“The UK’s medicines regulator is planning to make the registration of clinical trial results and publication of their results a legal requirement as part of a wider overhaul of UK clinical trial regulation.

 

 

Regulator MHRA proposes to enshrine three transparency requirements in law:

 

 

Trials must be registered

Trial results must be made public within 12 months of trial end

Trial participants must be informed of trial results…”

Study reveals strong demand for open-access science

“The study, published Feb. 23, 2022, in the Proceedings of the National Academy of Sciences (PNAS), analyzed the reasons for 1.6 million downloads of National Academies of Sciences, Engineering, and Medicine (NASEM) consensus reports, considered among the highest credibility science-based literature.

The resulting analysis, which included U.S. downloads only, is the first to look at who is using such information and why. Professor Diana Hicks, Assistant Professor Omar I. Asensio, and Ph.D. students Matteo Zullo and Ameet Doshi, all of Georgia Tech’s School of Public Policy, co-authored the study.

They found that while nearly half of the reports were downloaded for academic purposes, even more were accessed by people outside strictly educational settings, such as veterans, chaplains, and writers. The word “edification” appeared 3,700 times in the data set, signaling a strong desire for lifelong learning among users….”