Responsible Research Assessment | Open Science Talk

Abstract:  Felix Schönbrodt, Professor of Psychology at Ludwig-Maximilians-Universität (LMU) in Munich, tells about an initiative that he coordinates within the Deutsche Gesellschaft für Psychologie (German Psychological Society). Motivated by the Reproducibility Crisis and a rising frustration with the publishers of high-ranking journals, Schönbrodt has co-authored three position papers on the theme of responsible research assessment. The suggestion is to develop a two-stage evaluation system for hiring, the first of which will use responsible metrics with emphasis on open data, pre-registration and several aspects of reproducibility, whereas the second stage will focus on a qualitative (content-oriented) evaluation of selected candidates. The propositions of Schönbrodt’s group have so far led to published feedback from more than 40 different scholars. Besides his nation-wide work within the German Psychological Society, Schönbrodt is the managing director of LMU’s Open Science Centre, where scholars from different disciplines convene for workshops on various aspects of Open Science. Under the nickname «nicebread» (Schön = nice, Brodt = bread), he also runs a personal blog and a project webpage on GitHub.

 

Open Science: A New Paradigm for Economic Growth – Financial Times – Partner Content by Frontiers

“Restricting access to scientific knowledge bottlenecks urgently needed innovation and global economic growth. It’s comparable to throttling internet speed or reducing global penetration….

Even though the knowledge economy is the basis of our prosperity and well-being, few know that our science is locked up. Of the four million scientific papers published every year3, only a small fraction of scientists in the world have access to all the knowledge. Compared with internet penetration of around 66%4, the penetration of all scientific knowledge is less than 0.00125% of the world population. It is not globally shared, nor free to read, nor open to all. We are missing out on a vast publicly funded resource at a time when innovation, economic growth, and reducing inequality is more important than ever. It is also bad for business. The knowledge economy is failing to tap into the world’s largest market for ideas….”

Open Science: A New Paradigm for Economic Growth – Financial Times – Partner Content by Frontiers

“Restricting access to scientific knowledge bottlenecks urgently needed innovation and global economic growth. It’s comparable to throttling internet speed or reducing global penetration….

Even though the knowledge economy is the basis of our prosperity and well-being, few know that our science is locked up. Of the four million scientific papers published every year3, only a small fraction of scientists in the world have access to all the knowledge. Compared with internet penetration of around 66%4, the penetration of all scientific knowledge is less than 0.00125% of the world population. It is not globally shared, nor free to read, nor open to all. We are missing out on a vast publicly funded resource at a time when innovation, economic growth, and reducing inequality is more important than ever. It is also bad for business. The knowledge economy is failing to tap into the world’s largest market for ideas….”

How AI could reveal secrets of thousands of handwritten documents – from medieval manuscripts to hieroglyphics

“Ansund aims to use HTR [handwritten text recognition] to build an exhaustive, open-access digital corpus of Old English texts, that transcribes all surviving Old English for the first time, and in an unparalleled level of detail.”

New report “Recognising Digital Scholarly Outputs in the Humanities” | ALLEA

This report underscores the transformative impact of digital practices on humanities scholarship. It highlights the importance of recognising interdisciplinary work, innovative research methods, and non-traditional scholarly outputs. In the first part, the ALLEA Working Group E-Humanities addresses challenges in digital humanities, focusing on transparency in linking resources to publications, recognising updates as scholarly contributions, reevaluating authorship, fostering digital skills, and adjusting evaluation methods.

The second section offers recommendations for assessing specific digital outputs like editions, databases, infographics, code, blogs, and podcasts. Each case study includes practical examples and suggested readings.

 

Open-Science Guidance for Qualitative Research: An Empirically Validated Approach for De-Identifying Sensitive Narrative Data – Rebecca Campbell, McKenzie Javorka, Jasmine Engleton, Kathryn Fishwick, Katie Gregory, Rachael Goodman-Williams, 2023

Abstract:  The open-science movement seeks to make research more transparent and accessible. To that end, researchers are increasingly expected to share de-identified data with other scholars for review, reanalysis, and reuse. In psychology, open-science practices have been explored primarily within the context of quantitative data, but demands to share qualitative data are becoming more prevalent. Narrative data are far more challenging to de-identify fully, and because qualitative methods are often used in studies with marginalized, minoritized, and/or traumatized populations, data sharing may pose substantial risks for participants if their information can be later reidentified. To date, there has been little guidance in the literature on how to de-identify qualitative data. To address this gap, we developed a methodological framework for remediating sensitive narrative data. This multiphase process is modeled on common qualitative-coding strategies. The first phase includes consultations with diverse stakeholders and sources to understand reidentifiability risks and data-sharing concerns. The second phase outlines an iterative process for recognizing potentially identifiable information and constructing individualized remediation strategies through group review and consensus. The third phase includes multiple strategies for assessing the validity of the de-identification analyses (i.e., whether the remediated transcripts adequately protect participants’ privacy). We applied this framework to a set of 32 qualitative interviews with sexual-assault survivors. We provide case examples of how blurring and redaction techniques can be used to protect names, dates, locations, trauma histories, help-seeking experiences, and other information about dyadic interactions.

 

Humanities Guåhan to archive and digitize resource center | Guam News | postguam.com

“Humanities Guåhan received an award from the National Endowment for the Humanities Pacific Islands Cultural Initiative to fund the coestablishment of the Pacific Islands Humanities Network.

This funding will go toward developing a new digital resource center to preserve and enhance accessibility to valuable educational and cultural resources related to Guåhan, Micronesia and the broader Pacific region, according to Humanities Guåhan….”

The role of gender and coauthors in academic publication behavior – ScienceDirect

Abstract:  This paper contributes to the literature on differences in publication behavior of male and female scientists by examining two natural experiments in Germany that exogenously varied the attractiveness of journals. As a result of transformative open access publication agreements, journals published by Springer Nature and Wiley became more attractive as outlets for authors in Germany, while Elsevier journals lost some of their attractiveness within Germany due to substantial cancellations by university libraries. Studying 243,375 published articles in economics between 2015 and 2022, our findings suggests that men tend to seek reputation, while women favor visibility through open access, at least at the margin. While authorship in teams can dilute these behavioral patterns, female economists publish more single-authored papers. Overall female researchers appear to contribute more to the public good of open science, while their male colleagues focus on private reputation. These findings may offer an additional explanatory channel for the academic gender gap.

 

 

The Winners of the 2023 Einstein Foundation Award for Promoting Quality in Research – Einstein Foundation Award

“Initiative for Transparency in the Social Sciences (BITSS) advocates for ethical, transparent, and reproducible research in the social sciences in order to address the credibility crisis in the sciences and ensure that political decisions are based on robust data. BITSS develops the infrastructure required to support transparent social science research practice, such as the Preprint Service MetaArXiv and the Social Science Reproduction Platform (SSRP), which crowdsources attempts to assess and improve the reproducibility of social science research. The Initiative also conducts its own meta-research to validate scientific findings. BITSS has reached tens of thousands of social scientists across the world through its training and learning materials on Open Science practices. The Initiative was founded at the Center for Effective Global Action at the University of California in Berkeley in 2012 and has developed into one of the world’s most active organizations in the field of Open Science in the social sciences. The winner of the Institutional Award receives €200,000.”

[2311.09657] Open Access in Ukraine: characteristics and evolution from 2012 to 2021

This study investigates development of open access (OA) to publications produced by authors affiliated with Ukrainian universities and research organisations in the period 2012-2021. In order to get a comprehensive overview we assembled data from three popular databases: Dimensions, Web of Science (WoS) and Scopus. Our final dataset consisted of 187,135 records. To determine the OA status of each article, this study utilised Unpaywall data which was obtained via API. It was determined that 71.5% of all considered articles during the observed period were openly available at the time of analysis. Our findings show that gold OA was the most prevalent type of OA through a 10 years studied period. We also took a look at how OA varies by research fields, how dominant large commercial publishers are in disseminating national research and the preferences of authors regarding where to self-archive articles versions. We concluded that Ukraine needs to be thoughtful with engagement with large publishers and make sure academics control publishing, not for profit companies, which would monopolise research output distribution, leaving national publishers behind. Beyond that we put a special emphasis on the importance of FAIRness of national scholarly communication infrastructure in monitoring OA uptake.

Communication Scholarship and the Quest for Open Access

Abstract:  The advent of black, green, and gold open access publication models poses unique questions  for scholars of communication. Plato’s (1956) classic critique of writing in the legend of Theuth and Thamus warned that the printed word “rolls about all over the place, falling into the hands of those who have no concern with it” (pp. 69–70). More than 2 millennia later, scholars and administrators at all levels of the discipline face just such a phenomenon. As scholars of cyberspace debate whether “information wants to be free” (Levy, 2014), a communication perspective involves consideration of the importance of authorship and attribution amid an ever-shifting array of digital publishing options and subversions. The purpose of this study is to investigate the ongoing transformation of academic publishing by examining black, green, and gold open access models, the responses of the communication discipline, and ongoing questions surrounding the nature and extent of accessibility. As access options for research and publication continue to evolve, this study hopes to provide coordinates for administrators seeking to navigate questions concerning the what, how, and why of communication scholarship in a digital age. 

How to make research reproducible: psychology protocol gives 86% success rate

“In a bid to restore its reputation, experimental psychology has now brought its A game to the laboratory. A group of heavy-hitters in the field spent five years working on new research projects under the most rigorous and careful experimental conditions possible and getting each other’s labs to try to reproduce the findings.

Published today in Nature Human Behaviour1, the results show that the original findings could be replicated 86% of the time — significantly better than the 50% success rate reported by some systematic replication efforts….”

Transparency, openness and reproducible research practices are frequently underused in health economic evaluations

Objective: To investigate the extent to which articles of economic evaluations of healthcare interventions indexed in MEDLINE incorporate research practices that promote transparency, openness and reproducibility. Study design and setting: We evaluated a random sample of health economic evaluations indexed in MEDLINE during 2019. We included articles written in English reporting an incremental cost-effectiveness ratio (ICERs) in terms of costs per life years gained, quality-adjusted life years and/or disability-adjusted life years. Reproducible research practices, openness and transparency in each article was extracted in duplicate. We explored whether reproducible research practices were associated with self-report use of a guideline. Results: We included 200 studies published in 147 journals. Almost half were published as open access articles (n=93; 47%). Most studies (n=150; 75%) were model based economic evaluations. In 109 (55%) studies, authors self-reported use a guideline (e.g., for study conduct or reporting). Few studies (n=31; 16%) reported working from a protocol. In 112 (56%) studies, authors reported the data needed to recreate the ICERs for the base case analysis. This percentage was higher in studies using a guideline than studies not using a guideline (72/109 [66%] with guideline vs. 40/91 [44%] without guideline; risk ratio 1.50, 95% confidence interval 1.15 – 1.97). Only 10 (5%) studies mentioned access to raw data and analytic code for reanalyses. Conclusion: Transparency, openness and reproducible research practices are frequently underused in health economic evaluations. This study provides baseline data to compare future progress in the field. 

Evaluación de la ciencia en acceso abierto digital diamante – CLACSO

From Google’s English:  “Within the framework of the Diamond World Open Access Summit, which took place from October 23 to 27 in Toluca, Mexico, at the Autonomous University of the State of Mexico, headquarters of Redalyc, the Network of Scientific Journals of Latin America and the Caribbean, Spain and Portugal, and of which CLACSO is one of the organizers, we present this volume, which proposes a conceptual and methodological framework to characterize the knowledge published in Open Access Digital Diamante.

This book shows the application of the methodology to the scientific production of Social Sciences, Art and Humanities by authors from all over the world published in Ibero-American magazines between 2005 and 2022. In this, use is made of the database provided by Redalyc, the which accounts for a specific publication model: Diamond Digital Open Access.”