Open Science Symposium: Open Practices and Transparency in Business Studies and Economics

by Guido Scherp

On 27 April 2023, the ZBW – Leibniz Information Centre for Economics invited researchers of business studies and economics to Berlin for a day of exchange on Open Practices. In the morning a Meet-Up took place and in the afternoon a symposium on “Open Science – Research Transparency in Economics” (German). Both formats are part of the ZBW’s commitment to creating a discussion and networking space on the topic of research transparency for economics and business studies researchers.

The symposium was particularly dedicated to the topics of reproducibility and replication. Four national and international business researchers were invited to share their experiences with open practices for this topic and related food for thought. The event was moderated by Marianne Saam, Professor of Economics, in particular Digital Economics, at the Department of Socioeconomics at the University of Hamburg and head of the programme area “Open Economics” at the ZBW.

An open and collaborative approach pays off

Melanie Schienle, Professor of Statistical Methods and Econometrics at the Karlsruhe Institute of Technology, presented the Covid19-Nowcasthub in her talk “Transparent collaborative forecasts and nowcasts in real time” (German).

This is a platform for forecasting (forecast = 4 weeks, nowcast = estimate now-state) the incidence of hospitalization, which is based on a collaborative and combinatorial approach. To improve forecast quality, multiple models from different teams were combined using a standardized framework and an open, collaborative space for stakeholders to share was created. Jointly, the prediction quality and combination of the individual prediction models were assessed, evaluated, and interpreted. An attempt was also made to quantify the uncertainty that is always present in forecasts. Interestingly, the overall quality of the forecast combined from the different models performed better than the individual forecasts. The forecast data as well as the nowcast models are openly available on GitHub. The studies evaluating the forecast models or their combination were preregistered. In addition to quality assurance, this form of transparency and collaboration naturally also serves to promote trust in the results, says Schienle.

Through reuse, Open Science unfolds its true impact

oachim Gassen, Professor of Accounting and Auditing at Humboldt University of Berlin, shared his experiences from the “Open Science Data Center” (German) in his talk on “Making Open Science Matter: How can we increase the reusability and impact of research?” (PDF, German).

The center was created as part of the Collaborative Research Center “Accounting for Transparency” which is funded by the German Research Foundation. Gassen believes that there are still a number of hurdles to overcome on the way to Open Science. On the one hand, there is a lack of know-how, fear of mistakes, excessive demands, and also unresolved issues of licensing and data protection. On the other hand, there is a great interest in reusable data and tools, which is hindered by a lack of interoperability and content that is difficult to find. However, there is also the question of the impact of Open Science, or the question of what one wants to achieve with Open Science. The real impact of Open Science, according to Gassen, will ultimately unfold where others can build on the results and methods of previous research. An important, but unfortunately difficult issue is the recognition of corresponding achievements in career decisions.

Transparency and reproducibility do not end with publication

Dr Lars Vilhuber, Data Editor of the American Economic Association (AEA), reflected in his presentation “Continuous Transparency and Reproducibility in Academic Publications: Approaches and Techniques,” the experience of the AEA’s “Data and Code Availability Policy”, in effect since 2019.

With this policy, authors must provide so-called “Replication Packages” when submitting to the Association’s journals. A replication package should contain or reference the data and code underlying a publication and sufficiently describe the runtime environment. On this basis, Vilhuber’s team can perform reproducibility checks with the aim of obtaining computationally identical results. Since 2019, 1,500 publications have been checked in this way. However, Vilhuber points out limitations in this process. For example, programming errors are not checked, or complex computer environments can make computational reproducibility difficult or even impossible. Even if it is sometimes difficult to achieve today, the goal must be an all-encompassing replication package for the sake of transparency. When creating this package, authors must also think from the perspective of the reviewers, for example, what they need to carry out a reproduction (“computational empathy”), says Vilhuber. And transparency and reproducibility do not end with the journal publication; one has to get away from the concept of the final publication. For example, building replications can follow, including in the form of replication games. Vilhuber also emphasized that the implementation of replications must be better anchored in academic training.

Self-reflection must become part of economic science work

In the last presentation, “Economics needs self-reflection: Are transparency standards enough?” (PDF, German), Professor Jörg Ankel-Peters of RWI – Leibniz Institute for Economic Research and founding member of the Institute for Replication addressed the resilience of modern empirical economics. In principle, great progress has been made in implementing research transparency, Ankel-Peters said, for example in the area of data and code sharing and prespecification. But the road does not end with the creation of transparency standards, he said. A culture of meta-analysis (how can everything be put together to form a consolidated, generalized picture) and organized skepticism (methodological questioning of published results) is lacking in economics, he said. In this respect, there are signs of a “replication crisis” in economics, Ankel-Peters said. He followed this up with the question of what economic researchers actually replicate for, and brought the concept of “policing replications” into play. A “policing replication” is one that directly addresses and “challenges” a previously published paper. This can have a “regulatory effect,” Ankel-Peters said. However, the proportion of “policing replications” in published replications is very small, at 1- 3%, he said. And this type of replication is not without problems, he said, since they are both costly and highly socially and culturally sensitive. Nevertheless, Ankel-Peters called for critical self-reflection to become part of work in economics.

Continuing the discourse on Open Practices in business studies and economics

In the presentations, it became clear that economics researchers are currently quite aware of a further development of research transparency in economics and business studies. The examples presented show how Open Practices are becoming increasingly entrenched. In the discussion that followed, however, it also became clear that the journey is not yet over and that the discourse must continue, for example, on what should be achieved by creating transparency. For example, should more collaboration and reuse be achieved, should systematic quality assurance be pursued through reproducibility standards, or should targeted use of replications be the focus? In this context, a change in the work culture, for example the development of a different error culture, as well as the systematic teaching of necessary skills, also belong. In line with this, two groups have already formed in the Meet-Up to further develop the topics “Incentives, Culture, Conflict Readiness” and “Education”.

This text has been translated from German.

You may also find interesting:

About the Author:
Dr Guido Scherp is Head of the “Open-Science-Transfer” department at the ZBW – Leibniz Information Centre for Economics. He can also be found on LinkedIn and Twitter.
Portrait: ZBW©, photographer: Sven Wied

Photos: David Außerhofer, ZBW©

The post Open Science Symposium: Open Practices and Transparency in Business Studies and Economics first appeared on ZBW MediaTalk.

Research Integrity and Reproducibility are Two Aspects of the Same Underlying Issue – A Report from STM Week 2022

Observations on reproducibility and research integrity from London STM Week

The post Research Integrity and Reproducibility are Two Aspects of the Same Underlying Issue – A Report from STM Week 2022 appeared first on The Scholarly Kitchen.

RDA webinar on research software and reproducibility

Our Danish colleagues are organising an interesting webinar on research software and reproducibility – which is part of a longer series on FAIR code. This webinar will be the last one in a series of webinars about FAIR research software. The Danish and Swedish RDA nodes started a webinar series to bring outputs and the good…

Trust in Science: Views from Three Experts

Today’s guest post is a recap of the recent SSP webinar, Ask the Experts: Trust in Science, with Tracey Brown (Sense About Science), Richard Sever (Cold Spring Harbor Laboratory Press), and Eefke Smith (STM) by the moderator, Anita de Waard (Elsevier).

The post Trust in Science: Views from Three Experts appeared first on The Scholarly Kitchen.

PLOS ONE partners with the Children’s Tumor Foundation to trial Registered Reports

Today, the Children’s Tumor Foundation and PLOS ONE are pleased to announce a partnership to trial the integration of Registered Reports in the grant application and publication process:   The Children’s Tumor Foundation (CTF) and

Promoting reproducibility by emphasizing reporting: PLOS ONE’s approach

0000-0002-4592-9214As we celebrate PLOS ONE’s ten year anniversary, we continue our commitment to uphold rigorous standards for publications across all scientific disciplines. This is not a small goal. Several years ago, the staff editors at

New Paleontology Guidelines Enforce Ethics, Reproducibility

Ethics is a cornerstone of science, informing everything from how we design our experiments to what we do with the resulting data. Given the diverse nature of the research results published in PLOS ONE, no short and simple set of ethical guidelines can cover every situation. Thus, it is important for the journal to adapt and expand its ethical standards as the journal itself expands.

The field of paleontology, by its very nature, presents some special situations in ethics. Although the fossil subjects are long-dead, rendering matters of patient consent or laboratory animal care non-existent, other complicated concerns ranging from legalities to reproducibility must be taken into account. Any journal that hopes to be a major player in the study of fossils must confront these issues head-on.

As the volunteer section editor for paleontology at PLOS ONE, I am thrilled by the growth in the number of high-quality publications related to my field. I also want to make sure that all of these papers are held to the highest ethical standards, and many of my colleagues and I felt it was important to provide explicit ethical guidelines focused on paleontology. After extensive and thoughtful discussion with the journal’s internal editors and other interested parties, I am happy to announce that a specific set of editorial standards for paleontology submissions is now in place.

Critically, reproducible research in paleontology requires a long-term guarantee of accessibility and safety for fossils. This means that all fossils should be deposited in a permanent repository, such as a museum or university collection. Unfortunately, there is no guarantee that collections owned by private individuals—no matter how noble their intentions—will be accessible in the long-term. In one notable recent case, the family of a fossil enthusiast sold off the bulk of his scientifically important collection after he died. Some of the specimens had even been published in the peer-reviewed literature, but there is now little guarantee that any of those fossils will be accessible in fifty years, or even five. Moreover, not everything that calls itself a museum is a permanent collection; some are little more than showroom floors for a commercial fossil business. Some of these fossils do end up in permanent museum collections, but until this happens, it is extremely hazardous to publish on the specimens. Reproducibility and accessibility are key, as reflected in the new policies.

Ethical consideration is also critical for fossil collection in the field. Stories abound of skeletons in the Gobi Desert being looted for the most marketable parts (such as skulls or claws), which then end up for auction in Europe or North America. In fact, one recent PLOS ONE paper discussed dinosaur skin impressions salvaged from the mess left by fossil poachers who carted off more enticing pieces. Legal loopholes often mean that the specimens can then be traded or sold elsewhere, often accompanied by official-looking paperwork that purports to legitimize the original export. This horrible practice drains the world of its historical heritage and destroys scientific information. Thus, the new ethics policy explicitly prohibits publication of specimens that were obtained without permission or legal export.

This is a great day for paleontology at PLOS ONE, helping to ensure the journal’s future as a trustworthy publication with the highest ethical standards. I challenge everyone—authors, editors, readers, and reviewers—to carry the torch forward into a better world.

About the Author:  Dr. Andrew Farke is a vertebrate paleontologist and an academic editor at PLOS ONE. Andy also has his own blog, The Open Source Paleontologist, and can be found on Twitter @andyfarke.

Image: The fossil reptile Captorhinus (collection of Raymond M. Alf Museum of Paleontology)