This week is ‘Peer Review Week’ – a time to ‘celebrate’ that aspect of scientific publishing that academia loves to hate. – Research

“Imagine if every statistical analysis was accompanied by comments from a professional statistician; if every method described had been critiqued by a methodological expert; if interpretations could be published of the same analysis from a wide variety of people with different training, backgrounds or experience. How much richer would the research record be? How much more useful than each of us only passing our own personal judgement on each article, bounded by our own inevitably narrow experience, and unshared with others?

This was all part of my thinking when coming up with Octopus, the platform that is designed to be the new, digital-first primary research record for scientific work….”

Credibility of scientific information on social media: Variation by platform, genre and presence of formal credibility cues | Quantitative Science Studies | MIT Press

Abstract:  Responding to calls to take a more active role in communicating their research findings, scientists are increasingly using open online platforms, such as Twitter, to engage in science communication or to publicize their work. Given the ease with which misinformation spreads on these platforms, it is important for scientists to present their findings in a manner that appears credible. To examine the extent to which the online presentation of science information relates to its perceived credibility, we designed and conducted two surveys on Amazon’s Mechanical Turk. In the first survey, participants rated the credibility of science information on Twitter compared with the same information in other media, and in the second, participants rated the credibility of tweets with modified characteristics: presence of an image, text sentiment, and the number of likes/retweets. We find that similar information about scientific findings is perceived as less credible when presented on Twitter compared to other platforms, and that perceived credibility increases when presented with recognizable features of a scientific article. On a platform as widely distrusted as Twitter, use of these features may allow researchers who regularly use Twitter for research-related networking and communication to present their findings in the most credible formats.


Reproducibility: expect less of the scientific paper

“Many calls have been made to improve this scenario. Proposed measures include increasing sample sizes, preregistering protocols and using stricter statistical analyses. Another proposal is to introduce heterogeneity in methods and models to evaluate robustness — for instance, using more than one way to suppress gene expression across a variety of cell lines or rodent strains. In our work on the initiative, we have come to appreciate the amount of effort involved in following these proposals for a single experiment, let alone for an entire paper.

Even in a simple RT-PCR experiment, there are dozens of steps in which methods can vary, as well as a breadth of controls to assess the purity, integrity and specificity of materials. Specifying all of these steps in advance represents an exhaustive and sometimes futile process, because protocols inevitably have to be adapted along the way. Recording the entire method in an auditable way generates spreadsheets with hundreds of rows for every experiment.

We do think that the effort will pay off in terms of reproducibility. But if every paper in discovery science is to adopt this mindset, a typical high-profile article might easily take an entire decade of work, as well as a huge budget. This got us thinking about other, more efficient ways to arrive at reliable science….”

Open Grant Proposals · Business of Knowing, summer 2021

“One of those informal frontiers is crowdfunding for scientific research. For the past year, I’ve worked on Experiment, helping hundreds of scientists design and launch crowdfunding campaigns for their research questions. Experiment has been doing this for almost a decade, with more than 1,000 successfully funded projects on the platform. The process is very different than the grant funding mechanisms set up by agencies and foundations. It’s not big money yet, as the average fundraise is still ~$5,000. But in many ways, the process is better: faster, transparent, and more encouraging to early-career scientists. Of all the lessons learned, one stands out for broader consideration: grant proposals and processes should be open by default.

Grant proposals that meet basic requirements for scientific merit and rigor should be posted online, ideally in a standardized format, in a centralized (or several) database or clearinghouse. They should include more detail than just the abstract and dollar amount totals that are currently shown now on federal databases, especially in terms of budgets and costs. The proposals should include a DOI number so that future work can point back to the original question, thinking, and scope. A link to these open grant proposals should be broadly accepted as sufficient for submission to requests from agencies or foundations….

Open proposals would make research funding project-centric, rather than funder-centric….

Open proposals would promote more accurate budgets….

Open proposals would increase the surface area of collaboration….

Open proposals would improve citation metrics….

Open proposals would create an opportunity to reward the best question-askers in addition to the best question-answerers….

Open proposals would give us a view into the whole of science, including the unfunded proposals and the experiments with null results….”

A New Option for Scientific Exchange and an Alternative to the Commentary Format – Patricia J. Bauer, 2021

“Letters to the Editors will be disseminated online only to permit more rapid publication and to keep the discussion timely and responsive. They will be hosted on Figshare, which is an online open-access repository and the site that hosts all of the journal’s Supplemental Material. To further speed dissemination, accepted Letters to the Editors will not be copyedited or held for replies but instead will be disseminated as quickly as possible on acceptance. Letters to the Editors will have DOIs but, fitting their existence in the liminal space between a formal publication and an unmediated social media conversation, they will not be indexed (i.e., discoverable through PubMed, PsycInfo, etc.). To facilitate connections between the target article and Letters to the Editors, they will be linked to each other. …”

Lunch & Learn Discussion: The Demise of Documents? Structuring Content for AI | Industry Announcements and Events SSP-L

“A major shift in content consumption has transpired over the years and across many industries. For scholarly publishers the transition from journal-based collections to article-based content forced digital transformation for many organizations. We are experiencing another shift. No longer is content designed and optimized for human consumption. For content to be found by your target audience, it’s best served up in a semantically tagged structure in an open digital format. The foundation for today’s content must be machine-readable, which means structure and semantics. Join industry experts Mark Gross, Marjorie Hlava, and Jay Ven Eman for this informal lunch and learn session as they explore the topic of how you prepare for a world of machine readable data….”

Data journals and data reports – don’t miss out on this useful publishing format! | UCL Open@UCL Blog

“Why not publish a data report article?

For a researcher who produces large amounts of data or works heavily with software and code for analysis, getting proper credit for their efforts can be a problem. Traditionally, an academic article is written in a format where a hypothesis is tested, results produced and analysed, and ends with a conclusion. This format increasingly is a poor fit for the work of many and data journals are one solution to this issue. The goal of this kind of journal is to publish a type of article usually referred to as a data report which focusses on announcing and describing the output of research projects which are resources, raw data, databases or similar and can be of use to the research community in general.

Publishing with a data journal offers several benefits. First, a data report article is more formal than a publication of data files in a repository and is a peer reviewed publication which then contributes to a researcher’s publication record which is important for CVs and advancement for many. Second, they allow a more detailed explanation of a dataset and any analysis or code related to it than is usually otherwise possible. Third, the appearance of an article in a recognised journal can help to drive visibility of a dataset for other researchers. In practice it my often be the case that a repository will be used to host material which is discussed at length in a paper….”

“Positively Disrupt(ing) Research Culture for the Better”: An Interview with Alexandra Freeman of Octopus – The Scholarly Kitchen

“In early August, it was announced that UK Research and Innovation (UKRI) would provide significant funding for a new open publishing platform. Called Octopus, this initiative is not yet fully launched, but when it is it plans to “provide a new ‘primary research record’ for recording and appraising research “as it happens’”; UKRI calls Octopus “a ground-breaking global service which could positively disrupt research culture for the better.” I reached out to Octopus’s founder, Dr. Alexandra Freeman, to ask some questions about Octopus and its plans for the future….”

David Worlock | Developing digital strategies for the information marketplace | Supporting the migration of information providers and content players into the networked services world of the future.

“If STM publishers were successful in going [to fee-based gold] Open Access , and supporting a creator-pays business model , how will they cope with the next migration , if that is towards Open Platform , and funder pays in a context that does not really seem to require publishers in quite the same way at all…

But the really interesting part of the [Octopus] proposal is the break-up of the article itself . Dr Freeman sees it as dividing into eight different segments , each of them appearing on the platform as soon as they are ready , and thus each element being susceptible to review at that point . Her eight sections are :  Problem ; Hypothesis; Methodology/Protocol ; Data/Results ;Analysis; Interpretation : Real-world Implications ; Peer Review. It will be seen that the thinking leans towards the Open Science insistence in separating the publication of the first three elements in time prior to results being available . It also encompasses another strand of funder thinking – all the work that has been accepted and funded , through increasingly expensive selection processes , should subsequently appear on a platform and be peer-reviewed. The process of publisher/editor selection may not now be wanted on board ….”

Funding agreed for a platform that will change research culture – UKRI

“Research England grants £650,000 to help build Octopus into a new global service for scholarly communication.

Funding has been agreed to help develop a ground-breaking global service which could positively disrupt research culture for the better.

Announced today by the science minister, Amanda Solloway, Octopus Publishing Community Interest Company (CIC), in collaboration with Jisc, will receive £650,000 over three years from Research England’s emerging priorities fund.

The money will support development of a new platform for the scientific community. Called Octopus, it will provide a new ‘primary research record’ for recording and appraising research ‘as it happens’….”

Octopus. Built for Scientists.

“Scientific knowledge should not be locked behind paywalls, or only available to those who can read and write in English.

Scientific ideas and findings should be shared as quickly as possible.

Scientific work should be judged on its merits, and not on how good a “story” it tells: and so should scientific researchers.

These principles underlie the design of Octopus: a new way to share scientific work that recognises and rewards good practice, and serves the needs of both scientists and science itself….

In Octopus you publish work in units smaller than a “paper”.

You can write and share one of 8 kinds of publication (though we support custom types for different fields and research types):

Problem – a neatly defined scientific problem
Hypothesis/Theoretical Rationale – an original hypothesis relating to an existing published Problem or the rationale for how you think the Problem could be addressed
Method/Protocol – a practical method of testing an existing published Hypothesis
Data/Results – raw data or summarised results collected according to an existing published Method (can be linked to a data repository)
Analysis – a statistical or thematic analysis of existing published Data or Results
Interpretation – a discussion around an existing published Analysis
Translation/Application – “real world” applications arising from an existing published Interpretation
Review – a considered, detailed review of any of the above kinds of publication …”

Introducing Octopus – YouTube

“Dr Alexandra Freeman gives an overview of Octopus, A new way to publish your research that’s fast, free and fair.

Octopus focuses on recording primary research. It is not based on papers allowing users to publish smaller types of outputs instantly within a new structure, whether it is a hypothesis, a method, data, an analysis or a peer review …”

eLife authors relay their experiences with Executable Research Articles | Labs | eLife

“We are grateful to these authors for taking their time to share their feedback with us, and for helping us showcase how Executable Research Articles can help improve the transparency, reproducibility and discoverability of research content across a variety of research subjects. Executable Research Articles are an open-source technology available to all, and we encourage any authors or publishers interested in the format to [get in touch] for more information….”

Experimental Publishing collaboration with POP, the Politics of Patents project · COPIM

“The experimental publishing group at COPIM is collaborating with four research ?and book publishing projects:

?One focuses on POP and Data books ?working together with Mattering Press.

A second one, in collaboration with Open Humanities Press, explores the notion of Combinatorial Books that are made by reusing existing texts beyond established citation practices. Both involve innovative re-use of source data and texts. 

A third project, X-Sketchbook, in collaboration with TIB Hannover (Germany), The Bartlett School of Architecture (UCL, London, UK), and Open Book Publishers, will explore the state of the art of experimentation in architectural publishing.

And a fourth project, Citizen Science for Research Libraries—A Guide, in collaboration with TIB Hannover and the LIBER Citizen Science Working Group, will explore ways to assist research libraries in setting up Citizen Science programs at their institutions….”