Open Scholarship Priorities and Next Steps: Public Workshop Registration, Mon, Dec 5, 2022 at 8:30 AM | Eventbrite

“The National Academies of Sciences, Engineering, and Medicine’s Roundtable on Aligning Incentives for Open Scholarship will organize a one-day public workshop in conjunction with its Fall 2022 meeting. The workshop will explore actions being taken by various stakeholder organizations to foster the broad adoption of policies and practices in support of open scholarship. It will bring together participants from universities, scholarly societies, federal agencies, and private research funders. A Proceedings of a Workshop–in Brief will be prepared by designated rapporteurs and distributed broadly.

The public is invited to register to join virtually….”

Widespread use of National Academies consensus reports by the American public | PNAS

Significance

Advocates for open access argue that people need scientific information, although they lack evidence for this. Using Google’s recently developed deep learning natural language processing model, which offers unrivalled comprehension of subtle differences in meaning, 1.6 million people downloading National Academies reports were classified, not just into broad categories such as researchers and teachers but also precisely delineated small groups such as hospital chaplains, veterans, and science fiction authors. The results reveal adults motivated to seek out the most credible sources, engage with challenging material, use it to improve the services they provide, and learn more about the world they live in. The picture contrasts starkly with the dominant narrative of a misinformed and manipulated public targeted by social media.

Abstract

In seeking to understand how to protect the public information sphere from corruption, researchers understandably focus on dysfunction. However, parts of the public information ecosystem function very well, and understanding this as well will help in protecting and developing existing strengths. Here, we address this gap, focusing on public engagement with high-quality science-based information, consensus reports of the National Academies of Science, Engineering, and Medicine (NASEM). Attending to public use is important to justify public investment in producing and making freely available high-quality, scientifically based reports. We deploy Bidirectional Encoder Representations from Transformers (BERT), a high-performing, supervised machine learning model, to classify 1.6 million comments left by US downloaders of National Academies reports responding to a prompt asking how they intended to use the report. The results provide detailed, nationwide evidence of how the public uses open access scientifically based information. We find half of reported use to be academic—research, teaching, or studying. The other half reveals adults across the country seeking the highest-quality information to improve how they do their job, to help family members, to satisfy their curiosity, and to learn. Our results establish the existence of demand for high-quality information by the public and that such knowledge is widely deployed to improve provision of services. Knowing the importance of such information, policy makers can be encouraged to protect it.

Who Uses Open Access Research? Evidence from the use of US National Academies Reports   | Impact of Social Sciences

“A fundamental principle of open access is that publication technology enables the widest possible audience for research findings. However, the extent to which open research is used outside of academia is often underexplored. Drawing on a dataset covering over a million user comments about their use of US National Academies consensus study reports, Ameet Doshi, Diana Hicks, Matteo Zullo and Omar I. Asensio find widespread use of open research in the public sphere….

Our classification project reveals that the impact of these reports extend far beyond the research community (see Results, Fig 1). We find that half of all report downloads are used for non-academic purposes, including to improve the provision of services by medical professionals, local and regional planners, public health workers, and veterans’ advocates, to name just a few of the 64 total categories of report use.  Heavy use is made of Academies reports on STEM education and how people learn by teachers, school administrators and teachers’ coaches.  Other notable reports with their prominent users included Dying in America (chaplains), Nutrient Requirements for Beef Cattle (farmers), and Best Care at Lower Costs (clinicians and hospital administrators)….

Open access repositories require significant resources, both technological and human, to sustain and innovate. The National Academies Press, for example, has developed an engaging user interface to incentivize browsing and ease of access to NASEM publications. The PubMed Central server, developed and managed by the US National Institutes of Health (NIH), requires millions of dollars per year to operate. Our research indicates there is an identifiable payoff to society for these taxpayer investments into people, technology and design to support OA publishing….

Librarians and open access advocates have long presupposed that open access to high-quality scientific knowledge could and should be viewed as a public good. Our empirical research suggests that the initial utopian aspirations regarding the public use and societal impact of OA may indeed rest on sound footing.”

 

Board/Program Director – Government-University-Industry Research Roundtable

“The National Academies of Sciences, Engineering, and Medicine seeks a Program Director to develop and manage the portfolio of activities related to innovation and S&T competitiveness within the Policy and Global Affairs Division’s U.S. Science and Innovation Policy Theme. In particular, the incumbent will serve as the director of the Government-University-Industry Research Roundtable (GUIRR), a forum for dialogue among top government, university, and industry leaders of the national science and technology enterprise. …”

Study reveals strong demand for open-access science

“The study, published Feb. 23, 2022, in the Proceedings of the National Academy of Sciences (PNAS), analyzed the reasons for 1.6 million downloads of National Academies of Sciences, Engineering, and Medicine (NASEM) consensus reports, considered among the highest credibility science-based literature.

The resulting analysis, which included U.S. downloads only, is the first to look at who is using such information and why. Professor Diana Hicks, Assistant Professor Omar I. Asensio, and Ph.D. students Matteo Zullo and Ameet Doshi, all of Georgia Tech’s School of Public Policy, co-authored the study.

They found that while nearly half of the reports were downloaded for academic purposes, even more were accessed by people outside strictly educational settings, such as veterans, chaplains, and writers. The word “edification” appeared 3,700 times in the data set, signaling a strong desire for lifelong learning among users….”

Widespread use of National Academies consensus reports by the American public | PNAS

In seeking to understand how to protect the public information sphere from corruption, researchers understandably focus on dysfunction. However, parts of the public information ecosystem function very well, and understanding this as well will help in protecting and developing existing strengths. Here, we address this gap, focusing on public engagement with high-quality science-based information, consensus reports of the National Academies of Science, Engineering, and Medicine (NASEM). Attending to public use is important to justify public investment in producing and making freely available high-quality, scientifically based reports. We deploy Bidirectional Encoder Representations from Transformers (BERT), a high-performing, supervised machine learning model, to classify 1.6 million comments left by US downloaders of National Academies reports responding to a prompt asking how they intended to use the report. The results provide detailed, nationwide evidence of how the public uses open access scientifically based information. We find half of reported use to be academic—research, teaching, or studying. The other half reveals adults across the country seeking the highest-quality information to improve how they do their job, to help family members, to satisfy their curiosity, and to learn. Our results establish the existence of demand for high-quality information by the public and that such knowledge is widely deployed to improve provision of services. Knowing the importance of such information, policy makers can be encouraged to protect it.

Study Reveals Strong Demand for Open-Access Science

While a dominant narrative of American life paints a bleak picture of poorly informed internet partisans duking it out over a landscape denuded of anything resembling truth or reality, a new study from the Georgia Tech School of Public Policy offers a different take while also advancing the use of machine learning in the social sciences and an understanding of the importance of open-access, science-based information to everyday Americans.

The study, published Feb. 23, 2022, in the prestigious Proceedings of the National Academy of Sciences (PNAS), analyzed the reasons for 1.6 million downloads of National Academies of Sciences, Engineering, and Medicine (NASEM) consensus reports, considered among the highest credibility science-based literature.

 

Developing a Toolkit for Fostering Open Science Practices: Public Release Registration, Thu, Sep 30, 2021 at 3:30 PM | Eventbrite

“The National Academies of Sciences, Engineering, and Medicine’s Roundtable on Aligning Incentives for Open Science will host a public release of Developing a Toolkit for Fostering Open Science Practices: Proceedings of a Workshop on Thursday, September 30, 2021 from 3:30-4:30 pm EDT. Please register in advance to receive information on how to participate in the event.”

Developing a Toolkit for Fostering Open Science Practices: Proceedings of a Workshop | The National Academies Press

“The National Academies Roundtable on Aligning Incentives for Open Science, established in 2019, has taken on an important role in addressing issues with open science. The roundtable convenes critical stakeholders to discuss the effectiveness of current incentives for adopting open science practices, current barriers of all types, and ways to move forward in order to align reward structures and institutional values. The Roundtable convened a virtual public workshop on fostering open science practices on November 5, 2020. The broad goal of the workshop was to identify paths to growing the nascent coalition of stakeholders committed to reenvisioning credit/reward systems (e.g., academic hiring, tenure and promotion, and grants)to fully incentivize open science practices. The workshop explored the information and resource needs of researchers, research institutions, government agencies, philanthropies, professional societies, and other stakeholders interested in further supporting and implementing open science practices. This publication summarizes the presentations and discussion of the workshop.”

Micah Vandegrift Appointed ARL Visiting Program Officer for Accelerating Social Impact of Research – Association of Research Libraries

“The Association of Research Libraries (ARL) has named Micah Vandegrift as a visiting program officer in the Scholars & Scholarship program for July 2021–July 2022. Vandegrift is the open knowledge librarian at NC State University Libraries.

As visiting program officer, Vandegrift will design and deliver a pilot experience for a cohort of eight ARL member libraries that are advancing open research practices at their institutions. The pilot Accelerating the Social Impact of Research (ASIR) program will help participants develop a strategic approach for advancing the social impact of science, aimed at building and reinforcing institutional points of influence for open research practices. This initiative is in coordination with the US National Academy of Sciences, Engineering, and Medicine (NASEM) roundtable on Aligning Incentives for Open Science and with the NASEM Board on Research Data and Information (BRDI)….”

Guide to Supporting Open Scholarship for University Presidents and Provosts

“Open scholarship entails a culture shift in how research is conducted in universities. It requires action on the part of university administration, working in concert with faculty, sponsors and disciplinary communities.  Universities should consider steps in three areas: •  Policies:  Language and guidance should be reviewed for alignment with open scholarship, in particular: (1) academic hiring, review, tenure and promotion (valuing diverse types of research products; metrics that  incentivize the open dissemination of articles, data, and other research outputs; and valuing collaborative research); (2) intellectual property (ownership, licensing and distribution of data, software, materials and publications); (3) research data protection (for data to be stored and shared through repositories); (4) attribution (recognizing full range of contributions);  and (5) privacy (insuring that privacy obligations are met).  •  Services and Training:  Researchers need support to assure that data and other research objects are managed according to FAIR Principles: findable, accessible, interoperable and reusable.  While the specific solution must be tailored to the discipline and research, common standards, including Digital Object Identifiers (DOIs), must be followed. •  Infrastructure:  Archival storage is required for data, materials, specimens and publications to permit reuse.  Searchable portals are needed to register research products where they can be located and accessed. Universities can recognize efficiencies by utilizing external resources (including existing disciplinary repositories) and by developing shared resources that span the institution when external resources do not exist….”

Statement on the Scholarly Merit and Evaluation of Open Scholarship in Linguistics | Linguistic Society of America

“Open Scholarship can be a key component for a scholar’s portfolio in a number of situations, including but not limited to hiring, review, promotion, and awards. Because Open Scholarship can take many forms, evaluation of this work may need different tools and approaches from publications like journal articles and books.  In particular, citation counts, a common tool for evaluating publications, are not available for some kinds of Open Scholarship in the same form or from the same providers as they are from publications. Here we share recommendations on how to assess the use of Open Scholarship materials including and beyond citations, including materials that both have formal peer review and those that do not not.

For tenure & promotion committees, program managers, department chairs, hiring committees, and others tasked with evaluating Open Scholarship, NASEM has prepared a discipline-agnostic rubric that can be used as part of hiring, review, or promotion processes. Outside letters of evaluation can also provide insight into the significance and impact of Open Scholarship work. Psychologist Brian Nosek (2017) provides some insight into how a letter writer can evaluate Open Scholarship, and includes several ways that evaluation committees can ask for input specifically about contributions to Open Scholarship. Nosek suggests that letter writers and evaluators comment on ways that individuals have contributed to Open Scholarship through “infrastructure, service, metascience, social media leadership, and their own research practices.” We add that using Open Scholarship in the classroom, whether through open educational materials, open pedagogy, or teaching of Open Scholarship principles, should be included in this list. Evaluators can explicitly ask for these insights in requests to letter writers, for example by including the request to “Please describe the impact that [scholar name]’s openly available research outputs have had from the research, public policy, pedagogic, and/or societal perspectives.” These evaluations can be particularly important when research outputs are not formally peer reviewed.

For scholars preparing hiring, review, promotion, or other portfolios that include Open Scholarship, we recommend not only discussing the Open Scholarship itself, but also its documented and potential impacts on both the academic community as well as broader society. Many repositories housing Open Scholarship materials provide additional metrics such as views, downloads, comments, and forks (or reuse cases) alongside citations in published literature. The use and mention of material with a Digital Object Identifier (DOI) can be tracked using tools such as ImpactStory, Altmetric.com, and other alternative metrics. To aid with evaluation of this work, the creator should share these metrics where available, along with any other qualitative indicators (such as personal thank-yous, reuse stories, or online write-ups) that can give evaluators a sense of the impact of their work. The Metrics Toolkit provides examples and use cases for these kinds of metrics. This is of potential value when peer review of these materials may not take the same form as with published journals or books; thoughtful use and interpretation of metrics can help evaluators understand the impact and importance of the work.

The Linguistic Society of America reaffirms its commitment to fair review of Open Scholarship in hiring, tenure, and promotion, endorses all of these approaches to peer review and evaluation of Open Scholarship, and encourages scholars, departments, and personnel committees to take them into careful consideration and implement language about Open Scholarship in their evaluation processes.”