Metrics Literacy

Developing online material to educate researchers and research administrators about bibliometrics, usage metric, altmetrics, and technometrics.

Phase One

Stefanie Haustein, Tristan Lamonica

Scholarly metrics such as the impact factor and h-index are everywhere in academia, but their use and interpretation is often based on an incorrect understanding of what the indicators actually reflect. The Metrics Literacy project seeks to develop efficient and effective practices to educate researchers and research administrators in the application and interpretation of quantitative scholarly indicators. The project aims to increase the proper use of metrics in inapt contexts.

Using a mixed methods approach including qualitative and quantitative research, we will test and identify best practices to increase metrics literacy in academia. The goal is to improve the use of scholarly indicators and reduce the misuse of metrics such as the h-index and impact factor or comparing publication and citation counts across different disciplines. The project will build on currently available, but uncoordinated, resources with the objective to improve the understanding and appropriate use of scholarly metrics include the Leiden ManifestoMetrics Toolkit, and the recently published Measuring Research handbook. Expanding metrics literacy in a community which increasingly relies on quantitative indicators for review, tenure, and promotion will help to reduce the oversimplification of research evaluation and diminish associated adverse effects, such as profuse self-citations, self-plagiarism, or so-called ‘salami’ publishing, where authors split one manuscript into several to increase publication counts.

The first of four phases of a larger metrics literacy research program is currently funded by the University of Ottawa’s Seed Funding Opportunity and focuses on the two main objectives: to improve the understanding and use of scholarly indicators in research assessment and advance bibliometric indicator development. Multimedia resources will be targeted at specific audiences (e.g., researchers, funders) and address one of five main questions and four fields of application (i.e., usage metrics, altmetrics, bibliometrics, technometrics). Conducting A/B testing, entry/exit surveys, focus groups, and interviews, the Metrics Literacy project will identify the most efficient means to educate researchers and administrative staff about scholarly metrics. Metrics literacy educational resources will be made available using the Software Carpentry framework; research results to inform indicator development will be disseminated in peer-reviewed journals and at conferences.