Quality and peer review survey results, created by the science public interest charity Sense about Science and information analytics business Elsevier, published today at the start of international Peer Review Week
Ten years on from the influential peer review survey of 2009, new findings show researchers have confidence in peer review, but strains on the system – including the increasing volume of research outputs and platforms to access research – are causing them to worry about quality and reliability.
The survey found that researchers don’t want to replace the peer review process, as 75 percent are satisfied with it (compared to 69 percent in 2009). Researchers want to improve and extend the system. It is a question of when peer review should be used and knowing when it has been used. Over a third – 37 percent – admitted they had doubts over the quality of at least some research outputs they had encountered in the last week.
The study comes from the same teams from Sense about Science and Elsevier, as 2009, with over 3,000 respondents, randomly selected from the Scopus database of over 3 million published researchers.
Commenting on the survey, Sense about Science Director Tracey Brown said, “What this survey shows is that our inroads into getting wider understanding of peer review are now not enough for research publishing in the 2020s, or for people to understand what quality checks have been done.
“We urgently need to see a common language and greater transparency about what has been reviewed. Does it include data? What criteria were used? Is it even a published study? Remember many research users are arriving at information via search engines, not academic portals.”
Other results include:
Respondents want more information available alongside research papers, including an indicator to show whether someone else had tried to reproduce the research (82 percent). (see fig. 10, p20 of full report).
Most researchers (76 percent) believe that data and supporting material that accompany research articles should be reviewed. (see fig. 9, p19 of full report).
Two–thirds of participants feel that reviewers would benefit from clearer guidance on reviewing criteria, while many felt that recognition of reviewing work was important for maintenance of a healthy peer review system, most commonly from employers (see fig. 12, p22 of full report).
Few researchers (38 percent) agree that the public understand the concept of peer review. There is strong support (77 percent) for a clear indication that material has been reviewed; and most (70 percent) feel explaining research context, and the implications of research in lay terms would be most helpful.
Despite the trend towards using technology to evaluate evidence, few respondents (8 percent) felt the evaluation that relied on artificial intelligence would qualify as peer review.
There are four areas for urgent action:
Amid rising volume of research outputs and information platforms, researchers want improved signals, be able to cross-check work easily.
Better training, information, and more career-based recognition is raised by researchers in this and other studies. Finding a way to remove the current inconsistencies in reviewer instructions is paramount.
Agreement about the role of technology in peer review, e.g., using it to manage the rising volume of submissions and alleviate the burden on reviewers, without losing the benefits of human judgement? Without discussion, the use of AI may further disrupt people’s ability to trust content.
Ensure that research is put in context: articles include easy-to understand explanations of research; and a common language is used to communicate to both researchers and public alike what has been reviewed and the type of review done.
Adrian Mulligan, Research Director for Customer Insights, Elsevier, said, “This study highlights the growing pressure being placed on research communication and importantly the value of peer review. Maintaining the integrity of the peer review system is paramount so it’s important various stakeholders work together to ensure reviewers receive recognition, have clear guidance and quality tools to support them in their roles.”
Researchers are invited to read the full report on Sense about Science’s website: https://senseaboutscience.org/activities/peer-review-survey-2019
In 2009, Elsevier and Sense About Science joined forces for one of the largest ever international researcher surveys on the topic of peer review. It asked researchers for their thoughts on aspects of the process; for example, what it should do, how it was performing, and how it could be improved. The insights became an influential reference document for publishers and policy makers alike.
About Sense about Science
Sense about Science is an independent charity that promotes the public interest in sound science and evidence. www.senseaboutscience.org
About Elsevier
Elsevier is a global information analytics business that helps scientists and clinicians to find new answers, reshape human knowledge, and tackle the most urgent human crises. For 140 years, we have partnered with the research world to curate and verify scientific knowledge. Today, we’re committed to bringing that rigor to a new generation of platforms. Elsevier provides digital solutions and tools in the areas of strategic research management, R&D performance, clinical decision support, and professional education; including ScienceDirect, Scopus, SciVal, ClinicalKey and Sherpath. Elsevier publishes over 2,500 digitized journals, including The Lancet and Cell, 39,000 e-book titles and many iconic reference works, including Gray’s Anatomy. Elsevier is part of RELX, a global provider of information-based analytics and decision tools for professional and business customers. www.elsevier.com
SOURCE Elsevier and Sense about Science