Eugenia Shmidt, MD1, Michael Andreone, MD1, Michael Evans, MS1, Claire Neumann, MA, CMPP2, Neen Certo. LeMaster, MLIS2, Brian Jacobson, MD, MPH3 1University of Minnesota, Minneapolis, MN; 2American College of Gastroenterology, North Bethesda, MD; 3Massachusetts General Hospital, Harvard Medical School, Boston, MA Introduction: Unbiased peer review is essential for scientific research. Most journals employ a single-anonymous (SA) peer review process in which the identities of manuscript authors and institutions are known to anonymous reviewers. Recently, a minority of scientific journals switched to a double-anonymous (DA) review process but the impact of this change on the intended outcome of minimizing bias has not been well established. In January 2021, Clinical and Translational Gastroenterology (CTG) switched to a DA review process. The aim of this study is to determine the effect of SA vs DA review on reviewer bias. Methods: We performed an audit of full-length manuscripts submitted to CTG from January 1, 2021 to December 31, 2022. Submission data collected included country of origin, h-index of first authors (FA) and senior authors (SA), institutional prestige of the SA, number of reviewers invited per manuscript, standardized scoring of review quality, and time to review. Institutional prestige was determined by ranking among the Top 10 in Nature, Top 100 in EduRank or Top 200 in US News. Numeric and categorical outcome measures were compared between 2021 and 2022 using Wilcoxon rank-sum and chi-square tests. Results: In 2021, among 669 submitted manuscripts, 125 (19%) were accepted for publication. In 2022, 420 papers were submitted and 96 (23%) were accepted. Among all papers, the h-index of both first and senior authors significantly decreased from 2021 to 2022 (median h-index of FA was 7 vs 5, P=0.03; h-index of SA was 29 vs 24, P=< 0.001). This was also seen among the FAs of accepted papers (Figure). Among all papers, there were proportionally more SAs from prestigious institutions in 2022 vs 2021 (P=0.02). Of note, however, there was no difference in the prestige of SAs of accepted papers (P=0.99). Overall, a larger number of reviewers had to be invited in 2022 vs 2021 (among all papers, median 6 reviewers invited vs 9, P< 0.001; among accepted papers, 5 reviewers invited vs 8.5, P< 0.001). Discussion: Switching from a SA to a DA review policy was associated with more submissions and publications by authors with lower h-indices. Additionally, the DA review policy may have prevented an over-representation of senior authors from prestigious institutions among published papers. Lastly, reviewers may be less likely to review a manuscript when blinded to author and institution identities. Taken together, these findings suggest that a DA review process may decrease bias.
Figure: H-indices of FA and SA among all manuscripts and accepted manuscripts in 2021 vs 2022
Disclosures: Eugenia Shmidt: Bristol Myers Squibb – Grant/Research Support. Michael Andreone indicated no relevant financial relationships. Michael Evans indicated no relevant financial relationships. Claire Neumann: American College of Gastroenterology – Employee. Neen LeMaster indicated no relevant financial relationships. Brian Jacobson indicated no relevant financial relationships.
Eugenia Shmidt, MD1, Michael Andreone, MD1, Michael Evans, MS1, Claire Neumann, MA, CMPP2, Neen Certo. LeMaster, MLIS2, Brian Jacobson, MD, MPH3. P6195 - Double Anonymous Review May Decrease Bias: Experience From Clinical and Translational Gastroenterology, ACG 2025 Annual Scientific Meeting Abstracts. Phoenix, AZ: American College of Gastroenterology.
