ACS Publications. Most Trusted. Most Cited. Most Read
A Healthier Peer Review Process Would Improve Diversity
My Activity

Figure 1Loading Img
  • Free to Read
Comment

A Healthier Peer Review Process Would Improve Diversity
Click to copy article linkArticle link copied!

Open PDF

ACS Applied Materials & Interfaces

Cite this: ACS Appl. Mater. Interfaces 2020, 12, 37, 40987–40989
Click to copy citationCitation copied!
https://doi.org/10.1021/acsami.0c11528
Published September 1, 2020

Copyright © 2020 American Chemical Society. This publication is available under these Terms of Use.

This publication is licensed for personal use by The American Chemical Society.

Copyright © 2020 American Chemical Society
I appreciate the effective and timely actions taken by chemistry journals including ACS Applied Materials & Interfaces, (1)Angewandte Chemie, (2) and Chemical Sciences (3) in response to imbalances in gender equity, diversity, and inclusion. Historically, minorities such as women, people of color, and immigrants have been disproportionately harmed by societal prejudices and pressures that are only exacerbated in the laboratory environment. Together as a building block of the scientific community, we must take a stand to make the academic world an equitable, accepting, and safe environment. Achieving fair and unbiased peer review is an initial step that should be thoroughly considered by the scientific community, to facilitate progress in academic diversity in publications. (3) Limitations in the current practice of peer review and some potential approaches to overcome these issues are presented below.

Editorial Process

Click to copy section linkSection link copied!

The initial evaluation of manuscripts in many prestigious journals is conducted by editorial boards, who may be unintentionally biased (e.g., through a wide range of confirmational biases (4) including the halo effect (5)) about several parameters including the reputation of the authors and institutions, national origin, and/or publishing policies. Such bias in editorial boards, even though unintentional, may also affect the way they select potential reviewers and make decisions based on those reviewers’ reports. This might be one reason that some early career researchers look for opportunities to include “big-name” scientists in their work to increase their chances of publication. Clearly, there are many hidden variables in this practice that adversely affects the health and impact of the peer review process, including connections of any kind that make reviewers biased or opinionated in their review reports. Although honorary authorship is unethical and has been prohibited by both universities and publishers (e.g., the Ombuds office authorship guideline of Harvard Medical School (6)), the current literature is not devoid of honorary authors. All of these issues are in favor of excluding (i) concise evaluation of manuscripts and (ii) early career/less-famous scientists, in the absence of any strong scientific reasons.
There are some strategies that could address issues of confirmation bias. The first and foremost is designing suitable training to make editorial boards fully aware of the effects of such unnoticeable biases on the paper and reviewer selection processes. One possible model for such training might be drawn from the strategies proposed by Dr. Daniel Kahneman in his book titled Thinking, Fast and Slow. (7) Another possibility for journals would be to choose editors and editorial boards from outside academia (familiar examples of such policies are Nature and Science), who may be less affected by such confirmation biases. Although actively conducting research enables editors to provide a properly contextualized and nuanced view of research, their limited availability in terms of time might affect their decisions, specifically when dealing with the comments of reviewers who fail to meet ethical guidelines in providing unbiased and constructive comments in an appropriate way.
Another useful and informative strategy that could be applied at the publisher level is to conduct a thorough annual meta-analysis of the relation between the diversity of submitting authors and manuscript handling processes, including percentage of selected submissions from each author (based on their gender, race, etc.) encompassing the peer review process and final decisions on manuscripts. Such analyses would have to be performed at the publisher level, as the relevant information is available only to publishers. A recent example is the very comprehensive report on the gender gap in scholarly communications in the chemical sciences published by the Royal Society of Chemistry. (3,8) Such meta-analysis data inform publishers about the existence of such biases and therefore help them to create robust guidelines to minimize bias of any type during the manuscript review process. The other possibility is to establish an ombuds office within the journals (e.g., the one in place at the Lancet (3)) to investigate unfair situations/incidents. This would be totally distinct from appeal requests by researchers, as such appeals are considered in the editorial office, and the resulting decisions may be affected by the same confirmation biases. On the other hand, the existence of an independent ombuds office within a publisher—if not in individual journals—ensures fair and unbiased consideration of authors’ concerns.

Publishers’ Peer Review Policies/Guidelines

Click to copy section linkSection link copied!

The central goal of the conventional peer review system is to ensure scientific rigor and robustness of the information presented in manuscripts to improve or maintain scientific integrity and progress. (9,10) I believe the main issue facing us in achieving highly efficient outcomes of healthy peer review processes is that the true goal of peer review has to some extent been overshadowed by biases and other influencing factors, including (i) the absence of a code of conduct for reviewers that would make them more accountable and (ii) editors’ hesitancy to intervene in reviews that are unnecessarily discouraging and/or rude/harsh, which particularly affect early-career and minority researchers in academic settings. Because publishers are central stakeholders, by refocusing on the original goals of peer review for editors and reviewers, they can ameliorate the toxic peer review environment and improve both publishing diversity and scientific integrity. For example, the practice of publishing reviewers’ comments and the authors’ responses by journals like Nature Communications is a very direct approach to increasing the accountability of reviewers for their comments; for example, it reduces inappropriate encouragement of authors to cite the reviewers’ own manuscripts.
One strategy that should be revived and/or strengthened is for publishers to establish proper protocols for journals and editors to improve their accountability in eliciting positive reviewer behavior and constructive feedback, which are in line with the central goals of the peer review process. (11) In addition, journals and editors should exclude reviewers who fail to meet publishers’ minimum ethical expectations/guidelines. (11)
Another useful strategy could be making double-blind review the default, not an option. The current double-blind review option offered by some publishers (e.g., Nature Publishing Group) may not actually result in unbiased review, as some reviewers may see taking the double-blind option as a weakness in the manuscript.
Another long-term strategy would be an establishment of center of excellence to create golden standard and universal definitions and guidelines for healthier peer review process that will be followed by all publishers. This is essential in creating a robust and fair scientific environment which have a capacity in improving diversity.
The final major issue with the current peer review process is that some senior reviewers ask their lab members and/or postdocs to review manuscripts assigned to them. This is problematic for several reasons, including putting pressure on lab members to come up with criticisms to satisfy their PI—clearly not in line with the goals of the peer review process. This practice should not be tolerated by publishers or journals; not only is it unlikely to produce fair and constructive feedback, but the peer review credit goes to the PI rather than the person who actually did the work of reviewing the manuscript.
The main goal of this short commentary is to improve awareness regarding certain current issues in the peer review process. Awareness among stakeholders is an important initial step toward solving such problems. A good example is the efforts to raise awareness of and address the gender gap in science. Researchers, women scientists who have been discriminated against in scientific recognition and promotions, and supportive journalists and legislators worked together to address the question, “where are women in the sciences?” by documenting inequities and exploring the root causes. As a result, stakeholders, such as grant agencies and decision makers, felt more accountability and took some action. For example, many scientists (including the director of NIH (12)) refuse to speak or present on all-male science panels (also known as “manels”). As another example, the United Nations started a campaign called HeForShe for the advancement of gender equality. (13) In this case, many male scientists proactively worked to decrease the gender gap through blogs, research papers, and talks and by boycotting male-dominant conferences and workshops, and they published their outcomes in scientific and public forums to bring more awareness. Very recently, my collaborators and I counted male vs female scientific Nobel prize nominees and recipients and found that when women were nominated, they often received the prize. (14) Our findings suggest that one critical reason for the huge awardee gender gap is that qualified women are not equitably included in the nomination process. Therefore, we proposed that one solution to minimizing the gender gap would be more frequent nomination of qualified women scientists for Nobel prizes by the scientific community. (14) Increasing stakeholders’ awareness in the causality of gender discrepancy in scientific Nobel prize may pave a way to reduce gender imbalances in future Nobel prize recipients. Similarly, increasing stakeholders’ awareness of current issues in the peer review process may enable the scientific community to create an integrated functioning platform among stakeholders with a unique capacity to improve diversity and scientific integrity.
In summary, the problematic issues in the current peer review process deserve more honest and thorough discussion and require raising awareness among stakeholders, including scientists, publishers, editors, and reviewers. This may help the scientific community improve manuscript-handling protocols and guidelines that refocuses us on the fundamental goals of the peer review process. I therefore urgently propose the modification of journals’ ethical policies and guidelines, editorial training, and reviewer selection in a transparent and robust manner to address the current peer review issues, toward the larger of goal of improving diversity and integrity of all types in scientific publishing.

Author Information

Click to copy section linkSection link copied!

  • Corresponding Author
    • Notes
      The author declares no competing financial interest.

    References

    Click to copy section linkSection link copied!

    This article references 14 other publications.

    1. 1
      Burrows, C. J.; Huang, J.; Wang, S.; Kim, H. J.; Meyer, G. J.; Schanze, K.; Lee, T. R.; Lutkenhaus, J. L.; Kaplan, D.; Jones, C.; Bertozzi, C.; Kiessling, L.; Mulcahy, M. B.; Lindsley, C. W.; Finn, M. G.; Blum, J. D.; Kamat, P.; Choi, W.; Snyder, S.; Aldrich, C. C.; Rowan, S.; Liu, B.; Liotta, D.; Weiss, P. S.; Zhang, D.; Ganesh, K. N.; Atwater, H. A.; Gooding, J. J.; Allen, D. T.; Voigt, C. A.; Sweedler, J.; Schepartz, A.; Rotello, V.; Lecommandoux, S.; Sturla, S. J.; Hammes-Schiffer, S.; Buriak, J.; Steed, J. W.; Wu, H.; Zimmerman, J.; Brooks, B.; Savage, P.; Tolman, W.; Hofmann, T. F.; Brennecke, J. F.; Holme, T. A.; Merz, K. M.; Scuseria, G.; Jorgensen, W.; Georg, G. I.; Wang, S.; Proteau, P.; Yates, J. R.; Stang, P.; Walker, G. C.; Hillmyer, M.; Taylor, L. S.; Odom, T. W.; Carreira, E.; Rossen, K.; Chirik, P.; Miller, S. J.; Shea, J.-E.; McCoy, A.; Zanni, M.; Hartland, G.; Scholes, G.; Loo, J. A.; Milne, J.; Tegen, S. B.; Kulp, D. T.; Laskin, J. Confronting Racism in Chemistry Journals. ACS Appl. Mater. Interfaces 2020, 12, 2892528927,  DOI: 10.1021/acsami.0c10979
    2. 2
      An Open Letter to Our Community . Angewandte Chem., Int. Ed.https://onlinelibrary.wiley.com/journal/15213773 (accessed 2020-08-10).
    3. 3
      Day, A.; Corbett, P.; Boyle, J. Is there a gender gap in chemical sciences scholarly communication?. Chemical Science 2020, 11, 22772301,  DOI: 10.1039/C9SC04090K
    4. 4
      Nickerson, R. S. Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology 1998, 2, 175220,  DOI: 10.1037/1089-2680.2.2.175
    5. 5
      Nisbett, R. E.; Wilson, T. D. The halo effect: evidence for unconscious alteration of judgments. Journal of Personality and Social Psychology 1977, 35, 250,  DOI: 10.1037/0022-3514.35.4.250
    6. 6
      Harvard Medical School Authorship Guidelines.https://hms.harvard.edu/sites/default/files/assets/Sites/Ombuds/files/AUTHORSHIP%20GUIDELINES.pdf (accessed 2020-08-10).
    7. 7
      Kahneman, D. Thinking, Fast and Slow. Macmillan: New York, 2011.
    8. 8
      Is Publishing in the Chemical Sciences Gender Biased? Driving Change in Research Culture; Royal Society of Chemistry: Cambridge, U.K., 2019. https://www.rsc.org/globalassets/04-campaigning-outreach/campaigning/gender-bias/gender-bias-report-final.pdf (accessed 2020-08-10).
    9. 9
      Weller, A. C. Editorial Peer Review: Its Strengths and Weaknesses; Information Today: Medford, NJ, 2001.
    10. 10
      Gerwing, T. G.; Rash, J. A. Constructive and collegial peer-review as a necessary precursor to data-driven environmental policy. Marine Policy 2020, 111, 103721,  DOI: 10.1016/j.marpol.2019.103721
    11. 11
      Beaumont, L. J. Peer reviewers need a code of conduct too: emphasize ethics for peer reviewers. Nature 2019, 572, 439,  DOI: 10.1038/d41586-019-02492-w
    12. 12
      NIH director will no longer speak on all-male science panels. Washington Post , June 12, 2019. https://www.washingtonpost.com/health/nih-director-will-no-longer-speak-on-all-male-science-panels/2019/06/12/fe3b6386-8d2c-11e9-adf3-f70f78c156e8_story.html (accessed 2020-08-10).
    13. 13
      HeForShe. Wikipedia.https://en.wikipedia.org/wiki/HeForShe (accessed 2020-08-10).
    14. 14
      Mahmoudi, M.; Poorman, J. A.; Silver, J. K. Representation of women among scientific Nobel Prize nominees. Lancet 2019, 394, 19051906,  DOI: 10.1016/S0140-6736(19)32538-3

    Cited By

    Click to copy section linkSection link copied!
    Citation Statements
    Explore this article's citation statements on scite.ai

    This article is cited by 14 publications.

    1. Morteza Mahmoudi. Improving Diversity and Integrity in Funding Agency Peer Review Processes. Molecular Pharmaceutics 2025, 22 (1) , 1-2. https://doi.org/10.1021/acs.molpharmaceut.4c00815
    2. Shahriar Sharifi, Nigel Reuel, Nathaniel Kallmyer, Ethan Sun, Markita P. Landry, Morteza Mahmoudi. The Issue of Reliability and Repeatability of Analytical Measurement in Industrial and Academic Nanomedicine. ACS Nano 2023, 17 (1) , 4-11. https://doi.org/10.1021/acsnano.2c09249
    3. Michele Cascella, Thereza A. Soares. Bias Amplification in Gender, Gender Identity, and Geographical Affiliation. Journal of Chemical Information and Modeling 2022, 62 (24) , 6297-6301. https://doi.org/10.1021/acs.jcim.2c00533
    4. Igor V. Alabugin. How to Review a Paper. ACS Chemical Health & Safety 2021, 28 (1) , 14-18. https://doi.org/10.1021/acs.chas.0c00107
    5. Xiaoqing Li, Xiang Chen, Xiaoxin Pan, Yongcheng Wu, Jiayun Wei, Jie Tang, Jie Pan, Zhijia Huang, Haohan Qu, Cheng Huang, Zhengrong Wei, Jun Zhang, Jinxia Duan, Hao Wang. Low‐Temperature Stable CsPbI 2 Br Perovskite for Solar Photovoltaics and Blue Photodetection. Advanced Optical Materials 2024, 12 (32) https://doi.org/10.1002/adom.202401512
    6. Farid Saleh. Peer review bullying threatens diversity, equity, and inclusion. Trends in Ecology & Evolution 2024, 39 (11) , 975-978. https://doi.org/10.1016/j.tree.2024.09.001
    7. Anusmriti Guragain, Pathiyil Ravi Shankar, Ian G. Wilson. Examining authorship in publications in selected health professions education journals. Academia Medicine 2024, 1 (2) https://doi.org/10.20935/AcadMed6221
    8. Ouadoud Oumaima, Ouadoud Mohammed, Eddaoui Ahmed, Chafiq Tarik. Blockchain-Enhanced Peer Review: A Novel Approach to Academic Publishing. 2024, 51-58. https://doi.org/10.1007/978-3-031-66850-0_6
    9. Tony Wall, Maribel Blasco, Stella M Nkomo, Marton Racz, Marcela Mandiola. In Praise of Shadows : Exploring the hidden (responsibility) curriculum. Management Learning 2023, 54 (3) , 291-304. https://doi.org/10.1177/13505076231171371
    10. Giovanni Vescio, Juan Luis Frieiro, Andrés F. Gualdrón‐Reyes, Sergi Hernández, Iván Mora‐Seró, Blas Garrido, Albert Cirera. High Quality Inkjet Printed‐Emissive Nanocrystalline Perovskite CsPbBr 3 Layers for Color Conversion Layer and LEDs Applications. Advanced Materials Technologies 2022, 7 (7) https://doi.org/10.1002/admt.202101525
    11. Pallavi Ahluwalia, Bhavna Gupta. Evaluation of manuscripts- A guide to reviewers. Indian Journal of Clinical Anaesthesia 2021, 8 (1) , 1-4. https://doi.org/10.18231/j.ijca.2021.001
    12. Morteza Mahmoudi, Loraleigh Keashly. Filling the Space: A Framework for Coordinated Global Actions To Diminish Academic Bullying. Angewandte Chemie 2021, 133 (7) , 3378-3384. https://doi.org/10.1002/ange.202009270
    13. Morteza Mahmoudi, Loraleigh Keashly. Filling the Space: A Framework for Coordinated Global Actions To Diminish Academic Bullying. Angewandte Chemie International Edition 2021, 60 (7) , 3338-3344. https://doi.org/10.1002/anie.202009270
    14. Morteza Mahmoudi. Gender Parity Among the Altmetric Top 100 Publications on COVID-19. Future Science OA 2021, 7 (2) https://doi.org/10.2144/fsoa-2020-0175

    ACS Applied Materials & Interfaces

    Cite this: ACS Appl. Mater. Interfaces 2020, 12, 37, 40987–40989
    Click to copy citationCitation copied!
    https://doi.org/10.1021/acsami.0c11528
    Published September 1, 2020

    Copyright © 2020 American Chemical Society. This publication is available under these Terms of Use.

    Article Views

    3510

    Altmetric

    -

    Citations

    Learn about these metrics

    Article Views are the COUNTER-compliant sum of full text article downloads since November 2008 (both PDF and HTML) across all institutions and individuals. These metrics are regularly updated to reflect usage leading up to the last few days.

    Citations are the number of other articles citing this article, calculated by Crossref and updated daily. Find more information about Crossref citation counts.

    The Altmetric Attention Score is a quantitative measure of the attention that a research article has received online. Clicking on the donut icon will load a page at altmetric.com with additional details about the score and the social media presence for the given article. Find more information on the Altmetric Attention Score and how the score is calculated.

    • This publication has no figures.
    • References


      This article references 14 other publications.

      1. 1
        Burrows, C. J.; Huang, J.; Wang, S.; Kim, H. J.; Meyer, G. J.; Schanze, K.; Lee, T. R.; Lutkenhaus, J. L.; Kaplan, D.; Jones, C.; Bertozzi, C.; Kiessling, L.; Mulcahy, M. B.; Lindsley, C. W.; Finn, M. G.; Blum, J. D.; Kamat, P.; Choi, W.; Snyder, S.; Aldrich, C. C.; Rowan, S.; Liu, B.; Liotta, D.; Weiss, P. S.; Zhang, D.; Ganesh, K. N.; Atwater, H. A.; Gooding, J. J.; Allen, D. T.; Voigt, C. A.; Sweedler, J.; Schepartz, A.; Rotello, V.; Lecommandoux, S.; Sturla, S. J.; Hammes-Schiffer, S.; Buriak, J.; Steed, J. W.; Wu, H.; Zimmerman, J.; Brooks, B.; Savage, P.; Tolman, W.; Hofmann, T. F.; Brennecke, J. F.; Holme, T. A.; Merz, K. M.; Scuseria, G.; Jorgensen, W.; Georg, G. I.; Wang, S.; Proteau, P.; Yates, J. R.; Stang, P.; Walker, G. C.; Hillmyer, M.; Taylor, L. S.; Odom, T. W.; Carreira, E.; Rossen, K.; Chirik, P.; Miller, S. J.; Shea, J.-E.; McCoy, A.; Zanni, M.; Hartland, G.; Scholes, G.; Loo, J. A.; Milne, J.; Tegen, S. B.; Kulp, D. T.; Laskin, J. Confronting Racism in Chemistry Journals. ACS Appl. Mater. Interfaces 2020, 12, 2892528927,  DOI: 10.1021/acsami.0c10979
      2. 2
        An Open Letter to Our Community . Angewandte Chem., Int. Ed.https://onlinelibrary.wiley.com/journal/15213773 (accessed 2020-08-10).
      3. 3
        Day, A.; Corbett, P.; Boyle, J. Is there a gender gap in chemical sciences scholarly communication?. Chemical Science 2020, 11, 22772301,  DOI: 10.1039/C9SC04090K
      4. 4
        Nickerson, R. S. Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology 1998, 2, 175220,  DOI: 10.1037/1089-2680.2.2.175
      5. 5
        Nisbett, R. E.; Wilson, T. D. The halo effect: evidence for unconscious alteration of judgments. Journal of Personality and Social Psychology 1977, 35, 250,  DOI: 10.1037/0022-3514.35.4.250
      6. 6
        Harvard Medical School Authorship Guidelines.https://hms.harvard.edu/sites/default/files/assets/Sites/Ombuds/files/AUTHORSHIP%20GUIDELINES.pdf (accessed 2020-08-10).
      7. 7
        Kahneman, D. Thinking, Fast and Slow. Macmillan: New York, 2011.
      8. 8
        Is Publishing in the Chemical Sciences Gender Biased? Driving Change in Research Culture; Royal Society of Chemistry: Cambridge, U.K., 2019. https://www.rsc.org/globalassets/04-campaigning-outreach/campaigning/gender-bias/gender-bias-report-final.pdf (accessed 2020-08-10).
      9. 9
        Weller, A. C. Editorial Peer Review: Its Strengths and Weaknesses; Information Today: Medford, NJ, 2001.
      10. 10
        Gerwing, T. G.; Rash, J. A. Constructive and collegial peer-review as a necessary precursor to data-driven environmental policy. Marine Policy 2020, 111, 103721,  DOI: 10.1016/j.marpol.2019.103721
      11. 11
        Beaumont, L. J. Peer reviewers need a code of conduct too: emphasize ethics for peer reviewers. Nature 2019, 572, 439,  DOI: 10.1038/d41586-019-02492-w
      12. 12
        NIH director will no longer speak on all-male science panels. Washington Post , June 12, 2019. https://www.washingtonpost.com/health/nih-director-will-no-longer-speak-on-all-male-science-panels/2019/06/12/fe3b6386-8d2c-11e9-adf3-f70f78c156e8_story.html (accessed 2020-08-10).
      13. 13
        HeForShe. Wikipedia.https://en.wikipedia.org/wiki/HeForShe (accessed 2020-08-10).
      14. 14
        Mahmoudi, M.; Poorman, J. A.; Silver, J. K. Representation of women among scientific Nobel Prize nominees. Lancet 2019, 394, 19051906,  DOI: 10.1016/S0140-6736(19)32538-3