Search Submit
Close

Peer Review for Academic Research

Archives of Otorhinolaryngology-Head & Neck Surgery. 2022;6(1):2
DOI: 10.24983/scitemed.aohns.2022.00155
Article Type: Perspective, Opinion, or Commentary

Abstract

The peer review process is used to evaluate research papers and journal articles submitted for publication, providing a filter that helps to ensure the quality, credibility, and validity of published research. Reviews are generally conducted by peers who have a deep understanding of the knowledge domain and possess expertise in the methodologies and instruments involved in the research. Academics are honored to be invited to review a paper, and they undertake to fulfill their scientific obligations as well as to uphold the academic integrity of the organization. The primary goal of a peer reviewer is to determine whether a work falls within the scope of the journal. In addition, reviewers should ensure that the study has been conducted according to established methodologies. Reviewers also look at the ethical aspects of a study, evaluate the presentation and readability of the work, and apply tenets of logic to assess the integrity of its arguments and conclusions. According to statistics, the average reviewer works on approximately eight reports each year. However, there is a dearth of information regarding the effectiveness of the peer review process. This article summarizes aspects of peer review in an attempt to elucidate the state of academic publishing.

Keywords

  • Article processing charge; critique; maladaptation; open access; publish

Introduction

Peer reviews were first implemented 300 years ago when only a few centralized societies had royal permission to publish scientific works [1]. In 1731, the Royal Society of Edinburgh employed this process in the assessment of medical articles submitted for publication. However, it took time for the process to gain acceptance, and it was not until the 1950s that the process was extensively applied [2]. Peer reviews were not initiated by The Lancet until 1976 [3]. At present, peer reviews constitute the primary factor in publication decisions.

Traditionally, journal editors have been solely responsible for the selection of articles for publication. As the quantity of scientific research grows rapidly, journals are receiving an ever-increasing volume of manuscripts, often overwhelming editors' capacity to manage. Additionally, today's research has also become increasingly sophisticated, causing the concern that editors by themselves may not be able to adequately assess all submissions. As a result, it has become apparent that the opinions of other experts should be considered.

For the purpose of ensuring consistency in evaluation, systematic training in the art of assessing based on established criteria may be necessary. The main objective of this article is to provide a general overview of the peer review process in academic research as well as its underlying elements.

Motivations for Reviewing

The peer review process is emblematic of the scientific method, wherein the science community assesses the quality, credibility, and validity of work submitted for publication [4]. It has been observed, however, that there are numerous instances where an outstanding paper is rejected during the peer review process, while poorly written articles sail through [5]. It is not uncommon to find accusations of bias against individuals who have a well-established reputation in their field. Moreover, the time-consuming nature of peer review has an unnecessarily detrimental impact on research dissemination due to delays in the peer review process. There is also no existing plagiarism detection system that has been proven effective at discovering all forms of plagiarism. Based on the issues raised above, it is evident that a rigorous peer review system must be implemented in academia [6,7]. A thorough review by one or more experts in a specific area of research allows the journal to determine whether the manuscript is suitable for publication, should be rejected, or needs to be revised. This review process can serve to uphold the academic credibility of the journal as well as ensure that the paper meets the needs of its readers.

The key to establishing a rigorous peer review system lies in identifying qualified reviewers. Typically, it will be possible to find a reviewer within a few days, but in some instances, it may take up to several weeks [8-11]. This issue poses an additional challenge to publications as it can be quite time-consuming to find reviewers and wait for them to review papers. There will be a significant delay between submission and publication because of this. In particular, senior experts are in high demand, and many reviewers may not handle the reviewing responsibilities in an efficient manner due to their busy schedules. Since peer review is currently proceeding slowly, it is suggested that reviewers should be provided with feedback as motivation for their contribution to the review process.

For reviewers, a high level of motivation can be attained by engaging in professional discussion and debate during a review process, both socially and intellectually. Through participation in the peer-review process, academics can also achieve considerable prestige and even promotion. It is a unique opportunity for reviewers, both from the academic community and publishers, to be recognized for their contributions.

Considering the amount of time reviewers devote to this high-demanding process, it is fair to assume that they may not be adequately compensated due to a lack of monetary reward. There has been disappointment and frustration expressed by certain individuals regarding the lack of monetary compensation they are entitled to; however, their complaints are widely ignored because numerous applicants are willing to take part in peer reviews for free.

Although academics generally do not get paid for reviewing, being asked to be a reviewer is a sign of being recognised as a professional with expertise in that field. It can be a valuable addition to one’s academic career. The other non-monetary benefit is that publishers often grant reviewers free access to papers if the journals are not open access.

Peer Review in Practice

An initial step of the review process involves the editorial team of the publisher reviewing the submissions to prioritize which submissions should be reviewed. After that, reviewers are selected based upon their expertise required to review a particular paper. A request for review and an abstract of the manuscript are then emailed to the appropriate reviewers. The abstract should convey the principal findings of the article to the reviewers and enable them to decide whether to proceed with the manuscript.

Upon acceptance of a request, the editorial team sends an email with a link to the journal's online system for tracking and review. There are usually guidelines for writing reviews, and sometimes, specific questions are asked regarding certain aspects of the review. It is also possible that some journals do not provide guidelines, at which point the reviewers may assess the papers as they see fit [8].

Once the reviewers have completed their review of the manuscript, they submit their reviews to a tracking system. A feature of the tracking system allows reviewers to submit separate comments to authors as well as recommendations to editors. Once all reviews are completed, the editorial team compiles them and presents a summary report to the editor, who then decides whether the manuscript should be accepted, rejected, or revised.

The authors are usually required to make either minor or major corrections to their manuscript before their manuscript is accepted as final. The revised manuscript is sent to the original reviewers to give their opinion on whether they are satisfied with the revisions. Providing the requirements of the journal are met, the article will then be accepted for publication. The entire review process can take a considerable amount of time.

The editorial staff must perform due diligence when selecting the individuals who will review the manuscript to ensure the individuals possess the necessary expertise to provide an objective and unbiased review in a timely manner. Editors need to consider not only the time constraints, but also the expertise, and inclinations of all potential reviewers. In many cases, reviewers are too busy to submit their reviews on time. Individuals lacking expertise in that particular area may be unable to provide a learned and unbiased review. In such cases, it is far preferable to reject the request than to submit a half-baked review later. The reviewer must also evaluate whether they possess the inclination to conduct a fair review and report any conflicts of interest that they might have. Since most publishers do not pay for reviews, one's willingness to fulfil the assignment plays a key role in the success of the review.

Models of Peer Review

The peer review process is continually changing as new models are developed and changes to the traditional models are continuously explored [12]. Peer review may be conducted in different ways by different journals, and each of these methods has its own advantages and disadvantages (Table 1). The most common types of peer review are single-blind peer review, double-blind peer review, open peer review, and transparent peer review.

 

 

Single-Blind Review
The single-blind review process refers to a situation in which the authors are not aware of the identities of the reviewers, although the reviewers can see the authors' names. With the personal information of the reviewers protected, they can provide their frank and straightforward opinions without worrying about external influences or pressures. Though single-blind peer review is the most commonly used type of confidentiality agreement in peer review, there can still be a possibility of negative reviewer bias if there is any academic or professional competition between the reviewers and the authors [13]. It might be disappointing for the authors that their work is being unfairly judged by someone hiding behind anonymity.

Double-Blind Review
In a double-blind review, the names of the reviewers as well as the authors are withheld, thereby isolating the reviewers from bias based on the reputation of the authors. When there could be a conflict of interest or a competitive dynamic between the authors and the reviewers, authors feel more comfortable with the double-blind peer review process due to its neutrality. After implementing double-blind reviews in Behavioural Ecology in 2001, researchers were surprised to discover a notable increase in the number of papers published by females as first-authors [14].

Even so, there are still some downsides regarding a double-blind review. Although double-blind peer review is intended to prevent bias, in the case of specialized research some reviewers may still be able to discern the identities of the authors. In addition, a manuscript would need to have all references to the authors and their related work removed to ensure the double-blind review process. This can be a significant burden on the authors to revise their original manuscripts, and there will also be considerable delays in the review process because of the revisions.

Open Peer Review
In open peer reviews, the identities of the authors and the reviewers are disclosed to all participants during the review process. An open process may force reviewers to think more thoughtfully about the research subject and write more comprehensive assessments. Openness may also aid in exposing potential conflicts of interest when they arise. The openness of reviews usually encourages reviewers to tone down their opinions or to keep a polite attitude throughout their review. In some journals, the names of the reviewers are published along with the manuscript in recognition of their value and contribution to the manuscript [15].

However, the identification of both authors and reviewers does allow authors to influence reviewers and establish mutually beneficial relationships through these relationships. It is also worth noting that non-anonymous reviewing does not provide reviewers with protection from adverse reactions from authors if they make negative comments. Some reviewers may be reviewing the work of their friends or colleagues. In this case, they may be inclined to emphasize the positive aspects of the research as they may be concerned with the possible consequences that arise from their review.

Transparent Peer Review
Several peer-reviewed journals operate on a transparent basis [16,17]. In a transparent peer review system, the readers will be able to see from the inception of the peer review process all the way to the final decision in the peer review process. Upon publication, peer review reports are made available online in conjunction with the article. This means that readers of the article will be able to view reviewer reports, editor's decision letters, and the authors' responses. The reviewers may either choose to remain anonymous, or if they wish, sign their reports. Journals can still choose whether to carry out a single-blind, double-blind, or open peer review during the review process. Like open peer reviews, transparent peer reviews increase the accountability of the peer review process. However, the transparent review model allows early-career researchers who are put in the embarrassing position of having to make negative remarks about the work of senior academics in their fields to keep their identities anonymous to avoid causing themselves future problems [17].

Principles for Reviewing a Manuscript

The golden rule of a thorough review is that reviewers should review papers in the same way that they would like their own papers to be reviewed. Unfortunately, most academics never undergo formal training on performing peer reviews and must therefore learn by trial and error. Many publishers provide resources to improve the quality of reviews, and online training courses are also available [18]. Nonetheless, formal training, whether long-term or short-term, can be highly beneficial [19].

Familiarity with statistical methods and a willingness to invest time in a review (at least 3 hours) can make a big difference in assessing a paper [20]. Reviewers must try to keep an open mind, read the paper several times, invest mental effort, conduct themselves ethically, and follow the guidelines of publishers. Once the reviewer decides to proceed with a review, the following suggestions can be quite helpful [21, 22].

The publisher may provide guidelines for the review process. The first step in reviewing a manuscript is to thoroughly read the guidelines. If the guidelines and/or questions are not provided, then the reviewer performs a general assessment of the manuscript based on its merits. In some journals, reviewers are provided with a manuscript review form in which they must answer a series of questions (Table 2). A few prestigious journals may even specify exactly what they expect from their reviewers in a very concise and direct manner.

 

 

In the second step, the reviewers evaluate the value of the paper. The reviewers must review the paper quickly to identify the main hypothesis and determine whether it is pertinent and compelling enough to merit publication. An even cursory glance would suffice to reveal whether the language is appropriate and whether the research is in fact substantive. After a quick assessment, the reviewers should examine the paper meticulously and make a list of key points pertaining to the strengths and weaknesses of the paper. It is imperative for the reviewers to identify any flaws in the research. The reviewers are also required to note errors in layout, terminology, and grammar. Figures and tables are analysed to ensure that they support the text.

A brief third reading is generally required to see how the paper is structured. A paper must follow a logical order: study hypotheses or research issues, research methodology, study design, data collection, and analysis. The reviewer must determine whether the paper is organized properly, paying attention to sub-headings, data presentation, and the cited references. The merits of the paper and any gaps in research should be identified. Any information that is not relevant to the topic should be noted. There is a need for reviewers to remain vigilant in their efforts to avoid unsubstantiated claims. Every comment should be backed up with facts, references, and concrete recommendations. An imperative point to remember is that in cases where the reviewer is confronted with information outside of their expertise, the details must be made clear in the review.

It is interesting to note that the attitudes of reviewers and authors may change when their roles are reversed. Authors who are more tolerant of a few mistakes may become overly critical when acting as a reviewer. The main reason for this is that reviewers seek to provide an in-depth review of a manuscript, while authors are pursuing publication as the goal. It is not uncommon for novel ideas that contrast with conventional beliefs to get rebuffed. This occurs mainly because reviewers tend to view themselves as protectors of what they have learned in their field [23,24]. Ultimately, editors act as balancing agents who are committed to publishing research of high academic quality but will also ensure that highly innovative papers will not be overlooked due to negative reviews. For reviewers who are excessively critical, it would be prudent for them to tone down their criticisms when they write the review.

A Detailed Examination of a Paper

Title and Abstract
Having fully understood the principles for reviewing a manuscript discussed above, the next step is to conduct a formal review of the manuscript following a thorough examination. First, it is imperative for the reviewer to assess each section of a submission from the title to the reference section. The title should be brief and informative. The abstract needs to be clear and provide the key points pertaining to the research. The reviewer ought to assess whether it conveys the central question and the methodology by which findings were reached.

Introduction
In the introduction section of the article, there should be a check as to whether the authors adequately describe the background and context of the research. The introduction should also clearly state the research question and briefly describe its importance within the context of existing research. As an aid to clarify the research question of the study, the authors should provide a brief review of the literature concerning one or more of the following topics: limitations and/or flaws in previous studies; unresolved gaps in the research; and/or contradictory findings between studies. A detailed explanation of the objectives of the study should be included at the end of the introduction section and should be relevant to the research question.

Materials and Methods
In the materials and methods section, the reviewer should investigate the criteria used for the inclusion and exclusion of participants in the study/experimental groups, and the control groups (if applicable). A sufficient follow-up/observation period should be established to allow for the possibility of significant changes or events occurring. Study populations should be representative to allow generalization of the results from a small sample to a larger population. For a case-control or cohort study, the study and control groups should be homogeneous (i.e., nearly identical in terms of all the relevant variables) to avoid potential selection bias. An in-depth description of the sample collection process is essential for ensuring accountability and transparency. For randomized controlled trials, blinding refers to concealing patient or sample allocations from participants, care providers, and those assessing the outcomes so that selection bias is eliminated at the recruitment stage. Simply stating that a procedure is blind is not sufficient. The authors need to clarify who was blinded and in what manner. A randomization scheme is intended to minimize selection bias and confounding in cohort and case-control studies, thereby reducing the likelihood of prognostic differences between intervention groups. The reviewer needs to examine whether the authors applied statistical correction methods (including restriction, matching, stratification, and multivariate analysis) to adjust for potential confounders. The reviewer should know losing more than 20% of patients due to a lack of follow-up or withdrawal from a trial seriously undermines the validity of the findings. Only citations of information sources relating to materials and procedures are insufficient. The authors should provide in detail the procedures and materials for all experiments described. There should be a clear and concise definition of the ranges or cut-offs of normal and abnormal values in the test results. All variables of interest (including dependent and independent variables) should be well measured and controlled. The scales used for the measurements should be clearly defined. It is necessary to describe the statistical methods used in detail, and they must be appropriate for the topic as well.

Results
The reviewer should verify, in the results section, that the findings are consistent with the paper's stated objective(s). In cases in which the results appear unrealistic, the reviewer should communicate their suspicions to the authors. It is desirable to illustrate descriptive statistics using tables, charts, or graphs so that data can be analysed constructively. It is imperative to carefully review all the inferences and the statistical significance of the results (p-values) to ensure that the correlation cannot be attributed to random chance.

Discussion
The reviewer should ensure that the results or findings are interpreted cautiously and reasonably. Any overinterpretation of the findings should be avoided. In addition, findings should be compared with those of previous studies to evaluate their validity and reliability. When current findings are inconsistent with those that exist in the literature, possible explanations for these inconsistencies should be proposed. Discrepancies between statistical significance and practical significance should be clarified and addressed. Authors need to include a discussion addressing in what way current research can be applied to generate significant results in clinical settings. There may be a variety of biases introduced into a study at any time, including selection bias (a non-representative study population) and information bias (imprecise measurement of the outcome or incorrect documentation of results). There are several types of information bias, including ascertainment bias due to inconsistent data collection methods, diagnostic suspicion bias due to the use of less-than-rigorous diagnostic methods, family history bias because of unclear patient family histories, and recall bias resulting from patients' vague memories of past events. Authors should present a concise description of the strengths and limitations of their research, as well as the limitations of the research. Suggestions for future research may be beneficial to overcoming current research limitations.

Conclusion
A clear conclusion must be stated at the end of the manuscript. Several studies conclude with an interpretation of value judgments relating to certain practices. It is crucial that the conclusions be based on direct analysis of the results of the paper. Furthermore, research results can be interpreted from a variety of perspectives, and the study results should only be applied to similar populations and situations. It is important that researchers do not overgeneralize their own results. Finally, the authors should be aware that a conclusion is simply derived from a subjective interpretation of the data and is therefore not an unquestionable truth.

References
A literature review indicates whether the authors are aware of similar research in their field and are up to date with recent developments. The references section of a manuscript should include the most relevant literature. It is recommended that additional references be provided to the author if the reviewer believes they are necessary. Citations of scholarly journals and books are preferred over websites. It is imperative that any gaps in the literature review be clarified. Some researchers, especially young scholars, list a large number of references to demonstrate their scholarly abilities; therefore, reviewers must also determine if all the references were relevant to the research objective. An omission to include the most recent studies should be brought to the attention of the authors.

Language-Related Issues
There is no need to point out small grammatical errors in the review, since the reviewer is not a proofreader or editor. The reviewer should be able to provide feedback on the quality of the language and whether any subsequent revisions are required. It is true that non-English speakers may fail to convey their thoughts eloquently, however, a competent copy editor can enhance the authenticity of their writing. Upon reviewing a manuscript, the reviewer should focus first and foremost on the quality of the work and the value of it, rather than on the language used to convey it.

Elements of a Comprehensive Review

A comprehensive review report of a paper should consist of two major sections. For the first section, it is appropriate for a reviewer to give a brief overview of the paper. This summary should include the objective of the study, the assumptions and hypothesis, the supporting materials, and conclusions. The reviewer should also indicate whether the research warrants publication. A thorough review allows the editors to determine whether the paper conveys the message intended, and whether the readers will be able to comprehend the significance of the research. A key feature of the first section of the review report is the emphasis on the favourable aspects of the paper; criticism of it comes later.

For the second section of the review report, the following issues need to be addressed in further detail, along with an analysis of each one. An imperative first point for the reviewer is to ascertain whether the main hypothesis or the major research issue merits further investigation. The reviewer should then evaluate whether the research methodology and approach adopted by the authors are appropriate. The third point that the reviewer should consider is the validity of the methods used for collecting the data. In addition, the reviewer should consider whether the findings are supported by the data. A critical element of the review process is ensuring that the conclusion of the study provides an answer to the research hypotheses or issues that are outlined in the introduction. There is a need for the reviewer to identify the contributions of the paper and whether it adds to our understanding of the subject. Lastly, the reviewer recommends whether the paper be accepted in its submitted form, whether the paper be accepted with major or minor changes, or whether it should be rejected (Table 3).

 

 

If the reviewer suggests changes, the review should address how the paper could be improved. Each criticism should be accompanied by concrete suggestions about how the paper might be improved. When major changes are required, the reviewer needs to ensure that they are explained with references to previous research that supports the suggestions. A reviewer should be able to identify major flaws, but they should also be able to maintain control over their urge to point out every single error.

When a review is returned to a journal, the editor will ask the authors to respond to the points raised by the reviewers. It is common for the reviewer to be asked to re-assess the paper after the authors have made revisions to ensure that the problems have been addressed effectively. It is not necessary that the authors agree with every comment made by the reviewer. Instead, the authors should specify the reasons for not accepting a suggestion, and the reviewer can then decide whether the explanation is acceptable. It is recommended that reviewers sandwich their criticisms between positive feedback and constructive remarks.

Ethical Considerations in Peer Review

Scholarly publications are subject to peer review, which plays an important role in ensuring the integrity of the record. As part of the peer review process, the scholarly community must demonstrate trust and participation, and everyone involved must behave responsibly and ethically. This section explains how qualified reviewers should conduct their reviews in an ethical manner [25].

Review of Selected Papers
It is the responsibility of reviewers to accept only papers that are relevant to their field of expertise. In addition, a reviewer must refuse to review papers that present a conflict of interest or are too closely connected to their field of expertise. Reviewers should accept only those papers for which they can provide a fair and unbiased opinion. Unscrupulous practices have been reported and should be avoided, including impersonation, undeservedly harsh criticism of colleagues, and reciprocal relationships (in which authors provide complimentary reviews to each other).

Timely Review
The publication of a paper is often delayed by reviewers failing to send in their reviews on time. In some cases, this delay can take up to several months. Reviewers should endeavor to complete their reviews within the timeframe stipulated by the journal. Papers should only be accepted for review if the reviewer has enough time to complete the job within the specified deadline. Any anticipated delays should be disclosed to the journal in advance. If this cannot be achieved, the paper should be rejected at first assessment.

Conflicts of Interest Disclosure
The publishers should be made aware of reviewers' conflicts of interest and whether they might interfere with their ability to provide an honest review of a manuscript. Such interests could be due to a person’s beliefs, funding considerations, involvement in competing or similar work, or a relationship with any of the authors. Any person working at the same institution as the authors, or who has been a mentor, mentee, close collaborator, or joint grant holder within the last 3 years should not agree to review. In such cases, the invitation to review should be declined and brought to the notice of the publisher.

Avoiding Denigration or False Accusations
Reviewers are typically allowed to provide confidential comments to the editor in addition to comments for the authors. Reviewers should ensure their comments and recommendations for the editor are consistent with those in the report for the authors. Since the authors will not be able to see the confidential comments, this place should not be used for denigration and false accusations against the authors [26].

Unbiased Review
A biased opinion can easily lead reviewers astray. Reviewers may exhibit biases associated with intellectual conservatism, or with a preference for papers in their field of interest, or with papers which they agree with. Reviewers tend to approve papers that demonstrate large sample sizes, detailed statistical analysis, and a well-controlled design. Reviewers also favor authors from their own country. Biases of these types can lead to the loss of innovative ideas [27-29]. These practices can also lead to the publication of substandard or dubious papers [30]. The malpractices may partially explain why an otherwise excellent paper is rejected but an imperfect paper is accepted. An example is that of George Akerlof's paper, "The Market for 'Lemons'," which was rejected in peer review. The article was rejected by two journals that labeled it as being "too trivial", while another rejected it because it was deemed to be "too novel", as it challenged the very basis of economics [31]. That paper went on to receive the Nobel Prize in economics in 2001 for its significant contribution to behavioral economics.

Other Unethical Practices
There are several other unethical practices that have occasionally been reported. A particular case is where a reviewer rejects a certain paper, but then uses the same or similar data to write an entirely different piece of work. Furthermore, in some fields, academics have also been proven to operate as a clique, in which they accept each other's papers while being highly critical of those of other academics in the same field. Occasionally, even when double-blind reviews are being conducted, some reviewers are able to determine the identities of the authors and manipulate the process to favor them.

After the Review
Reviewers should be willing to re-examine the paper following revision to ensure that all the suggestions have been duly considered. In some cases, after the final review of a paper was submitted, the reviewers discovered new information pertinent to the papers they reviewed. This updated information must be brought to the notice of the publisher, particularly if it could affect the original review. Additionally, reviewers must maintain confidentiality throughout a blind review process and do not reveal their participation even after publication.

Flaws of Peer Review

Even though peer review has become a widely accepted practice, there are still several obvious shortcomings that raise doubts regarding the process. One noteworthy example of misconduct is that of Schön [32]. Jan Hendrik Schön is a German physicist who has published over 80 papers in reputable journals such as Nature and Science. His apparent breakthroughs in semiconductors were later proved to be fraudulent. False data were found in at least 17 papers he published while working at Bell Laboratories in Murray Hill, New Jersey. Nevertheless, none of the reviewers ever discovered discrepancies in his research.

There have been other studies that report the inability of reviewers to detect weaknesses in a manuscript [33,34]. Observations conducted on papers which had intentionally made mistakes demonstrate that it is not uncommon for reviewers to overlook major flaws when reviewing papers [35]. These examples have stirred considerable discussion in the scientific community regarding the responsibility and impact of reviewers on scientific work. It is crucial that peer reviews not only aim at determining the relevance and originality of articles, but also focus on detecting instances of intentional fraud.

Peer review presents several limitations such as the inaccurate detection of flaws or fraud, its time-consuming, subjective nature, prone to bias, and open to misuse [36]. Richard Smith, former editor of BMJ, remarked to the Royal Society in 2015 that the peer review process is faith-based (not evidence-based), slow, wasteful, ineffective, largely a lottery, easily abused, prone to bias, unable to detect fraud, and irrelevant [37]. There is a risk that these flaws in peer review result in the belief that obtaining publication in a highly reputable journal or receiving a grant for research is subject to luck, similar to the process of dice rolling [38].

It has been proposed that draft papers be made available online on preprint servers for peer review so that peers can comment on them [12]. This method could make the process of obtaining reviews and suggestions more efficient and direct. It should be noted, however, that this type of system can be misused by authors who might ask friendly colleagues to give high ratings to their manuscripts.

Despite the flaws in the peer review system, there remains an important place for peer review in academia. Research papers may benefit from academic involvement in the review process, and the use of blind review processes may provide greater objective evaluations of papers. In order to improve the current peer review process, it is crucial to develop an online training program as well as accreditation of reviewers.

Reviewer Compensation

Journals charge an article processing fee, which can range from $100 to $3000. Article processing fees may also include charges for the peer review process. Nevertheless, most reviewers do not receive any of these fees, which is unfair to them since they have spent considerable time and effort on the review. In 2020, it was estimated that reviewers worldwide spent more than 100 million hours performing peer reviews, equivalent to nearly fifteen thousand years of labor. The time spent by US-based reviewers on reviews in 2020 is estimated to have been worth over 1.5 billion dollars. In the case of Chinese reviewers, the estimate exceeded 600 million US dollars; in the case of British reviewers, it was close to 400 million US dollars [39].

Because peer reviewers' time is valuable, it is reasonable to expect that they should be paid for reviewing an article. However, many publishers are reluctant to pay reviewers since this would increase the cost of publication overall [3,40]. Reviewers are only rewarded for their contributions to journals by means of acknowledgment in the journals, by positions on editorial boards, by free journal access, and by discounts on article processing costs for future submissions. It is true that being recognized as an elite reviewer in a prestigious journal can be a significant incentive. However, incentivizing reviewers by paying them could be an effective tool to encourage others. There is a possibility that a rise in the number of companies providing financial compensation for peer reviewing may lead to more people accepting more papers and completing them on time [3,40].

The practice of paying for the review of a paper is a double-edged sword. Reviewers are busy professionals who deserve fair compensation for their work. Nevertheless, there is the possibility that paying an individual for reviewing an article may lead to sloppy reviews in an effort to maximize revenue. There have also been arguments that payments for peer review may compromise the objectivity of the reviewers and turn them into profit-making enterprises rather than services. To combat this problem, several academics have come up with the idea of the Reviewer Index (RI), Reviewer Index Directory (RID) and Global Reviewer Index Directory (GRID), which are aimed at strengthening scientific research by focusing on both authors and reviewers at the same time [6].

Conclusions

Peer review is a process where researchers receive feedback from other researchers within their field. This process enables editors to decide whether to publish given manuscripts in light of expert input. A comprehensive review emphasizes an article's significance and explains how it contributes to scientific research. The reasons for the rejection should be clearly explained. A competent reviewer keeps an open mind and strives to put biases aside constantly. Finally, reviewers must keep in mind that they are also authors. Therefore, exhibiting a degree of empathy serves the highest interest of both authors and reviewers.

References

  1. Ferreira C, Bastille-Rousseau G, Bennett AM, et al. The evolution of peer review as a basis for scientific publication: Directional selection towards a robust discipline? Biol Rev Camb Philos Soc 2016;91(3):597-610. [View Article]
  2. Spier R. The history of the peer-review process. Trends Biotechnol 2002;20(8):357-358. [View Article]
  3. Benos DJ, Bashari E, Chaves JM, et al. The ups and downs of peer review. Adv Physiol Educ 2007;31(2):145-152. [View Article]
  4. Salasche SJ. How to "peer review" a medical journal manuscript. Dermatol Surg 1997;23(6):423-428. [View Article]
  5. Altman DG. Poor-quality medical research: What can journals do? JAMA 2002;287(21):2765-2767. [View Article]
  6. Kachewar SG, Sankaye SB. Reviewer index: A new proposal of rewarding the reviewer. Mens Sana Monogr 2013;11(1):274-284. [View Article]
  7. Jain AK. Peer review: Heart and soul of scientific publication. Indian J Orthop 2009;43(1):3-5. [View Article]
  8. Ali PA, Watson R. Peer review and the publication process. Nurs Open 2016;3(4):193-202. [View Article]
  9. Provenzale JM, Stanley RJ. A systematic guide to reviewing a manuscript. J Nucl Med Technol 2006;34(2):92-99. [View Article]
  10. Peh WC, Ng KH. Role of the manuscript reviewer. Singapore Med J 2009;50(10):931-933; quiz 934. [View Article]
  11. Polak JF. The role of the manuscript reviewer in the peer review process. AJR Am J Roentgenol 1995;165(3):685-688. [View Article]
  12. Tennant JP, Dugan JM, Graziotin D, et al. A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Res 2017;6:1151. [View Article]
  13. Tvina A, Spellecy R, Palatnik A. Bias in the peer review process: Can we do better? Obstet Gynecol 2019;133(6):1081-1083. [View Article]
  14. Budden AE, Tregenza T, Aarssen LW, Koricheva J, Leimu R, Lortie CJ. Double-blind review favours increased representation of female authors. Trends Ecol Evol 2008;23(1):4-6. [View Article]
  15. Godlee F. Making reviewers visible: Openness, accountability, and credit. JAMA 2002;287(21):2762-2765. [View Article]
  16. Transparent peer review one year on. Nat Commun 2016;7:13626. [View Article]
  17. Cosgrove A, Flintoft L. Trialing transparent peer review. Genome Biol 2017;18(1):173. [View Article]
  18. Gough NR. Training for peer review. Sci Signal 2009;2(85):tr2. [View Article]
  19. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: Randomised controlled trial. BMJ 2004;328(7441):673. [View Article]
  20. Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA 1998;280(3):231-233. [View Article]
  21. Allen TW. Peer review guidance: How do you write a good review? J Am Osteopath Assoc 2013;113(12):916-920. [View Article]
  22. Lovejoy TI, Revenson TA, France CR. Reviewing manuscripts for peer-review journals: A primer for novice and seasoned reviewers. Ann Behav Med 2011;42(1):1-13. [View Article]
  23. Weinstein R. How to write a manuscript for peer review. J Clin Apher 2020;35(4):358-366. [View Article]
  24. Gross AF, Wiechers IR, Stern TA. Peer review by early career psychiatrists: An opportunity for development. J Clin Psychiatry 2009;70(11):1600-1601. [View Article]
  25. COPE Council. COPE Ethical guidelines for peer reviewers — English. Version 2: September 2017. [View Article]
  26. O'Brien BC, Artino AR, Jr., Costello JA, Driessen E, Maggio LA. Transparency in peer review: Exploring the content and tone of reviewers' confidential comments to editors. PLoS One 2021;16(11):e0260558. [View Article]
  27. Horrobin DF. The philosophical basis of peer review and the suppression of innovation. JAMA 1990;263(10):1438-1441. [View Article]
  28. Lee KP, Boyd EA, Holroyd-Leduc JM, Bacchetti P, Bero LA. Predictors of publication: Characteristics of submitted manuscripts associated with acceptance at major biomedical journals. Med J Aust 2006;184(12):621-626. [View Article]
  29. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990;263(10):1385-1389. [View Article]
  30. Ioannidis JP. Why most published research findings are false. PLoS Med 2005;2(8):e124. [View Article]
  31. Akerlof GA. Writing the “The Market for 'Lemons'”: A personal interpretive essay. Accessed November 10, 2021. [View Article]
  32. Consoli L. Scientific misconduct and science ethics: A case study based approach. Sci Eng Ethics 2006;12(3):533-541. [View Article]
  33. Godlee F, Gale CR, Martyn CN. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: A randomized controlled trial. JAMA 1998;280(3):237-240. [View Article]
  34. Nylenna M, Riis P, Karlsson Y. Multiple blinded reviews of the same two manuscripts. Effects of referee characteristics and publication language. JAMA 1994;272(2):149-151. [View Article]
  35. Baxt WG, Waeckerle JF, Berlin JA, Callaham ML. Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Ann Emerg Med 1998;32(3 Pt 1):310-317. [View Article]
  36. Smith R. Peer review: A flawed process at the heart of science and journals. J R Soc Med 2006;99(4):178-182. [View Article]
  37. Kravitz RL, Feldman MD. Rumors of the demise of peer review are premature. J Gen Intern Med 2015;30(12):1717-1721. [View Article]
  38. Cole S, Cole JR, Simon GA. Chance and consensus in peer review. Science 1981;214(4523):881-886. [View Article]
  39. Aczel B, Szaszi B, Holcombe AO. A billion-dollar donation: Estimating the cost of researchers' time spent on peer review. Res Integr Peer Rev 2021;6(1):14. [View Article]
  40. Benitez-Bribiesca L. The ups and downs of the impact factor: The case of Archives of Medical Research. Arch Med Res 2002;33(2):91-94. [View Article]

Editorial Information

Publication History

Received date: December 01, 2021
Accepted date: January 06, 2022
Published date: January 20, 2022

Ethics Approval and Consent to Participate

The study is in accordance with the ethical standards of the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of Interest

The author reports no financial or other conflict of interest relevant to this article, which is the intellectual property of the author.

Publisher Disclaimer

It is pertinent to note that all opinions and statements made by the author(s) throughout this article are solely those of the author(s). They may not be representative of those of their affiliated organizations, the publishing house, editors, or other reviewers since they are the opinions and statements of the author(s) alone. The publisher does not guarantee or endorse any statements made by the manufacturer of any product mentioned in this article or the author's evaluation.

Copyright

© 2022 The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY). In accordance with accepted academic practice, anyone may use, distribute, or reproduce this material, so long as the original author(s), the copyright holder(s), and the original publication of this journal are credited, and this publication is cited as the original. To the extent permitted by these terms and conditions of license, this material may not be compiled, distributed, or reproduced in any manner that is inconsistent with those terms and conditions.

  1. Department of Otolaryngology-Head and Neck Surgery, Taipei Veterans General Hospital, Taipei, Taiwan
  2. Institute of Brain Science, National Yang-Ming University School of Medicine, Taipei, Taiwan
  3. Department of Otolaryngology, National Yang-Ming University School of Medicine, Taipei, Taiwan
  1. Department of Otolaryngology-Head and Neck Surgery, Taipei Veterans General Hospital, Taipei, Taiwan
  2. Institute of Brain Science, National Yang-Ming University School of Medicine, Taipei, Taiwan
  3. Department of Otolaryngology, National Yang-Ming University School of Medicine, Taipei, Taiwan

    Address: No.201, Sec. 2, Shipai Rd., Beitou District, 11217 Taipei City, Taiwan
Table 1.jpgCommon Models of Peer Review

Table 2.jpgA Five-Point Likert Type Scale of Responses for Reviewing a Paper

Table 3.jpgReviewer Recommendations

Reviewer 1 Comments

  1. This article examines aspects of peer review to explain the current state of academic publishing. This article provides valuable information that can be used by scholars for conducting a critical review of research papers. It is my opinion that this paper deserves to be published, but I believe one problem needs to be addressed before it can be accepted for publication.
    ResponseYour insightful comments are greatly appreciated.
     
  2. There is a suggestion that reviewers should be provided with feedback as a motivation to make their contribution to the review process as meaningful as possible. Yet the reviewers may not be sufficiently compensated due to the lack of monetary rewards. There have been a few individuals who have complained about not receiving any monetary compensation. However, these complaints have been ignored simply because many people are willing to participate in peer reviews for free. Would it be possible for anything to be done to change this?
    ResponseIn recognition of the value of peer reviewers' time, it is reasonable to expect that they should be compensated for their efforts. However, many publishers are hesitant to compensate reviewers since this would increase the overall cost of publication. Typically, reviewers are rewarded for their contributions to journals by being acknowledged in the journals, by serving on editorial boards, by being provided with free access to journals, and by receiving discounts on article processing costs for future submissions. There is no doubt that being acknowledged as an elite reviewer in a prestigious journal can be a significant incentive. It may however be effective to compensate reviewers to encourage others to contribute. Peer reviews may be improved by more companies providing financial compensation, which may result in more papers being completed on time. Nevertheless, the practice of paying for a paper to be reviewed is a two-edged sword. It is possible that paying individuals to review articles may result in sloppy reviews to maximize revenue. Payments for peer review may compromise the objectivity of reviewers and turn them into profit-making businesses rather than services. To combat this problem, academics have come up with ideas such as the Reviewer Index (RI), Reviewer Index Directory (RID) and Global Reviewer Index Directory (GRID), which aim to strengthen scientific research by focusing on both authors and reviewers simultaneously.

Reviewer 2 Comments

  1. The article is well-written and presents several principles of peer review. The article outlines the major components of a comprehensive manuscript review. It also describes some of the basic responsibilities of the reviewer and illustrates some of the rewards and responsibilities that are part of this process. I believe that this paper will be of interest to the readers of this journal and recommend that it be accepted once two minor points have been clarified.
    ResponseThanks for your thoughtful comments.
     
  2. It is noteworthy that attitudes towards reviewers and authors can change when their roles are reversed. Those authors who are more tolerant of mistakes may be too critical when serving as a reviewer. Can you recommend a way to minimize the influences on the review process stemming from reviewers' biases?
    ResponseThe attitudes of reviewers and authors may change considerably when the roles of reviewers and authors are reversed. Authors who are more forgiving of mistakes may become overly critical when acting as a reviewer. The main reason for this is that reviewers seek to provide an in-depth review of a manuscript, while authors are pursuing publication as the goal. In addition, it is not uncommon for novel ideas that contrast with conventional beliefs to get rebuffed. This occurs mainly because reviewers tend to view themselves as protectors of what they have learned in their field. Ultimately, editors act as balancing agents who are committed to publishing research of high academic quality but will also ensure that highly innovative papers will not be overlooked due to negative reviews. For reviewers who are excessively critical, it would be prudent for them to tone down their criticisms when they write the review.
     
  3. Numerous studies have indicated that reviewers are unable to identify weaknesses in a submitted manuscript during the review process. It has been proposed that draft papers be published online on preprint servers for peer review purposes, allowing them to be commented upon by their peers. It might be beneficial to use this method of acquiring reviews and suggestions to make the process more efficient and direct. Further discussion of this topic is needed in the article.
    ResponseObservations conducted on papers which had intentionally made mistakes demonstrate that it is not uncommon for reviewers to overlook major flaws when reviewing papers. These examples have stirred considerable discussion in the scientific community regarding the responsibility and impact of reviewers on scientific work. It is crucial that peer reviews not only aim at determining the relevance and originality of articles, but also focus on detecting instances of intentional fraud. It has been proposed that draft papers be made available online on preprint servers for peer review so that peers can comment on them. This method could make the process of obtaining reviews and suggestions more efficient and direct. It should be noted, however, that this type of system can be misused by authors who might ask friendly colleagues to give high ratings to their manuscripts. Despite the flaws in the peer review system, peer review remains an invaluable component of academia. To improve the current peer review process, it is crucial to develop an online training program as well as accreditation of reviewers.

Kuo CL. Peer review for academic research. Arch Otorhinolaryngol Head Neck Surg. 2022;6(1):2. https://doi.org/10.24983/scitemed.aohns.2022.00155