research review journals

Review articles: purpose, process, and structure

• Published: 02 October 2017
• Volume 46 , pages 1–5, ( 2018 )

Cite this article

• Robert W. Palmatier 1 ,
• Mark B. Houston 2 &
• John Hulland 3  

241k Accesses

487 Citations

64 Altmetric

Explore all metrics

Avoid common mistakes on your manuscript.

Many research disciplines feature high-impact journals that are dedicated outlets for review papers (or review–conceptual combinations) (e.g., Academy of Management Review , Psychology Bulletin , Medicinal Research Reviews ). The rationale for such outlets is the premise that research integration and synthesis provides an important, and possibly even a required, step in the scientific process. Review papers tend to include both quantitative (i.e., meta-analytic, systematic reviews) and narrative or more qualitative components; together, they provide platforms for new conceptual frameworks, reveal inconsistencies in the extant body of research, synthesize diverse results, and generally give other scholars a “state-of-the-art” snapshot of a domain, often written by topic experts (Bem 1995 ). Many premier marketing journals publish meta-analytic review papers too, though authors often must overcome reviewers’ concerns that their contributions are limited due to the absence of “new data.” Furthermore, relatively few non-meta-analysis review papers appear in marketing journals, probably due to researchers’ perceptions that such papers have limited publication opportunities or their beliefs that the field lacks a research tradition or “respect” for such papers. In many cases, an editor must provide strong support to help such review papers navigate the review process. Yet, once published, such papers tend to be widely cited, suggesting that members of the field find them useful (see Bettencourt and Houston 2001 ).

In this editorial, we seek to address three topics relevant to review papers. First, we outline a case for their importance to the scientific process, by describing the purpose of review papers . Second, we detail the review paper editorial initiative conducted over the past two years by the Journal of the Academy of Marketing Science ( JAMS ), focused on increasing the prevalence of review papers. Third, we describe a process and structure for systematic ( i.e. , non-meta-analytic) review papers , referring to Grewal et al. ( 2018 ) insights into parallel meta-analytic (effects estimation) review papers. (For some strong recent examples of marketing-related meta-analyses, see Knoll and Matthes 2017 ; Verma et al. 2016 ).

Purpose of review papers

In their most general form, review papers “are critical evaluations of material that has already been published,” some that include quantitative effects estimation (i.e., meta-analyses) and some that do not (i.e., systematic reviews) (Bem 1995 , p. 172). They carefully identify and synthesize relevant literature to evaluate a specific research question, substantive domain, theoretical approach, or methodology and thereby provide readers with a state-of-the-art understanding of the research topic. Many of these benefits are highlighted in Hanssens’ ( 2018 ) paper titled “The Value of Empirical Generalizations in Marketing,” published in this same issue of JAMS.

The purpose of and contributions associated with review papers can vary depending on their specific type and research question, but in general, they aim to

Resolve definitional ambiguities and outline the scope of the topic.

Provide an integrated, synthesized overview of the current state of knowledge.

Identify inconsistencies in prior results and potential explanations (e.g., moderators, mediators, measures, approaches).

Evaluate existing methodological approaches and unique insights.

Develop conceptual frameworks to reconcile and extend past research.

Describe research insights, existing gaps, and future research directions.

Not every review paper can offer all of these benefits, but this list represents their key contributions. To provide a sufficient contribution, a review paper needs to achieve three key standards. First, the research domain needs to be well suited for a review paper, such that a sufficient body of past research exists to make the integration and synthesis valuable—especially if extant research reveals theoretical inconsistences or heterogeneity in its effects. Second, the review paper must be well executed, with an appropriate literature collection and analysis techniques, sufficient breadth and depth of literature coverage, and a compelling writing style. Third, the manuscript must offer significant new insights based on its systematic comparison of multiple studies, rather than simply a “book report” that describes past research. This third, most critical standard is often the most difficult, especially for authors who have not “lived” with the research domain for many years, because achieving it requires drawing some non-obvious connections and insights from multiple studies and their many different aspects (e.g., context, method, measures). Typically, after the “review” portion of the paper has been completed, the authors must spend many more months identifying the connections to uncover incremental insights, each of which takes time to detail and explicate.

The increasing methodological rigor and technical sophistication of many marketing studies also means that they often focus on smaller problems with fewer constructs. By synthesizing these piecemeal findings, reconciling conflicting evidence, and drawing a “big picture,” meta-analyses and systematic review papers become indispensable to our comprehensive understanding of a phenomenon, among both academic and practitioner communities. Thus, good review papers provide a solid platform for future research, in the reviewed domain but also in other areas, in that researchers can use a good review paper to learn about and extend key insights to new areas.

This domain extension, outside of the core area being reviewed, is one of the key benefits of review papers that often gets overlooked. Yet it also is becoming ever more important with the expanding breadth of marketing (e.g., econometric modeling, finance, strategic management, applied psychology, sociology) and the increasing velocity in the accumulation of marketing knowledge (e.g., digital marketing, social media, big data). Against this backdrop, systematic review papers and meta-analyses help academics and interested managers keep track of research findings that fall outside their main area of specialization.

JAMS’ review paper editorial initiative

With a strong belief in the importance of review papers, the editorial team of JAMS has purposely sought out leading scholars to provide substantive review papers, both meta-analysis and systematic, for publication in JAMS . Many of the scholars approached have voiced concerns about the risk of such endeavors, due to the lack of alternative outlets for these types of papers. Therefore, we have instituted a unique process, in which the authors develop a detailed outline of their paper, key tables and figures, and a description of their literature review process. On the basis of this outline, we grant assurances that the contribution hurdle will not be an issue for publication in JAMS , as long as the authors execute the proposed outline as written. Each paper still goes through the normal review process and must meet all publication quality standards, of course. In many cases, an Area Editor takes an active role to help ensure that each paper provides sufficient insights, as required for a high-quality review paper. This process gives the author team confidence to invest effort in the process. An analysis of the marketing journals in the Financial Times (FT 50) journal list for the past five years (2012–2016) shows that JAMS has become the most common outlet for these papers, publishing 31% of all review papers that appeared in the top six marketing journals.

As a next step in positioning JAMS as a receptive marketing outlet for review papers, we are conducting a Thought Leaders Conference on Generalizations in Marketing: Systematic Reviews and Meta-Analyses , with a corresponding special issue (see www.springer.com/jams ). We will continue our process of seeking out review papers as an editorial strategy in areas that could be advanced by the integration and synthesis of extant research. We expect that, ultimately, such efforts will become unnecessary, as authors initiate review papers on topics of their own choosing to submit them to JAMS . In the past two years, JAMS already has increased the number of papers it publishes annually, from just over 40 to around 60 papers per year; this growth has provided “space” for 8–10 review papers per year, reflecting our editorial target.

Consistent with JAMS ’ overall focus on managerially relevant and strategy-focused topics, all review papers should reflect this emphasis. For example, the domains, theories, and methods reviewed need to have some application to past or emerging managerial research. A good rule of thumb is that the substantive domain, theory, or method should attract the attention of readers of JAMS .

The efforts of multiple editors and Area Editors in turn have generated a body of review papers that can serve as useful examples of the different types and approaches that JAMS has published.

Domain-based review papers

Domain-based review papers review, synthetize, and extend a body of literature in the same substantive domain. For example, in “The Role of Privacy in Marketing” (Martin and Murphy 2017 ), the authors identify and define various privacy-related constructs that have appeared in recent literature. Then they examine the different theoretical perspectives brought to bear on privacy topics related to consumers and organizations, including ethical and legal perspectives. These foundations lead in to their systematic review of privacy-related articles over a clearly defined date range, from which they extract key insights from each study. This exercise of synthesizing diverse perspectives allows these authors to describe state-of-the-art knowledge regarding privacy in marketing and identify useful paths for research. Similarly, a new paper by Cleeren et al. ( 2017 ), “Marketing Research on Product-Harm Crises: A Review, Managerial Implications, and an Agenda for Future Research,” provides a rich systematic review, synthesizes extant research, and points the way forward for scholars who are interested in issues related to defective or dangerous market offerings.

Theory-based review papers

Theory-based review papers review, synthetize, and extend a body of literature that uses the same underlying theory. For example, Rindfleisch and Heide’s ( 1997 ) classic review of research in marketing using transaction cost economics has been cited more than 2200 times, with a significant impact on applications of the theory to the discipline in the past 20 years. A recent paper in JAMS with similar intent, which could serve as a helpful model, focuses on “Resource-Based Theory in Marketing” (Kozlenkova et al. 2014 ). The article dives deeply into a description of the theory and its underlying assumptions, then organizes a systematic review of relevant literature according to various perspectives through which the theory has been applied in marketing. The authors conclude by identifying topical domains in marketing that might benefit from additional applications of the theory (e.g., marketing exchange), as well as related theories that could be integrated meaningfully with insights from the resource-based theory.

Method-based review papers

Method-based review papers review, synthetize, and extend a body of literature that uses the same underlying method. For example, in “Event Study Methodology in the Marketing Literature: An Overview” (Sorescu et al. 2017 ), the authors identify published studies in marketing that use an event study methodology. After a brief review of the theoretical foundations of event studies, they describe in detail the key design considerations associated with this method. The article then provides a roadmap for conducting event studies and compares this approach with a stock market returns analysis. The authors finish with a summary of the strengths and weaknesses of the event study method, which in turn suggests three main areas for further research. Similarly, “Discriminant Validity Testing in Marketing: An Analysis, Causes for Concern, and Proposed Remedies” (Voorhies et al. 2016 ) systematically reviews existing approaches for assessing discriminant validity in marketing contexts, then uses Monte Carlo simulation to determine which tests are most effective.

Our long-term editorial strategy is to make sure JAMS becomes and remains a well-recognized outlet for both meta-analysis and systematic managerial review papers in marketing. Ideally, review papers would come to represent 10%–20% of the papers published by the journal.

Process and structure for review papers

In this section, we review the process and typical structure of a systematic review paper, which lacks any long or established tradition in marketing research. The article by Grewal et al. ( 2018 ) provides a summary of effects-focused review papers (i.e., meta-analyses), so we do not discuss them in detail here.

Systematic literature review process

Some review papers submitted to journals take a “narrative” approach. They discuss current knowledge about a research domain, yet they often are flawed, in that they lack criteria for article inclusion (or, more accurately, article exclusion), fail to discuss the methodology used to evaluate included articles, and avoid critical assessment of the field (Barczak 2017 ). Such reviews tend to be purely descriptive, with little lasting impact.

In contrast, a systematic literature review aims to “comprehensively locate and synthesize research that bears on a particular question, using organized, transparent, and replicable procedures at each step in the process” (Littell et al. 2008 , p. 1). Littell et al. describe six key steps in the systematic review process. The extent to which each step is emphasized varies by paper, but all are important components of the review.

Topic formulation . The author sets out clear objectives for the review and articulates the specific research questions or hypotheses that will be investigated.

Study design . The author specifies relevant problems, populations, constructs, and settings of interest. The aim is to define explicit criteria that can be used to assess whether any particular study should be included in or excluded from the review. Furthermore, it is important to develop a protocol in advance that describes the procedures and methods to be used to evaluate published work.

Sampling . The aim in this third step is to identify all potentially relevant studies, including both published and unpublished research. To this end, the author must first define the sampling unit to be used in the review (e.g., individual, strategic business unit) and then develop an appropriate sampling plan.

Data collection . By retrieving the potentially relevant studies identified in the third step, the author can determine whether each study meets the eligibility requirements set out in the second step. For studies deemed acceptable, the data are extracted from each study and entered into standardized templates. These templates should be based on the protocols established in step 2.

Data analysis . The degree and nature of the analyses used to describe and examine the collected data vary widely by review. Purely descriptive analysis is useful as a starting point but rarely is sufficient on its own. The examination of trends, clusters of ideas, and multivariate relationships among constructs helps flesh out a deeper understanding of the domain. For example, both Hult ( 2015 ) and Huber et al. ( 2014 ) use bibliometric approaches (e.g., examine citation data using multidimensional scaling and cluster analysis techniques) to identify emerging versus declining themes in the broad field of marketing.

Reporting . Three key aspects of this final step are common across systematic reviews. First, the results from the fifth step need to be presented, clearly and compellingly, using narratives, tables, and figures. Second, core results that emerge from the review must be interpreted and discussed by the author. These revelatory insights should reflect a deeper understanding of the topic being investigated, not simply a regurgitation of well-established knowledge. Third, the author needs to describe the implications of these unique insights for both future research and managerial practice.

A new paper by Watson et al. ( 2017 ), “Harnessing Difference: A Capability-Based Framework for Stakeholder Engagement in Environmental Innovation,” provides a good example of a systematic review, starting with a cohesive conceptual framework that helps establish the boundaries of the review while also identifying core constructs and their relationships. The article then explicitly describes the procedures used to search for potentially relevant papers and clearly sets out criteria for study inclusion or exclusion. Next, a detailed discussion of core elements in the framework weaves published research findings into the exposition. The paper ends with a presentation of key implications and suggestions for the next steps. Similarly, “Marketing Survey Research Best Practices: Evidence and Recommendations from a Review of JAMS Articles” (Hulland et al. 2017 ) systematically reviews published marketing studies that use survey techniques, describes recent trends, and suggests best practices. In their review, Hulland et al. examine the entire population of survey papers published in JAMS over a ten-year span, relying on an extensive standardized data template to facilitate their subsequent data analysis.

Structure of systematic review papers

There is no cookie-cutter recipe for the exact structure of a useful systematic review paper; the final structure depends on the authors’ insights and intended points of emphasis. However, several key components are likely integral to a paper’s ability to contribute.

Depth and rigor

Systematic review papers must avoid falling in to two potential “ditches.” The first ditch threatens when the paper fails to demonstrate that a systematic approach was used for selecting articles for inclusion and capturing their insights. If a reader gets the impression that the author has cherry-picked only articles that fit some preset notion or failed to be thorough enough, without including articles that make significant contributions to the field, the paper will be consigned to the proverbial side of the road when it comes to the discipline’s attention.

Authors that fall into the other ditch present a thorough, complete overview that offers only a mind-numbing recitation, without evident organization, synthesis, or critical evaluation. Although comprehensive, such a paper is more of an index than a useful review. The reviewed articles must be grouped in a meaningful way to guide the reader toward a better understanding of the focal phenomenon and provide a foundation for insights about future research directions. Some scholars organize research by scholarly perspectives (e.g., the psychology of privacy, the economics of privacy; Martin and Murphy 2017 ); others classify the chosen articles by objective research aspects (e.g., empirical setting, research design, conceptual frameworks; Cleeren et al. 2017 ). The method of organization chosen must allow the author to capture the complexity of the underlying phenomenon (e.g., including temporal or evolutionary aspects, if relevant).

Replicability

Processes for the identification and inclusion of research articles should be described in sufficient detail, such that an interested reader could replicate the procedure. The procedures used to analyze chosen articles and extract their empirical findings and/or key takeaways should be described with similar specificity and detail.

We already have noted the potential usefulness of well-done review papers. Some scholars always are new to the field or domain in question, so review papers also need to help them gain foundational knowledge. Key constructs, definitions, assumptions, and theories should be laid out clearly (for which purpose summary tables are extremely helpful). An integrated conceptual model can be useful to organize cited works. Most scholars integrate the knowledge they gain from reading the review paper into their plans for future research, so it is also critical that review papers clearly lay out implications (and specific directions) for research. Ideally, readers will come away from a review article filled with enthusiasm about ways they might contribute to the ongoing development of the field.

Helpful format

Because such a large body of research is being synthesized in most review papers, simply reading through the list of included studies can be exhausting for readers. We cannot overstate the importance of tables and figures in review papers, used in conjunction with meaningful headings and subheadings. Vast literature review tables often are essential, but they must be organized in a way that makes their insights digestible to the reader; in some cases, a sequence of more focused tables may be better than a single, comprehensive table.

In summary, articles that review extant research in a domain (topic, theory, or method) can be incredibly useful to the scientific progress of our field. Whether integrating the insights from extant research through a meta-analysis or synthesizing them through a systematic assessment, the promised benefits are similar. Both formats provide readers with a useful overview of knowledge about the focal phenomenon, as well as insights on key dilemmas and conflicting findings that suggest future research directions. Thus, the editorial team at JAMS encourages scholars to continue to invest the time and effort to construct thoughtful review papers.

Barczak, G. (2017). From the editor: writing a review article. Journal of Product Innovation Management, 34 (2), 120–121.

Article   Google Scholar  

Bem, D. J. (1995). Writing a review article for psychological bulletin. Psychological Bulletin, 118 (2), 172–177.

Bettencourt, L. A., & Houston, M. B. (2001). Assessing the impact of article method type and subject area on citation frequency and reference diversity. Marketing Letters, 12 (4), 327–340.

Cleeren, K., Dekimpe, M. G., & van Heerde, H. J. (2017). Marketing research on product-harm crises: a review, managerial implications. Journal of the Academy of Marketing Science, 45 (5), 593–615.

Grewal, D., Puccinelli, N. M., & Monroe, K. B. (2018). Meta-analysis: error cancels and truth accrues. Journal of the Academy of Marketing Science, 46 (1).

Hanssens, D. M. (2018). The value of empirical generalizations in marketing. Journal of the Academy of Marketing Science, 46 (1).

Huber, J., Kamakura, W., & Mela, C. F. (2014). A topical history of JMR . Journal of Marketing Research, 51 (1), 84–91.

Hulland, J., Baumgartner, H., & Smith, K. M. (2017). Marketing survey research best practices: evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Science. https://doi.org/10.1007/s11747-017-0532-y .

Hult, G. T. M. (2015). JAMS 2010—2015: literature themes and intellectual structure. Journal of the Academy of Marketing Science, 43 (6), 663–669.

Knoll, J., & Matthes, J. (2017). The effectiveness of celebrity endorsements: a meta-analysis. Journal of the Academy of Marketing Science, 45 (1), 55–75.

Kozlenkova, I. V., Samaha, S. A., & Palmatier, R. W. (2014). Resource-based theory in marketing. Journal of the Academy of Marketing Science, 42 (1), 1–21.

Littell, J. H., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis . New York: Oxford University Press.

Book   Google Scholar  

Martin, K. D., & Murphy, P. E. (2017). The role of data privacy in marketing. Journal of the Academy of Marketing Science, 45 (2), 135–155.

Rindfleisch, A., & Heide, J. B. (1997). Transaction cost analysis: past, present, and future applications. Journal of Marketing, 61 (4), 30–54.

Sorescu, A., Warren, N. L., & Ertekin, L. (2017). Event study methodology in the marketing literature: an overview. Journal of the Academy of Marketing Science, 45 (2), 186–207.

Verma, V., Sharma, D., & Sheth, J. (2016). Does relationship marketing matter in online retailing? A meta-analytic approach. Journal of the Academy of Marketing Science, 44 (2), 206–217.

Voorhies, C. M., Brady, M. K., Calantone, R., & Ramirez, E. (2016). Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies. Journal of the Academy of Marketing Science, 44 (1), 119–134.

Watson, R., Wilson, H. N., Smart, P., & Macdonald, E. K. (2017). Harnessing difference: a capability-based framework for stakeholder engagement in environmental innovation. Journal of Product Innovation Management. https://doi.org/10.1111/jpim.12394 .

Download references

Author information

Authors and affiliations.

Foster School of Business, University of Washington, Box: 353226, Seattle, WA, 98195-3226, USA

Robert W. Palmatier

Neeley School of Business, Texas Christian University, Fort Worth, TX, USA

Mark B. Houston

Terry College of Business, University of Georgia, Athens, GA, USA

John Hulland

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Robert W. Palmatier .

Rights and permissions

Reprints and permissions

About this article

Palmatier, R.W., Houston, M.B. & Hulland, J. Review articles: purpose, process, and structure. J. of the Acad. Mark. Sci. 46 , 1–5 (2018). https://doi.org/10.1007/s11747-017-0563-4

Download citation

Published : 02 October 2017

Issue Date : January 2018

DOI : https://doi.org/10.1007/s11747-017-0563-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

• Search Search
• CN (Chinese)
• DE (German)
• ES (Spanish)
• FR (Français)
• JP (Japanese)
• Open science
• Booksellers
• Peer Reviewers
• Springer Nature Group ↗
• Springer journals
• Nature Portfolio journals
• Adis journals
• Academic journals on nature.com
• Palgrave Macmillan journals
• Journal archives
• Open access journals
• eBook collections
• Book archives
• Open Access Books
• Proceedings
• Reference Modules
• Databases & Solutions
• AdisInsight
• Data Solutions
• protocols.io
• SpringerMaterials
• SpringerProtocols
• Springer Nature Experiments
• Corporate & Health
• Springer Nature Video
• Research Data Services
• Nature Masterclasses
• Products overview
• Academic, Government & Corporate
• Request a trial
• Request a demo
• Request a quote
• Corporate & Health solutions
• eBook and Journal collections
• Content on Demand (CoD)
• Quote, trial or demo
• Journals catalog
• Serials Update
• Publishing Model Changes
• Cessations & Transfers
• Licensing A-Z
• How it works
• Talk to a Licensing Manager
• Digital preservation
• Licensing overview
• Open Research
• Discovery at Springer Nature
• MARC Records
• Librarian Portal
• Remote Access
• Content Promotion
• Library Promotion
• Tutorials & User Guides
• Webinars & Podcasts
• White papers
• Support your users
• Account Development
• Usage reporting
• Tools & Services overview
• News & Initiatives
• System Updates
• Overview Page
• Stay informed

Nature Reviews

Research in perspective. Reviews, News & Opinion content, often referred to as non-primary content, plays a crucial part in enhancing science education, communication, and collaboration. By offering a diverse range of formats, this content bridges the gap between researchers, students, policymakers, and the broader public. This proprietary content — not available elsewhere — features:

• Review articles summarising the current state of understanding on a topic within a certain discipline.
• Perspectives, opinion, and commentary bringing original points of view to an issue and proposing a solution.
• Community-focused articles exploring community issues, such as career development, representation, sustainability, and more.
• Article summaries short , focused articles about recent scholarship.
• Feature articles giving depth and detail to a topical issue.

Licensing options

Help your users make their own discoveries

Filtering, enhancing & amplifying research

The Nature Reviews journals act as a filter to help busy students, researchers and clinicians keep up to date with the latest scientific literature. This carefully curated content educates and inspires future scientists, supports global collaboration, and addresses the world’s greatest challenges. It is widely read, acting as a catalyst for debate and change bringing about real-world benefits. Specialist Nature Reviews content includes:

• Technical Reviews that provide an outline of the scientific and technical challenges and opportunities in a certain field.
• Consensus Statements that give a comprehensive analysis and recommendations from a group of experts on a scientific or medical issue.
• Primer articles that provide introductions to diseases or disorders, or scientific methods.

Guiding researchers to important primary research beyond their immediate expertise

Why choose Reviews journals?

• Widely used by students and faculty for their educational value.
• Improves accessibility of original research by summarizing latest developments in various fields.
• Editors select leading experts to write on important and timely topics.
• Reviewed by expert peer reviewers.

“Review articles provide an overall picture of the topics that I bring up in lectures, as well as become material for discussion in our student groups.”

- Asst. Professor, Asia

“As research becomes more collaborative, our journals distill the diversity of perspectives to guide readers through the literature, providing researchers with the tools they need to push the boundary of discovery." 

- Mina Razzak, Editorial Director, Nature Reviews

What authors have to say

What authors really think about publishing in Nature Reviews

Get an inside look at what authors really think about publishing in Nature Reviews journals. Hear firsthand about their experiences, from working with editors to navigating peer review. 

Dive into the world of scientific publishing alongside the authors themselves and read the blog post written by Editorial Director, Mina Razzak.

Download the Infographic

All Reviews journals by subject area

Find the right journals for your users

The complete journal list.

Nature Portfolio includes a wide selection of journals covering clinical, life, physical, social and earth scientists.

Nature – Science journalism at its best

Essential science news, reviews and insights into developments affecting research communities globally.

Stay up to date

Here to foster information exchange with the library community

Springer Nature's LibraryZone is a community developed to foster sharing of information with the library community. Enjoy!

Connect with us on LinkedIn and stay up to date with news and development.

News, information on our forthcoming books, author content and discounts.

• Tools & Services
• Sales and account contacts
• Professional
• Press office
• Locations & Contact

We are a world leading research, educational and professional publisher. Visit our main website for more information.

• © 2024 Springer Nature
• General terms and conditions
• Your US State Privacy Rights
• Your Privacy Choices / Manage Cookies
• Accessibility
• Legal notice
• Help us to improve this site, send feedback.

RESEARCH REVIEW International Journal of Multidisciplinary

About the journal.

RESEARCH REVIEW International Journal of Multidisciplinary (RRIJM) is an international Double-blind peer-reviewed [refereed] open access online journal. Too often a journal’s decision to publish a paper is dominated by what the editor/s think is interesting and will gain greater readership-both of which are subjective judgments and lead to decisions which are frustrating and delay the publication of your work. RRIJM will rigorously peer-review your submissions and publish all papers that are judged to be technically sound. Judgments about the importance of any particular paper are then made after publication by the readership (who are the most qualified to determine what is of interest to them).

Most conventional journals publish papers from tightly defined subject areas, making it more difficult for readers from other disciplines to read them. RRIJM has no such barriers, which helps your research reach the entire scientific community.

• Title:  RESEARCH REVIEW International Multidisciplinary Research Journal
• ISSN:  2455-3085 (Online)
• Impact Factor: 6.849
• Crossref DOI:  10.31305/rrijm
• Frequency of Publication:  Monthly  [12 issues per year]
• Languages:  English/Hindi/Gujarat  [Multiple Languages]
• Accessibility:  Open Access
• Peer Review Process:  Double Blind Peer Review Process
• Subject:  Multidisciplinary
• Plagiarism Checker:  Turnitin (License)
• Publication Format:  Online
• Contact No.:  +91- 99784 40833
• Email:  [email protected]
• Old Website: https://old.rrjournals.com/
• New Website: https://rrjournals.com/ 
• Address:  15, Kalyan Nagar, Shahpur, Ahmedabad, Gujarat 380001

Key Features of RRIJM

• Journal was listed in  UGC  with  Journal No. 44945 (Till 14-06-2019)
• Journal Publishes online every month
• Online article submission
• Standard peer review process

Current Issue

Industrialization and Economic Prosperity: The Strategic Vision of Pandit Madan Mohan Malaviya

The final check: a strategy of patient engagement, are prospective teachers prepared for the new normal- reflections from bihar, india, elementary education among muslims in india: current status and policy implications, the gold market and covid-19: an indian perspective, a comparative analysis of modern and traditional herbal health care systems in the tribal dominated landscapes of pithoragarh district, uttarakhand, study on e-commerce growth and impact on indian economic development, a study on consumer buying behavior towards cosmetics products with respect to social media users, evaluating the impact and awareness of the ayushman bharat scheme in bihar, a comparative study of the adjustment of athlete and non-athlete police officers ખેલાડી અને બિન ખેલાડી પોલીસ ભાઈઓના સમાયોજનનો તુલનાત્મક અભ્યાસ, information.

• For Readers
• For Authors
• For Librarians

Make a Submission

RESEARCH REVIEW International Journal of Multidisciplinary is licensed under a Creative Commons Attribution 4.0 International License .

Click here to go to Old Website

International Journal of Research and Review

International Journal of Research and Review (IJRR)

Issn: 2349-9788 (online) issn: 2454-2237 (print).

International Journal of Research and Review (E-ISSN: 2349-9788; P-ISSN: 2454-2237)is a double-blind, Indexed peer-reviewed, open access international journal dedicated to promotion of research in multidisciplinary areas. We define Open Access-journals as journals that use a funding model that does not charge readers or their institutions for access. From the BOAI definition of "Open Access" users shall have the right to "read, download, copy, distribute, print, search, or link" to the full texts of articles. The journal publishes original research article from broad areas like Accountancy, Agriculture, Anthropology, Anatomy, Architecture, Arts, Biochemistry, Bioinformatics, Biology, Bioscience, Biostatistics, Biotechnology, Botany, Chemistry, Commerce, Computer Science, Dairy Technology, Dentistry, Ecology, Economics, Education, Engineering, Environmental Science, Food & Nutrition, Forensic Science, Forestry, Geology, Geography, Health Sciences, History, Home Science, Journalism & Mass Communication, Language, Law, Life Science, Literature, Management, Marine Science, Mathematics, Medical Science, Microbiology, Pathology, Paramedical Science, Pharmacy, Philosophy, Physical Education, Physiotherapy, Physics, Political Science, Public Health, Psychology, Science, Social Science, Sociology, Sports Medicine, Statistics, Tourism, Veterinary Science & Animal Husbandry, Yoga, Zoology etc.

The International Journal of Research and Review (IJRR) provides rapid publication of articles in all areas of research.

Frequency: Monthly Language: English

Scope of Journal

International Journal of Research and Review (IJRR) is a double-blind Indexed peer-reviewed open access journal which publishes original articles, reviews and short communications that are not under consideration for publication elsewhere. The journal publishes papers based on original research that are judged by critical reviews, to make a substantial contribution in the field. It aims at rapid publication of high quality research results while maintaining rigorous review process. The Journal welcomes the submission of manuscripts that meet the general criteria of significance and academic excellence. Papers are published approximately one month after acceptance.

IJRR is dedicated to promote high quality research work in multidisciplinary field.

Index Copernicus Value (ICV) for 2022: 100.00; ICV 2021: 100.00; ICV 2020: 100.00; ICV 2019: 96.58; ICV 2018: 95.79; ICV 2017: 85.08; ICV 2016: 67.22; ICV 2015: 73.64; ICV 2014: 52.85

Salient features of our journal.

✵ Journal is indexed with Index Copernicus and other indexing agencies.

✵ Print version (hard copy) with P-ISSN number is available.

✵ Fast Publication Process.

✵ Free e-Certificate of Publication on author's request.

How to Submit Manuscript:

Thesis publication, research articles.

Nitin V Vichare, Rakesh Maggon

Smita Deshkar, Niranjan Patil, Pranali Balmiki

Ashish Choudhary, Riyaz Farooq, Aamir Rashid Purra, Fayaz Ahmed Ahangar

Joshi Medhavi H., Desai Devangi S, Singh Lalli M, Joshi Vaibhavi A

Jasvir Singh

Mariya Thomas Chacko, Tima Babu, Jenin Vincent Parambi, Irene T. Ivan, Sheik Haja Sherief

Click Here to Read more...

Anwar Sholeh, Alwi Thamrin Nasution, Radar Radius Tarigan

Arnaldi Fernando, Dairion Gatot, Yan Indra Fajar Sitepu

Iji Clement O, Age Terungwa James

Varsha S. Dhage, Pratibha Chougule

Uzma Khan, Arindam Ghosh

Dr. Uzma Khan, Dr. Arindam Ghosh, Dr. John Pramod

Our Journals

Visit Website

International Journal of Science and Healthcare Research

Galore International Journal of Health Sciences and Research

Galore International Journal of Applied Sciences and Humanities

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Writing a good review article

• 3 minute read
• 95.9K views

Table of Contents

As a young researcher, you might wonder how to start writing your first review article, and the extent of the information that it should contain. A review article is a comprehensive summary of the current understanding of a specific research topic and is based on previously published research. Unlike research papers, it does not contain new results, but can propose new inferences based on the combined findings of previous research.

Types of review articles

Review articles are typically of three types: literature reviews, systematic reviews, and meta-analyses.

A literature review is a general survey of the research topic and aims to provide a reliable and unbiased account of the current understanding of the topic.

A systematic review , in contrast, is more specific and attempts to address a highly focused research question. Its presentation is more detailed, with information on the search strategy used, the eligibility criteria for inclusion of studies, the methods utilized to review the collected information, and more.

A meta-analysis is similar to a systematic review in that both are systematically conducted with a properly defined research question. However, unlike the latter, a meta-analysis compares and evaluates a defined number of similar studies. It is quantitative in nature and can help assess contrasting study findings.

Tips for writing a good review article

Here are a few practices that can make the time-consuming process of writing a review article easier:

• Define your question: Take your time to identify the research question and carefully articulate the topic of your review paper. A good review should also add something new to the field in terms of a hypothesis, inference, or conclusion. A carefully defined scientific question will give you more clarity in determining the novelty of your inferences.
• Identify credible sources: Identify relevant as well as credible studies that you can base your review on, with the help of multiple databases or search engines. It is also a good idea to conduct another search once you have finished your article to avoid missing relevant studies published during the course of your writing.
• Take notes: A literature search involves extensive reading, which can make it difficult to recall relevant information subsequently. Therefore, make notes while conducting the literature search and note down the source references. This will ensure that you have sufficient information to start with when you finally get to writing.
• Describe the title, abstract, and introduction: A good starting point to begin structuring your review is by drafting the title, abstract, and introduction. Explicitly writing down what your review aims to address in the field will help shape the rest of your article.
• Be unbiased and critical: Evaluate every piece of evidence in a critical but unbiased manner. This will help you present a proper assessment and a critical discussion in your article.
• Include a good summary: End by stating the take-home message and identify the limitations of existing studies that need to be addressed through future studies.
• Ask for feedback: Ask a colleague to provide feedback on both the content and the language or tone of your article before you submit it.
• Check your journal’s guidelines: Some journals only publish reviews, while some only publish research articles. Further, all journals clearly indicate their aims and scope. Therefore, make sure to check the appropriateness of a journal before submitting your article.

Writing review articles, especially systematic reviews or meta-analyses, can seem like a daunting task. However, Elsevier Author Services can guide you by providing useful tips on how to write an impressive review article that stands out and gets published!

What are Implications in Research?

How to Write the Results Section: Guide to Structure and Key Points

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Scholarly Sources: What are They and Where can You Find Them?

Input your search keywords and press Enter.

• Insights blog

What is a review article?

Learn how to write a review article.

What is a review article? A review article can also be called a literature review, or a review of literature. It is a survey of previously published research on a topic. It should give an overview of current thinking on the topic. And, unlike an original research article, it will not present new experimental results.

Writing a review of literature is to provide a critical evaluation of the data available from existing studies. Review articles can identify potential research areas to explore next, and sometimes they will draw new conclusions from the existing data.

Why write a review article?

To provide a comprehensive foundation on a topic.

To explain the current state of knowledge.

To identify gaps in existing studies for potential future research.

To highlight the main methodologies and research techniques.

Did you know? 

There are some journals that only publish review articles, and others that do not accept them.

Make sure you check the  aims and scope  of the journal you’d like to publish in to find out if it’s the right place for your review article.

How to write a review article

Below are 8 key items to consider when you begin writing your review article.

Check the journal’s aims and scope

Make sure you have read the aims and scope for the journal you are submitting to and follow them closely. Different journals accept different types of articles and not all will accept review articles, so it’s important to check this before you start writing.

Define your scope

Define the scope of your review article and the research question you’ll be answering, making sure your article contributes something new to the field. 

As award-winning author Angus Crake told us, you’ll also need to “define the scope of your review so that it is manageable, not too large or small; it may be necessary to focus on recent advances if the field is well established.” 

Finding sources to evaluate

When finding sources to evaluate, Angus Crake says it’s critical that you “use multiple search engines/databases so you don’t miss any important ones.” 

For finding studies for a systematic review in medical sciences,  read advice from NCBI . 

Writing your title, abstract and keywords

Spend time writing an effective title, abstract and keywords. This will help maximize the visibility of your article online, making sure the right readers find your research. Your title and abstract should be clear, concise, accurate, and informative. 

For more information and guidance on getting these right, read our guide to writing a good abstract and title  and our  researcher’s guide to search engine optimization . 

Introduce the topic

Does a literature review need an introduction? Yes, always start with an overview of the topic and give some context, explaining why a review of the topic is necessary. Gather research to inform your introduction and make it broad enough to reach out to a large audience of non-specialists. This will help maximize its wider relevance and impact. 

Don’t make your introduction too long. Divide the review into sections of a suitable length to allow key points to be identified more easily.

Include critical discussion

Make sure you present a critical discussion, not just a descriptive summary of the topic. If there is contradictory research in your area of focus, make sure to include an element of debate and present both sides of the argument. You can also use your review paper to resolve conflict between contradictory studies.

What researchers say

Angus Crake, researcher

As part of your conclusion, include making suggestions for future research on the topic. Focus on the goal to communicate what you understood and what unknowns still remains.

Use a critical friend

Always perform a final spell and grammar check of your article before submission. 

You may want to ask a critical friend or colleague to give their feedback before you submit. If English is not your first language, think about using a language-polishing service.

Find out more about how  Taylor & Francis Editing Services can help improve your manuscript before you submit.

What is the difference between a research article and a review article?

Before you submit your review article…

Complete this checklist before you submit your review article:

Have you checked the journal’s aims and scope?

Have you defined the scope of your article?

Did you use multiple search engines to find sources to evaluate?

Have you written a descriptive title and abstract using keywords?

Did you start with an overview of the topic?

Have you presented a critical discussion?

Have you included future suggestions for research in your conclusion?

Have you asked a friend to do a final spell and grammar check?

Expert help for your manuscript

Taylor & Francis Editing Services  offers a full range of pre-submission manuscript preparation services to help you improve the quality of your manuscript and submit with confidence.

Related resources

How to edit your paper

Writing a scientific literature review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved September 3, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Explore millions of high-quality primary sources and images from around the world, including artworks, maps, photographs, and more.

Explore migration issues through a variety of media types

• Part of The Streets are Talking: Public Forms of Creative Expression from Around the World
• Part of The Journal of Economic Perspectives, Vol. 34, No. 1 (Winter 2020)
• Part of Cato Institute (Aug. 3, 2021)
• Part of University of California Press
• Part of Open: Smithsonian National Museum of African American History & Culture
• Part of Indiana Journal of Global Legal Studies, Vol. 19, No. 1 (Winter 2012)
• Part of R Street Institute (Nov. 1, 2020)
• Part of Leuven University Press
• Part of UN Secretary-General Papers: Ban Ki-moon (2007-2016)
• Part of Perspectives on Terrorism, Vol. 12, No. 4 (August 2018)
• Part of Leveraging Lives: Serbia and Illegal Tunisian Migration to Europe, Carnegie Endowment for International Peace (Mar. 1, 2023)
• Part of UCL Press

Harness the power of visual materials—explore more than 3 million images now on JSTOR.

Enhance your scholarly research with underground newspapers, magazines, and journals.

Explore collections in the arts, sciences, and literature from the world’s leading museums, archives, and scholars.

• Introduction
• Conclusions
• Article Information

BP indicates bisphosphonate; NSAID, nonsteroidal anti-inflammatory drug; and WJ-MSC, Wharton jelly–derived mesenchymal stem cell.

a Refers to the 4 prospective comparative studies.

A, Network plot of studies included in network meta-analysis on short-term pain management during activity. Circle size is proportional to the number of trials for each intervention, and the line thickness is proportional to the number of trials comparing the interventions. B, Network meta-analysis for short-term pain management during activity comparing pharmacological interventions with placebo. Data are presented as the standardized mean difference (SMD) and 95% CI for pain during activity. Values below 0 indicate that the treatment mentioned first (before the vs) is favored, whereas values above 0 indicate that the treatment mentioned last (after the vs) is favored. NSAID indicates nonsteroidal anti-inflammatory drug.

A, Network plot of studies included in network meta-analysis of pharmacological interventions regarding long-term pain management. Circle size is proportional to the number of trials for each intervention, and the line thickness is proportional to the number of trials comparing the interventions. B, Network meta-analysis for long-term pain management (nonspecified) comparing pharmacological interventions with nonsteroidal anti-inflammatory drug (NSAID). Data are presented as the standardized mean difference (SMD) and 95% CI for long-term pain outcomes. Values below 0 indicate that the treatment mentioned first (before the vs) is favored, whereas values above 0 indicate that the treatment mentioned last (after the vs) is favored.

A, Network plot of studies included in network meta-analysis of brace treatment regarding long-term pain management. Circle size is proportional to the number of trials for each intervention, and the line thickness is proportional to the number of trials comparing the interventions. B, Network meta-analysis for long-term pain management (nonspecified) comparing the treatment with various braces with no brace treatment. Data are presented as the standardized mean difference (SMD) and 95% CI for long-term pain. Values below 0 indicate that the treatment mentioned first (before the vs) is favored, whereas values above 0 indicate that the treatment mentioned last (after the vs) is favored regarding pain outcomes.

eAppendix . Supplemental methods

eTable 1. Search strategy

eTable 2. Inclusion criteria

eTable 3. Framework for the GRADE assessment

eTable 4. List of excluded studies

eTable 5. Characteristics of the included studies

eTable 6. Study quality of the included prospective, comparative studies

eTable 7. Estimates of effects and GRADE quality ratings for comparison of different pharmacological interventions for short-term pain during activity

eTable 8. Adverse events of the included studies

eFigure 1. Risk of bias assessment

eFigure 2. Node splitting: Short-term pain during activity

eFigure 3. P-score ranking: Short-term pain during activity

eFigure 4. Sensitivity analyses: Short-term pain during activity (walking and rising up)

eFigure 5. Sensitivity analyses: Short-term pain during activity (only walking)

eFigure 6. Node splitting: Pharmacological interventions (long-term pain)

eFigure 7. P-score ranking: Pharmacological interventions for long-term pain

eFigure 8. Node splitting: Braces (long-term pain)

eFigure 9. P-score ranking: Braces for long-term pain

eReferences

Data Sharing Statement

See More About

Sign up for emails based on your interests, select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Download PDF
• X Facebook More LinkedIn

Alimy A , Anastasilakis AD , Carey JJ, et al. Conservative Treatments in the Management of Acute Painful Vertebral Compression Fractures : A Systematic Review and Network Meta-Analysis . JAMA Netw Open. 2024;7(9):e2432041. doi:10.1001/jamanetworkopen.2024.32041

Manage citations:

© 2024

• Permissions

Conservative Treatments in the Management of Acute Painful Vertebral Compression Fractures : A Systematic Review and Network Meta-Analysis

• 1 Department of Trauma and Orthopaedic Surgery, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
• 2 European Calcified Tissue Society Clinical Practice Action Group, Brussels, Belgium
• 3 Department of Endocrinology, 424 General Military Hospital, Thessaloniki, Greece
• 4 Department of Rheumatology, Galway University Hospitals, Galway, Ireland
• 5 Interdisciplinary Department of Medicine, University of Bari Aldo Moro, Bari, Italy
• 6 Unit of Metabolic Bone and Thyroid Diseases, Fondazione Policlinico Universitario Campus Bio-Medico, Rome, Italy
• 7 Department of Rheumatology, MABLab ULR 4490, CHU Lille, University Lille, Lille, France
• 8 First Department of Propedeutic and Internal Medicine Centre of Expertise for Rare Endocrine Diseases, Medical School National and Kapodistrian University of Athens, Athens, Greece
• 9 Department of Rheumatology, Amsterdam University Medical Center, Amsterdam, The Netherlands

Question   Which conservative treatment option is most beneficial regarding pain-related outcomes in acute osteoporotic vertebral compression fractures (VCFs)?

Findings   Using data from 20 trials involving 2102 patients, this systematic review and network meta-analysis evaluated conservative interventions, including bisphosphonates, calcitonin, teriparatide, nonsteroidal anti-inflammatory drugs (NSAIDs), and braces, for the treatment of acute VCF. The short-term assessment indicated that calcitonin and NSAIDs were associated with decreased pain during activity, and the long-term assessment revealed that teriparatide was associated with lower pain levels compared with bisphosphonates.

Meaning   These findings and currently established treatment strategies highlight the value of NSAIDs and teriparatide in managing pain in acute VCF, while emphasizing the need for further research.

Importance   Osteoporotic vertebral compression fractures (VCFs) frequently cause substantial pain and reduced mobility, posing a major health problem. Despite the critical need for effective pain management to restore functionality and improve patient outcomes, the value of various conservative treatments for acute VCF has not been systematically investigated.

Objective   To assess and compare different conservative treatment options in managing acute pain related to VCF.

Data Sources   On May 16, 2023, 4 databases—PubMed, Embase, Scopus, and CINAHL—were searched. In addition, a gray literature search within Scopus and Embase was also conducted.

Study Selection   Included studies were prospective comparative and randomized clinical trials that assessed conservative treatments for acute VCF.

Data Extraction and Synthesis   Data extraction and synthesis were performed by 2 authors according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Network Meta-Analyses recommendations. A frequentist graph-theoretical model and a random-effects model were applied for the meta-analysis.

Main Outcomes and Measures   Primary outcomes were short-term (4 weeks) pain during activity and long-term (latest available follow-up) nonspecified pain in patients with acute VCF.

Results   The study included 20 trials, encompassing 2102 patients, and evaluated various interventions for managing VCF. Calcitonin (standardized mean difference [SMD], −4.86; 95% CI, −6.87 to −2.86) and nonsteroidal anti-inflammatory drugs (NSAIDs; SMD, −3.94; 95% CI, −7.30 to −0.58) were beneficial regarding short-term pain during activity compared with placebo. For long-term nonspecific pain management, bisphosphonates were associated with inferior pain outcomes compared with daily (SMD, 1.21; 95% CI, 0.11 to 2.31) or weekly (SMD, 1.13; 95% CI, 0.05 to 2.21) administration of teriparatide, with no treatment being superior to NSAIDs. The qualitative analysis of adverse events highlighted that typical adverse events associated with these medications were observed.

Conclusions and Relevance   NSAIDs and teriparatide may be the preferred treatment options for pain management in acute osteoporotic VCF. Although calcitonin also proved to be beneficial, its safety profile and potential adverse effects restrict its widespread application. The limited evidence on braces and analgesics underscores the urgent need for future research.

Osteoporotic vertebral compression fractures (VCFs) are the most common type of osteoporotic fractures and occur primarily in women and men older than 50 years. 1 , 2 Although it has been reported that 2 of 3 VCFs remain asymptomatic, those fractures that do present with symptoms often lead to substantial pain, reduced mobility, and quality of life. 3 Therefore, effective pain management is crucial for restoring functionality and patient outcomes. Regardless of their impact, an optimal and universally accepted reference standard treatment to address pain has not yet been established. Initially, percutaneous surgical techniques gained popularity owing to the promise of effective pain reduction. 4 - 6 However, various studies 7 - 9 showed conflicting results concerning the efficacy of these techniques. Because both commonly performed surgical cement augmentation procedures, vertebroplasty and kyphoplasty, have failed to demonstrate consistent benefits in terms of pain relief in previous studies, optimized conservative pain management should be applied first for treating affected patients.

For this reason, the utility of conservative therapies to treat acute painful VCF has been explored in the past. 9 - 11 These include braces, analgesics such as nonsteroidal anti-inflammatory drugs (NSAIDs) or opioids, but also various antiosteoporotic drugs, including antiresorptives (eg, bisphosphonates, calcitonin, and denosumab) or osteoanabolic agents (eg, teriparatide). 12 - 18 However, the impact of the studies published to date is limited by a lack of comparative analyses, conflicting results, and low statistical power. Given these challenges, there is a critical need for a systematic review and network meta-analysis to thoroughly assess conservative treatment options and provide clinicians with clear, evidence-based guidelines for managing VCF.

Therefore, this systematic review and network meta-analysis aims to assess and compare different conservative treatment options in managing acute pain related to VCF. On the basis of the findings, we provide evidence to guide clinical decision-making and optimize pain management strategies for patients with acute painful VCF.

The systematic review and network meta-analysis was conducted according to the guidelines of the Cochrane Collaboration and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Network Meta-Analysis recommendations. 19 , 20 The review protocol was registered in the PROSPERO database ( CRD42023423189 ). 21 Informed consent and institutional review board approval were not required because the study does not constitute human participants research, in accordance with 45 CFR §46. The last search was performed on May 16, 2023, in the PubMed, Embase, Scopus, and CINAHL databases. In addition, we also conducted a gray literature search within Scopus and Embase. We further manually reviewed the reference lists of published systematic reviews and included studies. The search strategy was designed according to the Participants, Intervention, Comparator, and Outcome (PICO) model (eTables 1 and 2 in Supplement 1 ).

The inclusion and exclusion criteria were designed in accordance with the PICO model: Eligible studies included randomized clinical trials (RCTs) or prospective comparative studies (PCSs) that examined patients with acute painful VCF. 22 In these studies, conservative treatment modalities, such as analgesics, antiosteoporotic drugs, other pharmacological interventions, or braces, had to be investigated and compared against no treatment, placebo, or other conservative treatments. To be considered for inclusion, studies were required to assess outcomes related to pain. Furthermore, studies had to be published after January 1996 to align with current medical standards and treatments, particularly with regard to osteoporosis medication. 23 , 24 We did not apply further limits regarding the age of patients or the language of publication. Details on data collection and abstraction are presented in the eAppendix in Supplement 1 .

Each included study was independently assessed for risk of bias by 2 authors (A.R.A. and T.R.). For RCTs, the assessment was conducted using the second version of the Cochrane tool to assess the risk of bias in randomized trials (RoB2). 25 For PCSs, the Newcastle-Ottawa Scale was used. 26 In addition, the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach, tailored for network meta-analysis, was used to appraise the overall certainty of the evidence regarding each outcome (eTable 3 in Supplement 1 ). 27 Further details on the assessment of study quality are presented in the eAppendix in Supplement 1 .

The meta-analysis was conducted using the netmeta package in R statistical software version 4.3.3 (R Project for Statistical Computing), using a frequentist graph-theoretical approach. 28 For continuous outcomes, the standardized mean difference (SMD) was calculated to serve as the effect size in the network meta-analysis. This was performed using the Comprehensive Meta-Analysis software version 4 (Biostat). A random-effects model with inverse variance weighting was implemented to accommodate heterogeneity across studies, given the inherent heterogeneity of treatment protocols, dosages, and durations. To evaluate consistency within the network, the net splitting technique was used, along with the separate indirect from direct evidence method using the back-calculation method. 29 This method divides network estimates into direct and indirect evidence comparisons, facilitating the identification of inconsistencies in individual comparison estimates. Although a subgroup analysis based on the risk of bias assessment was initially planned, it was deemed unfeasible because most studies were classified with an intermediate risk of bias. Furthermore, although funnel plots were considered for assessing publication bias, this was ultimately not pursued owing to insufficient statistical power, primarily because most subgroups comprised fewer than 10 trials. 19 Treatment ranking was conducted by using P-scores, which estimate the probability of one treatment being superior to the others on a continuous scale from 0 to 1. 30

Our search identified 18 791 articles, and after duplicate removal, we reviewed 15 762 titles and abstracts, resulting in 287 full-text articles evaluated for eligibility (eTable 4 in Supplement 1 ). Moreover, citation searching led to the assessment of an additional 5 studies. This ultimately resulted in the final inclusion of 20 studies in our analysis, 16 , 31 - 49 including 16 RCTs and 4 prospective comparative studies ( Figure 1 and eTable 5 in Supplement 1 ). These trials included a variety of different types of interventions related to the conservative management of VCF, including bisphosphonates, calcitonin, teriparatide, NSAIDs, opioids, ipriflavone, vitamin D supplementation, pain rehabilitation, and braces. They were conducted across various global regions, including Asia (Japan, 32 - 36 , 43 - 46 , 49 Korea, 37 , 38 Bangladesh, 42 and Hong Kong 42 ), Europe (France, 31 , 39 Italy, 41 and Greece 16 , 40 ), and the US. 47 The majority of the studies were conducted in hospital environments, 16 , 31 , 35 - 37 , 39 , 40 , 43 , 46 , 48 , 49 a few also took place in clinics and private health care facilities, 38 , 42 , 44 , 45 and 1 study 32 included both the hospital and clinic setting.

Most included studies were evaluated as having intermediate quality according to the RoB2 criteria (eFigure 1 in Supplement 1 ). Among these, one study was identified with a low risk of bias and another with a high risk of bias. The included prospective comparative studies were all evaluated as being of good quality (eTable 6 in Supplement 1 ). A total of 2102 patients with acute VCF were included in this systematic review and network meta-analysis.

Our primary outcome of interest was short-term pain during activity, because early mobilization is one of the most critical objectives in the treatment of VCF. Here, our analysis encompassed a range of treatments, such as calcitonin, bisphosphonates, teriparatide, NSAIDs, and placebo ( Figure 2 A). Activities included walking, rising from a lying position, and standing. The use of calcitonin was associated with decreased pain compared with bisphosphonates and placebo (SMD, −4.86; 95% CI, −6.87 to −2.86) ( Figure 2 B and eFigure 2 in Supplement 1 ). Similarly, NSAIDs demonstrated benefits regarding pain relief compared with placebo (SMD, −3.94; 95% CI, −7.30 to −0.58) ( Figure 2 B). However, neither teriparatide (SMD, −1.01; 95% CI, −4.87 to 2.85) nor bisphosphonates (SMD, −0.91; 95% CI, −3.68 to 1.85) revealed differences regarding pain relief compared with placebo ( Figure 2 B). Overall, when ranking treatments for short-term pain management during activity, calcitonin emerged as the most favorable (P-score = 0.92), followed by NSAIDs (P-score = 0.76) (eFigure 3 in Supplement 1 ). However, it is important to note that most of the results were of low or very low certainty of evidence (eTable 7 in Supplement 1 ). To further check the robustness of our analysis, we also performed sensitivity analyses by including only walking and rising from a lying position and only walking as activities. Here, similar results were demonstrated (eFigures 4 and 5 in Supplement 1 ), which strengthens our initial analysis.

Our secondary outcome of interest was longer-term general (ie, nonspecified) pain, aimed at evaluating the association between conservative treatments and patients’ overall pain levels at the latest available follow-up, which was a mean (SD) of 11.5 (9.0) weeks in the included studies. Here, our network consisted of various treatments, including calcitonin, bisphosphonates, teriparatide, and NSAIDs ( Figure 3 A). Our analysis revealed that both daily (SMD, 1.22; 95% CI, 0.12 to 2.32) and weekly (SMD, 1.13; 95% CI, 0.05 to 2.21) teriparatide had similar superiority over bisphosphonates (eFigure 6 in Supplement 1 ). However, comparisons with NSAIDs revealed no advantage for either daily (SMD, −1.05; 95% CI, −2.54 to 0.45) or weekly (SMD, −0.96; 95% CI, −2.43 to 0.52) teriparatide ( Figure 3 B). Similarly, no benefits were observed for the combination of calcitonin and bisphosphonates (SMD, −0.40; 95% CI, −1.54 to 0.75), calcitonin alone (SMD, −0.36; 95% CI, −1.09 to 0.37), or bisphosphonates alone (SMD, 0.17; 95% CI, −0.84 to 1.18) compared with NSAIDs ( Figure 3 B). When ranking the pharmacological interventions according to their association with long-term pain relief, teriparatide emerged as the most favorable (daily teriparatide, P-score = 0.83; weekly teriparatide, P-score = 0.78), followed by a combination of calcitonin and bisphosphonate (P-score = 0.52) (eFigure 7 in Supplement 1 ). However, it is important to note that most of the results were of low and very low certainty of evidence ( Table ).

In addition to pharmacological interventions, we evaluated braces for their association with pain relief for VCF. Here, our network consisted of 4 studies that included rigid, soft, and semirigid active braces and no brace treatment with a mean (SD) latest available follow-up of 22.00 (19.18) weeks ( Figure 4 A). Regarding the use of braces for long-term nonspecified pain management, our findings indicated no benefit for semirigid braces (SMD, −1.51; 95% CI, −3.26 to 0.25), soft braces (SMD, −0.54; 95% CI, −2.01 to 0.93), or rigid braces (SMD, −0.26; 95% CI, −1.73 to 1.21) compared with no brace ( Figure 4 B and eFigure 8 in Supplement 1 ). For the use of braces, despite not showing benefits over no brace use, a comparative ranking was performed. Here, the semirigid brace was ranked highest (P-score = 0.96), followed by the soft brace (P-score = 0.51) (eFigure 9 in Supplement 1 ). However, again, most results were of low or very low certainty of evidence ( Table ).

Adverse events were assessed qualitatively, because pooling of adverse events of varying severity was deemed inappropriate. Our qualitative analysis revealed no apparent differences between the different interventions but highlighted that typical adverse events associated with these medications were frequently observed (eTable 8 in Supplement 1 ). For instance, gastrointestinal complaints were often reported with calcitonin use; in particular, upper gastrointestinal disorders were related to NSAIDs in 1 patient. 16 , 32 , 39

Osteoporotic VCF is one of the leading causes of pain and disability worldwide. Despite the major impact of these fractures, a definitive pain management treatment standard has yet to be established, contributing to uncertainty among physicians and patients. Although surgical options like kyphoplasty and vertebroplasty are available, their value compared with conservative or sham treatments is still debated. 7 - 9 , 11 , 50 Consequently, conservative management has emerged as a viable option to treat acute pain for many patients, dictating the need for further investigation into its applicability, benefits, and risks. However, to date, there has been no conclusive answer regarding different conservative treatment modalities for VCF and their comparative analysis of pain outcomes.

In this systematic review and network meta-analysis, we focused primarily on short-term pain during activity as the main outcome parameter because of its critical importance from a clinical perspective. Avoiding prolonged bed rest can reduce the risks associated with immobility, such as worsening osteopenia, musculoskeletal function, and neurological compromise. 51 Effectively managing pain during activity and, thus, facilitating early mobilization is essential to improving patient care and was, therefore, one primary focus of our study.

Our findings revealed that calcitonin was associated with superior outcomes in managing particularly short-term pain across activity. This suggests the potential for calcitonin as a pharmacological intervention in managing acute pain in VCF, although further research is warranted for a comprehensive understanding. In this context, it is essential to consider the adverse effects of calcitonin, such as nausea, vomiting, and headaches, in the clinical decision-making process, which were also observed in our qualitative analysis of adverse events. Moreover, the European Medicines Agency’s decision to revoke intranasal calcitonin-containing product’s approval for osteoporosis treatment, because of the associated long-term cancer risk, underscores the need for caution. 52 , 53 Nevertheless, calcitonin remains approved for preventing acute bone loss in cases of sudden immobilization, like recent osteoporotic fractures, although its limited availability also restrains its use in this setting. Given our results, calcitonin should be further evaluated in terms of its potential use and safety profile in acute VCF, ensuring that a comprehensive risk-benefit analysis informs treatment decisions.

Given the favorable results of calcitonin in our study, it appeared crucial to examine the performance of more commonly used antiosteoporotic drugs, such as antiresorptive and osteoanabolic agents. It can be speculated that short-term pain-related outcomes are primarily associated with the direct reduction of pain, whereas long-term outcomes are more likely influenced by the decrease in the incidence of new VCF. Surprisingly, bisphosphonates were not associated with decreased pain regarding both short-term pain during activity and long-term nonspecified pain. This was the case for both oral and intravenous bisphosphonates, as our analysis incorporated various forms of bisphosphonates, including pamidronate (intravenous), 31 , 39 alendronate (oral), 34 , 43 risedronate (oral), 43 , 46 and minodronic acid hydrate (oral). 44 Therefore, our data suggest that all available bisphosphonates have similar overall results for VCF pain.

Importantly, both daily and weekly administration of teriparatide were found to be more beneficial than bisphosphonates regarding long-term nonspecified pain. It has been proposed that teriparatide might exert its effects through anabolic mechanisms that promote fracture healing. 43 , 54 In addition, teriparatide could directly alleviate pain by decreasing proinflammatory cytokines. 55 Its superior results regarding long-term pain relief compared with bisphosphonates, which function primarily through antiresorptive mechanisms, might derive from its anabolic properties. The analgesic properties of bisphosphonates have been proposed to result from osteoclast inactivation and reduction of the acidic environment, in contrast to teriparatide, which promotes bone formation and may, therefore, be better tailored for VCF than bisphosphonates. 56

Our results highlight a notable gap regarding evidence on analgesics for acute painful VCF. Our extensive search identified only 2 studies evaluating NSAIDs 32 , 45 and 1 study 47 assessing the role of opioids. Notably, NSAIDs were associated with less pain during activity compared with placebo, suggesting an important role for NSAIDs in acute short-term pain control of VCF. The study 47 examining opioids, specifically tapentadol and oxycodone, failed to find any difference compared with placebo but was prematurely terminated, limiting its capacity to detect meaningful differences at all. This indicates a pressing need for further research to comprehensively evaluate the role of both nonopioid and opioid analgesics in managing pain for patients with acute VCF.

In evaluating braces, we found no benefit in long-term pain management, aligning with other studies. 57 However, it is important to interpret these results with caution, because pain in VCF may last for only 6 to 8 weeks. 58 , 59 Therefore, although braces may be appropriate and beneficial for initial immobilization, our findings do not support long-term benefits from their use. Nevertheless, additional high-quality RCTs are necessary to enable a comprehensive assessment of braces, particularly regarding their short-term outcomes.

Our initial motivation to perform this meta-analysis was based on the observation of one of the authors (W.F.L.), while preparing a presentation for the European Calcified Tissue Society 2023 annual meeting, that there was surprisingly little evidence on conservative treatment for the management of acute painful VCF. Above all, the high burden of affected patients, as well as the inconsistent results of surgical procedures such as vertebroplasty and kyphoplasty, should be reason enough to investigate the outcomes related to conservative treatment modalities thoroughly. The results of this systematic review and network meta-analysis may indicate a favorable role of teriparatide, which also shows excellent results in increasing vertebral bone mineral density and reducing the risk of vertebral fractures. 60 This finding is critical because it shows that an antiosteoporotic drug may also have a pain-reducing effect, which supports its use promptly after a fracture occurs. However, we were surprised by the poor evidence for other commonly used antiosteoporotic drugs, such as denosumab, calling for the need to conduct high-quality RCTs on the role of denosumab in pain management after an acute VCF. Furthermore, on the basis of the promising data on teriparatide, the outcomes related to romosozumab, the most recently approved osteoporosis medication with a dual osteoanabolic-antiresorptive effect, should also be evaluated in this context. 61 In our opinion, the lack of benefits of braces regarding long-term pain relief is clinically highly relevant, and more high-quality studies are needed, particularly evaluating their role in short-term pain relief. Another critical point is that there were only 3 studies 32 , 45 , 47 on analgesics, which is why future studies should specifically evaluate the outcomes of targeted pain protocols using analgesics such as NSAIDs and opioids. We would like to motivate both researchers and funding agencies to conduct and support studies on conservative pain management in patients with acute VCF, using controlled study designs that will evaluate specific treatment modalities against each other or placebo. The development of evidence-based treatment guidelines depends on more high-quality clinical research, especially RCTs.

Our study has several limitations. The included studies exhibited high clinical heterogeneity. For instance, the studies used various forms of calcitonin (salmon, eel, and synthetic human derivatives) and bisphosphonates (oral, intravenous, and various preparations). Although this diversity broadens our analysis, providing a comprehensive overview of the overall value regarding pain relief, it also increases heterogeneity, potentially hindering our ability to detect true differences between treatments. Furthermore, we included both RCTs and PCSs, which introduces variability in study quality and inevitably increases the risk of bias. However, the PCSs included were all of high quality, contributing to a more robust and comprehensive analysis. Another limitation of our study is the potential for confounders owing to varying treatment protocols among the studies. Differences in patients receiving prior antiosteoporotic therapy and the permitted use of NSAIDs or the additional application of braces for early immobilization also contribute to heterogeneity. Although these practices are ethically justified and reflect real-world clinical settings, they may limit the comparability of study outcomes. In addition, our primary outcome, pain during activity, presents an inherent limitation, because activities such as walking, rising from a lying position, and standing were not uniformly defined but were pooled together. This also leads to increased heterogeneity. However, we have conducted sensitivity analyses regarding this outcome to strengthen the robustness of our findings and mitigate the impact of different activities pooled together. A further limitation of our study is that the generalizability and applicability of our results is certainly limited, which is reflected by our GRADE assessment, which revealed low to very low certainty of evidence for most of our results but emphasizes, therefore, once again the need for more high-quality research. In addition, our review lacks not only an analysis of functional outcomes but also quality-of-life outcomes because they were not consistently assessed and reported across studies. Therefore, future studies should include these outcomes to allow for a more comprehensive pooling of data in the future.

Furthermore, our results on braces, as reported in our findings, should be interpreted with caution. The data regarding braces often rely on patient compliance, which can be ensured only to a certain extent. Despite their frequent use in clinical practice and positive anecdotal evidence, our study underscores the need for more research to provide more definitive evidence that can guide clinical practice and confirm or challenge the clinical value of braces.

Taken together, our study provides a comprehensive analysis of various conservative treatments for acute VCF. We identified that calcitonin, in its various forms, and teriparatide stand out for their ability in managing both short-term and long-term pain management associated with these fractures. Given the limitations arising from the included studies’ quality and heterogeneity and the lack of sufficient studies, especially on analgesics, our analysis calls for further high-quality RCTs to support clinical decision-making.

Accepted for Publication: July 11, 2024.

Published: September 6, 2024. doi:10.1001/jamanetworkopen.2024.32041

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2024 Alimy AR et al. JAMA Network Open .

Corresponding Author: Tim Rolvien, MD, PhD, Department of Trauma and Orthopaedic Surgery, University Medical Center Hamburg-Eppendorf, Martinistrasse 52, 20246, Hamburg, Germany ( [email protected] ).

Author Contributions: Dr Rolvien had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Lems and Rolvien jointly supervised this work and share last authorship.

Concept and design: Alimy, D’Oronzo, Naciu, Yavropoulou, Lems, Rolvien.

Acquisition, analysis, or interpretation of data: Alimy, Anastasilakis, Carey, Paccou, Rolvien.

Drafting of the manuscript: Alimy, Anastasilakis, Naciu, Rolvien.

Critical review of the manuscript for important intellectual content: Alimy, Anastasilakis, Carey, D’Oronzo, Paccou, Yavropoulou, Lems.

Statistical analysis: Alimy.

Administrative, technical, or material support: Anastasilakis, Rolvien.

Supervision: Carey, D’Oronzo, Naciu, Paccou, Yavropoulou, Lems, Rolvien.

Conflict of Interest Disclosures: Dr Carey reported receiving grants from UCB, grants and personal fees from Amgen, and personal fees from Consilient outside the submitted work; serving as president of the International Society for Clinical Densitometry (ISCD); being a member of the faculty for all ISCD densitometry courses; and being a member of Position Development Conference task forces and expert review panel and a speaker at the ISCD annual meetings from 2008 to 2024. Dr Lems reported receiving personal fees from UCB, Amgen, Pfizer, and Galapagos outside the submitted work. No other disclosures were reported.

Meeting Presentation: This paper was presented at the European Calcified Tissue Society annual meeting; May 27, 2024; Marseille, France.

Data Sharing Statement: See Supplement 2 .

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 07 September 2024

Progress on the research and development of plague vaccines with a call to action

• E. Diane Williamson   ORCID: orcid.org/0000-0002-0322-5972 1 ,
• Paul B. Kilgore 2 ,
• Emily K. Hendrix 2 ,
• Blake H. Neil   ORCID: orcid.org/0009-0002-8091-0604 2 ,
• Jian Sha 2 &
• Ashok K. Chopra 2 , 3 , 4 , 5 , 6  

npj Vaccines volume  9 , Article number:  162 ( 2024 ) Cite this article

1 Altmetric

Metrics details

• Bacterial infection

There is a compelling demand for approved plague vaccines due to the endemicity of Yersinia pestis and its potential for pandemic spread. Whilst substantial progress has been made, we recommend that the global funding and health security systems should work urgently to translate some of the efficacious vaccines reviewed herein to expedite clinical development and to prevent future disastrous plague outbreaks, particularly caused by antimicrobial resistant Y. pestis strains.

Content includes material subject to Crown Copyright © 2024.This is an open access article under the Open Government License ( http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/ ).

Similar content being viewed by others

Vaccine development for emerging infectious diseases

Looking beyond COVID-19 vaccine phase 3 trials

Reverse development of vaccines against antimicrobial-resistant pathogens

Epidemiology of plague.

Plague, caused by the gram-negative bacterium Yersinia pestis , is notorious for its involvement in three of the seven deadliest pandemics recorded in global history, including the recent COVID-19 pandemic. The three historic plague pandemics, the most infamous of which was the Black Death of the Middle Ages, collectively caused an estimated 200 million deaths 1 , 2 . Unfortunately, plague is still an endemic disease in parts of the world, with outbreaks being reported to the WHO from over 33 countries including Madagascar, Democratic Republic of the Congo (DRC), India, China, Peru, and occasionally, the south-western USA 3 . In these regions, disease is maintained by the existence of infected animal (mostly rodent) reservoirs of Y. pestis 4 , 5 .

Transmission to humans is predominantly promoted by flea bite; those fleas having fed on infected rodents (Fig. 1 ). However, Y. pestis is an obligate parasite and even if the rat population is reduced, the organism can infect mice, prairie dogs, rabbits, and members of the cat family, including the domestic cat 4 , 5 , 6 .

The figure depicts various routes for the flea-vectored transmission of plague to man. The figure is reproduced from Williamson and Westlake (2019) 8 with permission (License 5753521304382, Oxford University Press).

Infection through flea bite causes bubonic plague, which if undiagnosed, can develop into a septicemic infection or secondary pneumonic plague. Pneumonic plague is highly contagious requiring prompt antibiotic therapy for survival, as the mortality rate approaches 100% if untreated 6 , 7 , 8 . In pneumonic plague, Y. pestis can be transmitted to healthy individuals who are in close contact by respiratory droplets, establishing further cases of primary pneumonic plague, leading to disease outbreaks which may transform into epidemics and pandemics 6 .

With this epidemiology, poor living conditions augment the endemicity of plague, which requires close contact with a rodent population. However, in endemic regions such as Madagascar, the lack of an approved vaccine means that outbreaks have to be controlled by antibiotic therapy, administered to the patients and those in immediate contact with the infected individuals. Whilst timely antibiotic therapy is effective in treating the infection, case fatality rates still reached up to 8.6% during the 2017 Madagascar outbreak despite aggressive antibiotic therapy 9 . Additionally, there is a demonstrable risk of the development of antibiotic resistance. Indeed, antimicrobial resistant (AMR) Y. pestis strains have been identified in Peru and Madagascar 10 , 11 . Therefore, there is a clear need for a safe, effective, and licensed vaccine for use in endemic regions to control or prevent infection, as well as to protect military and civilians at large from potential biothreat attacks.

Emergence of Yersinia pestis as a dangerous pathogen

Y. pestis has evolved from the relatively mild gastrointestinal Y. pseudotuberculosis (notably serotype 1b) between 1500 and 20,000 years ago 12 , although archaeological evidence has suggested that the plague-causing bacterium existed long before previous estimates 13 .

The evolution of Y. pestis has resulted in the inactivation of genes required for an enteric lifestyle and by the acquisition of plasmids encoding new virulence factor-encoding genes 14 . In common with other pathogenic yersiniae ( e.g., Y. pseudotuberculosis and Y. enterocolitica ), Y. pestis possesses a 70-kilobase (kb) virulence plasmid designated as pYV/pCD1 that carries a Type III secretion system (TTSS) operon 15 , 16 . However, Y. pestis has acquired two additional unique plasmids, including a 9.5-kb pPCP1/pPla/pPst encoding a bacterial surface-bound protease (plasminogen activator, Pla), which has potent fibrinolytic activity 1 . In addition, this plasmid possesses pesticin and coagulase encoding genes which enable bacterial transmission from the flea 17 . The other 100–110 kb pFra/pMT1 plasmid 18 codes for two important proteins, Fraction 1 (F1) antigen and a phospholipase D known as murine toxin. The F1 antigen forms a polymeric anti-phagocytic capsule around the bacteria 18 whilst murine toxin has a role in preserving Y. pestis in the flea gut 19 . During its evolution from enteric to flea-vectored pathogen, Y. pestis has lost intestinal adhesin and invasin genes, but has retained the heme locus and possesses a number of chromosomal-encoded genes such as the ph6/psa fimbrial and attachment-invasion locus ( ail ) which promote colonization to the host cells 19 , 20 , 21 .

Evasion of host responses

In the process of acquiring a new mechanism of infection, Y . pestis has also activated genes which enable the pathogen to evade the defenses of its successive hosts. In purified or recombinant forms, some of these encoded gene products have provided vaccine targets and are therefore summarized here.

Y. pestis can survive and grow in the flea’s (most notably the rat flea Xenopsylla cheopis ) foregut, leading to ‘blockage’ of the flea. The proper functioning of the bacterial hemin storage system is thought to play an important role in the formation of this blockage 19 , which during the flea bite, results in the regurgitation of a dense bolus of bacteria 5 into a new host. Y. pestis expresses other genes in the flea gut such as a ‘murine’ toxin with phospholipase D activity 20 and a lipopolysaccharide (LPS) core modification locus, which together are required for biofilm formation and blockage of the flea 20 , 21 , 22 . However, transcriptional analysis of Y. pestis in the flea gut has identified a wide range of additional genes, such as insecticidal-like toxin genes, which are differentially regulated such that bacteria regurgitated into a new host have increased resistance to innate immune effectors 23 .

Upon infection of a new mammalian host, the plague bacilli are vulnerable to phagocytosis by polymorphonuclear leukocytes (PMNs or neutrophils) and/or monocytes. The bacteria may be killed within PMNs, but can persist within monocytes and express various virulence determinants, allowing Y. pestis growth and eventual release from the monocytes 24 . The fibrillar adhesin pH6 antigen is induced by low phagosomal pH (4.5) 25 and promotes bacterial adhesion to host cells, thereby enhancing resistance to phagocytosis 26 . Secretion of the F1 antigen with capsule formation is triggered by a temperature shift from 28 °C in the flea to 37 °C in humans or other mammals. The F1 capsule also plays a key role in avoiding phagocytosis 27 . However, non-capsulated Y. pestis retains its full capability to cause pneumonic infection in animals, while having reduced virulence during bubonic infection 28 .

The dominant anti-host effects are due to a temperature shift induction of the TTSS carried on the virulence plasmid pYV/pCD1. TTSS effectors, historically called Yersinia outer membrane proteins (Yops), have cytotoxic and phagocyte regulatory effects, are secreted through an injectosome after Y. pestis makes contact with the host cell, and are delivered into target cells 15 . The function of many of the Yops has been delineated for this well-characterized secretion system, and serves as a paradigm for other bacterial TTSS’s 15 . For example, the YopE protein is a cytotoxin and the YopH protein is a tyrosine phosphatase with anti-phagocytic activity 29 . The V (or Low calcium response V, LcrV) antigen plays a pivotal role by orchestrating intracellular Yop low calcium response protein G (LcrG) elaboration of the injectosome and then itself being delivered through this needle-like structure to be assembled as a pentamer at the tip 30 . Additionally, V antigen secreted from Y. pestis exerts a local immunomodulatory effect in the host by down-regulating the production of interferon-γ (IFNγ) and tumor necrosis factor-α (TNFα) 31 , 32 .

Plasminogen activator (Pla) is another major virulence factor in Y. pestis . Pla is an outer membrane-located protease, which breaks down the physical barriers of connective tissue in the host, thus promoting the systemic dissemination of the bolus of Y. pestis injected by the flea. The requirement for Pla has driven the selection in Y. pestis of the “rough” phenotype of LPS, which lacks an O antigen 33 , 34 , a rare phenomenon amongst gram-negative bacteria which has possibly resulted from the bacterium’s transmission through the flea, but which is necessary for Pla to be functional 35 , 36 . Inactivation of the O-antigens on Y. pestis LPS exposes the LPS core, so that Y. pestis can interact with C-type lectin receptors on host macrophages, promoting its uptake, and thus accelerating bacterial dissemination in the host 37 . Our study has also shown that the Δ pla mutant is unable to survive efficiently in murine and human macrophages, unlike the wild-type Y. pestis 38 .

The bacteria disseminate from the site of primary infection into draining regional lymph nodes. Within the lymph node, further growth of the bacteria accompanied by a massive inflammatory reaction leads to lymphadenopathy and the formation of buboes, typically in the groin or axillae. In the bubo, bacteria are predominantly extracellular, mainly due to the TTSS which is highly expressed in the lymph node 39 . An ability to proliferate in the bubo 40 is enabled by the efficient and abundant iron acquisition systems possessed by Y. pestis 41 .

Eventually, the bacteria are disseminated by the lymphatic system, gain access to the blood stream, and colonize pulmonary tissues, which may lead to development of the pneumonic form of the disease. When left untreated, pneumonic plague induces an overwhelming septicemia which triggers septic shock in the host. However, the precise mechanisms that lead to the death of the host have not been identified but involve multi-organ failure, during which the systemic induction of nitric oxide synthase may contribute, as seen with other gram-negative septicemias 42 .

Whilst pigmentation ( pgm )-negative strains of Y. pestis are usually avirulent and attenuated, the risk of reversion to virulence was highlighted by the fatal case of a laboratory worker who was unknowingly suffering from hemochromatosis and was exposed to the attenuated pgm - Y. pestis laboratory strain KIM. This individual developed plague and died, presumably due to his hemochromatosis-induced iron overload condition providing the infecting KIM strain, attenuated through defects in its iron acquisition ability, with sufficient iron to render it virulent 43 .

Early vaccines

The early use of an inactivated whole cell vaccine for plague by Haffkine between 1897 and 1935 successfully curtailed plague outbreaks in India. This was the first demonstration that components of Y. pestis , even when inactivated, could be immunogenic. Haffkine’s heat-killed whole cell (KWC) vaccine was administered to the human population in an estimated 24 million doses 44 (Table 1 ).

During the 1990s, there were several commercial suppliers of the KWC vaccine against plague. Subsequently, plague vaccine USP (United States Pharmacopeia; 1939–1999), containing formaldehyde-killed bacteria, was manufactured by Cutter Laboratories, USA. In 1994, the manufacturing was transferred to Greer Laboratories Inc., USA. In 1999, the production of this vaccine was discontinued largely because of severe side effects and its protection against bubonic but limited efficacy against pneumonic plague 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 . An alternative heat-killed (KWC) vaccine was also manufactured by the Commonwealth Serum Laboratories (CSL, Australia) 53 and until November 2005, was licensed for clinical use in Australia. Additionally, a Y. pestis isolate (EV76-NIIEG Y. pestis ) 54 , which is attenuated due to deletion of the pigmentation locus ( pgm ), has been used as a vaccine for many years and is licensed for use in China and Russia specifically 55 where plague is endemic. The vaccine can be administered by various routes; however, the vaccine is fully virulent under iron-overload conditions, i.e ., in individuals with hemochromatosis 56 , 57 (Table 1 ).

Virulence factors as vaccine antigens

The seminal observation in 1956 by Bacon and Burrows that Pasteurella pestis (now Y. pestis ) could be anti-phagocytic in the absence of capsule, led to the identification of a new virulence antigen, which they named the V antigen 58 . This paved the way for subsequent research to the present day on the immunogenic and protective potential of this and other virulence factors of Y. pestis 59 , 60 . Building on the observation that the F1 antigen-containing Cutter KWC vaccine needed the addition of a recombinant V (rV) antigen to fully protect mice against pneumonic plague 61 , Williamson, et al. demonstrated the synergistic effect of F1 and V in combination. Whilst vaccines lacking the V antigen may protect against bubonic plague, several groups showed that the inclusion of the V antigen was an essential requirement for protection against pneumonic plague 61 , 62 .

Much work has been carried out to determine the protective potential of other antigens derived from Y. pestis in native or recombinant form in addition to F1 and V, such as Pla, a protein constituent of the injectisome known as Yersinia secretory factor F (YscF), and a range of other Yops, and their various combinations 63 , 64 , 65 , 66 , 67 . Whilst some of these imparted partial protective efficacy and are useful adjuncts in some vaccine formulations (see below), to date F1 and V remain the key proteins which individually have protective efficacy, but which in combination, are consistently synergistic and, therefore, form the core building blocks of most vaccine approaches.

Vaccines for plague

Currently, there are more than 21 candidate vaccines in the preclinical phase 3 . Below, we have reviewed the pre-clinical candidates (Tables 1 – 5 ) and subsequently those that are in early clinical development with a timeline (Fig. 2 ). The pre-clinical candidates can be broadly categorised as subunit, live attenuated, vectored (bacterial or viral), DNA, or messenger RNA (mRNA).

*Adjuvant not specified. Ages of study participants ranges from 18 to 55 years. All vaccines were given in 2–3 doses intramuscularly over a range of 6 months. The EV 76 NIIEG vaccine was given 1–4 times at intervals of 1–3 months. ?data not published; !data not conclusive.

Many groups have now shown that immunization with the F1 and V subunit antigens provides a high degree of protection against infection caused by Y. pestis in a range of animal models 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 . The use of F1 and V in combination (F1 + V) or as a genetic fusion (F1-V) has the advantage that protection can be maintained against acapsular (F1-negative) Y. pestis strains which still retain virulence 79 . Many different formulations have been researched with a view to finding one that provides comprehensive protection with the least number of doses, is stable, and maintains immunogenicity when escalated up the species from mice to humans. These candidate vaccines which have been studied in much more detail are summarized in Table 2 .

Adjuvants used in preclinical rF1V vaccine development studies include the toll-like receptor 5 (TLR5) ligand flagellin and protollin 73 , 74 , 75 . Packaged in a polyanhydride nanoparticle with cyclic dinucleotide and delivered as a single dose intranasally, rF1V protected mice against pneumonic plague 76 . A truncated form of rV (rV10) has also been shown to be protective, as has rV10 in manganese silicate nanoparticles 71 , 77 . Similarly, the peptidoglycan-free outer membrane vesicles (OMV) with a phage lytic system has been demonstrated to be efficacious 78 , as have microvesicles derived from human commensal gut bacteria for immunization with V antigen 80 or OMV’s from Y. pseudotuberculosis 81 . A dry formulation of rF1+rV delivered on calcium phosphate-decorated microparticles demonstrated enhanced immunogenicity and efficacy 82 .

A study has shown that the polymeric form of F1 led to rapid protective humoral immune response by activating innate-like B1b cells 83 (Table 2 ), and further observations suggested that this activation was unaffected by the presence of the V antigen in an admixture of F1 and V 83 . Recent research has evaluated the impact of the administration of synthetic immunomodulating peptides on the survival of mice and guinea pigs subsequently exposed to virulent Y. pestis 84 . Administered in three doses prior to animal challenge, two immunomodulators were found to have a positive impact on survival; these were an azoximer bromide (polyoxidonium) and rIFNγ 84 . Another study has shown that co-formulation of the rF1-V vaccine with recombinant human (rhIL2) and/or recombinant murine GM-CSF in alhydrogel enhanced immunogenicity and efficacy against a lethal aerosol challenge in mice 85 .

In April 2024, the Russian state regulator granted a marketing authorization 86 for a single sub-cutaneous dose microencapsulated molecular plague vaccine (PMMM) comprising 25–30 µg each of rF1 and rV in 4–6 mg polylactide, with the excipients polyvinyl alcohol, alhydrogel, polyvinyl pyrrolidine, polysorbate in phosphate buffered saline and containing 30–60 µg thiomersal.

Live attenuated

The live attenuated vaccine (LAV) strain EV76-NIIEG has been used for human vaccination in Russia and China for many years to prevent or curtail outbreaks of plague 3 . Recently, the experimental evaluation of polyoxidonium co-administered with EV76 in a murine model has been shown to improve efficacy 87 .

In addition to the live vaccine strain EV76, various deletion mutants of Y. pestis CO92 have been demonstrated to be efficacious in rodent models of pneumonic plague 88 , 89 , 90 , 91 (Table 3 ). Among these, LMA and LMP mutants (deleted for genes encoding Braun lipoprotein [Lpp], methylacyl transferase B [MsbB], and either Attachment-invasion locus [Ail] or Plasminogen-activating protease [Pla], were of note, as they triggered robust humoral and cell-mediated immune responses in mice and were eliminated from the animals within 12–24 h 88 , 89 , 90 , 91 . Importantly, these mutants remained avirulent under iron-overload conditions 56 . Further, a heterologous prime-boost strategy using one dose each of LMA or LMP and replication-defective adenovius5-based three component vaccine containing genes for YscF, F1, and LcrV (Ad5-YFV) administered in any order was highly efficacious with complete protection in mice in a pneumonic plague model 56 providing safety and combined benefits of subunit and live-attenuated vaccines (Table 5 ). Likewise, EV76 vaccine deleted for Pla 92 has shown promise. Recent work has also addressed the possibility of further attenuation of Y. pestis to serve as a vaccine 93 . The two most protective vaccine candidates were Y. pestis CO92 mutants that were either cured for the pgm locus and the pPst plasmid or deleted for the yscN gene. These mutants completely protected BALB/c mice against subcutaneous and aerosol challenge with Y. pestis 93 , 94 , 95 (Table 3 ).

Since the potential to harness rDNA technology to produce vaccines in the 1980s, many more candidate plague vaccines have been pursued 45 , 46 , 52 , 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , including the use of attenuated bacterial or viral vectors to deliver antigens derived from Y. pestis . Vectors being evaluated include: Salmonella , Yersinia pseudotuberculosis , Lactobacillus , adenovirus, vesicular stomatitis virus, and vaccinia virus. Some of these well-studied vaccine candidates have been summarized in Table 4 .

A common advantage of these vectors is that because they are live, but replication-deficient, only 1 or 2 doses of vaccine may be required to achieve protective immunity. A second advantage is that these vectored vaccines can be multivalent, expressing antigens from different pathogens and can deliver these antigens intracellularly, mimicking infection and inducing appropriate immunity. All of these vaccine vectors require an in vivo promoter to switch on the expression of a heterogenous antigen(s) to induce an immune response. The efficiency of the promoter and the molecular size of the expressed protein-encoding genes, together with need for post-transcriptional modifications such as glycosylation, determine the level and potency of the expressed vaccine antigens. Potential disadvantages of live vaccine vectors are the necessity of stable attenuation, the risk of use in immunocompromised individuals, the possibility of inducing immunity, or pre-existing immunity to the vector itself; however, the latter can be overcome by modification of the vector or by employing a heterologous prime-boost approach to prevent reduced responses on repeated use of the same vector 99 , 100 as we have recently shown 52 . Further, our study in non-human primates showed that inducing pre-existing antibodies to Ad5 did not alter protective immune responses in a pneumonic plague model 114 .

A substantial amount of research has been devoted to the development of Y. pseudotuberculosis as a vaccine for plague by deleting three essential virulence factors (High Pathogenicity Island, pH6 antigen, and YopK toxin) and by the insertion of the caf operon into the chromosome, allowing the production of an F1 pseudocapsule 101 , 102 , 103 , 104 . A Y. pseudotuberculosis construct (VTnF1) modified to maximize stability was immunogenic and efficacious against pneumonic plague in mice after a single oral dose 102 , and generated humoral and cell-mediated immune responses 103 . Subsequently, the VTnF1 vaccine has been shown to be effective in mice after subcutaneous injection and protects fully against injected (10 4 LD 50 ) or 80% of animals against aerosolized (3300 LD 50 ) Y. pestis CO92 104 (Table 4 ).

Likewise, outer membrane vesicles (OMV) produced from a mutated version of Y. pseudotuberculosis expressing V and a modified version of LPS have been shown to be protective in mice against pneumonic plague 105 (Table 4 ).

There has also been substantial research investment in Salmonella Typhi as a vaccine vector, particularly with its potential as an oral vaccine for plague. Early studies showed that the successful carriage by S . Typhimurium of the F1-encoding plasmid resulted in F1 protein secretion by S . Typhimurium, with visualization of the capsule surrounding the bacteria 106 . However, whilst immunogenicity and efficacy were achieved, sustaining the vector in vivo to retain plasmids with sufficient gene expression over time without causing salmonellosis, has been an enduring challenge 107 , 108 . More recently, combinations of F1, Psn (pesticin receptor), and V antigen delivered orally to mice using mutant strains of S . Typhimurium have provided 100% protection against subcutaneous challenge with 570 LD 50 of Y. pestis CO92, but only 40-60% efficacy against 50 LD 50 of aerosolized Y. pestis CO92 109 . Moreover, S . Typhimurium deleted for the genes lpp and msbB and used to express F1, LcrV, a combination of F1 and LcrV, and a combination of YscF and YopD, protected mice against Y. pestis CO92 infection in a pneumonic plague model 45 , 110 .

Expression of genes encoding F1 and LcrV of Y. pestis and protective antigen (PA) of Bacillus anthracis in a Francisella tularensis LVS vaccine strain provided protection to mice against all three Tier-1 select agents, raising the prospect of a polyvalent biodefense vaccine 111 (Table 4 ).

Bacteriophage T4 serves as an excellent nanoparticle platform to deliver plague immunogens (F1 and V as well as PA antigen of B. anthracis ). Both mice and rats immunized with T4 phages without any adjuvant and harboring Y. pestis and B. anthracis immunogens were protective against pneumonic plague and lethal toxin intoxication when administered sequentially or simultaneously 112 (Table 4 ).

Recent preclinical studies have shown that vaccination of mice with Ad5-YFV provided complete protection to mice in a pneumonic plague model when challenge occurred with the F1-minus strain of Y. pestis CO92 113 . This is when compared to animals that were vaccinated with the monovalent, Ad5-LcrV-based vaccine, and challenged with the F1-minus strain of Y. pestis CO92 where anti-F1-antibodies were rendered ineffective 113 . This Ad5-YFV vaccine resulted in robust humoral and cell-mediated immune responses 114 . The above vaccine also provided 100% protection to Cynomolgus macaques at a very high challenge dose of Y. pestis CO92 administered by the aerosol route 114 (Tables 4 and 5 ). An earlier version of Ad5-based vaccine harbored genes for F1 and LcrV and was shown to be protective in a murine pneumonic plague model 115 , 116 .

A chimpanzee adenovirus vector (ChAdOX1) vaccine expressing F1 and V has been developed by the Oxford Vaccine Group. The ChAdOX1 vector is a replication-deficient adenoviral vector based on the simian adenovirus type Y25, originally chosen to avoid pre-existing adenovirus immunity in the human population 117 . The phase 1 clinical trial started on this ChAdOX1 plague vaccine in 2021 118 (Fig. 2 ).

DNA-based plague vaccines comprising F1 and LcrV have also been tested and found to be immunogenic and protective (Table 4 ). In an earlier study, we have shown that mice immunized with plasmid vectors containing genes for F1, LcrV, with a gene for LT (heat-labile enterotoxin as an adjuvant) were protective against pneumonic plague 45 . In this report, mice were immunized with recombinant plasmids coated with 1.6-μm gold particles and shot with the gene gun on the ears. The animals were immunized on days 0 and 3 months before intranasal challenge after 8 months following the last boost with Y. pestis CO92 45 .

A DNA vaccine designed to protect against both anthrax and plague was evaluated in mice 119 . DNA constructs comprising fusions of V with a truncated anthrax lethal factor (LF) or LF with F1 or V alone, were coated to gold nanoparticles and delivered by gene gun to A/J mice and were shown to protect fully against challenge 21days later wth aerosolized B. anthracis and to 80% against aerosolised Y. pestis 119 .

More recently, mRNA technology has been extended to produce candidate plague vaccines. An mRNA vaccine expressing a circularly permutated form of F1 delivered in lipid nanoparticles (LNP) protected (100%) of mice against bubonic disease after only a single dose 120 , whilst a self-amplifying mRNA LNP vaccine expressing both F1 and V was immunogenic in 2 doses and also protected outbred mice against a recent clinical isolate of Y. pestis from Madagascar in a bubonic plague model 121 . Both of these vaccines induced humoral and cell-mediated immune responses (Table 4 ) and show promise for future development, allowing a very flexible platform into which additional or modified RNA could be added, if needed. Furthermore, rapid advances in large scale manufacturing and formulation achieved for mRNA vaccines for SARS-CoV-2 are now readily transferrable to the plague application 122 .

Translation to clinical development

Some vaccine approaches discussed above have transitioned to early clinical development (Fig. 2 ). Currently, a formulation of recombinant F1V (rF1V) in alum, supplemented with CpG 1018 is in Phase 2 clinical trial (Dynavax, USA) 3 . This form of CpG has already been incorporated in a Hepatitis B vaccine and approved by the FDA for clinical use in adults 123 . Also in phase 2 clinical trial is a formulation of native F1 with rV in alum (Lanzhou Institute and Jiangsu Provincial CDC, China) 3 . The third subunit vaccine currently in Phase 1 clinical trial comprises rF1V adjuvanted with flagellin (National Institute of Allergy and Infectious Diseases, USA) 3 , 124 .

The EV76-NIIEG vaccine is still approved only in China and Russia where it is in Phase 4 clinical trial. Since 2002, there have been vaccination campaigns with EV76 in 16-18 provinces of Mongolia by the National Centre for Zoonotic Diseases, Ulaanbaatar, Mongolia 3 . In a vaccination campaign report from plague-endemic foci in Mongolia, an adverse event rate of 7.3% with a 5.6% breakthrough in protection has been reported 3 . In October 2023, there was an ongoing campaign with EV76 in Mongolia in response to an outbreak of plague 3 .

Strategies to gain evidence of vaccine efficacy

As with nearly all clinical prophylaxes or therapies, the pathway to regulatory approval is time-consuming, expensive and difficult, requiring evidence (direct, indirect or deduced) of human efficacy. This is especially challenging for vaccines against Tier-1 select agents such as Y. pestis , as human challenge studies are unethical. Further, due to the endemic nature of the disease, the number of infected patients is not large enough to draw meaningful conclusions on vaccine efficacy. Here we review the strategies available to demonstrate vaccine efficacy for plague.

Animal data to support licensing

Because of the pathogenicity of Y. pestis , its potential for epidemic spread, and the unpredictable nature and size of regional plague outbreaks, the feasibility of Phase 3 trials, whether preventive or reactive in nature, is under discussion 3 . Even in Madagascar, where the plague season is well known, the number of cases involved varies, so a Phase 3 trial may not be sufficiently powered, unless successive seasons are used. Pathways to licensure may therefore comprise the scrutiny of immunogenicity and efficacy data generated in animal models under Good Laboratory Practice (GLP) using the FDA’s Animal rule 125 with the human immunogenicity data generated in clinical trials ( i.e . immunobridging) 126 (Fig. 3 ).

The figure depicts the use of percentage survival in vaccinated animals, which correlates with an immunological readout(s), to compare with the same immunological readout determined in a clinical trial, to predict vaccine efficacy in human.

Evidence of efficacy could then be gained post-licensure. There is precedent for this with the Marketing Authorization under exceptional circumstances of an Ebola vaccine by the European Medicines Agency on the basis of human serum antibody data 127 . The FDA also has an accelerated approval pathway but cautions that the bar for approval of a vaccine for the pneumonic indication would be set higher than for the bubonic indication 3 .

Immunobridging from animal models to human

The vaccine approaches reviewed here have predominantly been screened in mice (outbred as well as inbred) 128 , with some in Brown Norway rats 112 , 129 and a few in NHPs 48 , 68 , 69 , 114 , 130 all of which are authentic models for plague and evaluate both antibody and cell-mediated immunity 131 . The most consistent NHP model is the cynomolgus macaque 3 , 114 . However, increasing diversity in response occurs with escalation up to NHPs and the human population 132 . Table 5 summarises the studies which have been performed in NHPs to determine the efficacy of vaccines which are in preclinical development.

Immune correlates of protection and surrogate markers of efficacy

Many researchers have now shown that antibody titers to the F1 and V proteins correlate with protection against bubonic and pneumonic plague in a range of animal models 66 , 68 , 69 , but the induction of cell-mediated immunity (CMI) 131 , 132 and particularly a balanced Th1/Th2 response 56 , 93 , 103 , 133 , 134 provides an optimal strategy for protection. The observation, inter alia , that mice immunized with the rF1-V vaccine and depleted of TNFα and IFNγ just prior to challenge, had poor survival compared with immunized controls which were not depleted 135 indicated key roles for these Th1 cytokines in the development of protective immunity and these cumulative data have spurred the formulation of vaccine candidates which induce appropriately balanced immunity.

To enable effective immunobridging of animal data to human, it is preferable that researchers and developers use similar approaches to the measurement of antibody and cell-mediated responses 3 , 126 , 132 . The assay of specific antibody titers by quantitative ELISA (including the species-agnostic BRIDGE ELISA) 69 is clearly important and provides a convenient surrogate marker of efficacy in the clinic.

Of equal importance is to assess cell-mediated immunity by ex vivo recall assay on animal tissue or human whole blood samples (by ELIspot or by flow cytometry) 8 , 103 , 136 to determine the establishment of immune memory, and hence the need and spacing of booster doses. The ability also to assay for the induction of functional antibodies has been facilitated by the development of neutralizing monoclonal antibodies, particularly to V, enabling the development of competitive ELISAs 82 , 137 , 138 , 139 , which may be important aids in the down-selection of promising candidates in research. There is also ongoing effort to establish human reference serum for plague to provide an international standard as a reference point for serological surrogate markers of efficacy and thus to enable vaccine development 140 .

The WHO has published a draft target product profile (TPP) for a future plague vaccine, which sets out the qualifying criteria in terms of schedule, administration route, presentation, target efficacies in reactive and preventive modes, stability, and coverage, which would be applied to any plague vaccine candidate 141 .

Future prospects

As highlighted in this review, there are some very promising vaccine candidates in the development pipeline with the potential to prevent plague in vulnerable populations. Here, we have also highlighted the epidemic potential of this disease and of Y. pestis , which in the absence of an approved vaccine, remains a serious biothreat. Seasonal outbreaks in Madagascar and other endemic regions cause fatalities every year. The potential for climate change to enhance this human vulnerability to plague in endemic regions or beyond is also being closely monoitored 142 , 143 . Climate change has already affected other zoonoses by extending the vector species to cause outbreaks of chikungunya and zika viruses in Central and South America 144 , 145 .

Aside from the difficulties of achieving statistically valid evidence of human efficacy, candidate vaccines may also fail because their manufacturing cannot be achieved at scale due to expense or feasibility. Thus, these promising candidate vaccines for plague are vulnerable to languish in the ‘valley of death’ without sustained and sufficient funding for clinical development and manufacturing at scale. Post COVID-19, the WHO has recently commented that ‘despite some recent progress, public heath vaccines are not available in all global regions and vaccines which have been prioritized by the WHO are not being developed or fully invested in, due to limited profit potential.’ 146 The Immunization Agenda 2030 endorsed by the WHO has a goal to reduce the incidence of, or to prevent, epidemics caused by vector-borne diseases by 2030 147 . These goals seem particularly relevant to plague prevention.

To date, there has been a significant investment in time and money in the research and early development of new plague vaccines. It is to be hoped that all the R&D activity on plague vaccines, which emerged after the decline in use of KWC vaccines, will be sustained and together with the regulatory pathways currently being mapped out, will lead to the approval of some new, safe, and fully efficacious vaccines to reduce disease prevalence. We recommend that global funding and health security systems take ambitious action to realize the potential of this investment in an approved vaccine(s) for plague.

Brubaker, B. Factors promoting acute and chronic diseases caused by yersiniae. Clin. Microbiol Rev. 4 , 309–324 (1991).

Article   PubMed   PubMed Central   CAS   Google Scholar  

Pollitzer, R. Plague. World Health Organ Monogr. Ser. 22 , 1–698 (1954).

Google Scholar  

Global consultation on Plague Vaccines, WHO https://www.who.int/news-room/events/detail/2023/10/12/default-calendar/global-consultation-on-plague-vaccines (2023).

Doll, J. M. et al. Cat-transmitted fatal pneumonic plague in a person who traveled from Colorado to Arizona. Am. J. Trop. Med. Hyg. 5 , 109–114 (1994).

Article   Google Scholar  

Perry, R. D. & Fetherston, J. D. Yersinia pestis—etiologic agent of plague. Clin. Microbiol Rev. 10 , 35–66 (1997).

Butler T. Plague and Other Yersinia Infections. (New York, Plenum Press, 1983).

Prevention of plague: Recommendations of the Advisory Committee on Immunization Practices (ACIP). Morb. Mortal Wkly. Rep. 45 , 1–15 (1996).

Williamson, E. D. & Westlake, G. E. Vaccines for emerging pathogens: prospects for licensure. Review series on Vaccines for emerging pathogens: from research to the clinic Part 1. Clin. Exp. Immunol. 198 , 170–183 (2019).

Randremanana, R. et al. Epidemiological characteristics of an urban plague epidemic in Madagascar, August-November, 2017: an outbreak report. Lancet Infect. Dis. 19 , 537–545 (2019).

Article   PubMed   PubMed Central   Google Scholar  

Guiyoule, A. et al. Transferable plasmid-mediated resistance to streptomycin in a clinical isolate of Yersinia pestis. Emerg. Infect. Dis. 7 , 43–48 (2001).

Galimand, M., Carniel, E. & Courvalin, P. Resistance of Y. pestis to antimicrobial agents. Antimicrob. Agents Chemother. 50 , 3233–3236 (2006).

Achtman, M. et al. Yersinia pestis , the cause of plague, is a recently emerged clone of Yersinia pseudotuberculosis . Proc. Natl Acad. Sci. USA 96 , 14043–14048 (1999).

Poinar G. A new genus of fleas with associated microorganisms in Dominican amber. J.Med. Entomol . 126 https://doi.org/10.1093/jme/tjv (2015).

Parkhill, J. et al. Genome sequence of Yersinia pestis , the causative agent of plague. Nature 413 , 523–552 (2001).

Article   PubMed   CAS   Google Scholar  

Cornelis, G. R. et al. The virulence plasmid of Yersinia, an antihost genome. Microbiol. Mol. Biol. Rev. 62 , 1315–1352 (1998).

Cornelis, G. R. et al. The Yersinia yop regulon. Mol. Microbiol 3 , 1455–1459 (1989).

Mikula K. M., Kolodziejczyk R., Goldman A. Yersinia infection tools—characterization of structure and function of adhesins. Frontiers in Cellular and Infection Microbiology 2 , article 16; https://doi.org/10.3389/fcimb.2012.00169 (2013).

Du, Y., Rosqvist, R. & Forsberg, A. Role of fraction 1 antigen of Yersinia pestis in inhibition of phagocytosis. Infect. Immun. 70 , 1453–146 (2002).

Hinnesbusch, B. J., Perry, R. D. & Schwan, T. G. Role of the Yersinia pestis hemin storage (hms) locus in the transmission of plague by fleas. Science 273 , 367–370 (1996).

Hinnebusch, B. J. et al. Role of Yersinia murine toxin in survival of Yersinia pestis in the midgut of the flea vector. Science 296 , 733–735 (2002).

Hinnebusch, B. J. Transmision factors: Yersinia pestis genes required to infect the flea vactor of plague. Adv. Exp. Med. Biol. 529 , 55–62 (2003).

Article   PubMed   Google Scholar  

Darby, C., Ananth, S. L., Tan, L. & Hinnebusch, B. J. Identification of gmhA, a Yersinia pestis gene required for flea blockage, by using a Caenorhabditis elegans biofilm system. Infect. Immun. 73 , 7236–7242 (2005).

Vadyvaloo, V. et al. Transit through the Flea Vector Induces a pretransmission innate immunity resistance phenotype in Yersinia pestis. PLoS Pathog. 6 , e1000783 (2010).

Cavanaugh, D. C. & Randall, R. The role of multiplication of Pasteurella pestis in mononuclear phagocytes in the pathogenesis of flea-borne plague. J. Immunol. 83 , 348–363 (1959).

Lindler, L. E., Klemper, M. S. & Straley, S. C. Yersinia pestis pH 6 antigen: Genetic, biochemical and virulence characterisation of a protein involved in the pathogenesis of bubonic plague. Infect. Immun. 58 , 2569–2577 (1990).

Huang, X.-Z. & Lindler, L. E. The pH6 antigens is an antiphagocytic factor produced by Yersinia pestis independent of Yersinia outer proteins and capsular antigen. Infect. Immun. 72 , 7212–7219 (2004).

Williams, R. C., Gewurz, H. & Quie, P. G. Effects of fraction 1 from Yersinia pestis on phagocytosis in vitro. J. Infect. Dis. 126 , 235–241 (1972).

Sha, J. et al. Characterization of an F1 deletion mutant of Yersinia pestis CO92, pathogenic role of F1 antigen in bubonic and pneumonic plague, and evaluation of sensitivity and specificity of F1 antigen capture-based dipsticks. J. Clin. Microbiol. 49 , 1708–1715 (2011).

Straley, S. C., Skrzypek, E., Plano, G. V. & Bliska, J. B. Yops of Yersinia spp. pathogenic for humans. Infect. Immun. 61 , 3105–3110 (1993).

Mueller, C. A. et al. The V-antigens of Yersinia forms a distinct structure at the tip of injectisome needles. Science 310 , 674–676 (2005).

Nakajima, R. & Brubaker, R. R. Association between virulence of Yersinia pestis and supression of gamma interferon and tumor necrosis factor alpha. Infect. Immun. 61 , 23–31 (1993).

Nakajima, R., Motin, V. L. & Brubaker, R. R. Suppression of cytokines in mice by protein A-V antigen fusion peptide and restoration of synthesis by active immunisation. Infect. Immun. 6 , 3021–3029 (1995).

Skurnik, M., Peippo, A. & Ervela, E. Characterization of the O antigen gene clusters of Yersinia pseudotuberculosis and the cryptic O antigen gene cluster of Yersinia pestis shows that the plague bacillus is most closely related to and has evolved from Y. pseudotuberculosis serotype O:1b. Mol. Microbiol. 37 , 316–330 (2000).

Prior, J. L. et al. The failure of different strains of Yersinia pestis to produce lipopolysaccharide O antigen under different growth conditions is due to mutations in the O-antigen gene cluster. FEMS Microbiol Lett. 197 , 229–233 (2001).

Suomalainen, M. et al. Temperature-induced changes in the lipopolysaccharide of Yersinia pestis affect plasminogen activation by the pla surface protease. Infect. Immun. 78 , 2644–2652 (2010).

Kukkonen, M. et al. Lack of O-antigen is essential for plasminogen activation by Yersinia pestis and Salmonella enterica . Mol. Microbiol. 51 , 215–225 (2004).

Kun, Y. et al. Yersinia pestis Interacts with SIGNR1 (CD209b) for promoting host dissemination and infection. Front. Immunol. 10 , 96 (2019).

van Lier C. J. et al. Further characterization of a highly attenuated Yersinia pestis CO92 mutant deleted for the genes encoding Braun lipoprotein and plasminogen activator protease in murine alveolar and primary human macrophages. Microb. Pathog . 80 , 27–38 https://doi.org/10.1016/j.micpath.2015.02.005 .

Sebbane, F. et al. The Yersinia pestis caf1M1A1 fimbrial capsule operon promotes transmission by flea bite in a mouse model of bubonic plague. Infect. Immun. 77 , 1222–1229 (2009).

Siggins, M. K. & Sriskandan, S. Bacterial Lymphatic Metastasis in Infection and Immunity. Cells 2022 , 33 (2022).

Chaaban, T., Mohsen, Y., Ezzeddine, Z. & Ghssein, G. Overview of Yersinia pestis Metallophores: Yersiniabactin and Yersinopine. Biology 12 , 598 (2023).

Paul-Clark, M. J. et al. Differential effects of Gram-positive versus Gram-negative bacteria on NOSII and TNF in macrophages: role of TLRs in synergy between the two. Brit J. Pharm. 148 , 1067–1075 (2006).

Article   CAS   Google Scholar  

Anon. Fatal Laboratory-Acquired Infection with an Attenuated Yersinia pestis Strain. Morbidity Mortal. Wkly. Rep. (MMWR) 60 , 201–205 (2011).

Hawgood, B. J. & Haffkine, W. M. CIE (1860–1930): Prophylactic vaccination against cholera and bubonic plague in British India. J. Med. Biogr. 1 , 9–19 (2007).

Rosenzweig, J. A. et al. Progress on plague vaccine development. Appl Microbiol Biotechnol. 91 , 265–286 (2011).

Feodorova, V. A. & Motin, V. L. Plague vaccines: current developments and future perspectives. Emerg. Microbes Infect. 1 , e36 (2012).

Rosenzweig, J. A. & Chopra, A. K. The future of plague vaccines: hopes raised by a surrogate, live-attenuated recombinant vaccine candidate. Expert Rev. Vaccines. 11 , 659–661 (2012).

Feodorova V. A. et al. Russian vaccines against especially dangerous bacterial pathogens. Emerg Microbes Infect . 3 https://doi.org/10.1038/emi.2014.82 . (2014).

Feodorova, V. A., Sayapina, L. V. & Motin, V. L. Assessment of live plague vaccine candidates. Methods Mol. Biol. 1403 , 487–498 (2016).

Verma S. K. and Tuteja U. Plague vaccine development: Current research and future trends. Frontiers in Immunology 7 ; https://doi.org/10.3389/fimmu.2016.00602 (2016).

Sun, W. & Singh, A. K. Plague vaccine: recent progress and prospects. NPJ Vaccines. 4 , 11 (2019).

Rosenzweig, J. A., Hendrix, E. K. & Chopra, A. K. Plague vaccines: new developments in an ongoing search. Appl. Microbiol. Biotechnol. 105 , 4931–4941 (2021).

The Australian Immunisation Handbook, 8th edition, NHMRC, Australia. Australian Immunisation Handbook, 8th Edition Part 3: Vaccines Listed by Disease (wordpress.com) (2003).

Sagiyev, Z. et al. Human response to live plague vaccine EV, Almaty region, Kazakhstan, 2014–2015. PloS ONE 14 , e0218366 (2019).

Feodorova, V. A. & Corbel, M. J. Prospects for new plague vaccines. Expert Rev. Vaccines. 8 , 1721–1738 (2009).

Kilgore, P. B. et al. Combinatorial viral vector-based and live attenuated vaccines without an adjuvant to generate broader immune responses to effectively combat pneumonic plague. mBio 12 , e03223–21 (2021).

Quenee, L. E. et al. Hereditary Hemochromatosis Restores the Virulence of Plague Vaccine Strains. J. Infect. Dis. 206 , 1050–1058 (2012).

Bacon, G. A. & Burrows, T. W. The basis of virulence in Pasteurella pestis: an antigen determining virulence. Br. J. Exp. Pathol. 37 , 481–493 (1956).

PubMed   PubMed Central   CAS   Google Scholar  

Williamson, E. D. et al. A new improved sub-unit vaccine for plague: the basis of protection. FEMS Immunol. Med Microbiol. 12 , 223–230 (1995).

Heath, D. G. et al. Protection against experimental bubonic and pneumonic plague by a recombinant capsular F1-V antigen fusion protein vaccine. Vaccine 16 , 1131–1137 (1998).

Williamson, E. D. et al. Kinetics of the immune response to the (F1 + V) vaccine in models of bubonic and pneumonic plague. Vaccine 25 , 1142–1148 (2007).

Quenee, L. E. et al. Plague in Guinea pigs and its prevention by subunit vaccines. Am. J. Pathol. 178 , 1689–1700 (2011).

Demeure, C. E. et al. Yersinia pestis and plague: an updated view on evolution, virulence determinants, immune subversion, vaccination, and diagnostics. Genes Immun. 20 , 357–370 (2019).

Erova T. E. et al. Evaluation of Protective Potential of Yersinia pestis Outer Membrane Protein Antigens as Possible Candidates for a New-Generation Recombinant Plague Vaccine. Clin. Vaccine Immunol. 20 https://doi.org/10.1128/CVI.00597-12 (2013).

Matson, J. S. et al. Immunization of mice with YscF provides protection from Yersinia pestis infections. BMC Microbiol. 5 , 38–55 (2005).

Ivanov M. I. et al.. Vaccination of Mice with a Yop Translocon Complex Elicits Antibodies That Are Protective against Infection with F1−Yersinia pestis. Infect Immun 76 https://doi.org/10.1128/iai.00189-08 (2008).

Swietncki, W., Powell, B. S. & Goodwin, J. Yersinia pestis Yop secretion protein F: purification, characterization and protective efficacy against bubonic plague. Protein Expr. Purif. 42 , 166–172 (2005).

Williamson, E. D. et al. Recombinant (F1 + V) vaccine protects cynomolgus macaques against pneumonic plague. Vaccine 29 , 4771–4777 (2011).

Fellows, P. et al. Characterization of a cynomolgus macaque model of pneumonic plague for evaluation of vaccine efficacy. Clin. Vaccin. Immunol. 22 , 1070–1078 (2015).

Fellows, P. et al. Protection in mice passively immunized with serum from cynomolgus macaques and humans vaccinated with recombinant plague vaccine (rF1V). Vaccine 28 , 7748–7756 (2010).

Quenee, L. E. et al. Prevention of pneumonic plague in mice, rats, guinea pigs and non-human primates with clinical grade rV10, rV10-2 or F1-V vaccines. Vaccine 29 , 6572–6583 (2011).

Aftalion, M. et al. Rapid induction of protective immunity against pneumonic plague by Yersinia pestis Polymeric F1 and LcrV Antigens. Vaccines 11 , 581 (2023).

Jones, T. et al. Intranasal Protollin/F1-V vaccine elicits respiratory and serum antibody responses and protects mice against lethal aerosolized plague infection. Vaccine 6 , 1625–1632 (2006).

Honko, A. N. et al. Flagellin is an effective adjuvant for immunization against lethal respiratory challenge with Yersinia pestis. Infect. Immun. 74 , 1113–1120 (2006).

Mizel, S. B. et al. Flagellin-F1-V fusion protein is an effective plague vaccine in mice and two species of nonhuman primates. Clin. Vaccin. Immunol. 16 , 21–28 (2009).

Wagner, D. A. et al. Single-dose combination nanovaccine induces both rapid and long-lived protection against pneumonic plague. Acta Biomater. 100 , 326–337 (2019).

Yang, X. O. et al. Manganese-based nanoparticle vaccine for combating fatal bacterial pneumonia. Adv. Mater. 35 , 2304514 (2023).

Dentovskaya, S. V. et al. Peptidoglycan-Free Bacterial Ghosts Confer Enhanced Protection against Yersinia pestis Infection. Vaccines 10 , 51 (2022).

Anderson, G. W. J. et al. Recombinant V antigen protects mice against pneumonic and bubonic plague caused by F1-capsule-positive and -negative strains of Yersinia pestis. Infect. Immun. 64 , 4580–4585 (1996).

Carvalho, A. et al. Use of Bioengineered Human Commensal Gut Bacteria-Derived Microvesicles for Mucosal Plague Vaccine Delivery and Immunization. Clin. Exp. Immunol. 196 , 287–304 (2019).

Byvalov, A. A., Konyshev, I. V., Uversky, V. N., Dentovskaya, S. V. & Anisimov, A. P. Yersinia Outer Membrane Vesicles as Potential Vaccine Candidates in Protecting against Plague. Biomolecules 10 , 1694 (2020).

Moore, B. D. et al. Predictors of survival after vaccination in a pneumonic plague model. Vaccines 10 , 145 (2022).

Levy, Y. et al. Targeting of the Yersinia pestis F1 capsular antigen by innate-like B1b cells mediates a rapid protective response against bubonic plague. NPJ Vaccines 3 , 52 (2018).

Goncharova, A. et al. Assessment of the effect of using immunomodulatory drugs for emergency prevention of experimental plague caused by a virulent strain of the main subspecies Yersinia pestis . J. Microbiol., Epidemiol. Immunobiol. 101 , 2024 (2024).

Galloway, D. R. et al. Co-formulation of the rF1V plague vaccine with depot-formulated cytokines enhances immunogenicity and efficacy to elicit protective responses against aerosol challenge in mice. Front. Immunol. 15 , 1277526 (2024).

Marketing authorisation from the State Research Centre for Applied Microbiology amnd Biotechnology of the Federal Service for the Supervision of Consumer Rights and Protection (Russia); Registration LP-004808 Microencapsulated molecular plague vaccine (PMMM, 2024) https://www.rlsnet.ru/regdoc/vakcina-cumnaya-molekulyarnaya-mikroinkapsulirovannaya-vcmm-lp-004808-74621 .

Goncharova, A. et al. Experimental Evaluation of Application of the Vaccine Strain Yersinia pestis EV NIIEG in Combination with Immune-Modulators. Problems of Particularly Dangerous Infections . 71-77. https://doi.org/10.21055/0370-1069-2020-2-71-77 . (2020).

Tiner, B. L. et al. Immunisation of two rodent species with new live-attenuated mutants of Yersinia pestis CO92 induces protective long-term humoral- and cell-mediated immunity against pneumonic plague. NPJ Vaccines. 1 , 16020 (2016).

Tiner, B. L. et al. Combinational deletion of three membrane protein-encoding genes highly attenuates yersinia pestis while retaining immunogenicity in a mouse model of pneumonic plague. Infect. Immun. 83 , 1318–1338 (2015A).

Tiner, B. L. et al. Intramuscular Immunization of Mice with a Live-Attenuated Triple Mutant of Yersinia pestis CO92 Induces Robust Humoral and Cell-Mediated Immunity To Completely Protect Animals against Pneumonic Plague. Clin. Vaccin. Immunol. 22 , 1255–1268 (2015B).

Van Lier, C. J. et al. Deletion of Braun lipoprotein and plasminogen-activating protease-encoding genes attenuates Yersinia pestis in mouse models of bubonic and pneumonic plague. Infect. Immun. 82 , 2485–2503 (2014).

Junxia F. et al. Construction of a Live-Attenuated Vaccine Strain of Yersinia pestis EV76-B-SHUΔpla and Evaluation of Its Protection Efficacy in a Mouse Model by Aerosolized Intratracheal Inoculation. Cell. Infect. Microbiol ., 10 https://doi.org/10.3389/fcimb.2020.00473 (2020).

Cote, C. K. et al. Protection Elicited by Attenuated Live Yersinia pestis Vaccine Strains against Lethal Infection with Virulent Y. pestis . Vaccines 9 , 161 (2021).

WHO R&D blueprint: Plague vaccines workshop https://www.who.int/blueprint/what/norms-standards/Plague_vaccines_workshop-23-april-2018/en/ (2018).

Demeure, C. et al. Yersinia pestis and plague: an updated view on evolution, virulence determinants, immune subversion, vaccination and diagnostics. Microbes Infect. 21 , 202–212 (2019).

Dentovskaya, S. V. et al. Molecular bases of vaccine-prevention of plague. Mol. Genet. Microbiol. Virol. 28 , 87–98 (2013).

Andersson, J. A. et al. Identification of New Virulence Factors and Vaccine Candidates for Yersinia pestis . Front Cell Infect. Microbiol. 7 , 448 (2017).

Wang, X., Zhang, X., Zhou, D. & Yang, R. Live-attenuated Yersinia pestis vaccines. Expert Rev. Vaccines. 12 , 677–686 (2013).

Humphreys, I. R. & Sebastian, S. Novel viral vectors in infectious diseases. Immunology 153 , 1–9 (2018).

Wang, S. et al. Viral vectored vaccines: design, development, preventive and therapeutic applications in human diseases. Sig Transduct. Target Ther. 8 , 149 (2023).

Derbise, A. et al. An encapsulated Yersinia pseudotuberculosis is a highly efficient vaccine against pneumonic plague. PLoS Negl. Trop. Dis. 6 , e1528 (2012).

Derbise A., et al. Complete Protection against Pneumonic and Bubonic Plague after a Single Oral Vaccination. PLoS Negl Trop Dis . 16 https://doi.org/10.1371/journal.pntd.0004162 (2015).

Demeure, C. E. et al. Humoral and cellular immune correlates of protection against bubonic plague by a live Yersinia pseudotuberculosis vaccine. Vaccine 3 , 123–129 (2019).

Derbise, A. et al. Subcutaneous vaccination with a live attenuated Yersinia pseudotuberculosis plague vaccine. Vaccine 38 , 1888–1892 (2020).

Wang, X. et al. Remodeling Yersinia pseudotuberculosis to generate a highly immunogenic outer membrane vesicle vaccine against pneumonic plague. PNAS 119 , e2109667119 (2022).

Morton, M. et al. A Salmonella enterica serovar Typhi vaccine expressing Yersinia pestis F1 antigen on its surface provides protection against plague in mice. Vaccine 22 , 2524–2532 (2004).

Garmory, H. S. et al. Antibiotic-free plasmid stabilization by operator-repressor titration for vaccine delivery by using live Salmonella enterica Serovar typhimurium. Infect. Immun. 73 , 2005–2011 (2005).

Leckenby, M. W. et al. Enhanced vaccine antigen delivery by Salmonella with antibiotic free Operator Repressor Titration-based plasmid stabilisation compared to chromosomal integration. Microb. Pathog. 46 , 201–206 (2009).

Sanapala, S. et al. Multiple antigens of Yersinia pestis delivered by live recombinant attenuated Salmonella vaccine strains elicit protective immunity against plague. Vaccine 5 , 2410–2416 (2016).

Sha, J. et al. The two murine lipoproteins of Salmonella enterica serovar Typhimurium contribute to the virulence of the organism. Infect. Immun. 72 , 3987–4003 (2004).

Qingmei, J. et al. Single vector platform vaccine protects against lethal respiratory challenge with Tier 1 select agents of anthrax, plague, and tularemia. Sci. Rep. 8 , 7009 (2018).

Tao, P. et al. A Bacteriophage T4 Nanoparticle-Based Dual Vaccine against Anthrax and Plague. mBio 9 , e01926–18 (2018).

Kilgore, P. B. et al. A new generation needle- and adjuvant-free trivalent plague vaccine utilizing adenovirus-5 nanoparticle platform. Npj vaccines 6 , 21 (2021).

Sha, J. et al. A Replication-Defective Human type 5 adenovirus-based trivalent vaccine confers complete protection against plague in mice and nonhuman primates. Clin. Vaccin. Immunol. 23 , 586–600 (2016).

Chiuchiolo, M. J. et al. Protective immunity against respiratory tract challenge with Yersinia pestis in mice immunized with an adenovirus-based vaccine vector expressing V antigen. J. Infect. Dis. 194 , 1249–1257 (2006).

Boyer, J. L. et al. Protective immunity against a lethal respiratory Yersinia pestis challenge induced by V antigen or the F1 capsular antigen incorporated into adenovirus capsid. Hum. Gene Ther. 21 , 891–901 (2010).

Folegatti, P. M. et al. Benefit-Risk Assessment of Vaccines by Technology Working Group BRAVATO, ex-V3SWG). Vaccines based on the replication-deficient simian adenoviral vector ChAdOx1: Standardized template with key considerations fora risk/benefit assessment. Vaccine 40 , 5248–5262 (2022).

WHO Landscape of plague vaccine candidates (2023).

Albrecht, M. T. et al. Immunogenicity and efficacy of an anthrax/plague DNA fusion vaccine in a mouse model. FEMS Immunol. Med. Microbiol. 65 , 505–509 (2012).

Kon E. et al. A single-dose F1-based mRNA-LNP vaccine provides protection against the lethal plague bacterium. Sci Adv . 9 https://doi.org/10.1126/sciadv.adg1036 (2023).

Shattock, R. J. et al. A self-amplifying RNA vaccine provides protection in a murine model of bubonic plague. Front. Microbiol. 14 , 1247041 (2023).

Zhang, G. et al. mRNA vaccines in disease prevention and treatment. Sig Transduct. Target Ther. 8 , 365 (2023).

Lee, G. H. & Lim, S. G. CpG-Adjuvanted Hepatitis B Vaccine (HEPLISAV-B®) Update. Expert Rev. Vaccines. 20 , 487–495 (2021).

Hartley, L., Harold, S. & Hawe, E. The efficacy, safety and immunogenicity of plague vaccines: a systemic literature review. Curr. Res. Immunol. 4 , 100072 (2023).

21CFR 601.90-95. Evidence needed to demonstrate effectiveness of new drugs when human efficacy studies are not ethical or feasible Fed. Register 67 , 37988-37998 (2002)

Williamson, E. D. et al. Recent advances in predictive models and correlates of protection in testing biodefence vaccines. Expert Rev. Vaccines 9 , 527–537 (2010).

European Medicines Agency. https://www.ema.europa.eu/en/human-regulatory-overview/public-health-threats/ebola (2024).

Mogil, J. S., Philip, V. M., Tuttle, A. H. & Chesler, E. J. Comparing phenotypic variation between inbred and outbred mice. Nat. Methods 15 , 994–996 (2018).

Majumder, S. et al. Protection Induced by Oral Vaccination with a Recombinant Yersinia pseudotuberculosis Delivering Yersinia pestis LcrV and F1 Antigens in Mice and Rats against Pneumonic Plague. Infect. Immu. 90 , e0016522 (2022).

Chichester, J. A. et al. A single component two-valent LcrV-F1 vaccine protects non-human primates against pneumonic plague. Vaccine 27 , 3471–3474 (2009).

Smiley, S. T. Immune defense against pneumonic plague. Immunol. Rev. 22 , 256–271 (2008).

Williamson, E. D. & Oyston, P. C. F. in Plotkins Vaccines (8th edition) (Ed. Orenstein, W. A., Offit, P. A., Edwards, K. M. & Plotkin, S. A.) Chapter 46 (Elsevier, 2023).

Parent, M. A. et al. Gamma interferon, tumor necrosis factor alpha, and nitric oxide synthase 2, key elements of cellular immunity, perform critical protective functions during humoral defense against lethal pulmonary Yersinia pestis infection. Infect. Immun. 74 , 3381–3386 (2006).

Elvin, S. J. & Williamson, E. D. Stat 4 but not Stat 6 mediated immune mechanisms are essential in protection against plague. Micro. Pathog. 3 , 177–184 (2004).

Levy, Y. et al. T cells play an essential role in anti-F1 mediated rapid protection against bubonic plague. Vaccine 29 , 6866–6873 (2011).

Hamzabegovic, F. et al. Flagellin adjuvanted F1/V subunit plague vaccine induces T cell and functional antibody responses with unique gene signatures. NPJ Vaccines. 5 , 6 (2020).

Amemiya, K. et al. Binding sites of anti-LcrV monoclonal antibodies are more critical than the avidities and affinities for passive protection against Yersinia pestis infection in a bubonic plague model. Antibodies 9 , 37 (2020).

Andrianaivoarimanana, V. et al. Potential human immunotherapeutics for plague. Immunother. Adv. 1 , 1–8 (2021).

Biryukov, S. S. et al. Functional assays to screen and select monoclonal antibodies that target Yersinia pestis . Hum. Vaccines Immunotherapeutics 19 , 2 (2023).

International Standard Plague antiserum[Almond, Rajerison, Williamson, personal communication].

Raveros X. Target Product Profiles Plague vaccines WHO https://cdn.who.int/media/docs/default-source/documents/r-d-blueprint-meetings/global-consultation-on-plague-vaccines/3_ximena-riveros_plague-tpp.pdf?sfvrsn=edbe8135_3 (2023).

Rocke, T. E. et al. Effects of climate change on plague exposure pathways and resulting disease dynamics. DoD. Final Rep. 16 , RC01–RC012 (2020).

Xu, L. et al. Climate-driven marmot-plague dynamics in Mongolia and China. Sci. Rep. 13 , 11906 (2023).

Leal Filho, W. et al. A Review of Concepts, Definitions, and Bibliometrics. Int J. Environ. Res. Public Health 19 , 893 (2022).

Bartlow, A. W. et al. Forecasting Zoonotic Infectious Disease Response to Climate Change: Mosquito Vectors and a Changing Environment. Vet. Sci. 6 , 40 (2019).

PubMed   PubMed Central   Google Scholar  

WHO releases first data on global vaccine market since COVID-19 https://www.who.int/news/item/09-11-2022-who-releases-first-data-on-global-vaccine-market-since-covid-19 .

WHO The Immunisation Agenda 2030 https://www.who.int/publications/i/item/9789240062726 .

Download references

Acknowledgements

E.D.W. would like to thank Prof. Tim Atkins, Prof. Riccardo d’Elia and Dr. Tom Laws for their review of the draft. Funding through the NIH (AI064389, AI153524 and AI071634) awarded to A.K.C. is gratefully acknowledged. Funding for PBK through an NIH T32 AI179595 postdoctoral fellowship in the Antimicrobial Resistance Training Program of the Texas Medical Center (AMR-TPT) is also acknowledged.

Author information

Authors and affiliations.

Defence Science and Technology Laboratory, Porton Down, Salisbury, SP4 0JQ, UK

E. Diane Williamson

Department of Microbiology and Immunology, UTMB, Galveston, TX, 77555, USA

Paul B. Kilgore, Emily K. Hendrix, Blake H. Neil, Jian Sha & Ashok K. Chopra

Sealy Institute for Vaccine Sciences, UTMB, Galveston, TX, 77555, USA

Ashok K. Chopra

Institute for Human Infections and Immunity, UTMB, Galveston, TX, 77555, USA

Center for Biodefense and Emerging Infectious Diseases, UTMB, Galveston, TX, 77555, USA

Galveston National Laboratory, UTMB, Galveston, TX, 77555, USA

You can also search for this author in PubMed   Google Scholar

Contributions

E.D.W. and A.K.C contributed equally to the preparation of the manuscript. All authors have read and approved the manuscript. P.B.K., E.K.H., B.H.N., and J.S. performed literature search, prepared tables and Fig. 2 and edited the review.

Corresponding authors

Correspondence to E. Diane Williamson or Ashok K. Chopra .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Williamson, E.D., Kilgore, P.B., Hendrix, E.K. et al. Progress on the research and development of plague vaccines with a call to action. npj Vaccines 9 , 162 (2024). https://doi.org/10.1038/s41541-024-00958-1

Download citation

Received : 23 May 2024

Accepted : 21 August 2024

Published : 07 September 2024

DOI : https://doi.org/10.1038/s41541-024-00958-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Peer Reviewed

GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation

Article metrics.

CrossRef Citations

Altmetric Score

PDF Downloads

Academic journals, archives, and repositories are seeing an increasing number of questionable research papers clearly produced using generative AI. They are often created with widely available, general-purpose AI applications, most likely ChatGPT, and mimic scientific writing. Google Scholar easily locates and lists these questionable papers alongside reputable, quality-controlled research. Our analysis of a selection of questionable GPT-fabricated scientific papers found in Google Scholar shows that many are about applied, often controversial topics susceptible to disinformation: the environment, health, and computing. The resulting enhanced potential for malicious manipulation of society’s evidence base, particularly in politically divisive domains, is a growing concern.

Swedish School of Library and Information Science, University of Borås, Sweden

Department of Arts and Cultural Sciences, Lund University, Sweden

Division of Environmental Communication, Swedish University of Agricultural Sciences, Sweden

Research Questions

• Where are questionable publications produced with generative pre-trained transformers (GPTs) that can be found via Google Scholar published or deposited?
• What are the main characteristics of these publications in relation to predominant subject categories?
• How are these publications spread in the research infrastructure for scholarly communication?
• How is the role of the scholarly communication infrastructure challenged in maintaining public trust in science and evidence through inappropriate use of generative AI?

research note Summary

• A sample of scientific papers with signs of GPT-use found on Google Scholar was retrieved, downloaded, and analyzed using a combination of qualitative coding and descriptive statistics. All papers contained at least one of two common phrases returned by conversational agents that use large language models (LLM) like OpenAI’s ChatGPT. Google Search was then used to determine the extent to which copies of questionable, GPT-fabricated papers were available in various repositories, archives, citation databases, and social media platforms.
• Roughly two-thirds of the retrieved papers were found to have been produced, at least in part, through undisclosed, potentially deceptive use of GPT. The majority (57%) of these questionable papers dealt with policy-relevant subjects (i.e., environment, health, computing), susceptible to influence operations. Most were available in several copies on different domains (e.g., social media, archives, and repositories).
• Two main risks arise from the increasingly common use of GPT to (mass-)produce fake, scientific publications. First, the abundance of fabricated “studies” seeping into all areas of the research infrastructure threatens to overwhelm the scholarly communication system and jeopardize the integrity of the scientific record. A second risk lies in the increased possibility that convincingly scientific-looking content was in fact deceitfully created with AI tools and is also optimized to be retrieved by publicly available academic search engines, particularly Google Scholar. However small, this possibility and awareness of it risks undermining the basis for trust in scientific knowledge and poses serious societal risks.

Implications

The use of ChatGPT to generate text for academic papers has raised concerns about research integrity. Discussion of this phenomenon is ongoing in editorials, commentaries, opinion pieces, and on social media (Bom, 2023; Stokel-Walker, 2024; Thorp, 2023). There are now several lists of papers suspected of GPT misuse, and new papers are constantly being added. 1 See for example Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . While many legitimate uses of GPT for research and academic writing exist (Huang & Tan, 2023; Kitamura, 2023; Lund et al., 2023), its undeclared use—beyond proofreading—has potentially far-reaching implications for both science and society, but especially for their relationship. It, therefore, seems important to extend the discussion to one of the most accessible and well-known intermediaries between science, but also certain types of misinformation, and the public, namely Google Scholar, also in response to the legitimate concerns that the discussion of generative AI and misinformation needs to be more nuanced and empirically substantiated  (Simon et al., 2023).

Google Scholar, https://scholar.google.com , is an easy-to-use academic search engine. It is available for free, and its index is extensive (Gusenbauer & Haddaway, 2020). It is also often touted as a credible source for academic literature and even recommended in library guides, by media and information literacy initiatives, and fact checkers (Tripodi et al., 2023). However, Google Scholar lacks the transparency and adherence to standards that usually characterize citation databases. Instead, Google Scholar uses automated crawlers, like Google’s web search engine (Martín-Martín et al., 2021), and the inclusion criteria are based on primarily technical standards, allowing any individual author—with or without scientific affiliation—to upload papers to be indexed (Google Scholar Help, n.d.). It has been shown that Google Scholar is susceptible to manipulation through citation exploits (Antkare, 2020) and by providing access to fake scientific papers (Dadkhah et al., 2017). A large part of Google Scholar’s index consists of publications from established scientific journals or other forms of quality-controlled, scholarly literature. However, the index also contains a large amount of gray literature, including student papers, working papers, reports, preprint servers, and academic networking sites, as well as material from so-called “questionable” academic journals, including paper mills. The search interface does not offer the possibility to filter the results meaningfully by material type, publication status, or form of quality control, such as limiting the search to peer-reviewed material.

To understand the occurrence of ChatGPT (co-)authored work in Google Scholar’s index, we scraped it for publications, including one of two common ChatGPT responses (see Appendix A) that we encountered on social media and in media reports (DeGeurin, 2024). The results of our descriptive statistical analyses showed that around 62% did not declare the use of GPTs. Most of these GPT-fabricated papers were found in non-indexed journals and working papers, but some cases included research published in mainstream scientific journals and conference proceedings. 2 Indexed journals mean scholarly journals indexed by abstract and citation databases such as Scopus and Web of Science, where the indexation implies journals with high scientific quality. Non-indexed journals are journals that fall outside of this indexation. More than half (57%) of these GPT-fabricated papers concerned policy-relevant subject areas susceptible to influence operations. To avoid increasing the visibility of these publications, we abstained from referencing them in this research note. However, we have made the data available in the Harvard Dataverse repository.

The publications were related to three issue areas—health (14.5%), environment (19.5%) and computing (23%)—with key terms such “healthcare,” “COVID-19,” or “infection”for health-related papers, and “analysis,” “sustainable,” and “global” for environment-related papers. In several cases, the papers had titles that strung together general keywords and buzzwords, thus alluding to very broad and current research. These terms included “biology,” “telehealth,” “climate policy,” “diversity,” and “disrupting,” to name just a few.  While the study’s scope and design did not include a detailed analysis of which parts of the articles included fabricated text, our dataset did contain the surrounding sentences for each occurrence of the suspicious phrases that formed the basis for our search and subsequent selection. Based on that, we can say that the phrases occurred in most sections typically found in scientific publications, including the literature review, methods, conceptual and theoretical frameworks, background, motivation or societal relevance, and even discussion. This was confirmed during the joint coding, where we read and discussed all articles. It became clear that not just the text related to the telltale phrases was created by GPT, but that almost all articles in our sample of questionable articles likely contained traces of GPT-fabricated text everywhere.

Evidence hacking and backfiring effects

Generative pre-trained transformers (GPTs) can be used to produce texts that mimic scientific writing. These texts, when made available online—as we demonstrate—leak into the databases of academic search engines and other parts of the research infrastructure for scholarly communication. This development exacerbates problems that were already present with less sophisticated text generators (Antkare, 2020; Cabanac & Labbé, 2021). Yet, the public release of ChatGPT in 2022, together with the way Google Scholar works, has increased the likelihood of lay people (e.g., media, politicians, patients, students) coming across questionable (or even entirely GPT-fabricated) papers and other problematic research findings. Previous research has emphasized that the ability to determine the value and status of scientific publications for lay people is at stake when misleading articles are passed off as reputable (Haider & Åström, 2017) and that systematic literature reviews risk being compromised (Dadkhah et al., 2017). It has also been highlighted that Google Scholar, in particular, can be and has been exploited for manipulating the evidence base for politically charged issues and to fuel conspiracy narratives (Tripodi et al., 2023). Both concerns are likely to be magnified in the future, increasing the risk of what we suggest calling evidence hacking —the strategic and coordinated malicious manipulation of society’s evidence base.

The authority of quality-controlled research as evidence to support legislation, policy, politics, and other forms of decision-making is undermined by the presence of undeclared GPT-fabricated content in publications professing to be scientific. Due to the large number of archives, repositories, mirror sites, and shadow libraries to which they spread, there is a clear risk that GPT-fabricated, questionable papers will reach audiences even after a possible retraction. There are considerable technical difficulties involved in identifying and tracing computer-fabricated papers (Cabanac & Labbé, 2021; Dadkhah et al., 2023; Jones, 2024), not to mention preventing and curbing their spread and uptake.

However, as the rise of the so-called anti-vaxx movement during the COVID-19 pandemic and the ongoing obstruction and denial of climate change show, retracting erroneous publications often fuels conspiracies and increases the following of these movements rather than stopping them. To illustrate this mechanism, climate deniers frequently question established scientific consensus by pointing to other, supposedly scientific, studies that support their claims. Usually, these are poorly executed, not peer-reviewed, based on obsolete data, or even fraudulent (Dunlap & Brulle, 2020). A similar strategy is successful in the alternative epistemic world of the global anti-vaccination movement (Carrion, 2018) and the persistence of flawed and questionable publications in the scientific record already poses significant problems for health research, policy, and lawmakers, and thus for society as a whole (Littell et al., 2024). Considering that a person’s support for “doing your own research” is associated with increased mistrust in scientific institutions (Chinn & Hasell, 2023), it will be of utmost importance to anticipate and consider such backfiring effects already when designing a technical solution, when suggesting industry or legal regulation, and in the planning of educational measures.

Recommendations

Solutions should be based on simultaneous considerations of technical, educational, and regulatory approaches, as well as incentives, including social ones, across the entire research infrastructure. Paying attention to how these approaches and incentives relate to each other can help identify points and mechanisms for disruption. Recognizing fraudulent academic papers must happen alongside understanding how they reach their audiences and what reasons there might be for some of these papers successfully “sticking around.” A possible way to mitigate some of the risks associated with GPT-fabricated scholarly texts finding their way into academic search engine results would be to provide filtering options for facets such as indexed journals, gray literature, peer-review, and similar on the interface of publicly available academic search engines. Furthermore, evaluation tools for indexed journals 3 Such as LiU Journal CheckUp, https://ep.liu.se/JournalCheckup/default.aspx?lang=eng . could be integrated into the graphical user interfaces and the crawlers of these academic search engines. To enable accountability, it is important that the index (database) of such a search engine is populated according to criteria that are transparent, open to scrutiny, and appropriate to the workings of  science and other forms of academic research. Moreover, considering that Google Scholar has no real competitor, there is a strong case for establishing a freely accessible, non-specialized academic search engine that is not run for commercial reasons but for reasons of public interest. Such measures, together with educational initiatives aimed particularly at policymakers, science communicators, journalists, and other media workers, will be crucial to reducing the possibilities for and effects of malicious manipulation or evidence hacking. It is important not to present this as a technical problem that exists only because of AI text generators but to relate it to the wider concerns in which it is embedded. These range from a largely dysfunctional scholarly publishing system (Haider & Åström, 2017) and academia’s “publish or perish” paradigm to Google’s near-monopoly and ideological battles over the control of information and ultimately knowledge. Any intervention is likely to have systemic effects; these effects need to be considered and assessed in advance and, ideally, followed up on.

Our study focused on a selection of papers that were easily recognizable as fraudulent. We used this relatively small sample as a magnifying glass to examine, delineate, and understand a problem that goes beyond the scope of the sample itself, which however points towards larger concerns that require further investigation. The work of ongoing whistleblowing initiatives 4 Such as Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . , recent media reports of journal closures (Subbaraman, 2024), or GPT-related changes in word use and writing style (Cabanac et al., 2021; Stokel-Walker, 2024) suggest that we only see the tip of the iceberg. There are already more sophisticated cases (Dadkhah et al., 2023) as well as cases involving fabricated images (Gu et al., 2022). Our analysis shows that questionable and potentially manipulative GPT-fabricated papers permeate the research infrastructure and are likely to become a widespread phenomenon. Our findings underline that the risk of fake scientific papers being used to maliciously manipulate evidence (see Dadkhah et al., 2017) must be taken seriously. Manipulation may involve undeclared automatic summaries of texts, inclusion in literature reviews, explicit scientific claims, or the concealment of errors in studies so that they are difficult to detect in peer review. However, the mere possibility of these things happening is a significant risk in its own right that can be strategically exploited and will have ramifications for trust in and perception of science. Society’s methods of evaluating sources and the foundations of media and information literacy are under threat and public trust in science is at risk of further erosion, with far-reaching consequences for society in dealing with information disorders. To address this multifaceted problem, we first need to understand why it exists and proliferates.

Finding 1: 139 GPT-fabricated, questionable papers were found and listed as regular results on the Google Scholar results page. Non-indexed journals dominate.

Most questionable papers we found were in non-indexed journals or were working papers, but we did also find some in established journals, publications, conferences, and repositories. We found a total of 139 papers with a suspected deceptive use of ChatGPT or similar LLM applications (see Table 1). Out of these, 19 were in indexed journals, 89 were in non-indexed journals, 19 were student papers found in university databases, and 12 were working papers (mostly in preprint databases). Table 1 divides these papers into categories. Health and environment papers made up around 34% (47) of the sample. Of these, 66% were present in non-indexed journals.

Finding 2: GPT-fabricated, questionable papers are disseminated online, permeating the research infrastructure for scholarly communication, often in multiple copies. Applied topics with practical implications dominate.

The 20 papers concerning health-related issues are distributed across 20 unique domains, accounting for 46 URLs. The 27 papers dealing with environmental issues can be found across 26 unique domains, accounting for 56 URLs.  Most of the identified papers exist in multiple copies and have already spread to several archives, repositories, and social media. It would be difficult, or impossible, to remove them from the scientific record.

As apparent from Table 2, GPT-fabricated, questionable papers are seeping into most parts of the online research infrastructure for scholarly communication. Platforms on which identified papers have appeared include ResearchGate, ORCiD, Journal of Population Therapeutics and Clinical Pharmacology (JPTCP), Easychair, Frontiers, the Institute of Electrical and Electronics Engineer (IEEE), and X/Twitter. Thus, even if they are retracted from their original source, it will prove very difficult to track, remove, or even just mark them up on other platforms. Moreover, unless regulated, Google Scholar will enable their continued and most likely unlabeled discoverability.

A word rain visualization (Centre for Digital Humanities Uppsala, 2023), which combines word prominences through TF-IDF 5 Term frequency–inverse document frequency , a method for measuring the significance of a word in a document compared to its frequency across all documents in a collection. scores with semantic similarity of the full texts of our sample of GPT-generated articles that fall into the “Environment” and “Health” categories, reflects the two categories in question. However, as can be seen in Figure 1, it also reveals overlap and sub-areas. The y-axis shows word prominences through word positions and font sizes, while the x-axis indicates semantic similarity. In addition to a certain amount of overlap, this reveals sub-areas, which are best described as two distinct events within the word rain. The event on the left bundles terms related to the development and management of health and healthcare with “challenges,” “impact,” and “potential of artificial intelligence”emerging as semantically related terms. Terms related to research infrastructures, environmental, epistemic, and technological concepts are arranged further down in the same event (e.g., “system,” “climate,” “understanding,” “knowledge,” “learning,” “education,” “sustainable”). A second distinct event further to the right bundles terms associated with fish farming and aquatic medicinal plants, highlighting the presence of an aquaculture cluster.  Here, the prominence of groups of terms such as “used,” “model,” “-based,” and “traditional” suggests the presence of applied research on these topics. The two events making up the word rain visualization, are linked by a less dominant but overlapping cluster of terms related to “energy” and “water.”

The bar chart of the terms in the paper subset (see Figure 2) complements the word rain visualization by depicting the most prominent terms in the full texts along the y-axis. Here, word prominences across health and environment papers are arranged descendingly, where values outside parentheses are TF-IDF values (relative frequencies) and values inside parentheses are raw term frequencies (absolute frequencies).

Finding 3: Google Scholar presents results from quality-controlled and non-controlled citation databases on the same interface, providing unfiltered access to GPT-fabricated questionable papers.

Google Scholar’s central position in the publicly accessible scholarly communication infrastructure, as well as its lack of standards, transparency, and accountability in terms of inclusion criteria, has potentially serious implications for public trust in science. This is likely to exacerbate the already-known potential to exploit Google Scholar for evidence hacking (Tripodi et al., 2023) and will have implications for any attempts to retract or remove fraudulent papers from their original publication venues. Any solution must consider the entirety of the research infrastructure for scholarly communication and the interplay of different actors, interests, and incentives.

We searched and scraped Google Scholar using the Python library Scholarly (Cholewiak et al., 2023) for papers that included specific phrases known to be common responses from ChatGPT and similar applications with the same underlying model (GPT3.5 or GPT4): “as of my last knowledge update” and/or “I don’t have access to real-time data” (see Appendix A). This facilitated the identification of papers that likely used generative AI to produce text, resulting in 227 retrieved papers. The papers’ bibliographic information was automatically added to a spreadsheet and downloaded into Zotero. 6 An open-source reference manager, https://zotero.org .

We employed multiple coding (Barbour, 2001) to classify the papers based on their content. First, we jointly assessed whether the paper was suspected of fraudulent use of ChatGPT (or similar) based on how the text was integrated into the papers and whether the paper was presented as original research output or the AI tool’s role was acknowledged. Second, in analyzing the content of the papers, we continued the multiple coding by classifying the fraudulent papers into four categories identified during an initial round of analysis—health, environment, computing, and others—and then determining which subjects were most affected by this issue (see Table 1). Out of the 227 retrieved papers, 88 papers were written with legitimate and/or declared use of GPTs (i.e., false positives, which were excluded from further analysis), and 139 papers were written with undeclared and/or fraudulent use (i.e., true positives, which were included in further analysis). The multiple coding was conducted jointly by all authors of the present article, who collaboratively coded and cross-checked each other’s interpretation of the data simultaneously in a shared spreadsheet file. This was done to single out coding discrepancies and settle coding disagreements, which in turn ensured methodological thoroughness and analytical consensus (see Barbour, 2001). Redoing the category coding later based on our established coding schedule, we achieved an intercoder reliability (Cohen’s kappa) of 0.806 after eradicating obvious differences.

The ranking algorithm of Google Scholar prioritizes highly cited and older publications (Martín-Martín et al., 2016). Therefore, the position of the articles on the search engine results pages was not particularly informative, considering the relatively small number of results in combination with the recency of the publications. Only the query “as of my last knowledge update” had more than two search engine result pages. On those, questionable articles with undeclared use of GPTs were evenly distributed across all result pages (min: 4, max: 9, mode: 8), with the proportion of undeclared use being slightly higher on average on later search result pages.

To understand how the papers making fraudulent use of generative AI were disseminated online, we programmatically searched for the paper titles (with exact string matching) in Google Search from our local IP address (see Appendix B) using the googlesearch – python library(Vikramaditya, 2020). We manually verified each search result to filter out false positives—results that were not related to the paper—and then compiled the most prominent URLs by field. This enabled the identification of other platforms through which the papers had been spread. We did not, however, investigate whether copies had spread into SciHub or other shadow libraries, or if they were referenced in Wikipedia.

We used descriptive statistics to count the prevalence of the number of GPT-fabricated papers across topics and venues and top domains by subject. The pandas software library for the Python programming language (The pandas development team, 2024) was used for this part of the analysis. Based on the multiple coding, paper occurrences were counted in relation to their categories, divided into indexed journals, non-indexed journals, student papers, and working papers. The schemes, subdomains, and subdirectories of the URL strings were filtered out while top-level domains and second-level domains were kept, which led to normalizing domain names. This, in turn, allowed the counting of domain frequencies in the environment and health categories. To distinguish word prominences and meanings in the environment and health-related GPT-fabricated questionable papers, a semantically-aware word cloud visualization was produced through the use of a word rain (Centre for Digital Humanities Uppsala, 2023) for full-text versions of the papers. Font size and y-axis positions indicate word prominences through TF-IDF scores for the environment and health papers (also visualized in a separate bar chart with raw term frequencies in parentheses), and words are positioned along the x-axis to reflect semantic similarity (Skeppstedt et al., 2024), with an English Word2vec skip gram model space (Fares et al., 2017). An English stop word list was used, along with a manually produced list including terms such as “https,” “volume,” or “years.”

• Artificial Intelligence
• / Search engines

Cite this Essay

Haider, J., Söderström, K. R., Ekström, B., & Rödl, M. (2024). GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation. Harvard Kennedy School (HKS) Misinformation Review . https://doi.org/10.37016/mr-2020-156

• / Appendix B

Bibliography

Antkare, I. (2020). Ike Antkare, his publications, and those of his disciples. In M. Biagioli & A. Lippman (Eds.), Gaming the metrics (pp. 177–200). The MIT Press. https://doi.org/10.7551/mitpress/11087.003.0018

Barbour, R. S. (2001). Checklists for improving rigour in qualitative research: A case of the tail wagging the dog? BMJ , 322 (7294), 1115–1117. https://doi.org/10.1136/bmj.322.7294.1115

Bom, H.-S. H. (2023). Exploring the opportunities and challenges of ChatGPT in academic writing: A roundtable discussion. Nuclear Medicine and Molecular Imaging , 57 (4), 165–167. https://doi.org/10.1007/s13139-023-00809-2

Cabanac, G., & Labbé, C. (2021). Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology , 72 (12), 1461–1476. https://doi.org/10.1002/asi.24495

Cabanac, G., Labbé, C., & Magazinov, A. (2021). Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals . arXiv. https://doi.org/10.48550/arXiv.2107.06751

Carrion, M. L. (2018). “You need to do your research”: Vaccines, contestable science, and maternal epistemology. Public Understanding of Science , 27 (3), 310–324. https://doi.org/10.1177/0963662517728024

Centre for Digital Humanities Uppsala (2023). CDHUppsala/word-rain [Computer software]. https://github.com/CDHUppsala/word-rain

Chinn, S., & Hasell, A. (2023). Support for “doing your own research” is associated with COVID-19 misperceptions and scientific mistrust. Harvard Kennedy School (HSK) Misinformation Review, 4 (3). https://doi.org/10.37016/mr-2020-117

Cholewiak, S. A., Ipeirotis, P., Silva, V., & Kannawadi, A. (2023). SCHOLARLY: Simple access to Google Scholar authors and citation using Python (1.5.0) [Computer software]. https://doi.org/10.5281/zenodo.5764801

Dadkhah, M., Lagzian, M., & Borchardt, G. (2017). Questionable papers in citation databases as an issue for literature review. Journal of Cell Communication and Signaling , 11 (2), 181–185. https://doi.org/10.1007/s12079-016-0370-6

Dadkhah, M., Oermann, M. H., Hegedüs, M., Raman, R., & Dávid, L. D. (2023). Detection of fake papers in the era of artificial intelligence. Diagnosis , 10 (4), 390–397. https://doi.org/10.1515/dx-2023-0090

DeGeurin, M. (2024, March 19). AI-generated nonsense is leaking into scientific journals. Popular Science. https://www.popsci.com/technology/ai-generated-text-scientific-journals/

Dunlap, R. E., & Brulle, R. J. (2020). Sources and amplifiers of climate change denial. In D.C. Holmes & L. M. Richardson (Eds.), Research handbook on communicating climate change (pp. 49–61). Edward Elgar Publishing. https://doi.org/10.4337/9781789900408.00013

Fares, M., Kutuzov, A., Oepen, S., & Velldal, E. (2017). Word vectors, reuse, and replicability: Towards a community repository of large-text resources. In J. Tiedemann & N. Tahmasebi (Eds.), Proceedings of the 21st Nordic Conference on Computational Linguistics (pp. 271–276). Association for Computational Linguistics. https://aclanthology.org/W17-0237

Google Scholar Help. (n.d.). Inclusion guidelines for webmasters . https://scholar.google.com/intl/en/scholar/inclusion.html

Gu, J., Wang, X., Li, C., Zhao, J., Fu, W., Liang, G., & Qiu, J. (2022). AI-enabled image fraud in scientific publications. Patterns , 3 (7), 100511. https://doi.org/10.1016/j.patter.2022.100511

Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods , 11 (2), 181–217.   https://doi.org/10.1002/jrsm.1378

Haider, J., & Åström, F. (2017). Dimensions of trust in scholarly communication: Problematizing peer review in the aftermath of John Bohannon’s “Sting” in science. Journal of the Association for Information Science and Technology , 68 (2), 450–467. https://doi.org/10.1002/asi.23669

Huang, J., & Tan, M. (2023). The role of ChatGPT in scientific communication: Writing better scientific review articles. American Journal of Cancer Research , 13 (4), 1148–1154. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10164801/

Jones, N. (2024). How journals are fighting back against a wave of questionable images. Nature , 626 (8000), 697–698. https://doi.org/10.1038/d41586-024-00372-6

Kitamura, F. C. (2023). ChatGPT is shaping the future of medical writing but still requires human judgment. Radiology , 307 (2), e230171. https://doi.org/10.1148/radiol.230171

Littell, J. H., Abel, K. M., Biggs, M. A., Blum, R. W., Foster, D. G., Haddad, L. B., Major, B., Munk-Olsen, T., Polis, C. B., Robinson, G. E., Rocca, C. H., Russo, N. F., Steinberg, J. R., Stewart, D. E., Stotland, N. L., Upadhyay, U. D., & Ditzhuijzen, J. van. (2024). Correcting the scientific record on abortion and mental health outcomes. BMJ , 384 , e076518. https://doi.org/10.1136/bmj-2023-076518

Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023). ChatGPT and a new academic reality: Artificial Intelligence-written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology, 74 (5), 570–581. https://doi.org/10.1002/asi.24750

Martín-Martín, A., Orduna-Malea, E., Ayllón, J. M., & Delgado López-Cózar, E. (2016). Back to the past: On the shoulders of an academic search engine giant. Scientometrics , 107 , 1477–1487. https://doi.org/10.1007/s11192-016-1917-2

Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A multidisciplinary comparison of coverage via citations. Scientometrics , 126 (1), 871–906. https://doi.org/10.1007/s11192-020-03690-4

Simon, F. M., Altay, S., & Mercier, H. (2023). Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harvard Kennedy School (HKS) Misinformation Review, 4 (5). https://doi.org/10.37016/mr-2020-127

Skeppstedt, M., Ahltorp, M., Kucher, K., & Lindström, M. (2024). From word clouds to Word Rain: Revisiting the classic word cloud to visualize climate change texts. Information Visualization , 23 (3), 217–238. https://doi.org/10.1177/14738716241236188

Swedish Research Council. (2017). Good research practice. Vetenskapsrådet.

Stokel-Walker, C. (2024, May 1.). AI Chatbots Have Thoroughly Infiltrated Scientific Publishing . Scientific American. https://www.scientificamerican.com/article/chatbots-have-thoroughly-infiltrated-scientific-publishing/

Subbaraman, N. (2024, May 14). Flood of fake science forces multiple journal closures: Wiley to shutter 19 more journals, some tainted by fraud. The Wall Street Journal . https://www.wsj.com/science/academic-studies-research-paper-mills-journals-publishing-f5a3d4bc

The pandas development team. (2024). pandas-dev/pandas: Pandas (v2.2.2) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.10957263

Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science , 379 (6630), 313–313. https://doi.org/10.1126/science.adg7879

Tripodi, F. B., Garcia, L. C., & Marwick, A. E. (2023). ‘Do your own research’: Affordance activation and disinformation spread. Information, Communication & Society , 27 (6), 1212–1228. https://doi.org/10.1080/1369118X.2023.2245869

Vikramaditya, N. (2020). Nv7-GitHub/googlesearch [Computer software]. https://github.com/Nv7-GitHub/googlesearch

This research has been supported by Mistra, the Swedish Foundation for Strategic Environmental Research, through the research program Mistra Environmental Communication (Haider, Ekström, Rödl) and the Marcus and Amalia Wallenberg Foundation [2020.0004] (Söderström).

Competing Interests

The authors declare no competing interests.

The research described in this article was carried out under Swedish legislation. According to the relevant EU and Swedish legislation (2003:460) on the ethical review of research involving humans (“Ethical Review Act”), the research reported on here is not subject to authorization by the Swedish Ethical Review Authority (“etikprövningsmyndigheten”) (SRC, 2017).

This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided that the original author and source are properly credited.

Data Availability

All data needed to replicate this study are available at the Harvard Dataverse: https://doi.org/10.7910/DVN/WUVD8X

Acknowledgements

The authors wish to thank two anonymous reviewers for their valuable comments on the article manuscript as well as the editorial group of Harvard Kennedy School (HKS) Misinformation Review for their thoughtful feedback and input.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Healthcare (Basel)
• PMC11354114

Adenomyosis—A Call for Awareness, Early Detection, and Effective Treatment Strategies: A Narrative Review

Objective: To provide a brief summary of the high incidence, symptomatology, different types, and diagnosis of adenomyosis and to explore various aspects of the disease, with the primary aim of raising awareness among gynecologists for appropriate and early detection. Background: Adenomyosis, a benign gynecological condition characterized by the infiltration of endometrial tissue into the myometrium, poses significant challenges to women’s reproductive health. Methods: A narrative review was conducted by searching PubMed, Scopus, and Cochrane databases and offering a non-systematic summary and critical analysis of current knowledge on the impact of adenomyosis on women’s health. Articles published in the English language up to May 2023, including original scientific papers, clinical trials, meta-analyses, and reviews focusing on various aspects of adenomyosis, were included in the synthesis of this review. Conclusions: Approximately 20% of women are affected by adenomyosis, which manifests with various subtypes, distinct epidemiological profiles, symptomatology, and treatment responses. Despite its clinical significance, adenomyosis remains understudied, resulting in a significant disparity in research and the literature compared to other gynecological conditions. The severity of adenomyosis is compounded when coexisting with endometriosis, particularly deep-infiltrating endometriosis (DIE), leading to exacerbated fertility issues and severe symptomatology. The wide range of symptoms, including adverse pregnancy outcomes such as pre-eclampsia, highlights its wider impact and emphasizes the need for increased awareness of the condition. Adenomyosis is frequently associated with treatment failure in endometriosis, contributing to dienogest resistance, elevated discontinuation rates, and persistent pain post-endometriosis surgery. Additionally, the lack of specific treatments tailored to adenomyosis poses a considerable challenge in clinical management.

1. Introduction

Adenomyosis is a prevalent yet often underrecognized gynecological condition involving the invasion of endometrial tissue into the myometrium. This condition significantly impacts women’s reproductive health, contributing to a wide array of symptoms and complications. Despite its high incidence and clinical importance, adenomyosis does not receive adequate clinical attention and research compared to other gynecological disorders. The aim of this narrative review is to consolidate current knowledge about adenomyosis, focusing on its incidence, subtypes, diagnostic techniques, symptomatology and complications, co-occurrence with endometriosis and resistance to treatment in endometriosis patients as well as the lack of specific treatment modalities. By doing so, we aim to enhance awareness among healthcare providers and encourage early and accurate diagnosis. This article is presented in accordance with the narrative review reporting checklist (available at https://gpm.amegroups.org/article/view/10.21037/gpm-20-57/rc , accessed on 15 May 2024) [ 1 ].

We conducted a thorough literature search on PubMed, Scopus, and Cochrane databases using MeSH or index terms for the keyword: “adenomyosis”. We included articles published in the English language from the last thirty years, from May 1994 to May 2024, from a diverse range of sources such as original scientific papers, clinical trials, meta-analyses, and reviews. We found 3625 articles from PubMed, 2170 from Scopus, and 194 from Cochrane, and 43 articles were selected by the authors and additional references were also searched from the bibliographies of those chosen articles.

3. Pathogenesis

Sampson was a pioneering figure in the classification and understanding of adenomyosis, initially describing the condition as ‘adenomyoma’ [ 2 ]. He proposed three potential pathways for the development of adenomyomas: invasion of normal endometrium into the myometrium, invasion from extrinsic sources such as endometrial cysts, or misplacement of endometrial tissue within the uterine wall. Over time, various pathophysiological theories have emerged to explain the formation of adenomyosis. These include microtrauma theory, which focuses on tissue injury and repair mechanisms at the endometrial–myometrial interface; the theory of invasion of endometrial basalis cells into the myometrium; the de novo metaplasia or Mullerian remnant theory; and the outside to inside theory. Each of these theories offers insights into the complex mechanisms underlying the development of adenomyosis, contributing to our evolving understanding of this enigmatic condition.

Going through the most prominent theory, the mechanism of tissue injury and repair (TIAR) plays a crucial role. This suggest that chronic peristaltic contractions of the myometrium may lead to auto-traumatization of the predisposed myometrium, potentially initiated by traumatized endometrial–myometrial interfaces, which are often observed following uterine manipulating procedures like hysteroscopy or cesarean section. This chronic peristalsis induces chronic microtrauma and inflammation, releasing cytokines and promoting increased estrogen production. This hyperestrogenic environment may lead to invagination of the endometrial basal layer into the uterine muscular layer, forming adenomyotic lesions.

Hence, a vicious cycle is reinforced, where hyperperistalsis in the junctional zone induces repeated auto-traumatization cycles where the constant disruption of the myometrial muscular fibers and the fibroblasts at the endometrial–myometrial connection leads to an invagination of the endometrial basal layer to the muscular layer of the uterus [ 3 ]. While the prevailing theory suggests that adenomyosis arises from the invasion of endometrial tissue into the myometrium, certain cases challenge this hypothesis. For instance, in patients with Mayer–Rokitansky–Kuster–Hauser syndrome, where functional endometrium is absent, adenomyosis has been histologically confirmed, suggesting alternative mechanisms. Thus, it is proposed that adenomyosis may develop through metaplastic transformation of embryonic epithelial progenitors or differentiation of adult endometrial stem cells migrating to and integrating within the myometrium [ 4 ]. Additionally, in cases lacking direct histological evidence linking adenomyosis to the endometrium, the conventional TIAR mechanism may not fully apply. Instead, the pathology may originate from the invasion of endometrium-like structures from outside the uterus, disrupting the uterine serosa.

4. Symptomatology

The main symptoms of adenomyosis are pain (e.g., dysmenorrhea and pelvic pain) and abnormal uterine bleeding (AUB). Accompanying diseases with comparable symptomatology are often encountered such as uterine fibroids and endometriosis, making it challenging to differentiate which pathology is accountable for individual symptoms.

Rates of dysmenorrhea among patients with adenomyosis range from 30% to 68% [ 5 , 6 ]. Patients with adenomyosis may often present with deep dyspareunia, often mimicking the dyspareunia of patients with deep-infiltrating endometriosis. The pathophysiology of the pain is not fully understood but chronic inflammation and prostaglandins may play an important role [ 7 ]. Regarding AUB, the International Federation of Gynecology and Obstetrics (FIGO) recognizes adenomyosis as a distinct entity within the PALM- COEIN classification system. Uterine adenomyosis is a potential cause of abnormal uterine bleeding, occurring in 20% to 35% of cases of AUB [ 8 ]. This classification system categorizes causes of AUB into specific groups, with “A” representing adenomyosis. Natalin et al. reported a significant increase in menstrual blood loss with an increase in the disease burden [ 9 ]. To further complicate things, about one-third of patients with adenomyosis have no symptoms [ 10 ].

5. Key Reasons for Increasing Awareness of Adenomyosis

• High Incidence:

Historically, adenomyosis diagnosis relied primarily on histological examination post-hysterectomy, which often underestimated the condition’s prevalence and significance. With advancements in imaging technologies like transvaginal ultrasound and MRI, diagnosis has significantly improved, revealing a broader demographic of affected individuals, particularly younger women of reproductive age. This shift has highlighted a higher than previously recognized prevalence of adenomyosis, estimated to affect 15–20% [ 11 , 12 ] of young women, often coexisting with endometriosis and uterine fibroids. Similarly, in cases where hysterectomies are performed for urogynecological reasons, the prevalence ranges from 20% to 30% [ 13 , 14 ]. In women suffering from dysmenorrhea and abnormal uterine bleeding, prevalence rates can be notably higher, ranging from 30% to 59%, as reported in some studies [ 15 , 16 ]. Notably, 40% of women diagnosed with endometriosis also have adenomyosis present, while 80% of infertile women diagnosed with endometriosis have also adenomyosis, with over 40% of these cases involving deep-infiltrating endometriosis (DIE) lesions [ 17 ]. In women undergoing assisted reproductive technologies (ARTs), adenomyosis is prevalent in approximately 20% to 25% of cases [ 18 ]. Increased recognition is crucial for appropriate diagnostic and management approaches;

• Disparity in the Research and the Literature:

Despite its clinical significance, adenomyosis remains understudied compared to endometriosis, as evidenced by the disparity in the scientific literature, with only 3982 entries for adenomyosis versus 35,132 for endometriosis on PubMed (as of 1 May 2024) Historically, even comprehensive guidelines for endometriosis seldom addressed adenomyosis, reflecting an oversight in clinical protocols. The recent emergence of guidelines on adenomyosis such as from the Asian Society of Endometriosis and Adenomyosis and the Society of Obstetricians and Gynecologists of Canada (SOGC) [ 19 , 20 ] mark the progress in standardizing the management and treatment of adenomyosis. Furthermore, efforts are underway to systematically address the research gap, such as the “ Development of a core outcome set and outcome definitions for studies on uterus-sparing treatments of adenomyosis (COSAR): an international multistakeholder-modified Delphi consensus study ”. This initiative aims to standardize research outcomes to enhance the quality and comparability of studies on adenomyosis;

• Different Types and Classifications of Adenomyosis:

Recent advancements in imaging technologies, such as transvaginal ultrasound and magnetic resonance imaging, have significantly transformed the landscape of adenomyosis diagnosis and also reveal different patterns of adenomyosis: most classification models typically delineate between focal and disseminated disease, as well as adenomyosis of the inner and outer myometrium. While various classification systems are available, one of the most comprehensive and widely utilized models is the one proposed by Kishi et al. [ 21 ]. In this pioneering work, Kishi et al. introduced a comprehensive classification system for uterine adenomyosis based on MRI analysis. Recognizing the two primary forms of adenomyosis, diffuse and focal, and their dual localizations relative to the junctional zone, as initially reported by Kishi et al., is essential. The extrinsic form of adenomyosis typically spares inner structures such as the endometrium and junctional zone but often disrupts the serosa; in contrast, the intrinsic form affects the inner structures, underscoring the importance of distinguishing between these manifestations for a comprehensive understanding of adenomyosis pathology and symptomatology.

Emerging data indicate that different subtypes of adenomyosis—especially intrinsic versus extrinsic and diffuse versus focal—have different etiologies and clinical profiles. Intrinsic adenomyosis, more common in older patients and often associated with abnormal uterine bleeding (AUB) and prior uterine surgery, contrasts with extrinsic adenomyosis, which is typically found in younger, nulligravida women and is closely associated with deep-infiltrating endometriosis and primary infertility [ 22 , 23 ].

In 2015, the international Morphological Uterus Sonographic Assessment (MUSA) group issued a consensus regarding the appropriate terminology for describing myometrial lesions observed on ultrasonography [ 24 ];

• Compounded Severity with Endometriosis:

Chapron et al. in 2017 introduced the “outside to inside invasion” theory, proposing that ectopic endometrial cells migrate from posterior endometriosis nodules into the myometrium. Their 2017 prospective observational study revealed a statistically significant correlation between focal adenomyosis of the outer myometrium (FOAM) and deep-infiltrating endometriosis phenotypes, with a co-occurrence rate of 66.3% (110 cases; p < 0.001). Conversely, the co-occurrence rates with superficial peritoneal endometriosis and endometriomas were lower, at 7.5% (3 cases) and 19.3%, respectively [ 25 ]. Parker et al. revealed a significant association between adenomyosis and specific sites affected by endometriosis, particularly deep retrocervical endometriosis (60%; p = 0.01) and involvement of the rectosigmoid (49.2%; p = 0.03) [ 26 ].

The same study group in a prospective observational study involving 255 symptomatic deep-infiltrating endometriosis patients discovered a prevalence of 56.5% of focal adenomyosis of the outer myometrium (FOAM). The prevalence of multiple DIE lesions was found to be notably higher in the FAOM-positive group compared to the FAOM-negative group. Specifically, among the FAOM(+) participants, 82.6% (119 out of 144) exhibited multiple lesions, while in the FAOM(−) group the prevalence was 58.6% (65 out of 111), indicating a strong association between the presence of FAOM and the increased likelihood of multiple DIE lesions [ 27 ]. The presence of adenomyosis in patients with endometriosis often results in more severe symptoms, complicating the management of endometriosis and necessitating integrated therapeutic strategies that address both conditions;

• Association with Treatment Failure in Endometriosis:

Adenomyosis is a major cause of persistent pelvic pain and can lead to higher rates of treatment failure, both surgically and medically, especially in patients being treated for endometriosis [ 28 , 29 ]. This highlights the necessity for clinicians to consider adenomyosis in their differential diagnoses after endometriosis treatment, as well as in managing ongoing symptoms. Medical therapies, such as dienogest used for treating endometriosis, may be less effective when adenomyosis is also present, as high discontinuation rates are reported due to persistent bleeding [ 22 , 30 ]. The often limited effectiveness of progesterone-based therapies in adenomyosis involve progesterone resistance, which stems from a reduction in the expression of PGR, particularly the PGR-B isoform [ 31 , 32 ]. The presence of adenomyosis has also been established as an independent risk factor for complications in deep endometriosis laparoscopic surgery [ 33 ];

• Lack of Specific Treatments:

There are currently no drugs specifically labeled for the treatment of adenomyosis, which complicates management strategies and highlights the need for innovative therapeutic development and tailored treatment approaches. Medical treatment serves as the primary option for patients with adenomyosis who aim to preserve fertility or are advised for individuals ineligible for surgery due to concurrent medical conditions because of the high morbidity and recurrence rates [ 34 ]. Regarding medical therapies, potential treatment options encompass combined oral contraceptive (COC) pills, progestins, the levonorgestrel-releasing intrauterine system (LNG-IUS), and gonadotropin-releasing hormone (GnRH) agonists and antagonists [ 35 ]. The effectiveness of these therapies varies, with the LNG- IUS being the most effective—although in off-label use—in reducing the pain and the abnormal uterine bleeding by inducing decasualization and atrophy of the endometrium and down-regulating estrogen receptors through increased continuous local progesterone release [ 36 ].

As a second step for patients with treatment resistance or severe symptoms and in cases where organ preservation is warranted, a range of interventional procedures are available, including uterine artery embolization (UAE) and a broad category of hyperthermic treatments. Hyperthermic treatments encompass procedures utilizing energy sources such as high-intensity focused ultrasound (HIFU), percutaneous microwave ablation (PMWA), and radiofrequency ablation (RFA). Two recent systematic reviews and meta-analyses showed that hyperthermic therapy including RFA may be effective and safe minimally invasive therapies for symptomatic adenomyosis [ 37 , 38 ].

Other uterine-sparing surgeries include uterine artery ligation and adenomyosis excision. Surgical resection of diffuse and focal adenomyosis, excluding adenomyomas, demands significant surgical expertise and is associated with notable perioperative risks. The last treatment option for severe adenomyosis and completed family planning remains hysterectomy;

• Adenomyosis is linked to infertility and negative outcomes during pregnancy.

The study of Mishra et al. [ 39 ] was the first systematic review to evaluate the prevalence of both isolated adenomyosis and adenomyosis with coexisting endometriosis and/or fibroids in women with subfertility. The pooled prevalence was 10% for isolated adenomyosis, 1% for adenomyosis with coexisting fibroids, 6% for adenomyosis with coexisting endometriosis, and 7% for adenomyosis with endometriosis. The answer remains unclear for which type of adenomyosis has the worst fertility outcome. There is some recent evidence, though, that suggests that the focal type of the disease might have a more significant negative effect than other forms but more evidence is definitely needed [ 40 ].

Beyond its impact on pregnancy rates, adenomyosis has been linked to negative obstetric outcomes. In a recent systematic review and metanalysis, Nirgianakis et al. reported that following treatment with assisted reproductive technology (ART), there was a notably higher miscarriage rate (odds ratio [OR] 2.17; 95% confidence interval [CI] 1.25–3.79) when adenomyosis was present. Furthermore, adenomyosis was further found to be significantly linked with an elevated risk of adverse obstetric outcomes, including pre-eclampsia. Τhese findings were also confirmed by the studies of Horton et al. and Razavi et al., with ORs ranging from 4.35 to 7.87 [ 41 , 42 , 43 ].

Several other complications seem also to be increased in patients with adenomyosis including an increased prevalence of small for gestational age infants, preterm delivery [ 41 , 42 , 43 , 44 ], caesarean section [ 41 , 43 ], fetal malpresentation [ 42 , 43 ], and postpartum hemorrhage [ 43 ]. These associations remained significant even after adjustments for age and mode of conception were made, highlighting the intrinsic risks associated with the presence of adenomyosis in pregnancy. Last but not least, women with adenomyosis situated on the posterior side are identified to experience severe obstetric complications, including placenta previa, placenta accreta, preeclampsia, and preterm birth according to Shi et al. [ 45 ]. These data emphasize the importance of careful monitoring and management of pregnant women diagnosed with adenomyosis, to mitigate these risks and improve pregnancy outcomes [ 43 ].

6. Conclusions

This narrative review has several limitations that have to be acknowledged. The non-systematic methodology used in this review does not follow a standardized protocol for article selection and data extraction, potentially leading to inconsistent inclusion criteria, incomplete coverage, and variable study quality. This review and commentary offer an initial framework for assessing the impact of adenomyosis on women’s health and underscores the need for awareness of the disease, early detection, and improved treatment approaches for this often overlooked condition. However, several critical research questions persist.

7. Call for Awareness

With its profound impact on the quality of life of affected women, it is imperative that healthcare professionals, researchers, and policymakers unite to address the challenges posed by adenomyosis. By raising awareness among both healthcare providers and the general public, we can ensure timely diagnosis and intervention, ultimately improving outcomes and quality of life for affected individuals. Early detection through improved diagnostic techniques and screening protocols is essential for timely intervention and preventing disease progression. Moreover, concerted efforts are needed to develop and implement more effective treatment strategies tailored to individual patient needs. Through collaborative research and education, we can strive towards better recognition, understanding, and management of adenomyosis, ultimately enhancing the well-being of the affected women worldwide.

Funding Statement

This research received no external funding.

Author Contributions

The concept of the article was developed by G.K. Methodology and synthesis were performed by G.K. and I.D. The first draft of the manuscript was written by G.K. and all authors commented on previous versions of the manuscript. Supervision was carried out by M.M. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

No ethics board approval was requested as the data were extracted from published papers.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors declare no conflicts of interest.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.