literature review management software

Start your free trial

Arrange a trial for your organisation and discover why FSTA is the leading database for reliable research on the sciences of food and health.

REQUEST A FREE TRIAL

• Research Skills Blog

5 software tools to support your systematic review processes

By Dr. Mina Kalantar on 19-Jan-2021 13:01:01

Systematic reviews are a reassessment of scholarly literature to facilitate decision making. This methodical approach of re-evaluating evidence was initially applied in healthcare, to set policies, create guidelines and answer medical questions.

Systematic reviews are large, complex projects and, depending on the purpose, they can be quite expensive to conduct. A team of researchers, data analysts and experts from various fields may collaborate to review and examine incredibly large numbers of research articles for evidence synthesis. Depending on the spectrum, systematic reviews often take at least 6 months, and sometimes upwards of 18 months to complete.

The main principles of transparency and reproducibility require a pragmatic approach in the organisation of the required research activities and detailed documentation of the outcomes. As a result, many software tools have been developed to help researchers with some of the tedious tasks required as part of the systematic review process.

hbspt.cta._relativeUrls=true;hbspt.cta.load(97439, 'ccc20645-09e2-4098-838f-091ed1bf1f4e', {"useNewLoader":"true","region":"na1"});

The first generation of these software tools were produced to accommodate and manage collaborations, but gradually developed to help with screening literature and reporting outcomes. Some of these software packages were initially designed for medical and healthcare studies and have specific protocols and customised steps integrated for various types of systematic reviews. However, some are designed for general processing, and by extending the application of the systematic review approach to other fields, they are being increasingly adopted and used in software engineering, health-related nutrition, agriculture, environmental science, social sciences and education.

Software tools

There are various free and subscription-based tools to help with conducting a systematic review. Many of these tools are designed to assist with the key stages of the process, including title and abstract screening, data synthesis, and critical appraisal. Some are designed to facilitate the entire process of review, including protocol development, reporting of the outcomes and help with fast project completion.

As time goes on, more functions are being integrated into such software tools. Technological advancement has allowed for more sophisticated and user-friendly features, including visual graphics for pattern recognition and linking multiple concepts. The idea is to digitalise the cumbersome parts of the process to increase efficiency, thus allowing researchers to focus their time and efforts on assessing the rigorousness and robustness of the research articles.

This article introduces commonly used systematic review tools that are relevant to food research and related disciplines, which can be used in a similar context to the process in healthcare disciplines.

These reviews are based on IFIS' internal research, thus are unbiased and not affiliated with the companies.

This online platform is a core component of the Cochrane toolkit, supporting parts of the systematic review process, including title/abstract and full-text screening, documentation, and reporting.

The Covidence platform enables collaboration of the entire systematic reviews team and is suitable for researchers and students at all levels of experience.

From a user perspective, the interface is intuitive, and the citation screening is directed step-by-step through a well-defined workflow. Imports and exports are straightforward, with easy export options to Excel and CVS.

Access is free for Cochrane authors (a single reviewer), and Cochrane provides a free trial to other researchers in healthcare. Universities can also subscribe on an institutional basis.

Rayyan is a free and open access web-based platform funded by the Qatar Foundation, a non-profit organisation supporting education and community development initiative . Rayyan is used to screen and code literature through a systematic review process.

Unlike Covidence, Rayyan does not follow a standard SR workflow and simply helps with citation screening. It is accessible through a mobile application with compatibility for offline screening. The web-based platform is known for its accessible user interface, with easy and clear export options.

Function comparison of 5 software tools to support the systematic review process

EPPI-Reviewer

EPPI-Reviewer is a web-based software programme developed by the Evidence for Policy and Practice Information and Co-ordinating Centre  (EPPI) at the UCL Institute for Education, London .

It provides comprehensive functionalities for coding and screening. Users can create different levels of coding in a code set tool for clustering, screening, and administration of documents. EPPI-Reviewer allows direct search and import from PubMed. The import of search results from other databases is feasible in different formats. It stores, references, identifies and removes duplicates automatically. EPPI-Reviewer allows full-text screening, text mining, meta-analysis and the export of data into different types of reports.

There is no limit for concurrent use of the software and the number of articles being reviewed. Cochrane reviewers can access EPPI reviews using their Cochrane subscription details.

EPPI-Centre has other tools for facilitating the systematic review process, including coding guidelines and data management tools.

CADIMA is a free, online, open access review management tool, developed to facilitate research synthesis and structure documentation of the outcomes.

The Julius Institute and the Collaboration for Environmental Evidence established the software programme to support and guide users through the entire systematic review process, including protocol development, literature searching, study selection, critical appraisal, and documentation of the outcomes. The flexibility in choosing the steps also makes CADIMA suitable for conducting systematic mapping and rapid reviews.

CADIMA was initially developed for research questions in agriculture and environment but it is not limited to these, and as such, can be used for managing review processes in other disciplines. It enables users to export files and work offline.

The software allows for statistical analysis of the collated data using the R statistical software. Unlike EPPI-Reviewer, CADIMA does not have a built-in search engine to allow for searching in literature databases like PubMed.

DistillerSR

DistillerSR is an online software maintained by the Canadian company, Evidence Partners which specialises in literature review automation. DistillerSR provides a collaborative platform for every stage of literature review management. The framework is flexible and can accommodate literature reviews of different sizes. It is configurable to different data curation procedures, workflows and reporting standards. The platform integrates necessary features for screening, quality assessment, data extraction and reporting. The software uses Artificial Learning (AL)-enabled technologies in priority screening. It is to cut the screening process short by reranking the most relevant references nearer to the top. It can also use AL, as a second reviewer, in quality control checks of screened studies by human reviewers. DistillerSR is used to manage systematic reviews in various medical disciplines, surveillance, pharmacovigilance and public health reviews including food and nutrition topics. The software does not support statistical analyses. It provides configurable forms in standard formats for data extraction.

DistillerSR allows direct search and import of references from PubMed. It provides an add on feature called LitConnect which can be set to automatically import newly published references from data providers to keep reviews up to date during their progress.

The systematic review Toolbox is a web-based catalogue of various tools, including software packages which can assist with single or multiple tasks within the evidence synthesis process. Researchers can run a quick search or tailor a more sophisticated search by choosing their approach, budget, discipline, and preferred support features, to find the right tools for their research.

If you enjoyed this blog post, you may also be interested in our recently published blog post addressing the difference between a systematic review and a systematic literature review.

• FSTA - Food Science & Technology Abstracts
• IFIS Collections
• Resources Hub
• Diversity Statement
• Sustainability Commitment
• Company news
• Frequently Asked Questions
• Privacy Policy
• Terms of Use for IFIS Collections

Ground Floor, 115 Wharfedale Road,  Winnersh Triangle, Wokingham, Berkshire RG41 5RB

Get in touch with IFIS

© International Food Information Service (IFIS Publishing) operating as IFIS – All Rights Reserved     |     Charity Reg. No. 1068176     |     Limited Company No. 3507902     |     Designed by Blend

10 Best Literature Review Tools for Researchers

Boost your research game with these Best Literature Review Tools for Researchers! Uncover hidden gems, organize your findings, and ace your next research paper!

Researchers struggle to identify key sources, extract relevant information, and maintain accuracy while manually conducting literature reviews. This leads to inefficiency, errors, and difficulty in identifying gaps or trends in existing literature.

Table of Contents

Top 10 Literature Review Tools for Researchers: In A Nutshell (2023)

#1. Semantic Scholar – A free, AI-powered research tool for scientific literature

Not all scholarly content may be indexed, and occasional false positives or inaccurate associations can occur. Furthermore, the tool primarily focuses on computer science and related fields, potentially limiting coverage in other disciplines. 

#2. Elicit – Research assistant using language models like GPT-3

Elicit is a game-changing literature review tool that has gained popularity among researchers worldwide. With its user-friendly interface and extensive database of scholarly articles, it streamlines the research process, saving time and effort. 

However, users should be cautious when using Elicit. It is important to verify the credibility and accuracy of the sources found through the tool, as the database encompasses a wide range of publications. 

Additionally, occasional glitches in the search function have been reported, leading to incomplete or inaccurate results. While Elicit offers tremendous benefits, researchers should remain vigilant and cross-reference information to ensure a comprehensive literature review.

#3. Scite.Ai – Your personal research assistant

Scite.Ai is a popular literature review tool that revolutionizes the research process for scholars. With its innovative citation analysis feature, researchers can evaluate the credibility and impact of scientific articles, making informed decisions about their inclusion in their own work. 

However, while Scite.Ai offers numerous advantages, there are a few aspects to be cautious about. As with any data-driven tool, occasional errors or inaccuracies may arise, necessitating researchers to cross-reference and verify results with other reputable sources. 

Rayyan offers the following paid plans:

#4. DistillerSR – Literature Review Software

Despite occasional technical glitches reported by some users, the developers actively address these issues through updates and improvements, ensuring a better user experience. 

#5. Rayyan – AI Powered Tool for Systematic Literature Reviews

However, it’s important to be aware of a few aspects. The free version of Rayyan has limitations, and upgrading to a premium subscription may be necessary for additional functionalities. 

#6. Consensus – Use AI to find you answers in scientific research

With Consensus, researchers can save significant time by efficiently organizing and accessing relevant research material.People consider Consensus for several reasons. 

Consensus offers both free and paid plans:

#7. RAx – AI-powered reading assistant

#8. lateral – advance your research with ai.

Additionally, researchers must be mindful of potential biases introduced by the tool’s algorithms and should critically evaluate and interpret the results. 

#9. Iris AI – Introducing the researcher workspace

Researchers are drawn to this tool because it saves valuable time by automating the tedious task of literature review and provides comprehensive coverage across multiple disciplines. 

#10. Scholarcy – Summarize your literature through AI

Scholarcy’s ability to extract key information and generate concise summaries makes it an attractive option for scholars looking to quickly grasp the main concepts and findings of multiple papers.

Scholarcy’s automated summarization may not capture the nuanced interpretations or contextual information presented in the full text. 

Final Thoughts

In conclusion, conducting a comprehensive literature review is a crucial aspect of any research project, and the availability of reliable and efficient tools can greatly facilitate this process for researchers. This article has explored the top 10 literature review tools that have gained popularity among researchers.

Q1. What are literature review tools for researchers?

Q2. what criteria should researchers consider when choosing literature review tools.

When choosing literature review tools, researchers should consider factors such as the tool’s search capabilities, database coverage, user interface, collaboration features, citation management, annotation and highlighting options, integration with reference management software, and data extraction capabilities. 

Q3. Are there any literature review tools specifically designed for systematic reviews or meta-analyses?

Meta-analysis support: Some literature review tools include statistical analysis features that assist in conducting meta-analyses. These features can help calculate effect sizes, perform statistical tests, and generate forest plots or other visual representations of the meta-analytic results.

Q4. Can literature review tools help with organizing and annotating collected references?

Integration with citation managers: Some literature review tools integrate with popular citation managers like Zotero, Mendeley, or EndNote, allowing seamless transfer of references and annotations between platforms.

By leveraging these features, researchers can streamline the organization and annotation of their collected references, making it easier to retrieve relevant information during the literature review process.

Leave a Comment Cancel reply

Covidence website will be inaccessible as we upgrading our platform on Monday 23rd August at 10am AEST, / 2am CEST/1am BST (Sunday, 15th August 8pm EDT/5pm PDT) 

The World's #1 Systematic Review Tool

See your systematic reviews like never before

Faster reviews.

An average 35% reduction in time spent per review, saving an average of 71 hours per review.

Expert, online support

Easy to learn and use, with 24/7 support from product experts who are also seasoned reviewers!

Seamless collaboration

Enable the whole review team to collaborate from anywhere.

Suits all levels of experience and sectors

Suitable for reviewers in a variety of sectors including health, education, social science and many others.

Supporting the world's largest systematic review community

See how it works.

Step inside Covidence to see a more intuitive, streamlined way to manage systematic reviews.

Unlimited use for every organization

With no restrictions on reviews and users, Covidence gets out of the way so you can bring the best evidence to the world, more quickly.

Covidence is used by world-leading evidence organizations

Whether you’re an academic institution, a hospital or society, Covidence is working for organizations like yours right now.

See a list of organizations already using Covidence →

How Covidence has enabled living guidelines for Australians impacted by stroke

Clinical guidelines took 7 years to update prior to moving to a living evidence approach. Learn how Covidence streamlined workflows and created real time savings for the guidelines team.

University of Ottawa Drives Systematic Review Excellence Across Many Academic Disciplines

University of Ottawa

Top Ranked U.S. Teaching Hospital Delivers Effective Systematic Review Management

Top Ranked U.S. Teaching Hospital

See more Case Studies

Better systematic review management

Head office, working for an institution or organisation.

Find out why over 350 of the world’s leading institutions are seeing a surge in publications since using Covidence!

Request a consultation with one of our team members and start empowering your researchers: 

By using our site you consent to our use of cookies to measure and improve our site’s performance. Please see our Privacy Policy for more information. 

Accelerate your research with the best systematic literature review tools

The ideal literature review tool helps you make sense of the most important insights in your research field. ATLAS.ti empowers researchers to perform powerful and collaborative analysis using the leading software for literature review.

Finalize your literature review faster with comfort

ATLAS.ti makes it easy to manage, organize, and analyze articles, PDFs, excerpts, and more for your projects. Conduct a deep systematic literature review and get the insights you need with a comprehensive toolset built specifically for your research projects.

Figure out the "why" behind your participant's motivations

Understand the behaviors and emotions that are driving your focus group participants. With ATLAS.ti, you can transform your raw data and turn it into qualitative insights you can learn from. Easily determine user intent in the same spot you're deciphering your overall focus group data.

Visualize your research findings like never before

We make it simple to present your analysis results with meaningful charts, networks, and diagrams. Instead of figuring out how to communicate the insights you just unlocked, we enable you to leverage easy-to-use visualizations that support your goals.

Paper Search – Access 200+ Million Papers with AI-Powered Insights

Unlock access to over 200 million scientific papers and streamline your research with our cutting-edge AI-powered Paper Search 2.0. Easily find, summarize, and integrate relevant papers directly into your ATLAS.ti Web workspace.

Everything you need to elevate your literature review

Import and organize literature data.

Import and analyze any type of text content – ATLAS.ti supports all standard text and transcription files such as Word and PDF.

Analyze with ease and speed

Utilize easy-to-learn workflows that save valuable time, such as auto coding, sentiment analysis, team collaboration, and more.

Leverage AI-driven tools

Make efficiency a priority and let ATLAS.ti do your work with AI-powered research tools and features for faster results.

Visualize and present findings

With just a few clicks, you can create meaningful visualizations like charts, word clouds, tables, networks, among others for your literature data.

The faster way to make sense of your literature review. Try it for free, today.

A literature review analyzes the most current research within a research area. A literature review consists of published studies from many sources:

• Peer-reviewed academic publications
• Full-length books
• University bulletins
• Conference proceedings
• Dissertations and theses

Literature reviews allow researchers to:

• Summarize the state of the research
• Identify unexplored research inquiries
• Recommend practical applications
• Critique currently published research

Literature reviews are either standalone publications or part of a paper as background for an original research project. A literature review, as a section of a more extensive research article, summarizes the current state of the research to justify the primary research described in the paper.

For example, a researcher may have reviewed the literature on a new supplement's health benefits and concluded that more research needs to be conducted on those with a particular condition. This research gap warrants a study examining how this understudied population reacted to the supplement. Researchers need to establish this research gap through a literature review to persuade journal editors and reviewers of the value of their research.

Consider a literature review as a typical research publication presenting a study, its results, and the salient points scholars can infer from the study. The only significant difference with a literature review treats existing literature as the research data to collect and analyze. From that analysis, a literature review can suggest new inquiries to pursue.

Identify a focus

Similar to a typical study, a literature review should have a research question or questions that analysis can answer. This sort of inquiry typically targets a particular phenomenon, population, or even research method to examine how different studies have looked at the same thing differently. A literature review, then, should center the literature collection around that focus.

Collect and analyze the literature

With a focus in mind, a researcher can collect studies that provide relevant information for that focus. They can then analyze the collected studies by finding and identifying patterns or themes that occur frequently. This analysis allows the researcher to point out what the field has frequently explored or, on the other hand, overlooked.

Suggest implications

The literature review allows the researcher to argue a particular point through the evidence provided by the analysis. For example, suppose the analysis makes it apparent that the published research on people's sleep patterns has not adequately explored the connection between sleep and a particular factor (e.g., television-watching habits, indoor air quality). In that case, the researcher can argue that further study can address this research gap.

External requirements aside (e.g., many academic journals have a word limit of 6,000-8,000 words), a literature review as a standalone publication is as long as necessary to allow readers to understand the current state of the field. Even if it is just a section in a larger paper, a literature review is long enough to allow the researcher to justify the study that is the paper's focus.

Note that a literature review needs only to incorporate a representative number of studies relevant to the research inquiry. For term papers in university courses, 10 to 20 references might be appropriate for demonstrating analytical skills. Published literature reviews in peer-reviewed journals might have 40 to 50 references. One of the essential goals of a literature review is to persuade readers that you have analyzed a representative segment of the research you are reviewing.

Researchers can find published research from various online sources:

• Journal websites
• Research databases
• Search engines (Google Scholar, Semantic Scholar)
• Research repositories
• Social networking sites (Academia, ResearchGate)

Many journals make articles freely available under the term "open access," meaning that there are no restrictions to viewing and downloading such articles. Otherwise, collecting research articles from restricted journals usually requires access from an institution such as a university or a library.

Evidence of a rigorous literature review is more important than the word count or the number of articles that undergo data analysis. Especially when writing for a peer-reviewed journal, it is essential to consider how to demonstrate research rigor in your literature review to persuade reviewers of its scholarly value.

Select field-specific journals

The most significant research relevant to your field focuses on a narrow set of journals similar in aims and scope. Consider who the most prominent scholars in your field are and determine which journals publish their research or have them as editors or reviewers. Journals tend to look favorably on systematic reviews that include articles they have published.

Incorporate recent research

Recently published studies have greater value in determining the gaps in the current state of research. Older research is likely to have encountered challenges and critiques that may render their findings outdated or refuted. What counts as recent differs by field; start by looking for research published within the last three years and gradually expand to older research when you need to collect more articles for your review.

Consider the quality of the research

Literature reviews are only as strong as the quality of the studies that the researcher collects. You can judge any particular study by many factors, including:

• the quality of the article's journal
• the article's research rigor
• the timeliness of the research

The critical point here is that you should consider more than just a study's findings or research outputs when including research in your literature review.

Narrow your research focus

Ideally, the articles you collect for your literature review have something in common, such as a research method or research context. For example, if you are conducting a literature review about teaching practices in high school contexts, it is best to narrow your literature search to studies focusing on high school. You should consider expanding your search to junior high school and university contexts only when there are not enough studies that match your focus.

You can create a project in ATLAS.ti for keeping track of your collected literature. ATLAS.ti allows you to view and analyze full text articles and PDF files in a single project. Within projects, you can use document groups to separate studies into different categories for easier and faster analysis.

For example, a researcher with a literature review that examines studies across different countries can create document groups labeled "United Kingdom," "Germany," and "United States," among others. A researcher can also use ATLAS.ti's global filters to narrow analysis to a particular set of studies and gain insights about a smaller set of literature.

ATLAS.ti allows you to search, code, and analyze text documents and PDF files. You can treat a set of research articles like other forms of qualitative data. The codes you apply to your literature collection allow for analysis through many powerful tools in ATLAS.ti:

• Code Co-Occurrence Explorer
• Code Co-Occurrence Table
• Code-Document Table

Other tools in ATLAS.ti employ machine learning to facilitate parts of the coding process for you. Some of our software tools that are effective for analyzing literature include:

• Named Entity Recognition
• Opinion Mining
• Sentiment Analysis

As long as your documents are text documents or text-enable PDF files, ATLAS.ti's automated tools can provide essential assistance in the data analysis process.

• Help Center

GET STARTED

COLLABORATE ON YOUR REVIEWS WITH ANYONE, ANYWHERE, ANYTIME

Save precious time and maximize your productivity with a Rayyan membership. Receive training, priority support, and access features to complete your systematic reviews efficiently.

Rayyan Teams+ makes your job easier. It includes VIP Support, AI-powered in-app help, and powerful tools to create, share and organize systematic reviews, review teams, searches, and full-texts.

RESEARCHERS

Rayyan makes collaborative systematic reviews faster, easier, and more convenient. Training, VIP support, and access to new features maximize your productivity. Get started now!

Over 1 billion reference articles reviewed by research teams, and counting...

Intelligent, scalable and intuitive.

Rayyan understands language, learns from your decisions and helps you work quickly through even your largest systematic literature reviews.

WATCH A TUTORIAL NOW

Solutions for Organizations and Businesses

Rayyan Enterprise and Rayyan Teams+ make it faster, easier and more convenient for you to manage your research process across your organization.

• Accelerate your research across your team or organization and save valuable researcher time.
• Build and preserve institutional assets, including literature searches, systematic reviews, and full-text articles.
• Onboard team members quickly with access to group trainings for beginners and experts.
• Receive priority support to stay productive when questions arise.
• SCHEDULE A DEMO
• LEARN MORE ABOUT RAYYAN TEAMS+

RAYYAN SYSTEMATIC LITERATURE REVIEW OVERVIEW

LEARN ABOUT RAYYAN’S PICO HIGHLIGHTS AND FILTERS

Join now to learn why Rayyan is trusted by already more than 500,000 researchers

Individual plans, teams plans.

For early career researchers just getting started with research.

Free forever

• 3 Active Reviews
• Invite Unlimited Reviewers
• Import Directly from Mendeley
• Industry Leading De-Duplication
• 5-Star Relevance Ranking
• Advanced Filtration Facets
• Mobile App Access
• 100 Decisions on Mobile App
• Standard Support
• Revoke Reviewer
• Online Training
• PICO Highlights & Filters
• PRISMA (Beta)
• Auto-Resolver 
• Multiple Teams & Management Roles
• Monitor & Manage Users, Searches, Reviews, Full Texts
• Onboarding and Regular Training

Professional

For researchers who want more tools for research acceleration.

per month, billed annually

• Unlimited Active Reviews
• Unlimited Decisions on Mobile App
• Priority Support
• Auto-Resolver

For currently enrolled students with valid student ID.

per month, billed quarterly

For a team that wants professional licenses for all members.

per month, per user, billed annually

• Single Team
• High Priority Support

For teams that want support and advanced tools for members.

• Multiple Teams
• Management Roles

For organizations who want access to all of their members.

Annual Subscription

Contact Sales

• Organizational Ownership
• For an organization or a company
• Access to all the premium features such as PICO Filters, Auto-Resolver, PRISMA and Mobile App
• Store and Reuse Searches and Full Texts
• A management console to view, organize and manage users, teams, review projects, searches and full texts
• Highest tier of support – Support via email, chat and AI-powered in-app help
• GDPR Compliant
• Single Sign-On
• API Integration
• Training for Experts
• Training Sessions Students Each Semester
• More options for secure access control

———————–

ANNUAL ONLY

Rayyan Subscription

membership starts with 2 users. You can select the number of additional members that you’d like to add to your membership.

Total amount:

Click Proceed to get started.

Great usability and functionality. Rayyan has saved me countless hours. I even received timely feedback from staff when I did not understand the capabilities of the system, and was pleasantly surprised with the time they dedicated to my problem. Thanks again!

This is a great piece of software. It has made the independent viewing process so much quicker. The whole thing is very intuitive.

Rayyan makes ordering articles and extracting data very easy. A great tool for undertaking literature and systematic reviews!

Excellent interface to do title and abstract screening. Also helps to keep a track on the the reasons for exclusion from the review. That too in a blinded manner.

Rayyan is a fantastic tool to save time and improve systematic reviews!!! It has changed my life as a researcher!!! thanks

Easy to use, friendly, has everything you need for cooperative work on the systematic review.

Rayyan makes life easy in every way when conducting a systematic review and it is easy to use.

Something went wrong when searching for seed articles. Please try again soon.

No articles were found for that search term.

Author, year The title of the article goes here

LITERATURE REVIEW SOFTWARE FOR BETTER RESEARCH

“Litmaps is a game changer for finding novel literature... it has been invaluable for my productivity.... I also got my PhD student to use it and they also found it invaluable, finding several gaps they missed”

Varun Venkatesh

Austin Health, Australia

As a full-time researcher, Litmaps has become an indispensable tool in my arsenal. The Seed Maps and Discover features of Litmaps have transformed my literature review process, streamlining the identification of key citations while revealing previously overlooked relevant literature, ensuring no crucial connection goes unnoticed. A true game-changer indeed!

Ritwik Pandey

Doctoral Research Scholar – Sri Sathya Sai Institute of Higher Learning

Using Litmaps for my research papers has significantly improved my workflow. Typically, I start with a single paper related to my topic. Whenever I find an interesting work, I add it to my search. From there, I can quickly cover my entire Related Work section.

David Fischer

Research Associate – University of Applied Sciences Kempten

“It's nice to get a quick overview of related literature. Really easy to use, and it helps getting on top of the often complicated structures of referencing”

Christoph Ludwig

Technische Universität Dresden, Germany

“This has helped me so much in researching the literature. Currently, I am beginning to investigate new fields and this has helped me hugely”

Aran Warren

Canterbury University, NZ

“I can’t live without you anymore! I also recommend you to my students.”

Professor at The Chinese University of Hong Kong

“Seeing my literature list as a network enhances my thinking process!”

Katholieke Universiteit Leuven, Belgium

“Incredibly useful tool to get to know more literature, and to gain insight in existing research”

KU Leuven, Belgium

“As a student just venturing into the world of lit reviews, this is a tool that is outstanding and helping me find deeper results for my work.”

Franklin Jeffers

South Oregon University, USA

“Any researcher could use it! The paper recommendations are great for anyone and everyone”

Swansea University, Wales

“This tool really helped me to create good bibtex references for my research papers”

Ali Mohammed-Djafari

Director of Research at LSS-CNRS, France

“Litmaps is extremely helpful with my research. It helps me organize each one of my projects and see how they relate to each other, as well as to keep up to date on publications done in my field”

Daniel Fuller

Clarkson University, USA

As a person who is an early researcher and identifies as dyslexic, I can say that having research articles laid out in the date vs cite graph format is much more approachable than looking at a standard database interface. I feel that the maps Litmaps offers lower the barrier of entry for researchers by giving them the connections between articles spaced out visually. This helps me orientate where a paper is in the history of a field. Thus, new researchers can look at one of Litmap's "seed maps" and have the same information as hours of digging through a database.

Baylor Fain

Postdoctoral Associate – University of Florida

We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.

Advanced Literature Review Software

Synthesis provides advanced literature review software with analytical and automation functionality for delivering timely evidence-based information in hours, not months, for better decisions.

Strategic Analysis

Perform Scoping and Systematic Reviews quickly and accurately using the latest automation and information management algorithms.

Reference Management

Synthesis organizes and manages all your references and PDFs. You can then quickly search the Abstract and Full-Text PDFs for keywords and phrases.

Advanced Analytics

Quickly summarize the reference by searching and tagging for keywords, preform topic clustering or word clouds on the literature, and then graph all your data.

Multiple Databases

PubMed, PubMed Central, IEEE, US Patents, Ovid (Medline, Embase, Global Health), Web of Science, Scopus, ProQuest, and many others..

Distribution

Export capabilities for sharing the Knowledge that you have just created as either CSV files or for importing into Cite and Write managers.

Internationally Recognized

Synthesis is used in academic research universities, hospitals, government agencies, private corporations and non-governmental organziations throughout the world.

Synthesis applies the latest in automation and enhanced analytic functionality for improving the efficiency and effectiveness of conducting literature reviews...

How to get started

Explore the features of Synthesis to see what truly sets it apart from other approaches for managing and analyzing the academic and business literature.

Synthesis provides online embedded searching on major bibliographical databases, validated automated de-duplication of references, automated importing of PDFs, methods to analyze the literature, and many more features.

Synthesis is available for Windows, Macintosh, Linux and as a Java application that can be run on any platform.

Find out more

I want to have Access to the latest Literature in the Fastest Possible way and Quickly Assess it. Physician

We need Systems with Automation and Artificial Intelligence that Allows Literature Reviews to be conducted quickly and efficiently. Academic Researcher

We need a Computer System for Healthcare that Puts the Information at My Finger Tips and Tells Me Everything I Need to Know. Hospital Administrator

We need Information Systems that Aren't Based in 1970s Technology Medical Student

Keep Up to Date about Synthesis

Synthesis research inc..

Synthesis Research Inc is a software development company focused on improving the way that literature is managed and analyzed. This desire is based around the goal of providing the best synthesized knowledge for supporting evidence-based decision making.

Synthesis Research Inc applies the latest computer science algorithms based around automation and information retrieval and management for improving the efficiency and effectiveness of conducting literature reviews through automating manual processes and enhancing the workflow.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• JMIR Med Inform
• v.10(5); 2022 May

Web-Based Software Tools for Systematic Literature Review in Medicine: Systematic Search and Feature Analysis

Kathryn cowie.

1 Nested Knowledge, Saint Paul, MN, United States

Asad Rahmatullah

Nicole hardy, kevin kallmes, associated data.

Supplementary Table 1: Screening Decisions for SR (systematic review) Tools Reviewed in Full.

Supplementary Table 2: Inter-observer Agreement across (1) Systematic Review (SR) Tools and (2) Features Assessed.

Systematic reviews (SRs) are central to evaluating therapies but have high costs in terms of both time and money. Many software tools exist to assist with SRs, but most tools do not support the full process, and transparency and replicability of SR depend on performing and presenting evidence according to established best practices.

This study aims to provide a basis for comparing and selecting between web-based software tools that support SR, by conducting a feature-by-feature comparison of SR tools.

We searched for SR tools by reviewing any such tool listed in the SR Toolbox, previous reviews of SR tools, and qualitative Google searching. We included all SR tools that were currently functional and required no coding, and excluded reference managers, desktop applications, and statistical software. The list of features to assess was populated by combining all features assessed in 4 previous reviews of SR tools; we also added 5 features (manual addition, screening automation, dual extraction, living review, and public outputs) that were independently noted as best practices or enhancements of transparency and replicability. Then, 2 reviewers assigned binary present or absent assessments to all SR tools with respect to all features, and a third reviewer adjudicated all disagreements.

Of the 53 SR tools found, 55% (29/53) were excluded, leaving 45% (24/53) for assessment. In total, 30 features were assessed across 6 classes, and the interobserver agreement was 86.46%. Giotto Compliance (27/30, 90%), DistillerSR (26/30, 87%), and Nested Knowledge (26/30, 87%) support the most features, followed by EPPI-Reviewer Web (25/30, 83%), LitStream (23/30, 77%), JBI SUMARI (21/30, 70%), and SRDB.PRO (VTS Software) (21/30, 70%). Fewer than half of all the features assessed are supported by 7 tools: RobotAnalyst (National Centre for Text Mining), SRDR (Agency for Healthcare Research and Quality), SyRF (Systematic Review Facility), Data Abstraction Assistant (Center for Evidence Synthesis in Health), SR Accelerator (Institute for Evidence-Based Healthcare), RobotReviewer (RobotReviewer), and COVID-NMA (COVID-NMA). Notably, of the 24 tools, only 10 (42%) support direct search, only 7 (29%) offer dual extraction, and only 13 (54%) offer living/updatable reviews.

Conclusions

DistillerSR, Nested Knowledge, and EPPI-Reviewer Web each offer a high density of SR-focused web-based tools. By transparent comparison and discussion regarding SR tool functionality, the medical community can both choose among existing software offerings and note the areas of growth needed, most notably in the support of living reviews.

Introduction

Systematic review costs and gaps.

According to the Centre for Evidence-Based Medicine, systematic reviews (SRs) of high-quality primary studies represent the highest level of evidence for evaluating therapeutic performance [ 1 ]. However, although vital to evidence-based medical practice, SRs are time-intensive, taking an average of 67.3 weeks to complete [ 2 ] and costing leading research institutions over US $141,000 in labor per published review [ 3 ]. Owing to the high costs in researcher time and complexity, up-to-date reviews cover only 10% to 17% of primary evidence in a representative analysis of the lung cancer literature [ 4 ]. Although many qualitative and noncomprehensive publications provide some level of summative evidence, SRs—defined as reviews of “evidence on a clearly formulated question that use systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyze data from the studies that are included” [ 5 ]—are distinguished by both their structured approach to finding, filtering, and extracting from underlying articles and the resulting comprehensiveness in answering a concrete medical question.

Software Tools for Systematic Review

Software tools that assist with central SR activities—retrieval (searching or importing records), appraisal (screening of records), synthesis (content extraction from underlying studies), and documentation/output (presentation of SR outputs)—have shown promise in reducing the amount of effort needed in a given review [ 6 ]. Because of the time savings of web-based software tools, institutions and individual researchers engaged in evidence synthesis may benefit from using these tools in the review process [ 7 ].

Existing Studies of Software Tools

However, choosing among the existing software tools presents a further challenge to researchers; in the SR Toolbox [ 8 ], there are >240 tools indexed, of which 224 support health care reviews. Vitally, few of these tools can be used for each of the steps of SR, so comparing the features available through each tool can assist researchers in selecting an SR tool to use. This selection can be informed by feature analysis; for example, a previously published feature analysis compared 15 SR tools [ 9 ] across 21 subfeatures of interest and found that DistillerSR (Evidence Partners), EPPI-Reviewer (EPPI-Centre), SWIFT-Active Screener (Sciome), and Covidence (Cochrane) support the greatest number of features as of 2019. Harrison et al [ 10 ], Marshall et al [ 11 ], and Kohl et al [ 12 ] have completed similar analyses, but each feature assessment selected a different set of features and used different qualitative feature assessment methods, and none covered all SR tools currently available.

The SR tool landscape continues to evolve; as existing tools are updated, new software is made available to researchers, and new feature classes are developed. For instance, despite the growth of calls for living SRs, that is, reviews where the outputs are updated as new primary evidence becomes available, no feature analysis has yet covered this novel capability. Furthermore, the leading feature analyses [ 9 - 12 ] have focused on the screening phase of review, meaning that no comparison of data extraction capabilities has yet been published.

Feature Analysis of Systematic Review Tools

The authors, who are also the developers of the Nested Knowledge platform for SR and meta-analysis (Nested Knowledge, Inc) [ 13 ], have noted the lack of SR feature comparison among new tools and across all feature classes (retrieval, appraisal, synthesis, documentation/output, administration of reviews, and access/support features). To provide an updated feature analysis comparing SR software tools, we performed a feature analysis covering the full life cycle of SR across software tools.

Search Strategy

We searched the SR tools for assessment in 3 ways: first, we identified any SR tool that was published in existing reviews of SR tools (Table S1 in Multimedia Appendix 1 ). Second, we reviewed SR Toolbox [ 8 ], a repository of indexed software tools that support the SR process. Third, we performed a Google search for Systematic review software and identified any software tool that was among the first 5 pages of results. Furthermore, for any library resource pages that were among the search results, we included any SR tools mentioned by the library resource page that met our inclusion criteria. The search was completed between June and August 2021. Four additional tools, namely SRDR+ (Agency for Healthcare Research and Quality), Systematic Review Assistant-Deduplication Module (Institute for Evidence-Based Healthcare), Giotto Compliance, and Robotsearch (Robotsearch), were assessed in December 2021 following reviewer feedback.

Selection of Software Tools

The inclusion and exclusion criteria were determined by 3 authors (KK, KH, and KC). Among our search results, we queued up all software tools that had descriptions meeting our inclusion criteria for full examination of the software in a second round of review. We included any that were functioning web-based tools that require no coding by the user to install or operate, so long as they were used to support the SR process and can be used to review clinical or preclinical literature. The no coding requirement was established because the target audience of this review is medical researchers who are selecting a review software to use; thus, we aim to review only tools that this broad audience is likely to be able to adopt. We also excluded desktop applications, statistical packages, and tools built for reviewing software engineering and social sciences literature, as well as reference managers, to avoid unfairly casting these tools as incomplete review tools (as they would each score quite low in features that are not related to reference management). All software tools were screened by one reviewer (KC), and inclusion decisions were reviewed by a second (KK).

Selection of Features of Interest

We built on the previous comparisons of SR tools published by Van der Mierden et al [ 9 ], Harrison et al [ 10 ], Marshall et al [ 11 ], and Kohl et al [ 12 ], which assign features a level of importance and evaluate each feature in reference screening tools. As the studies by Van der Mierden et al [ 9 ] and Harrison et al [ 10 ] focus on reference screening, we supplemented the features with features identified in related reviews of SR tools (Table S1 in Multimedia Appendix 1 ). From a study by Kohl et al [ 12 ], we added database search, risk of bias assessment (critical appraisal), and data visualization. From Marshall et al [ 11 ], we added report writing.

We added 4 more features based on their importance to software-based SR: manual addition of records, automated full-text retrieval, dual extraction of studies, risk of bias (critical appraisal), living SR, and public outputs. Each addition represents either a best practice in SR [ 14 ] or a key feature for the accuracy, replicability, and transparency of SR. Thus, in total, we assessed the presence or absence of 30 features across 6 categories: retrieval, appraisal, synthesis, documentation/output, administration/project management, and access/support.

We adopted each feature unless it was outside of the SR process, it was required for inclusion in the present review, it duplicated another feature, it was not a discrete step for comparison, it was not necessary for English language reviews, it was not necessary for a web-based software, or it related to reference management (as we excluded reference managers from the present review). Table 1 shows all features not assessed, with rationale.

Features from systematic reviews not assessed in this review, with rationale.

Feature Assessment

To minimize bias concerning the subjective assessment of the necessity or desirability of features or of the relative performance of features, we used a binary assessment where each SR tool was scored 0 if a given feature was not present or 1 if a feature was present. Tools were assessed between June and August 2021. We assessed 30 features, divided into 6 feature classes. Of the 30 features, 77% (23/30) were identified in existing literature, and 23% (7/30) were added by the authors ( Table 2 ).

The criteria for each selected feature, as well as the rationale.

a API: application programming interface.

b Rationale only provided for features added in this review; all other features were drawn from existing feature analyses of Systematic Review Software Tools.

c RIS: Research Information System.

d PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

e AI: artificial intelligence.

Evaluation of Tools

For tools with free versions available, each of the researchers created an account and tested the program to determine feature presence. We also referred to user guides, publications, and training tutorials. For proprietary software, we gathered information on feature offerings from marketing webpages, training materials, and video tutorials. We also contacted all proprietary software providers to give them the opportunity to comment on feature offerings that may have been left out of those materials. Of the 8 proprietary software providers contacted, 38% (3/8) did not respond, 50% (4/8) provided feedback on feature offerings, and 13% (1/8) declined to comment. When providers provided feedback, we re-reviewed the features in question and altered the assessment as appropriate. One provider gave feedback after initial puplication, prompting issuance of a correction.

Feature assessment was completed independently by 2 reviewers (KC and AR), and all disagreements were adjudicated by a third (KK). Interobserver agreement was calculated using standard methods [ 19 ] as applied to binary assessments. First, the 2 independent assessments were compared, and the number of disagreements was counted per feature, per software. For each feature, the total number of disagreements was counted and divided by the number of software tools assessed. This provided a per-feature variability percentage; these percentages were averaged across all features to provide a cumulative interobserver agreement percentage.

Identification of SR Tools

We reviewed all 240 software tools offered on SR Toolbox and sent forward all studies that, based on the software descriptions, could meet our inclusion criteria; we then added in all software tools found on Google Scholar. This strategy yielded 53 software tools that were reviewed in full ( Figure 1 shows the PRISMA [Preferred Reporting Items for Systematic Reviews and Meta-Analyses]-based chart). Of these 53 software tools, 55% (29/53) were excluded. Of the 29 excluded tools, 17% (5/29) were built to review software engineering literature, 10% (3/29) were not functional as of August 2021, 7% (2/29) were citation managers, and 7% (2/29) were statistical packages. Other excluded tools included tools not designed for SRs (6/29, 21%), desktop applications (4/29, 14%), tools requiring users to code (3/29, 10%), a search engine (1/29, 3%), and a social science literature review tool (1/29, 3%). One tool, Research Screener [ 20 ], was excluded owing to insufficient information available on supported features. Another tool, the Health Assessment Workspace Collaborative, was excluded because it is designed to assess chemical hazards.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)-based chart showing the sources of all tools considered for inclusion, including 2-phase screening and reasons for all exclusions made at the full software review stage. SR: systematic review.

Overview of SR Tools

We assessed the presence of features in 24 software tools, of which 71% (17/24) are designed for health care or biomedical sciences. In addition, 63% (15/24) of the analyzed tools support the full SR process, meaning they enable search, screening, extraction, and export, as these are the basic capabilities necessary to complete a review in a single software tool. Furthermore, 21% (5/34) of the tools support the screening stage ( Table 3 ).

Breakdown of software tools for systematic review by process type (full process, screening, extraction, or visualization; n=24).

Data Gathering

Interobserver agreement between the 2 reviewers gathering data features was 86.46%, meaning that across all feature assessments, the 2 reviewers disagreed on <15% of the applications. Final assessments are summarized in Table 4 , and Table S2 in Multimedia Appendix 2 shows the interobserver agreement on a per–SR tool and per-feature basis. Interobserver agreement was ≥70% for every feature assessed and for all SR tools except 3: LitStream (ICF; 53.3%), RevMan Web (Cochrane; 50%), and SR Accelerator (Institute for Evidence-Based Healthcare; 53.3%); on investigation, these low rates of agreement were found to be due to name changes and versioning (LitStream and RevMan Web) and due to the modular nature of the subsidiary offerings (SR Accelerator). An interactive, updatable visualization of the features offered by each tool is available in the Systematic Review Methodologies Qualitative Synthesis.

Feature assessment scores by feature class for each systematic review tool analyzed. The total number of features across all feature classes is presented in descending order.

Giotto Compliance (27/30, 90%), DistillerSR (26/30, 87%), and Nested Knowledge (26/30, 87%) support the most features, followed by EPPI-Reviewer Web (25/30, 83%), LitStream (23/30, 77%), JBI SUMARI (21/30, 70%), and SRDB.PRO (VTS Software) (21/30, 70%).

The top 16 software tools are ranked by percent of features from highest to lowest in Figure 2 . Fewer than half of all features are supported by 7 tools: RobotAnalyst (National Centre for Text Mining), SRDR (Agency for Healthcare Research and Quality), SyRF (Systematic Review Facility), Data Abstraction Assistant (Center for Evidence Synthesis in Health, Institute for Evidence-Based Healthcare), SR-Accelerator, RobotReviewer (RobotReviewer), and COVID-NMA (COVID-NMA; Table 3 ).

Stacked bar chart comparing the percentage of supported features, broken down by their feature class (retrieval, appraisal, extraction, output, admin, and access), among all analyzed software tools.

Feature Assessment: Breakout by Feature Class

Of all 6 feature classes, administrative features are the most supported, and output and extraction features are the least supported ( Figure 3 ). Only 3 tools, Covidence (Cochrane), EPPI-Reviewer, and Giotto Compliance, offer all 4 extraction features ( Table 4 ). DistillerSR and Giotto support all 5 retrieval features, while Nested Knowledge supports all 5 documentation/output features. Colandr, DistillerSR, EPPI-Reviewer, Giotto Compliance, and PICOPortal support all 6 appraisal features.

Heat map of features observed in 24 analyzed software tools. Dark blue indicates that a feature is present, and light blue indicates that a feature is not present.

Feature Class 1: Retrieval

The ability to search directly within the SR tool was only present for 42% (10/24) of the software tools, meaning that for all other SR tools, the user is required to search externally and import records. The only SR tool that did not enable importing of records was COVID-NMA, which supplies studies directly from the providers of the tool but does not enable the user to do so.

Feature Class 2: Appraisal

Among the 19 tools that have title/abstract screening, all tools except for RobotAnalyst and SRDR+ enable dual screening and adjudication. Reference deduplication is less widespread, with 58% (14/24) of the tools supporting it. A form of machine learning/automation during the screening stage is present in 54% (13/24) of the tools.

Feature Class 3: Extraction

Although 75% (18/24) of the tools offer data extraction, only 29% (7/24) offer dual data extraction (Giotto Compliance, DistillerSR, SRDR+, Cadima [Cadima], Covidence, EPPI-Reviewer, and PICOPortal [PICOPortal]). A total of 54% (13/24) of the tools enable risk of bias assessments.

Feature Class 4: Output

Exporting references or collected data is available in 71% (17/24) of the tools. Of the 24 tools, 54% (13/24) generate figures or tables, 42% (10/24) of tools generate PRISMA flow diagrams, 32% (8/24%) have report writing, and only 13% (3/34) have in-text citations.

Feature Class 5: Admin

Protocols, customer support, and training materials are available in 71% (17/24), 79% (19/24), and 83% (20/24) of the tools, respectively. Of all administrative features, the least well developed are progress/activity monitoring, which is offered 67% (16/24) of the tools, and comments, which are available in 58% (14/24) of the tools.

Feature Class 6: Access

Access features cover both collaboration during the review, cost, and availability of outputs. Of the 24 software tools, 83% (20/24) permit collaboration by allowing multiple users to work on a project. COVID-NMA, RobotAnalyst, RobotReviewer, and SR-Accelerator do not allow multiple users. In addition, of the 24 tools, 71% (17/24) offer a free subscription, whereas 29% (7/24) require paid subscriptions or licenses (Covidence, DistillerSR, EPPI-Reviewer Web, Giotto Compliance, JBI Sumari, SRDB.PRO, and SWIFT-Active Screener). Only 54% (13/24) of the software tools support living, updatable reviews.

Principal Findings

Our review found a wide range of options in the SR software space; however, among these tools, many lacked features that are either crucial to the completion of a review or recommended as best practices. Only 63% (15/24) of the SR tools covered the full process from search/import through to extraction and export. Among these 15 tools, only 67% (10/15) had a search functionality directly built in, and only 47% (7/15) offered dual data extraction (which is the gold standard in quality control). Notable strengths across the field include collaborative mechanisms (offered by 20/24, 83% tools) and easy, free access (17/24, 71% of tools are free). Indeed, the top 4 software tools in terms of number of features offered (Giotto Compliance, DistillerSR, Nested Knowledge, and EPPI-Reviewer all offered between 83% and 90% of the features assessed. However, major remaining gaps include a lack of automation of any step other than screening (automated screening offered by 13/24, 54% of tools) and underprovision of living, updatable outputs.

Major Gaps in the Provision of SR Tools

Marshall et al [ 11 ] have previously noted that “the user should be able to perform an automated search from within the tool which should identify duplicate papers and handle them accordingly” [ 11 ]. Less than a third of tools (7/24, 29%) support search, reference import, and manual reference addition.

Study Selection

Screening of references is the most commonly offered feature and has the strongest offerings across features. All software tools that offer screening also support dual screening (with the exception of RobotAnalyst and SRDR+). This demonstrates adherence to SR best practices during the screening stage.

Automation and Machine Learning

Automation in medical SR screening has been growing. Some form of machine learning or other automation for screening literature is present in over half (13/24, 54%) of all the tools analyzed. Machine learning/screening includes reordering references, topic modeling, and predicting inclusion rates.

Data Extraction

In contrast to screening, extraction is underdeveloped. Although extraction is offered by 75% (18/24) tools, few tools adhere to SR best practices of dual extraction. This is a deep problem in the methods of review, as the error rate for manual extraction without dual extraction is highly variable and has even reached 50% in independent tests [ 16 ].

Although single extraction continues to be the only commonly offered method, the scientific community has noted that automating extraction would have value in both time savings and improved accuracy, but the field is as of yet underdeveloped. To quote a recent review on the subject of automated extraction, “[automation] techniques have not been fully utilized to fully or even partially automate the data extraction step of systematic review” [ 21 ]. The technologies to automate extraction have not achieved partial extraction at a sufficiently high accuracy level to be adopted; therefore, dual extraction is a pressing software requirement that is unlikely to be surpassed in the near future.

Project Management

Administrative features are well supported by SR software. However, there is a need for improved monitoring of review progress. Project monitoring is offered by 67% (16/24) of the tools, which is among the lowest of all admin features and likely the feature most closely associated with the quality of the outputs. As collaborative access is common and highly prized, SR software providers should recognize the barriers to collaboration in medical research; lack of mutual awareness, inertia in communication, and time management and capacity constraints are among the leading reasons for failure in interinstitutional research [ 22 ]. Project monitoring tools could assist with each of these pain points and improve the transparency and accountability within the research team.

Living Reviews

The scientific community has made consistent demands for SR processes to be rendered updatable, with the goal of improving the quality of evidence available to clinicians, health policymakers, and the medical public [ 23 , 24 ]. Despite these ongoing calls for change, living, updatable reviews are not yet standard in SR software tools. Only 54% (13/24) of the tools support living reviews, largely because living review depends on providing updatability at each step up through to outputs. However, until greater provision of living review tools is achieved, reviews will continue to fall out of date and out of sync with clinical practice [ 24 ].

Study Limitations

In our study design, we elected to use a binary assessment, which limited the bias induced by the subjective appeal of any given tool. Therefore, these assessments did not include any comparison of quality or usability among the SR tools. This also meant that we did not use the Desmet [ 25 ] method, which ranks features by level of importance. We also excluded certain assessments that may impact user choices such as language translation features or translated training documentation, which is supported by some technologies, including DistillerSR. We completed the review in August 2021 but added several software tools following reviewer feedback; by adding expert additions without repeating the entire search strategy, we may have missed SR tools that launched between August and December 2021. Finally, the authors of this study are the designers of one of the leading SR tools, Nested Knowledge, which may have led to tacit bias toward this tool as part of the comparison.

By assessing features offered by web-based SR applications, we have identified gaps in current technologies and areas in need of development. Feature count does not equate to value or usability; it fails to capture benefits of simple platforms, such as ease of use, effective user interface, alignment with established workflows, or relative costs. The authors make no claim about superiority of software based on feature prevalence.

Future Directions

We invite and encourage independent researchers to assess the landscape of SR tools and build on this review. We expect the list of features to be assessed will evolve as research changes. For example, this review did not include features such as the ability to search included studies, reuse of extracted data, and application programming interface calls to read data, which may grow in importance. Furthermore, this review assessed the presence of automation at a high level without evaluating details. A future direction might be characterizing specific types of automation models used in screening, as well as in other stages, for software applications that support SR of biomedical research.

The highest-performing SR tools were DistillerSR, EPPI-Reviewer Web, and Nested Knowledge, each of which offer >80% of features. The most commonly offered and robust feature class was screening, whereas extraction (especially quality-controlled dual extraction) was underprovided. Living reviews, although strongly advocated for in the scientific community, were similarly underprovided by the SR tools reviewed here. This review enables the medical community to complete transparent and comprehensive comparison of SR tools and may also be used to identify gaps in technology for further development by the providers of these or novel SR tools.

This review of web-based software review software tools represents an attempt to best capture information from software providers’ websites, free trials, peer-reviewed publications, training materials, or software tutorials. The review is based primarily on publicly available information and may not accurately reflect feature offerings, as relevant information was not always available or clear to interpret. This evaluation does not represent the views or opinions of any of the software developers or service providers, except those of the authors. The review was completed in August 2021, and readers should refer to the respective software providers’ websites to obtain updated information on feature offerings.

Acknowledgments

The authors acknowledge the software development team from Nested Knowledge, Stephen Mead, Jeffrey Johnson, and Darian Lehmann-Plantenberg for their input in designing Nested Knowledge. The authors thank the independent software providers who provided feedback on our feature assessment, which increased the quality and accuracy of the results.

Abbreviations

Multimedia Appendix 1

Multimedia appendix 2.

Authors' Contributions: All authors participated in the conception, drafting, and editing of the manuscript.

Conflicts of Interest: KC, NH, and KH work for and hold equity in Nested Knowledge, which provides a software application included in this assessment. AR worked for Nested Knowledge. KL works for and holds equity in Nested Knowledge, Inc, and holds equity in Superior Medical Experts, Inc. KK works for and holds equity in Nested Knowledge, and holds equity in Superior Medical Experts.

🛠️ Tools and Software for SLR

A systematic literature review (SLR) involves a comprehensive and structured approach to searching, selecting, and analyzing relevant research papers. To facilitate this process, various tools and software can be used to streamline tasks. Here is a list of tools commonly used for conducting a systematic literature review:

Reference Management Software :

EndNote : A popular reference management tool for organizing, storing, and citing references in SLRs.

Zotero : A free, open-source reference management software that helps collect, organize, and cite research materials.

Mendeley : Combines reference management, PDF annotation, and social networking features, aiding in SLR organization and collaboration.

Database Search Tools :

PubMed : A widely-used database for medical and life sciences research.

IEEE Xplore : Useful for computer science and engineering SLRs.

Web of Science : Provides access to a wide range of academic disciplines.

Scopus : Offers extensive coverage of scientific literature across various fields.

Google Scholar : A free tool for discovering scholarly articles, useful for broad searches.

Systematic Review Software :

Covidence : Specifically designed for managing systematic reviews, it facilitates screening, data extraction, and collaboration.

Rayyan : A web-based tool for systematic reviewers to screen and collaborate on study selection.

DistillerSR : DistillerSR is another proprietary platform that excels in meeting the essential criteria for systematic review work. Researchers often opt for DistillerSR due to its robust capabilities and user-friendly interface.

Screening and Data Extraction Tools :

EPPI-Reviewer : A tool for systematic review management, data extraction, and synthesis. It offers a suite of features that streamline the review process, ensuring thorough and accurate results.

JBI SUMARI : Software for systematic review, meta-analysis, and evidence synthesis. JBI SUMARI is a proprietary systematic review software known for its comprehensive functionality and user-friendly interface. Researchers favor it for its ability to handle various types of systematic reviews effectively.

Giotto Compliance : Giotto Compliance is a proprietary, web-based systematic review tool known for its comprehensive feature set, making it a preferred choice among researchers for conducting systematic reviews with high precision.

Nested Knowledge : Nested Knowledge is recognized for its effectiveness in handling complex systematic reviews. Its inclusion of a wide array of key features makes it a valuable tool for researchers tackling intricate research questions.

LitStream : LitStream stands out for its capacity to support researchers in conducting systematic reviews efficiently. Its incorporation of crucial features facilitates the review process and contributes to the quality of the results.

Citation Analysis and Visualization Tools :

VOSviewer : Used to visualize and analyze bibliographic data and co-citation networks.

CiteSpace : A tool for visualizing and analyzing trends, patterns, and emerging themes in scientific literature.

Excel or Google Sheets : Often used for data extraction and organization of study characteristics, results, and quality assessments.

Screening Forms and Templates : Custom-designed forms or templates in Microsoft Word or Google Docs for screening and data extraction.

Online Survey Tools : For collecting data on the risk of bias assessments or other relevant data points.

Statistical Software : If conducting meta-analyses, software like R or specialized meta-analysis software (e.g., RevMan ) may be needed.

Flowchart Diagram Tools : Tools like Lucidchart or draw.io can help create PRISMA flowcharts to visualize the screening process.

Collaboration and Communication Tools : Tools like Slack, Microsoft Teams, or Trello can facilitate collaboration among review team members.

Documentation and Reporting Tools : Microsoft Word or Google Docs for writing the systematic review report, adhering to PRISMA guidelines.

Screen Recording Tools : If collaboration involves virtual meetings and discussions, screen recording tools like Zoom or Microsoft Teams can be useful for documentation.

The choice of tools depends on your specific needs, budget, and preferences. Many researchers use a combination of these tools to conduct a systematic literature review efficiently and accurately.

Last updated 9 months ago

Buyer’s Guide To Literature Review Software

About this guide.

Our team has been developing literature review software for the world’s leading research organizations for over 15 years. Though the software has evolved dramatically over that period, the questions we are asked about the features and benefits of review software haven’t changed much.

In this guide, we present a comprehensive list of things to consider when evaluating a literature review software solution.

This guide will:

• Explain what literature review software does and how it is used
• Discuss where literature review software fits within the overall review process
• Provide a checklist of features to help you with the evaluation process

Who should read this guide?

If you are doing literature reviews today, you already know that they are increasingly required for regulatory compliance and safety monitoring. You also probably know that, while reviews sound simple on the surface, they are big projects that can consume significant amounts of time and resources. Doing reviews well can be a challenge.

This guide can benefit you if:

You are struggling with the amount of time it takes to conduct a review

If you are involved in the preparation of literature reviews for Clinical Evaluation Reports (CERs), Performance Evaluation Reports (PER), or if you track literature for safety monitoring, you need to be able to enforce standardized review processes and methods across your organization. Since your work could be subject to an audit, you need to be prepared.

You need to reduce the time it takes to conduct a review

You’re concerned about manual errors compromising the quality of your review

Did I make a transcription error? Did we forget to review that paper by Nosyk? Has any of my data changed? Worries like these can keep a researcher up at night and can seriously impact the quality and integrity of your review.

You’re not sure which literature review software is the best fit for you

What does literature review software do.

Today’s literature review software automates the many manual tasks involved in conducting a review. Literature reviews are process intensive and data heavy, and not so long ago they typically involved circulating paper copies of articles and screening forms to the review team who captured their work on spreadsheets.

Most reviewers currently use some form of technology to help manage the information and data in their review projects. In fact, a recent survey showed that the vast majority of reviewers still use spreadsheets at some stage of their review process.

Of course, it is possible to produce results using spreadsheets, or even paper forms. That said, each of these methods has a number of drawbacks that can have significant impact on both the quality and the volume of research produced.

Just Say No To Spreadsheets

When using spreadsheets for review tasks such as screening, data extraction, or storing references, you may find yourself dealing with some or all of the following:

• A reviewing “bottleneck” because each stage of the review must be completed before the next one is started
• Manual data entry errors that can be difficult or even impossible to catch
• Excessive manual work in checking for disagreements and creating reports
• Questions about the validity of your results due to lost files or undocumented processes

Where does literature review software fit in the process?

Literature review software is designed to reduce the manual work involved in conducting reviews and maintain a complete record of the work that’s been done on your review projects.

But how does it do this?

Once you’ve defined your research question and completed your search of relevant databases, you can typically import your search results into your literature review software and start your screening and data extraction processes.

Similar to the paper forms used in the past, literature review software uses electronic forms to record the answers to inclusion/exclusion questions. Some forms can be configured for data extraction. One of the main advantages that electronic forms provide is that they collect all your review data in one place, eliminating the need to manually cut and paste collate individual responses for processing and analysis. 

“Why input data twice when it only needs to be done once?”

Digital forms can be reused an unlimited number of times. Depending on the form and the reviewer, they can usually be completed faster than writing or typing since they can incorporate easy-to-use answer formats like checkboxes or radio buttons. They can also validate your data and even perform calculations before you submit it, giving you cleaner results and fewer errors.

Screening and data extraction are the most common review tasks facilitated by literature review software, but there are often other valuable features such as direct connection to popular databases such as PubMed, automated report generation, and reviewer roles and permissions management.

With regulatory bodies calling for continuous monitoring and assessment of safety data, having your entire review project and all its references, full text articles and audit trail stored within your literature review software can be a huge time saver when it comes time for updates.

As literature reviews have become a fundamental component of the risk management system for many organizations, they are increasingly scrutinized for thoroughness, standardized processes, and data integrity. By maintaining complete, accurate records of every reviewer action and decision, and allowing you to establish and enforce repeatable processes, literature review software makes it easier to deliver regulatory compliant, audit-ready literature reviews on time and on budget.

Top 5 Ways Systematic Review Software Can Help You

#1 compliance.

If there’s one thing that almost every reviewer wishes for, it’s more time. In our Survey of Literature Reviews, approximately one quarter of the respondents mentioned their greatest review challenge is the time involved in completing a review – to conduct searches, remove duplicates and irrelevant articles, complete screening, extract data, and prepare reports. In a recent survey of our user community, reviewers reported that literature review software reduced the time required to produce reviews by 40%-60%.

#3 Automation

No one wants to discover a mistake in their review right before – or worse, during – an audit.

Duplicate references, transcription errors, and data entry errors can skew, or even invalidate, your results. Literature review software can provide built-in automation and validation tools that dramatically reduce the potential for errors in your reviews.

#4 Compatibility

Although literature review software can help with many tasks throughout the review lifecycle, your process likely includes other tools for searching and storing references and data. You also likely need to use the information from your completed review in reports and submissions. Your literature review software should allow you to import and export your data in all the most common file formats, such as CSV, Excel, Word, PDF, RIS, and ENLX.

#5 Collaboration

Literature review software packages today are typically cloud-based and can be used from any browser on any device. With a centralized, shared data set, your team can collaborate in real time, regardless of location.

Your Literature Review Software Checklist

Deciding to adopt literature review software is more than just a monetary investment – it’s a commitment to a new way of doing things. And just like any significant purchase, it’s always a good idea to do your research first.

Make sure you conduct a thorough assessment of each of the available options to choose the software that is the best fit for your needs. Below is a list of features that may be offered by systematic review software packages.

This requirement applies to my assessement

Automatic reference updates to prevent the review from becoming out-of-date

Compatible with standard reference file types (RIS, CSV, and ENLX)

Direct integration with reference databases

Keyword highlighting for faster screening

Full-Text Retrieval

Data extraction, project management.

Real-time updates on project progress to inform stakeholders and facilitate planning

Live customer support, professional services offerings and training

Enterprise-Grade Software (High availability and redundancy,  scalable to handle hundreds of thousands of references per project, secure and regulatory compliant )

Download this Ebook

Learn more about distillersr.

Save up to 71 hours on your literature review.  

Join over 200,000 researchers and students finishing their review faster., ✓ no credit card required ✓ no time limit on free trial ✓ invite unlimited team members on paid plans, deliver your systematic literature reviews like never before, faster reviews, expert, online support.

Save an average of 35% time spent per review, or up to 71 hours per review. 

Easy to learn and use, with 24/7 support from product experts who are also seasoned reviewers!

Seamless collaboration

No expertise required.

Enable the whole review team to collaborate from anywhere.

Suitable for all levels of experienced reviewers in a variety of sectors including health, education, social science and many others.

Supporting the world's largest systematic literature review community

Researchers and students, universities, societies and hospitals, reviews started, take a look inside, watch a quick demo.

See how Covidence makes doing your review more intuitive, streamlined and fun. 

HOW IT WORKS

Start streamlining your review, import citations.

Covidence works seamlessly with your favourite reference managers like EndNote, Zotero, Refworks, Mendeley or any tool that support RIS, CSV or PubMedXML formats.

Screen titles & abstracts

Breeze through screening with keyword highlighting & a lightning quick interface. Covidence keeps full records of who voted and also supports single or dual screeners.

Upload references

Transfer PDFs stored in your reference manager to Covidence in a few clicks.

Screen full text

Decide quickly on studies in full text. Capture reasons for exclusion and any notes so you can resolve any disagreements quickly, with a click of a button.

Risk of bias

Automatically populate your risk of bias tables by highlighting and commenting on text directly in your PDF. No more cut and paste.

Data extraction & export

Create customised extraction templates to suit you, then export a single, machine-readable file that easily integrates into all the common statistics packages.

TESTIMONIALS

What other reviewers say.

We're a non-profit on a mission to streamline the review process while improving research quality.

We love our community, and they say some very nice things about us.

Syreeta Nolan @nolan_syreeta

"I love you @Covidence. I love the YouTube series Covidence put out and my librarian is my hero. Let others help you and be careful when you set Inclusion and exclusion as to when you want to adhere to it at what stage as some information may be more visible in full-text review."

Rebecca Venchiarutti

@RebeccaVenchers

"Big fan of using @Covidence for conducting systematic reviews - this is my first systematic review I'm leading from start to finish, and hard to see myself using another platform. Step by step, organised, and very user-friendly."

Stephanie Craig, PhD

@DrStephCraig

"Second using covidence. It's great for the abstract and full text review. Also helps you track rationale for removal which is needed in the methods"

Unlimited use for every organization

With no restrictions on reviews and users, Covidence gets out of the way so you can bring the best evidence to the world, more quickly.

Start delivering your systematic literature review faster and better today.

Copyright - Covidence 2020

View website

Terms & Conditions

Privacy Policy

• Resources Home 🏠
• Try SciSpace Copilot
• Search research papers
• Add Copilot Extension
• Try AI Detector
• Try Paraphraser
• Try Citation Generator
• April Papers
• June Papers
• July Papers

5 literature review tools to ace your research (+2 bonus tools)

Table of Contents

Your literature review is the lore behind your research paper . It comes in two forms, systematic and scoping , both serving the purpose of rounding up previously published works in your research area that led you to write and finish your own.

A literature review is vital as it provides the reader with a critical overview of the existing body of knowledge, your methodology, and an opportunity for research applications.

Some steps to follow while writing your review:

• Pick an accessible topic for your paper
• Do thorough research and gather evidence surrounding your topic
• Read and take notes diligently or you can use ChatPDF tool for this
• Create a rough structure for your review
• Synthesis your notes and write the first draft
• Edit and proofread your literature review

To make your workload a little lighter, there are many literature review AI tools. These tools can help you find academic articles through AI and answer questions about a research paper.  

Best literature review tools to improve research workflow

A literature review is one of the most critical yet tedious stages in composing a research paper. Many students find it an uphill task since it requires extensive reading and careful organization .

Using some of the best literature review tools listed here, you can make your life easier by overcoming some of the existing challenges in literature reviews. From collecting and classifying to analyzing and publishing research outputs, these tools help you with your literature review and improve your productivity without additional effort or expenses.

1. SciSpace

SciSpace is an AI for academic research that will help find research papers and answer questions about a research paper. You can discover, read, and understand research papers with SciSpace making it an excellent platform for literature review. Featuring a repository with over 270 million research papers, it comes with your AI research assistant called Copilot that offers explanations, summaries , and answers as you read.

Get started now:

Find academic articles through AI

SciSpace has a dedicated literature review tool that finds scientific articles when you search for a question. Based on semantic search, it shows all the research papers relevant for your subject. You can then gather quick insights for all the papers displayed in your search results like methodology, dataset, etc., and figure out all the papers relevant for your research.

Identify relevant articles faster

Abstracts are not always enough to determine whether a paper is relevant to your research question. For starters, you can ask questions to your AI research assistant, SciSpace Copilot to explore the content and better understand the article. Additionally, use the summarize feature to quickly review the methodology and results of a paper and decide if it is worth reading in detail.

Learn in your preferred language

A big barrier non-native English speakers face while conducting a literature review is that a significant portion of scientific literature is published in English. But with SciSpace Copilot, you can review, interact, and learn from research papers in any language you prefer — presently, it supports 75+ languages. The AI will answer questions about a research paper in your mother tongue.

Integrates with Zotero

Many researchers use Zotero to create a library and manage research papers. SciSpace lets you import your scientific articles directly from Zotero into your SciSpace library and use Copilot to comprehend your research papers. You can also highlight key sections, add notes to the PDF as you read, and even turn helpful explanations and answers from Copilot into notes for future review.

Understand math and complex concepts quickly

Come across complex mathematical equations or difficult concepts? Simply highlight the text or select the formula or table, and Copilot will provide an explanation or breakdown of the same in an easy-to-understand manner. You can ask follow-up questions if you need further clarification.

Discover new papers to read without leaving

Highlight phrases or sentences in your research paper to get suggestions for related papers in the field and save time on literature reviews. You can also use the 'Trace' feature to move across and discover connected papers, authors, topics, and more.

SciSpace Copilot is now available as a Chrome extension , allowing you to access its features directly while you browse scientific literature anywhere across the web.

Get citation-backed answers

When you're conducting a literature review, you want credible information with proper references.  Copilot ensures that every piece of information provided by SciSpace Copilot is backed by a direct reference, boosting transparency, accuracy, and trustworthiness.

Ask a question related to the paper you're delving into. Every response from Copilot comes with a clickable citation. This citation leads you straight to the section of the PDF from which the answer was extracted.

By seamlessly integrating answers with citations, SciSpace Copilot assures you of the authenticity and relevance of the information you receive.

2. Mendeley

Mendeley Citation Manager is a free web and desktop application. It helps simplify your citation management workflow significantly. Here are some ways you can speed up your referencing game with Mendeley.

Generate citations and bibliographies

Easily add references from your Mendeley library to your Word document, change your citation style, and create a bibliography, all without leaving your document.

Retrieve references

It allows you to access your references quickly. Search for a term, and it will return results by referencing the year, author, or source.

Add sources to your Mendeley library by dragging PDF to Mendeley Reference Manager. Mendeley will automatically remove the PDF(s) metadata and create a library entry.‌

Read and annotate documents

It helps you highlight and comment across multiple PDFs while keep them all in one place using Mendeley Notebook . Notebook pages are not tied to a reference and let you quote from many PDFs.

A big part of many literature review workflows, Zotero is a free, open-source tool for managing citations that works as a plug-in on your browser. It helps you gather the information you need, cite your sources, lets you attach PDFs, notes, and images to your citations, and create bibliographies.

Import research articles to your database

Search for research articles on a keyword, and add relevant results to your database. Then, select the articles you are most interested in, and import them into Zotero.

Add bibliography in a variety of formats

With Zotero, you don’t have to scramble for different bibliography formats. Simply use the Zotero-Word plug-in to insert in-text citations and generate a bibliography.

Share your research

You can save a paper and sync it with an online library to easily share your research for group projects. Zotero can be used to create your database and decrease the time you spend formatting citations.

Sysrev is an AI too for article review that facilitates screening, collaboration, and data extraction from academic publications, abstracts, and PDF documents using machine learning. The platform is free and supports public and Open Access projects only.

Some of the features of Sysrev include:

Group labels

Group labels can be a powerful concept for creating database tables from documents. When exported and re-imported, each group label creates a new table. To make labels for a project, go into the manage -> labels section of the project.

Group labels enable project managers to pull table information from documents. It makes it easier to communicate review results for specific articles.

Track reviewer performance

Sysrev's label counting tool provides filtering and visualization options for keeping track of the distribution of labels throughout the project's progress. Project managers can check their projects at any point to track progress and the reviewer's performance.

Tool for concordance

The Sysrev tool for concordance allows project administrators and reviewers to perform analysis on their labels. Concordance is measured by calculating the number of times users agree on the labels they have extracted.

Colandr is a free, open-source, internet-based analysis and screening software used as an AI for academic research. It was designed to ease collaboration across various stages of the systematic review procedure. The tool can be a little complex to use. So, here are the steps involved in working with Colandr.

Create a review

The first step to using Colandr is setting up an organized review project. This is helpful to librarians who are assisting researchers with systematic reviews.

The planning stage is setting the review's objectives along with research queries. Any reviewer can review the details of the planning stage. However, they can only be modified by the author for the review.

Citation screening/import

In this phase, users can upload their results from database searches. Colandr also offers an automated deduplication system.

Full-text screening

The system in Colandr will discover the combination of terms and expressions that are most useful for the reader. If an article is selected, it will be moved to the final step.

Data extraction/export

Colandr data extraction is more efficient than the manual method. It creates the form fields for data extraction during the planning stage of the review procedure. Users can decide to revisit or modify the form for data extraction after completing the initial screening.

Bonus literature review tools

SRDR+ is a web-based tool for extracting and managing systematic review or meta-analysis data. It is open and has a searchable archive of systematic reviews and their data.

7. Plot Digitizer

Plot Digitizer is an efficient tool for extracting information from graphs and images, equipped with many features that facilitate data extraction. The program comes with a free online application, which is adequate to extract data quickly.

Final thoughts

Writing a literature review is not easy. It’s a time-consuming process, which can become tiring at times. The literature review tools mentioned in this blog do an excellent job of maximizing your efforts and helping you write literature reviews much more efficiently. With them, you can breathe a sigh of relief and give more time to your research.

As you dive into your literature review, don’t forget to use SciSpace ResearchGPT to streamline the process. It facilitates your research and helps you explore key findings, summary, and other components of the paper easily.

Frequently Asked Questions (FAQs)

1. what is rrl in research.

RRL stands for Review of Related Literature and sometimes interchanged with ‘Literature Review.’ RRL is a body of studies relevant to the topic being researched. These studies may be in the form of journal articles, books, reports, and other similar documents. Review of related literature is used to support an argument or theory being made by the researcher, as well as to provide information on how others have approached the same topic.

2. What are few softwares and tools available for literature review?

• SciSpace Discover

• Mendeley

• Zotero

• Sysrev

• Colandr

• SRDR+

3. How to generate an online literature review?

The Scispace Discover tool, which offers an excellent repository of millions of peer-reviewed articles and resources, will help you generate or create a literature review easily. You may find relevant information by utilizing the filter option, checking its credibility, tracing related topics and articles, and citing in widely accepted formats with a single click.

4. What does it mean to synthesize literature?

To synthesize literature is to take the main points and ideas from a number of sources and present them in a new way. The goal is to create a new piece of writing that pulls together the most important elements of all the sources you read. Make recommendations based on them, and connect them to the research.

5. Should we write abstract for literature review?

Abstracts, particularly for the literature review section, are not required. However, an abstract for the research paper, on the whole, is useful for summarizing the paper and letting readers know what to expect from it. It can also be used to summarize the main points of the paper so that readers have a better understanding of the paper's content before they read it.

6. How do you evaluate the quality of a literature review?

• Whether it is clear and well-written.

• Whether Information is current and up to date.

• Does it cover all of the relevant sources on the topic.

• Does it provide enough evidence to support its conclusions.

7. Is literature review mandatory?

Yes. Literature review is a mandatory part of any research project. It is a critical step in the process that allows you to establish the scope of your research and provide a background for the rest of your work.

8. What are the sources for a literature review?

• Reports

• Theses

• Conference proceedings

• Company reports

• Some government publications

• Journals

• Books

• Newspapers

• Articles by professional associations

• Indexes

• Databases

• Catalogues

• Encyclopaedias

• Dictionaries

• Bibliographies

• Citation indexes

• Statistical data from government websites

9. What is the difference between a systematic review and a literature review?

A systematic review is a form of research that uses a rigorous method to generate knowledge from both published and unpublished data. A literature review, on the other hand, is a critical summary of an area of research within the context of what has already been published.

Suggested reads!

Types of essays in academic writing Citation Machine Alternatives — A comparison of top citation tools 2023

QuillBot vs SciSpace: Choose the best AI-paraphrasing tool

ChatPDF vs. SciSpace Copilot: Unveiling the best tool for your research

You might also like

Consensus GPT vs. SciSpace GPT: Choose the Best GPT for Research

Literature Review and Theoretical Framework: Understanding the Differences

Types of Essays in Academic Writing - Quick Guide (2024)

7 open source tools to make literature reviews easy

Opensource.com

A good literature review is critical for academic research in any field, whether it is for a research article, a critical review for coursework, or a dissertation. In a recent article, I presented detailed steps for doing  a literature review using open source software .

The following is a brief summary of seven free and open source software tools described in that article that will make your next literature review much easier.

1. GNU Linux

Most literature reviews are accomplished by graduate students working in research labs in universities. For absurd reasons, graduate students often have the worst computers on campus. They are often old, slow, and clunky Windows machines that have been discarded and recycled from the undergraduate computer labs. Installing a flavor of GNU Linux will breathe new life into these outdated PCs. There are more than 100 distributions , all of which can be downloaded and installed for free on computers. Most popular Linux distributions come with a "try-before-you-buy" feature. For example, with Ubuntu you can make a bootable USB stick that allows you to test-run the Ubuntu desktop experience without interfering in any way with your PC configuration. If you like the experience, you can use the stick to install Ubuntu on your machine permanently.

Linux distributions generally come with a free web browser, and the most popular is Firefox . Two Firefox plugins that are particularly useful for literature reviews are Unpaywall and Zotero. Keep reading to learn why.

3. Unpaywall

Often one of the hardest parts of a literature review is gaining access to the papers you want to read for your review. The unintended consequence of copyright restrictions and paywalls is it has narrowed access to the peer-reviewed literature to the point that even Harvard University is challenged to pay for it. Fortunately, there are a lot of open access articles—about a third of the literature is free (and the percentage is growing). Unpaywall is a Firefox plugin that enables researchers to click a green tab on the side of the browser and skip the paywall on millions of peer-reviewed journal articles. This makes finding accessible copies of articles much faster that searching each database individually. Unpaywall is fast, free, and legal, as it accesses many of the open access sites that I covered in my paper on using open source in lit reviews .

Formatting references is the most tedious of academic tasks. Zotero can save you from ever doing it again. It operates as an Android app, desktop program, and a Firefox plugin (which I recommend). It is a free, easy-to-use tool to help you collect, organize, cite, and share research. It replaces the functionality of proprietary packages such as RefWorks, Endnote, and Papers for zero cost. Zotero can auto-add bibliographic information directly from websites. In addition, it can scrape bibliographic data from PDF files. Notes can be easily added on each reference. Finally, and most importantly, it can import and export the bibliography databases in all publishers' various formats. With this feature, you can export bibliographic information to paste into a document editor for a paper or thesis—or even to a wiki for dynamic collaborative literature reviews (see tool #7 for more on the value of wikis in lit reviews).

5. LibreOffice

Your thesis or academic article can be written conventionally with the free office suite LibreOffice , which operates similarly to Microsoft's Office products but respects your freedom. Zotero has a word processor plugin to integrate directly with LibreOffice. LibreOffice is more than adequate for the vast majority of academic paper writing.

If LibreOffice is not enough for your layout needs, you can take your paper writing one step further with LaTeX , a high-quality typesetting system specifically designed for producing technical and scientific documentation. LaTeX is particularly useful if your writing has a lot of equations in it. Also, Zotero libraries can be directly exported to BibTeX files for use with LaTeX.

7. MediaWiki

If you want to leverage the open source way to get help with your literature review, you can facilitate a dynamic collaborative literature review . A wiki is a website that allows anyone to add, delete, or revise content directly using a web browser. MediaWiki is free software that enables you to set up your own wikis.

Researchers can (in decreasing order of complexity): 1) set up their own research group wiki with MediaWiki, 2) utilize wikis already established at their universities (e.g., Aalto University ), or 3) use wikis dedicated to areas that they research. For example, several university research groups that focus on sustainability (including mine ) use Appropedia , which is set up for collaborative solutions on sustainability, appropriate technology, poverty reduction, and permaculture.

Using a wiki makes it easy for anyone in the group to keep track of the status of and update literature reviews (both current and older or from other researchers). It also enables multiple members of the group to easily collaborate on a literature review asynchronously. Most importantly, it enables people outside the research group to help make a literature review more complete, accurate, and up-to-date.

Wrapping up

Free and open source software can cover the entire lit review toolchain, meaning there's no need for anyone to use proprietary solutions. Do you use other libre tools for making literature reviews or other academic work easier? Please let us know your favorites in the comments.

Related Content

How Literature Management Software Streamlines and Future-Proofs Systematic Review Workflows

The volume of published scientific and medical articles has exploded over the years, making the systematic literature review (SLR) process challenging for research organizations. They must review tens of thousands of research findings, often using disparate tools and manual workflows.

The National Library of Medicine reports that SLRs can cost research organizations hundreds of thousands of dollars annually. That’s because completing manual SLR workflows increases the chances of errors and slows internal productivity.

Literature management software (LMS) solutions can make the stages of a systematic review workflow more affordable, efficient, organized, and accurate. In this guide, we’ll explain how LMS platforms help you quickly find the right literature in the review process and set your business up for long-term success.

What is a Systematic Literature Review workflow?

Systematic literature review workflows are rigorous, evidence-based processes involving multiple subject matter experts within an organization. These experts meticulously evaluate research findings to align and agree on answers to specific research questions. Whether SLR workflows are manual or automated using literature review management software, they follow strict methodologies to ensure reproducible and unbiased outcomes.

How Literature Management Software Improves Workflows

Literature management software for systematic reviews can help organizations streamline and future-proof the process by:

Automating manual or disparate tools

Excel spreadsheets are often used to track and organize documents in manual systematic literature processes. Businesses also import documents from one platform into another to evaluate the relevance of research findings at different literature review stages. This complex and time-consuming workflow is no longer sustainable, given that thousands of articles come out weekly.

Integrated systematic review automation tools like ReadCube streamline the SLR process—from identifying and retrieving documents to managing references and shared libraries, screening and reviewing references, and creating reports using PRISMA charts and citation software.

Decreasing internal costs

Organizations save money by reducing the time investment required to complete the process— while boosting team productivity—when they automate and integrate the multi-stage workflow using one systematic review tool.

Facilitating team collaboration

Multiple reviewers may have conflicting opinions about whether an article should be included in a systematic literature review. Research workflow management solutions make it easy to spot conflicts and automatically flag them to reviewers, allowing teams to connect and make a group decision while ensuring productive collaboration.

Teams can also create and share project and reference libraries within some LMS platforms. Likewise, team leads can assign specific projects and literature reviews to colleagues directly from the LMS solution. Once a literature review is complete, team leads can email included references directly to colleagues to create reports and presentations.

Reducing the chance of errors

LMS solutions streamline and simplify industry-standard review procedures, reducing the risk of errors and improving team efficiencies. The accuracy and scalability provided by a single automated SLR workflow can also improve the discovery of novel solutions, such as drug discovery in the pharmaceutical sector.

Leveraging AI developments

Some LMS solutions now offer configurable AI capabilities to future-proof and support organizations with increasing volumes of information and stringent regulatory demands.

The Future of Systematic Literature Reviews

LMS platforms help you navigate the challenges of increasing article volumes and regulatory demands, ensuring better systematic literature review management. Literature Review by ReadCube is a systematic, simplified end-to-end platform that can streamline these workflows with efficiency and automation to set your business up for long-term success.

The ReadCube platform also improves team collaboration and reduces systematic literature review process errors. Furthermore, ReadCube customer support can help businesses set up custom workflows to optimize their systematic literature review software outcomes.

Download ReadCube’s “ Future of Systematic Literature Reviews ” white paper to learn more.

Published 08/29/2024 by Rachel Segal in Blog ,

Monday, June 14, 2021

Literature Reviews and Reference Management with MAXQDA

Literature Reviews with MAXQDA

Your comprehensive guide to working with bibliographic data and creating literature reviews with maxqda., get your copy now.

Download the guide

Start organizing your literature review and analysis with our new guide

More and more researchers use reference management software to keep track of the vast amount of literature they come across. Beyond allowing you to easily organize your literature and manage bibliographical data and excerpts, MAXQDA actually lets you treat everything as data! These unique features allow you not only to automatically create lists of references for your own publications, but to analyze and visualize patterns in your literature by applying the full range of qualitative, quantitative and Mixed Methods features that MAXQDA is famous for. This feature has been available since MAXQDA 2018.

Import bibliographical data into MAXQDA

Reference management software generally has a core functionality that only allows you to import, organize and export bibliographical data. With MAXQDA, you can import RIS-files from your university’s online catalog, or from programs such as Zotero, Endnote or Citavi.

Every book, article or other reference is treated as one document. The content of the RIS tags is inserted into the document and automatically coded. Additionally, the five most important tags (type, author, title, ID, and year of publication) are added as document variables.

Search and interrogate your data

This, for example, enables you to apply an advanced lexical search to all the abstracts of a specific time or by a specific author. You can also compare the frequency of certain terms using a dictionary, or visualize the co-occurrence of certain themes. Our advanced Mixed Methods features and tools for statistical analysis open up infinite possibilities to really find out what your literature is all about.

To store and manage your excerpts, you can either add memos or enter the text directly into the document. Of course, you can also export your bibliographical data in a RIS-format, in order to import it into programs such as Word.

You are currently viewing a placeholder content from YouTube . To access the actual content, click the button below. Please note that doing so will share data with third-party providers.

Editor’s note: this post has been updated from its original version published in March 2017.

MAXQDA Newsletter

Our research and analysis tips, straight to your inbox.

Similar Articles

• #ResearchforChange Grants (46)
• Conferences & Events (32)
• Field Work Diary (39)
• Learning MAXQDA (110)
• Research Projects (132)
• Tip of the Month (57)
• Uncategorized (12)
• Updates (66)
• VERBI News (71)

You are currently viewing a placeholder content from Brevo . To access the actual content, click the button below. Please note that doing so will share data with third-party providers.

You need to load content from reCAPTCHA to submit the form. Please note that doing so will share data with third-party providers.

You are currently viewing a placeholder content from Facebook . To access the actual content, click the button below. Please note that doing so will share data with third-party providers.

University of Tasmania, Australia

Systematic reviews for health: tools for systematic review.

• Handbooks / Guidelines for Systematic Reviews
• Standards for Reporting
• Registering a Protocol
• Tools for Systematic Review
• Online Tutorials & Courses
• Books and Articles about Systematic Reviews
• Finding Systematic Reviews
• Critical Appraisal
• Library Help
• Bibliographic Databases
• Grey Literature
• Handsearching
• Citation Searching
• 1. Formulate the Research Question
• 2. Identify the Key Concepts
• 3. Develop Search Terms - Free-Text
• 4. Develop Search Terms - Controlled Vocabulary
• 5. Search Fields
• 6. Phrase Searching, Wildcards and Proximity Operators
• 7. Boolean Operators
• 8. Search Limits
• 9. Pilot Search Strategy & Monitor Its Development
• 10. Final Search Strategy
• 11. Adapt Search Syntax
• Documenting Search Strategies
• Handling Results & Storing Papers

Tools for Systematic Reviews

Managing the selection process can be challenging, particularly in a large-scale systematic review that involves multiple reviewers. There are various free and subscription-based tools available that support the study selection process ( Cochrane Handbook, 4.6.6.1 ).

This page describes various tools available to help conduct a systematic review. The University of Tasmania has access to EndNote, Covidence and JBI SUMARI.

Covidence   is an online systematic review program developed by, and for, systematic reviewers. It can import citations from reference managers like EndNote, facilitate the screening of abstracts and full-text, populate risk of bias tables, assist with data extraction, and export to all common formats.

Covidence Demo video [3:24]

Covidence is a core component of Cochrane's review production toolkit and has also been endorsed by JBI.

Access to UTAS Covidence account

If you are the project leader, follow these steps to create a UTAS Covidence account:

• Create a UTAS Covidence account Step-by-step instructions on how to create at UTAS Covidence account

Once you have created your UTAS Covidence account, you can create a review and invite others to join the review.

If you are not the project leader, please wait for your invitation from your project leader to join the review (you don't need to create a UTAS Covidence account).  

Covidence Training & Help

• Need some help getting started? Covidence offers regular  training webinars . You also have the option to listen to the recording of a recent session.
• Work through the content of the Covidence Academy .
• Subscribe to the Covidence YouTube channel to find video tutorials and recorded webinars. The Step-by-step webinars playlist is particularly useful for in-depth guidance.
• Head to the Covidence knowledge base to get answers to FAQs.
• Contact [email protected] anytime to answer any of your questions.

Abstrackr   is a software for semi-automated abstract screening for systematic reviews. At present, Abstrackr is a free, open-source tool for facilitating the citation screening process. Upload your abstracts, invite reviewers, and get to screening!

Rayyan  is a free online tool that can be used for screening and coding of studies in a systematic review. It uses tagging and filtering to code and organise references.

The System for the Unified Management, Assessment and Review of Information ( SUMARI ) is  JBI 's software for the systematic review of literature.

I t is designed to assist researchers to conduct systematic reviews and facilitates the entire review process. SUMARI supports 10 review types. It is especially useful for new review types and qualitative reviews.

University of Tasmania researchers have access to SUMARI via the JBI EBP Database  under EBP Tools .

• JBI EBP Database via Ovid This link opens in a new window

SUMARI support:

• JBI SUMARI Video Tutorials
• JBI SUMARI Knowledge Base

Systematic Review Accelerator

The Systematic Review Accelerator (SRA) is a suite of automation tools developed by the Institute for Evidence-Based Healthcare at  Bond University. The SRA tools aim to make literature review and synthesis processes faster while maintaining and enhancing quality. The suite includes tools that can help with designing search strategies, title and abstract screening, citation tracking, and writing drafts for  methods and result sections.

The SRA tools are free and include extensive help pages .  

RevMan 5 is the software used for preparing and maintaining Cochrane Reviews. RevMan facilitates preparation of protocols and full reviews, including text, characteristics of studies, comparison tables, and study data. It can perform meta-analysis of the data entered, and present the results graphically.

RevMan 5 is no longer being developed, but they continue to support Cochrane authors.

RevMan Web   is the next generation of Cochrane's software for preparing and maintaining systematic reviews.  This web-based version of RevMan works across all platforms, is installation-free, and automatically updated. 

DistillerSR

DistillerSR is a systematic review software. It was designed from the ground up to provide a better review experience, faster project completion and transparent, audit-ready results.

What can you do in DistillerSR? Upload your references from any reference management software, create screening and data extraction forms, lay out workflow and assign reviewers, monitor study progress and review process, export results (incl PRISMA flowchart automation).

This software is more sophisticated and a bit harder to learn. DistillerSR attracts a fee .

The Systematic Review Toolbox is a community-driven, searchable, web-based catalogue of tools that support the systematic review process across multiple domains. The resource aims to help reviewers find appropriate tools based on how they provide support for the systematic review process. Users can perform a simple keyword search (i.e. Quick Search) to locate tools, a more detailed search (i.e. Advanced Search) allowing users to select various criteria to find specific types of tools and submit new tools to the database.

Need More Help? Book a consultation with a  Learning and Research Librarian  or contact  [email protected] .

• << Previous: Registering a Protocol
• Next: Online Tutorials & Courses >>
• Last Updated: Sep 3, 2024 10:18 AM
• URL: https://utas.libguides.com/SystematicReviews

MSU Libraries

• Need help? Ask Us

Systematic & Advanced Evidence Synthesis Reviews

• Our Services
• Choosing A Review Type
• Conducting A Review
• Systematic Reviews
• Scoping & Other Types of Advanced Reviews

Online Toolkits & Workbooks

Search strategies and citation chaining, citation management, deduplication, bibliography creation, and cite-while-you-write, screening results, creating prisma compliant flow charts, data analysis & abstraction, total workflow sr products, writing a manuscript.

• Contact Your Librarian For Help

This page lists commonly used software for Systematic Review's (SRs) and other advanced evidence synthesis reviews and should not be taken as MSU Libraries endorsing one program over another. The sections of the guide list fee-based as well as free and open-source software for different aspects of the review workflow.  All-inclusive workflow products are listed in this section.

• Wanner, Amanda. 2019. Getting started with your systematic or scoping review: Workbook & Endnote Instructions. Open Science Framework. This is a librarian created workbook on OSF that includes a pretty comprehensive workbook that walks you through all the steps and stages of creating a systematic or scoping review.
• What review is right for you? This tool is designed to provide guidance and supporting material to reviewers on methods for the conduct and reporting of knowledge synthesis. As a pilot project, the current version of the tool only identifies methods for knowledge synthesis of quantitative studies. A future iteration will be developed for qualitative evidence synthesis.
• Systematic Review Toolkit The Systematic Review Toolbox is a web-based catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. The toolbox aims to help researchers and reviewers find the following: Software tools, Quality assessment / critical appraisal checklists, Reporting standards, and Guidelines.

It is highly recommended that researchers partner with the academic librarian for their specialty to create search strategies for systematic and advanced reviews. Many guidance organizations recognize the invaluable contributions of information professionals to creating search strategies - the bedrock of synthesis reviews.

• Visualising systematic review search strategies to assist information specialists
• Gusenbauer, M., & Haddaway, N. R. (2019). Which Academic Search Systems are Suitable for Systematic Reviews or Meta‐Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 other Resources. Research Synthesis Methods.
• Citation Chaser Forward citation chasing looks for all records citing one or more articles of known relevance; backward citation chasing looks for all records referenced in one or more articles. This tool automates this process by making use of the Lens.org API. An input article list can be used to return a list of all referenced records, and/or all citing records in the Lens.org database (consisting of PubMed, PubMed Central, CrossRef, Microsoft Academic Graph and CORE)

How do you track your integration and resourcing for projects that require systematic searching, like systematic or scoping reviews? What, where, and how should you be tracking? Using a tool like Air Table can help you stay organized.

• Airtable for Systematic Search Tracking Talk from Whitney Townsend at the University of Michigan - April 6, 2022

Having a software program that can store citations from databases, deduplicating your results, and automating the creation and formatting of citations and a bibliography using a cite-while-you-write plugin will save a lot of time when doing any literature review. The software listed below can do all of these functions which are not found in the fee-based total systematic review workflow products.

You could also do most of the components of an SR in these software including screening.

• Endnote Guide Endnote Online is free and has basic functionality like importing citations and cite-while-you-write for Microsoft Word. The desktop version of Endnote is a separate individual purchase and is more robust then the online version particularly for organization of citations and ease of use with large citation libraries.
• Mendeley Guide Mendeley has all the standard features of a citation manager with the addition of a social community of scholars. Mendeley can be sluggish with large file sizes of multiple thousands of citations and the free version has limited collaborative features.

Screening the titles, abstracts, and full text of your results is one of the most time consuming components of any review. There are easy-to-use free software for this process but they won't have features like automatically creating the flow charts and inter-rater reliability kappa coefficient that you need to report in your methodology section. You will have to do this by hand.

Deduplication of results before importation into one of these tools and screening should be done in a citation management program like Endnote, Mendeley, or Zotero.

• Abstrackr Created by Brown University, Abstrackr is one of the best known and easiest to use free tools for screening results.
• Colandr Colandr is an open source screening tool. Like Abstrackr, deduplication is best done in a citation manager and then results imported into Colandr. There is a learning curve to this tool but a vibrant user community does exist for troubleshooting.
• Rayyan Built by the Qatar Foundation. It is a free web-tool (Beta) designed to help researchers working on systematic reviews and other knowledge synthesis projects. It has simple interface and a mobile app for screening-on-the-go.
• PRISMA Diagram Generator Using the PRISMA Diagram Generator you can produce a diagram easily in any of 10 different formats. The official PRISMA website only has the format as a .docx or .pdf option. Using the generator the diagram is produced using the Open Source dot program (part of graphviz), and this tool provides the source for your diagram if you wish to further tweak your diagram.
• PRISMA 2020: R Package and ShinyApp This free, online tool makes use of the DiagrammeR R package to develop a customisable flow diagram that conforms to PRISMA 2020 standards. It allows the user to specify whether previous and other study arms should be included, and allows interactivity to be embedded through the use of mouseover tooltips and hyperlinks on box clicks.

Tools for data analysis can help you categorize results such as outcomes of studies and perform metanalyses. The SRDR tool may be the easiest to use and has frequent functionality updates.

• OpenMeta[Analyst] Developed by Brown University using an AHRQ grant, OpenMeta[Analyst] is a no-frills approach to data analysis.
• SRDR Developed by the AHRQ, The Systematic Review Data Repository (SRDR) is a powerful and easy-to-use tool for the extraction and management of data for systematic review or meta-analysis. It is also an open and searchable archive of systematic reviews and their data.

Data abstraction commonly refers to the extraction, synthesis, and structured visualization of evidence characteristics. Evidence tables/table shells/reading grids are the core way article extraction analyses are displayed. It lists all the included data sources and their characteristics according to your inclusion/exclusion criteria. Tools like Covidence have modules to create your own data extraction form and export a table when finished.

• OpenAcademics: Reading Grid Template
• The National Academies Press: Sample Literature Table Shells
• Campbell Collaboration: Data Extraction Tips

There are several fee-based products that are a one-stop-shop for systematic reviews. They complete all the steps from importing citations, deduplicating results, screening, bibliography management, and some even perform metanalyses. These are best used by teams that have grant or departmental funding because they can be rather expensive. 

None of these tools offers a robust bibliography creation function or cite-while-you write option. You will still need to use a separate citation manager to do these aspects of review writing. We list commonly used citation management tools on this page.

• EPPI-Reviewer 4 EPPI-Reviewer 4 is a web-based software program for managing and analysing data in literature reviews. It has been developed for all types of systematic review (meta-analysis, framework synthesis, thematic synthesis etc) but also has features that would be useful in any literature review. It manages references, stores PDF files and facilitates qualitative and quantitative analyses such as meta-analysis and thematic synthesis. It also contains some new ‘text mining’ technology which is promising to make systematic reviewing more efficient. It also supports many different analytic functions for synthesis including meta-analysis, empirical synthesis and qualitative thematic synthesis. It does not have a bibliographic manager or cite-while-you-write feature.
• JBI-SUMARI Currently, this tool can only accept Endnote XML files for citation import. So you would need to download citations to Endnote, import them into SUMARI, and when screening is complete use Endnote as your bibliographic manager for any writing. SUMARI supports 10 review types, including reviews of effectiveness, qualitative research, economic evaluations, prevalence/incidence, aetiology/risk, mixed methods, umbrella/overviews, text/opinion, diagnostic test accuracy and scoping reviews. It facilitates the entire review process, from protocol development, team management, study selection, critical appraisal, data extraction, data synthesis and writing your systematic review report.

Using Excel

Some teams may choose to use Excel for their systematic review. This is not recommended because it can be extremely time consuming and is more prone to error. However, there is a basic template for Excel-based SR's online that is good quality and walks one through the entire workflow of completing an SR (excluding bibliography creation and citation management).

• PIECES Workbook This link will open an Excel workbook designed to help conduct, document, and manage a systematic review. Made by Margaret J. Foster, MS, MPH, AHIP Systematic Reviews Coordinator Associate Professor Medical Sciences Library, Texas A&M University
• Systematic Review Accelerator: Methods Wizard An tool to help you write consistent, reproducible methods sections according to common reporting structures.
• PRISMA Extensions Each PRISMA reporting extension has a manuscript checklist that lays out exactly how to write/report your review and what information to include.
• << Previous: Scoping & Other Types of Advanced Reviews
• Next: Contact Your Librarian For Help >>
• Last Updated: May 14, 2024 2:50 PM
• URL: https://libguides.lib.msu.edu/systematic_reviews

Literature Review Simplified

For Perfectionist Medical Writers On A Deadline

Your Time Is Too Valuable for Copy-Paste

We all know Systematic literature review is an essential part of any regulatory process. We also know it’s a complete labor-intensive pain to perform and maintain without making mistakes.

That’s why we built CiteMed , to keep our own medtech writing team from setting fire to their laptops trying to keep up with generating error-free SLRs day in and day out.

We didn’t have time to onboard or learn (or pay for) some absurdly complex enterprise package of software, so we built something simple. Our users can:

• Fire up a new review (or update a previous one) in 60 seconds. Less if you’ve had coffee.
• Run searches (on multiple databases), and automatically remove/record duplicates
• Perform a compliant (EU MDR/IVDR, FDA) literature review of abstracts and full-texts
• Click once to generate a submission-ready output (Word Doc and Tables), or Excel/RIS Archive

Built to Search, Review and Output. Not to Confuse

CiteMed.io was designed for you and your team to sit down and start reviewing. That’s it. You’re not going to find 3 weeks worth of training videos, multi-hour ‘strategy-calls’, or powerpoint presentations made by consultants who have never written so much as a postcard.

We know our workflows are great, because we’ve used them to successfully submit 100s of our own reviews and documents (see our track record at Cite Medical! ). So we suggest you give us a shot and see if our tool can work for you too. Free evaluation periods are always available to try it out.

P.S. If you’re a big team with a complex workflow, we’re not opposed to building you something special to suit your needs… but it won’t be today. CiteMed.io as it’s currently designed is for reviewers that are ready to get traction immediately.

15 Regulatory Experts and Software Engineers Walk Into A Bar…

We built CiteMed to solve our own problem, which was juggling a mountain of Systematic Literature Reviews for our medical device clients. It’s been forged through our mistakes, aspirations, and ideas over the course of 7 years and we don’t intend on stopping or selling out.

• Family owned and operated for 8 years and counting
• Zero pesky investors or controlling interests (we’re here to stay!)
• Built by EU MDR/IVDR/FDA medical writers at every step.

Enterprise Quality at Startup Speed

Whether you’re a department of one, or managing a team of 50 writers we’re set up to get you results without the fuss of enterprise level configuration and training. If you’re committed to performing the highest caliber of literature review (in any industry), we’ve got you covered.

For freelancers and small teams:

• Start today,  with  a monthly license
• Book in time directly with our Founders (seriously) to walk through your first real project.  
• Get reviewing in minutes!

For Enterprise and Big Teams:

• You can still start today (minus the red tape for licenses at your company)
• See our enterprise ready environment and compliance checklist to slash through that red tape
• Concierge on-boarding ( we’ll set up your projects for you and train your staff, free)

Beyond a Tool - Industry Leading Education

We don’t want to just send you a login and say ‘Ciao Ciao’, our experienced writing team (hundreds of successful submissions) continues to put together best-in-class instructionals on the latest Regs in EU and US, and a strong focus on evidence and literature review.

• Sign up for our free Systematic Literature Review Certification program
• Check out our whitepapers and extensive articles here
• Watch/Listen to our Podcasts scattered across the regulatory world

Better evidence means safer products, and healthier outcomes. Join us in the never-ending quest to process and evaluate evidence.

© 2024 CiteMed. All Rights Reserved.

• Book a Demo

Systematic Literature Review Tools – How To Choose One

A variety of factors (e.g., the new EU MDR & IVDR requirements, finding and retaining top talent, and tight budgets, just to name a few) have influenced Medical Device and Diagnostic manufacturers to invest in technology that streamlines and automates required compliance activities. Systematic Literature Review tools (SLR tools) can offer users many advantages over using “manual” methods such as Excel. However, with more and more SLR tools entering the market to meet the growing demand of the medical device industry, how do you know where to start?

The Celegence team is here to help. The following blog details six things to look for when choosing a systematic literature review tool to meet the MDR/IVDR requirements for your medical device or diagnostic portfolio.

#1: Ease of Use and Simple User Interface (UI)

Conducting a systematic literature review is quite time intensive. In fact, research shows that systematic literature reviews can take anywhere between 6 to 24 months to complete.[1–3] And, depending on the size of your team and product portfolio, you will likely need to execute multiple projects simultaneously. You will spend a lot of time using your chosen SLR tool or platform, so it should be easy to use and navigate. Furthermore, the composition of regulatory team members associated with any given project may change over time. Medical writers may transition, new reviewers could be assigned, and additional stakeholders from cross-functional teams might need to contribute. Given this dynamic environment, make sure that you look for a tool that is intuitive, recognizing the time commitment to implement it into your workflow. Review the provider’s training program too, ensuring ongoing support for your team well after the initial implementation.

The checklist highlights all of the documentation that you will need in place for certification of your IVD device and will serve as a guide to help you achieve ongoing compliance. In conjunction with this checklist, we are also able to provide you with bespoke strategies to bring your business up to speed. We are currently working with businesses from the United States, India, and throughout Europe to ensure that they are ready for the deadline in May of 2022.

#2: Systematic Literature Review Tools Collaboration Features

Systematic reviews are a collaborative effort. Any given systematic literature review may include multiple medical writers, inputs from medical and subject matter experts, and reviewers from regulatory, quality, and clinical departments. Your organization might even be outsourcing different regulatory functions to an external firm or consultant, so adopting an SLR tool that can accommodate internal and external users will be key. More importantly, your chosen platform should possess features that facilitate collaboration. Some key collaboration features you may want to consider include:

• Comments and Direct Messages: An avenue for threaded discussions and direct communication adds depth to collaboration.
• Full Audit Trail: Think of it as a digital breadcrumb trail – essential for traceability, discussions, and learning as projects evolve.
• Customizable Review Workflows: Flexibility to tailor workflows to your team’s unique dynamics ensures smooth progress.
• Dashboard Insights: Quick-glance dashboards brimming with project stats empower managers and executives to steer the ship confidently.

Consider all the key players involved in the post-market surveillance process for your device portfolio, document their needs, and review these needs against the feature set of the platform and the support available from the provider.

#3: Validated System

The medical device industry is highly regulated to ensure that the products that reach patients are safe and efficacious and work as intended. Once a product reaches the market, device and diagnostic manufacturers follow stringent policies and procedures to carry out the necessary post market surveillance (PMS) activities to collect, analyze, and interpret a massive amount of data about the product’s usage, performance, safety, etc.

Your chosen SLR tool will play a vital role in the ongoing lifecycle management activities of your marketed products. It will help you prove to regulators, through required PMS reports, that your device continues to be safe and efficacious. The stakes are extremely high when it comes to the data collected and reported upon within the PMS reports for any given device.

The Software as a Service (SaaS) model is becoming increasingly common within the life science industry, with several service providers available that offer platforms and tools that support the systematic literature review process. The SaaS model benefits end users by providing access to solutions that are less expensive and cheaper to maintain as the responsibility of software maintenance rests with the service provider. Nonetheless, this arrangement means that the SaaS provider retains the authority to alter and enhance elements such as the interface and functionality of the system, according to their timeline rather than yours.

As a buyer, you should seek out a validated system to ensure that the tool meets the needs of your team and works the way that it is supposed to work. Many tools that you consider will likely fall under the SaaS category. It is paramount to ascertain that the SaaS provider has a robust validation program in place that covers the process, coverage, and services that you are purchasing.

#4: Systematic Literature Review Tool Scalability

As you survey the landscape for potential SLR tools to implement, consider the current state of your team but also the future state of your organization. A cheaper “quick fix” solution may be tantalizing initially, but the solution should not only address your immediate needs, but also easily scale to support a growing product portfolio and align with your firm’s goals over the next three, five, and years beyond.

#5: SLR Tool Customer Support and Software Development

Unfortunately, no software tool is perfect, and bugs are bound to arise due to any number of conditions. It goes without saying how important PMS activities and meeting reporting deadlines for internal and external stakeholders (global Health Authorities, Notified Bodies , etc.) are for the lifecycle management of any medical product. Strong customer support will allow your team to navigate any potential hiccups that might occur with your SLR tool.

During the vetting process, ask the potential vendors about their standard response times, how often new releases or patches are issued, etc. to ensure that they are dedicated to responding to any issues that you may encounter. In addition to a strong customer support process, you should also inquire about the software vendor’s development team. You can ask questions such as: What is on the development roadmap? What features are planned to be implemented and when? How does the tool adapt to evolving regulatory requirements? How often is user feedback considered in the development cycle?

You may find that new features they’re brewing up could be a total game-changer for your team and the process that your organization follows when performing systematic literature reviews. Choosing a partner that will consider and adapt to your needs is pivotal, so don’t shy away from asking for examples or references that indicate strong customer and development support.

#6: Regulatory Expertise

A successful regulatory affairs department requires equal parts science and art; strong scientific or technical expertise is important, but so is the ability able to interpret and act upon the regulatory policy and guidance. When searching for the perfect SLR tool, you’re bound to find several tools offered by technology providers outside of the life science industry, or those that do not have regulatory experts on staff. Tread lightly.

Picture this: a vendor that employs a team of medical writers and regulatory professionals engaged in the same work that you will be doing on the platform. This will provide you with access to their goldmine of best practices and learnings from their own interactions with Notified Bodies. Furthermore, a vendor who routinely provides regulatory services to manufacturers in the life science industry will understand the challenges of your team and implement creative solutions into the tool to address these.

Researching SLR Tools

Researching and selecting the right tool for your team can be an arduous task. But fear not, these six tips will ease the burden of implementing a new SLR system for your organization. If you’ve any questions feel free to reach out to our expert team.

Meet CAPTIS  – AI Powered Systematic Literature Review Tools

Speaking of SLR software , meet CAPTIS – the end-to-end EU MDR/IVDR compliance platform developed by Celegence for the life science industry. CAPTIS was initially conceived to automate manual tasks and facilitate project collaboration for Celegence’s own medical writers, who still routinely perform systematic literature reviews and author PMS reports for medical device and diagnostic manufacturers.

Through regular monthly meetings with all CAPTIS users, Celegence actively listens to users’ needs and shares best practices for regulatory compliance using the tool. Moreover, user feedback, whether from real-world use or insights from Notified Body interactions, frequently shapes new features that benefit all customers. CAPTIS is a validated system with several automated features and an AI Assistant – CAPTIS Copilot , that helps users perform systematic reviews and create PMS report documentation in record time. To see if CAPTIS will be a good fit for your team, drop us a line today to book a demo .

Related Posts

New FDA Regulations for Laboratory Developed Tests (LDTs): What It Means to Laboratories

RAPS Regulatory Convergence 2024

Mastering PMCF and PMPF for Regulatory Compliance

Privacy overview.

Please complete the form:

• *I consent to celegence.com using my data from this form in accordance with their privacy policy.
• Phone This field is for validation purposes and should be left unchanged.

And our team will get back to you soon.

Never see this message again.