by identifying a gap in the existing literature a researcher

• Privacy Policy

Home » Research Gap – Types, Examples and How to Identify

Research Gap – Types, Examples and How to Identify

Table of Contents

Research Gap

Definition:

Research gap refers to an area or topic within a field of study that has not yet been extensively researched or is yet to be explored. It is a question, problem or issue that has not been addressed or resolved by previous research.

How to Identify Research Gap

Identifying a research gap is an essential step in conducting research that adds value and contributes to the existing body of knowledge. Research gap requires critical thinking, creativity, and a thorough understanding of the existing literature . It is an iterative process that may require revisiting and refining your research questions and ideas multiple times.

Here are some steps that can help you identify a research gap:

• Review existing literature: Conduct a thorough review of the existing literature in your research area. This will help you identify what has already been studied and what gaps still exist.
• Identify a research problem: Identify a specific research problem or question that you want to address.
• Analyze existing research: Analyze the existing research related to your research problem. This will help you identify areas that have not been studied, inconsistencies in the findings, or limitations of the previous research.
• Brainstorm potential research ideas : Based on your analysis, brainstorm potential research ideas that address the identified gaps.
• Consult with experts: Consult with experts in your research area to get their opinions on potential research ideas and to identify any additional gaps that you may have missed.
• Refine research questions: Refine your research questions and hypotheses based on the identified gaps and potential research ideas.
• Develop a research proposal: Develop a research proposal that outlines your research questions, objectives, and methods to address the identified research gap.

Types of Research Gap

There are different types of research gaps that can be identified, and each type is associated with a specific situation or problem. Here are the main types of research gaps and their explanations:

Theoretical Gap

This type of research gap refers to a lack of theoretical understanding or knowledge in a particular area. It can occur when there is a discrepancy between existing theories and empirical evidence or when there is no theory that can explain a particular phenomenon. Identifying theoretical gaps can lead to the development of new theories or the refinement of existing ones.

Empirical Gap

An empirical gap occurs when there is a lack of empirical evidence or data in a particular area. It can happen when there is a lack of research on a specific topic or when existing research is inadequate or inconclusive. Identifying empirical gaps can lead to the development of new research studies to collect data or the refinement of existing research methods to improve the quality of data collected.

Methodological Gap

This type of research gap refers to a lack of appropriate research methods or techniques to answer a research question. It can occur when existing methods are inadequate, outdated, or inappropriate for the research question. Identifying methodological gaps can lead to the development of new research methods or the modification of existing ones to better address the research question.

Practical Gap

A practical gap occurs when there is a lack of practical applications or implementation of research findings. It can occur when research findings are not implemented due to financial, political, or social constraints. Identifying practical gaps can lead to the development of strategies for the effective implementation of research findings in practice.

Knowledge Gap

This type of research gap occurs when there is a lack of knowledge or information on a particular topic. It can happen when a new area of research is emerging, or when research is conducted in a different context or population. Identifying knowledge gaps can lead to the development of new research studies or the extension of existing research to fill the gap.

Examples of Research Gap

Here are some examples of research gaps that researchers might identify:

• Theoretical Gap Example : In the field of psychology, there might be a theoretical gap related to the lack of understanding of the relationship between social media use and mental health. Although there is existing research on the topic, there might be a lack of consensus on the mechanisms that link social media use to mental health outcomes.
• Empirical Gap Example : In the field of environmental science, there might be an empirical gap related to the lack of data on the long-term effects of climate change on biodiversity in specific regions. Although there might be some studies on the topic, there might be a lack of data on the long-term effects of climate change on specific species or ecosystems.
• Methodological Gap Example : In the field of education, there might be a methodological gap related to the lack of appropriate research methods to assess the impact of online learning on student outcomes. Although there might be some studies on the topic, existing research methods might not be appropriate to assess the complex relationships between online learning and student outcomes.
• Practical Gap Example: In the field of healthcare, there might be a practical gap related to the lack of effective strategies to implement evidence-based practices in clinical settings. Although there might be existing research on the effectiveness of certain practices, they might not be implemented in practice due to various barriers, such as financial constraints or lack of resources.
• Knowledge Gap Example: In the field of anthropology, there might be a knowledge gap related to the lack of understanding of the cultural practices of indigenous communities in certain regions. Although there might be some research on the topic, there might be a lack of knowledge about specific cultural practices or beliefs that are unique to those communities.

Examples of Research Gap In Literature Review, Thesis, and Research Paper might be:

• Literature review : A literature review on the topic of machine learning and healthcare might identify a research gap in the lack of studies that investigate the use of machine learning for early detection of rare diseases.
• Thesis : A thesis on the topic of cybersecurity might identify a research gap in the lack of studies that investigate the effectiveness of artificial intelligence in detecting and preventing cyber attacks.
• Research paper : A research paper on the topic of natural language processing might identify a research gap in the lack of studies that investigate the use of natural language processing techniques for sentiment analysis in non-English languages.

How to Write Research Gap

By following these steps, you can effectively write about research gaps in your paper and clearly articulate the contribution that your study will make to the existing body of knowledge.

Here are some steps to follow when writing about research gaps in your paper:

• Identify the research question : Before writing about research gaps, you need to identify your research question or problem. This will help you to understand the scope of your research and identify areas where additional research is needed.
• Review the literature: Conduct a thorough review of the literature related to your research question. This will help you to identify the current state of knowledge in the field and the gaps that exist.
• Identify the research gap: Based on your review of the literature, identify the specific research gap that your study will address. This could be a theoretical, empirical, methodological, practical, or knowledge gap.
• Provide evidence: Provide evidence to support your claim that the research gap exists. This could include a summary of the existing literature, a discussion of the limitations of previous studies, or an analysis of the current state of knowledge in the field.
• Explain the importance: Explain why it is important to fill the research gap. This could include a discussion of the potential implications of filling the gap, the significance of the research for the field, or the potential benefits to society.
• State your research objectives: State your research objectives, which should be aligned with the research gap you have identified. This will help you to clearly articulate the purpose of your study and how it will address the research gap.

Importance of Research Gap

The importance of research gaps can be summarized as follows:

• Advancing knowledge: Identifying research gaps is crucial for advancing knowledge in a particular field. By identifying areas where additional research is needed, researchers can fill gaps in the existing body of knowledge and contribute to the development of new theories and practices.
• Guiding research: Research gaps can guide researchers in designing studies that fill those gaps. By identifying research gaps, researchers can develop research questions and objectives that are aligned with the needs of the field and contribute to the development of new knowledge.
• Enhancing research quality: By identifying research gaps, researchers can avoid duplicating previous research and instead focus on developing innovative research that fills gaps in the existing body of knowledge. This can lead to more impactful research and higher-quality research outputs.
• Informing policy and practice: Research gaps can inform policy and practice by highlighting areas where additional research is needed to inform decision-making. By filling research gaps, researchers can provide evidence-based recommendations that have the potential to improve policy and practice in a particular field.

Applications of Research Gap

Here are some potential applications of research gap:

• Informing research priorities: Research gaps can help guide research funding agencies and researchers to prioritize research areas that require more attention and resources.
• Identifying practical implications: Identifying gaps in knowledge can help identify practical applications of research that are still unexplored or underdeveloped.
• Stimulating innovation: Research gaps can encourage innovation and the development of new approaches or methodologies to address unexplored areas.
• Improving policy-making: Research gaps can inform policy-making decisions by highlighting areas where more research is needed to make informed policy decisions.
• Enhancing academic discourse: Research gaps can lead to new and constructive debates and discussions within academic communities, leading to more robust and comprehensive research.

Advantages of Research Gap

Here are some of the advantages of research gap:

• Identifies new research opportunities: Identifying research gaps can help researchers identify areas that require further exploration, which can lead to new research opportunities.
• Improves the quality of research: By identifying gaps in current research, researchers can focus their efforts on addressing unanswered questions, which can improve the overall quality of research.
• Enhances the relevance of research: Research that addresses existing gaps can have significant implications for the development of theories, policies, and practices, and can therefore increase the relevance and impact of research.
• Helps avoid duplication of effort: Identifying existing research can help researchers avoid duplicating efforts, saving time and resources.
• Helps to refine research questions: Research gaps can help researchers refine their research questions, making them more focused and relevant to the needs of the field.
• Promotes collaboration: By identifying areas of research that require further investigation, researchers can collaborate with others to conduct research that addresses these gaps, which can lead to more comprehensive and impactful research outcomes.

Disadvantages of Research Gap

While research gaps can be advantageous, there are also some potential disadvantages that should be considered:

• Difficulty in identifying gaps: Identifying gaps in existing research can be challenging, particularly in fields where there is a large volume of research or where research findings are scattered across different disciplines.
• Lack of funding: Addressing research gaps may require significant resources, and researchers may struggle to secure funding for their work if it is perceived as too risky or uncertain.
• Time-consuming: Conducting research to address gaps can be time-consuming, particularly if the research involves collecting new data or developing new methods.
• Risk of oversimplification: Addressing research gaps may require researchers to simplify complex problems, which can lead to oversimplification and a failure to capture the complexity of the issues.
• Bias : Identifying research gaps can be influenced by researchers’ personal biases or perspectives, which can lead to a skewed understanding of the field.
• Potential for disagreement: Identifying research gaps can be subjective, and different researchers may have different views on what constitutes a gap in the field, leading to disagreements and debate.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Research Design – Types, Methods and Examples

Data Collection – Methods Types and Examples

Research Report – Example, Writing Guide and...

Research Summary – Structure, Examples and...

Survey Instruments – List and Their Uses

What is a Hypothesis – Types, Examples and...

The Research Gap (Literature Gap)

I f you’re just starting out in research, chances are you’ve heard about the elusive research gap (also called a literature gap). In this post, we’ll explore the tricky topic of research gaps. We’ll explain what a research gap is, look at the four most common types of research gaps, and unpack how you can go about finding a suitable research gap for your dissertation, thesis or research project.

Overview: Research Gap 101

• What is a research gap
• Four common types of research gaps
• Practical examples
• How to find research gaps
• Recap & key takeaways

What (exactly) is a research gap?

Well, at the simplest level, a research gap is essentially an unanswered question or unresolved problem in a field, which reflects a lack of existing research in that space. Alternatively, a research gap can also exist when there’s already a fair deal of existing research, but where the findings of the studies pull in different directions , making it difficult to draw firm conclusions.

For example, let’s say your research aims to identify the cause (or causes) of a particular disease. Upon reviewing the literature, you may find that there’s a body of research that points toward cigarette smoking as a key factor – but at the same time, a large body of research that finds no link between smoking and the disease. In that case, you may have something of a research gap that warrants further investigation.

Now that we’ve defined what a research gap is – an unanswered question or unresolved problem – let’s look at a few different types of research gaps.

Types of research gaps

While there are many different types of research gaps, the four most common ones we encounter when helping students at Grad Coach are as follows:

• The classic literature gap
• The disagreement gap
• The contextual gap, and
• The methodological gap

Need a helping hand?

1. The Classic Literature Gap

First up is the classic literature gap. This type of research gap emerges when there’s a new concept or phenomenon that hasn’t been studied much, or at all. For example, when a social media platform is launched, there’s an opportunity to explore its impacts on users, how it could be leveraged for marketing, its impact on society, and so on. The same applies for new technologies, new modes of communication, transportation, etc.

Classic literature gaps can present exciting research opportunities , but a drawback you need to be aware of is that with this type of research gap, you’ll be exploring completely new territory . This means you’ll have to draw on adjacent literature (that is, research in adjacent fields) to build your literature review, as there naturally won’t be very many existing studies that directly relate to the topic. While this is manageable, it can be challenging for first-time researchers, so be careful not to bite off more than you can chew.

2. The Disagreement Gap

As the name suggests, the disagreement gap emerges when there are contrasting or contradictory findings in the existing research regarding a specific research question (or set of questions). The hypothetical example we looked at earlier regarding the causes of a disease reflects a disagreement gap.

Importantly, for this type of research gap, there needs to be a relatively balanced set of opposing findings . In other words, a situation where 95% of studies find one result and 5% find the opposite result wouldn’t quite constitute a disagreement in the literature. Of course, it’s hard to quantify exactly how much weight to give to each study, but you’ll need to at least show that the opposing findings aren’t simply a corner-case anomaly .

3. The Contextual Gap

The third type of research gap is the contextual gap. Simply put, a contextual gap exists when there’s already a decent body of existing research on a particular topic, but an absence of research in specific contexts .

For example, there could be a lack of research on:

• A specific population – perhaps a certain age group, gender or ethnicity
• A geographic area – for example, a city, country or region
• A certain time period – perhaps the bulk of the studies took place many years or even decades ago and the landscape has changed.

The contextual gap is a popular option for dissertations and theses, especially for first-time researchers, as it allows you to develop your research on a solid foundation of existing literature and potentially even use existing survey measures.

Importantly, if you’re gonna go this route, you need to ensure that there’s a plausible reason why you’d expect potential differences in the specific context you choose. If there’s no reason to expect different results between existing and new contexts, the research gap wouldn’t be well justified. So, make sure that you can clearly articulate why your chosen context is “different” from existing studies and why that might reasonably result in different findings.

4. The Methodological Gap

Last but not least, we have the methodological gap. As the name suggests, this type of research gap emerges as a result of the research methodology or design of existing studies. With this approach, you’d argue that the methodology of existing studies is lacking in some way , or that they’re missing a certain perspective.

For example, you might argue that the bulk of the existing research has taken a quantitative approach, and therefore there is a lack of rich insight and texture that a qualitative study could provide. Similarly, you might argue that existing studies have primarily taken a cross-sectional approach , and as a result, have only provided a snapshot view of the situation – whereas a longitudinal approach could help uncover how constructs or variables have evolved over time.

Practical Examples

Let’s take a look at some practical examples so that you can see how research gaps are typically expressed in written form. Keep in mind that these are just examples – not actual current gaps (we’ll show you how to find these a little later!).

Context: Healthcare

Despite extensive research on diabetes management, there’s a research gap in terms of understanding the effectiveness of digital health interventions in rural populations (compared to urban ones) within Eastern Europe.

Context: Environmental Science

While a wealth of research exists regarding plastic pollution in oceans, there is significantly less understanding of microplastic accumulation in freshwater ecosystems like rivers and lakes, particularly within Southern Africa.

Context: Education

While empirical research surrounding online learning has grown over the past five years, there remains a lack of comprehensive studies regarding the effectiveness of online learning for students with special educational needs.

As you can see in each of these examples, the author begins by clearly acknowledging the existing research and then proceeds to explain where the current area of lack (i.e., the research gap) exists.

How To Find A Research Gap

Now that you’ve got a clearer picture of the different types of research gaps, the next question is of course, “how do you find these research gaps?” .

Well, we cover the process of how to find original, high-value research gaps in a separate post . But, for now, I’ll share a basic two-step strategy here to help you find potential research gaps.

As a starting point, you should find as many literature reviews, systematic reviews and meta-analyses as you can, covering your area of interest. Additionally, you should dig into the most recent journal articles to wrap your head around the current state of knowledge. It’s also a good idea to look at recent dissertations and theses (especially doctoral-level ones). Dissertation databases such as ProQuest, EBSCO and Open Access are a goldmine for this sort of thing. Importantly, make sure that you’re looking at recent resources (ideally those published in the last year or two), or the gaps you find might have already been plugged by other researchers.

Once you’ve gathered a meaty collection of resources, the section that you really want to focus on is the one titled “ further research opportunities ” or “further research is needed”. In this section, the researchers will explicitly state where more studies are required – in other words, where potential research gaps may exist. You can also look at the “ limitations ” section of the studies, as this will often spur ideas for methodology-based research gaps.

By following this process, you’ll orient yourself with the current state of research , which will lay the foundation for you to identify potential research gaps. You can then start drawing up a shortlist of ideas and evaluating them as candidate topics . But remember, make sure you’re looking at recent articles – there’s no use going down a rabbit hole only to find that someone’s already filled the gap 🙂

Let’s Recap

We’ve covered a lot of ground in this post. Here are the key takeaways:

• A research gap is an unanswered question or unresolved problem in a field, which reflects a lack of existing research in that space.
• The four most common types of research gaps are the classic literature gap, the disagreement gap, the contextual gap and the methodological gap.
• To find potential research gaps, start by reviewing recent journal articles in your area of interest, paying particular attention to the FRIN section .

If you’re keen to learn more about research gaps and research topic ideation in general, be sure to check out the rest of the Grad Coach Blog . Alternatively, if you’re looking for 1-on-1 support with your dissertation, thesis or research project, be sure to check out our private coaching service .

You Might Also Like:

How To Review & Understand Academic Literature Quickly

Learn how to fast-track your literature review by reading with intention and clarity. Dr E and Amy Murdock explain how.

Dissertation Writing Services: Far Worse Than You Think

Thinking about using a dissertation or thesis writing service? You might want to reconsider that move. Here’s what you need to know.

Triangulation: The Ultimate Credibility Enhancer

Triangulation is one of the best ways to enhance the credibility of your research. Learn about the different options here.

The Harsh Truths Of Academic Research

Dr. Ethar Al-Saraf and Dr. Amy Murdock dive into the darker truths of academic research, so that you’re well prepared for reality.

Dissertation Paralysis: How To Get Unstuck

In this episode of the podcast, Dr. Ethar and Dr. Amy Murdock dive into how to get unstuck when you’re facing dissertation paralysis

📄 FREE TEMPLATES

Research Topic Ideation

Proposal Writing

Literature Review

Methodology & Analysis

Academic Writing

Referencing & Citing

Apps, Tools & Tricks

The Grad Coach Podcast

42 Comments

This post is REALLY more than useful, Thank you very very much

Very helpful specialy, for those who are new for writing a research! So thank you very much!!

I found it very helpful article. Thank you.

it very good but what need to be clear with the concept is when di we use research gap before we conduct aresearch or after we finished it ,or are we propose it to be solved or studied or to show that we are unable to cover so that we let it to be studied by other researchers ?

Just at the time when I needed it, really helpful.

Very helpful and well-explained. Thank you

VERY HELPFUL

We’re very grateful for your guidance, indeed we have been learning a lot from you , so thank you abundantly once again.

hello brother could you explain to me this question explain the gaps that researchers are coming up with ?

Am just starting to write my research paper. your publication is very helpful. Thanks so much

How to cite the author of this?

your explanation very help me for research paper. thank you

Very important presentation. Thanks.

Very helpful indeed

Best Ideas. Thank you.

I found it’s an excellent blog to get more insights about the Research Gap. I appreciate it!

Kindly explain to me how to generate good research objectives.

This is very helpful, thank you

How to tabulate research gap

Very helpful, thank you.

Thanks a lot for this great insight!

This is really helpful indeed!

This article is really helpfull in discussing how will we be able to define better a research problem of our interest. Thanks so much.

Reading this just in good time as i prepare the proposal for my PhD topic defense.

Very helpful Thanks a lot.

Thank you very much

This was very timely. Kudos

Great one! Thank you all.

Thank you very much.

This is so enlightening. Disagreement gap. Thanks for the insight.

How do I Cite this document please?

Research gap about career choice given me Example bro?

I found this information so relevant as I am embarking on a Masters Degree. Thank you for this eye opener. It make me feel I can work diligently and smart on my research proposal.

This is very helpful to beginners of research. You have good teaching strategy that use favorable language that limit everyone from being bored. Kudos!!!!!

This plat form is very useful under academic arena therefore im stil learning a lot of informations that will help to reduce the burden during development of my PhD thesis

This information is beneficial to me.

Insightful…

I have found this quite helpful. I will continue using gradcoach for research assistance

Doing research in PhD accounting, my research topic is: Business Environment and Small Business Performance: The Moderating Effect of Financial Literacy in Eastern Uganda. I am failing to focus the idea in the accounting areas. my supervisor tells me my research is more of in the business field. the literature i have surveyed has used financial literacy as an independent variable and not as a moderator. Kindly give me some guidance here. the core problem is that despite the various studies, small businesses continue to collapse in the region. my vision is that financial literacy is still one of the major challenges hence the need for this topic.

An excellent work, it’s really helpful

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Submit Comment

• Print Friendly

• Thesis Action Plan New
• Academic Project Planner

Literature Navigator

Thesis dialogue blueprint, writing wizard's template, research proposal compass.

• Why students love us
• Rebels Blog
• Why we are different
• All Products
• Coming Soon

Identifying Gaps in Literature: Methods and Strategies

Finding gaps in existing research is important for advancing knowledge and making new discoveries. By identifying what hasn't been studied yet, researchers can focus on new questions and problems. This article will explore different ways to find these gaps, including reviewing recent studies, analyzing contradictions, using advanced search methods, looking at key works, joining academic communities, and using critical thinking.

Key Takeaways

• Conducting a thorough literature review helps find recurring themes and unresolved issues.
• Analyzing contradictions in studies can reveal gaps in current knowledge.
• Advanced search techniques and keywords are vital for pinpointing research gaps.
• Exploring key studies and influential authors provides a solid foundation for identifying gaps.
• Engaging with scholarly communities can offer insights and collaboration opportunities for finding research gaps.

Conducting a Comprehensive Literature Review

Conducting a comprehensive literature review is your first step in identifying research gaps. As you search for journal articles, you will need to read critically across the breadth of the literature to identify these gaps. Your goal should be to find a ‘space’ or opening for contributing new research. The first step is gathering a broad range of research articles on your topic. You may want to look for research that approaches the topic from a variety of methods – qualitative, quantitative, or mixed methods.

Reviewing Recent Publications

Start by thoroughly reviewing existing literature related to your topic of interest. Pay close attention to recent publications and seminal works in the field. Identify recurring themes, unresolved issues, and areas with conflicting findings. Take note of gaps in knowledge, unanswered questions, and areas for further investigation.

Identifying Recurring Themes

Conducting an exhaustive literature review is your first step. As you search for journal articles, you will need to read critically across the breadth of the literature to identify these gaps. Your goal should be to find a ‘space’ or opening for contributing new research. The first step is gathering a broad range of research articles on your topic. You may want to look for research that approaches the topic from a variety of methods – qualitative, quantitative, or mixed methods.

Noting Unresolved Issues

Conducting a thorough literature review is a foundational step in identifying research gaps. By delving into existing literature within your field, you can pinpoint areas where conflicting findings, unanswered questions, or limited studies are prevalent. Pay close attention to gaps in knowledge or discrepancies between different studies, as these can highlight areas ripe for further exploration.

Analyzing Contradictions and Inconsistencies

Recognizing conflicting findings.

When you review literature, it's crucial to spot any conflicting findings. These discrepancies can highlight areas where further research is needed. For instance, if one study shows a treatment is effective while another does not, this inconsistency needs to be addressed. Identifying these contradictions can help you pinpoint gaps in the current understanding of a topic.

Evaluating Methodological Differences

Different studies often use varied methods, which can lead to different results. By evaluating these methodological differences, you can understand why certain studies may have conflicting outcomes. This step is essential in determining whether the inconsistencies are due to the methods used or other factors. Assessing these differences can provide insights into how future research should be designed.

Assessing Theoretical Discrepancies

Theoretical frameworks guide research, but they can also lead to different interpretations of the same data. When you come across theoretical discrepancies, it's important to note them. These differences can reveal underlying assumptions that need to be tested. By exploring these theoretical gaps, you can contribute to a more comprehensive understanding of the topic.

Utilizing Advanced Search Techniques

Effective search strategies.

When conducting a literature review, it's crucial to use effective search strategies. Start by using specific keywords related to your topic. This helps narrow down the vast amount of available information . You can also use Boolean operators like AND, OR, and NOT to refine your search results.

Using Keywords to Pinpoint Gaps

To identify a gap in the literature , include terms like "literature gap" or "future research" along with your subject keywords. This method can help you find articles that discuss unresolved issues or suggest areas for future research.

Leveraging Research Databases

Research databases are invaluable tools for finding scholarly articles. Use advanced search options to filter results by document type, publication type, or methodology. For example, you can search for "video games" AND "literature reviews" to find specific types of studies. This approach can make your search more efficient and targeted.

Exploring Seminal Works and Key Studies

Understanding foundational research.

When diving into a new research area, it's crucial to identify and understand the foundational research. These seminal works are often cited repeatedly and form the basis of further studies. By grasping these key studies, you can build a solid foundation for your own research. Use databases like Web of Science to trace the research trail and discover other articles that have cited these pivotal works.

Identifying Influential Authors

Influential authors in your field often contribute significantly to the body of knowledge. Recognizing these key figures can help you understand the evolution of research trends and methodologies. Follow their publications and see how their work has shaped the field. This can also guide you in identifying potential mentors or collaborators.

Examining Landmark Studies

Landmark studies are those that have made significant contributions to the field and are frequently referenced. These studies often introduce new theories, methodologies, or findings that have a lasting impact. Examining these works can provide insights into the major developments and ongoing debates within your research area. Make sure to review the reference sections of these papers to uncover additional important resources.

Engaging with Scholarly Communities

Engaging with scholarly communities is essential for identifying gaps in literature and advancing your research. By actively participating in academic conferences, networking with experts, and collaborating on research projects, you can gain valuable insights and feedback that may not be immediately apparent from literature alone.

Applying Critical Analysis

Evaluating study limitations.

When you engage in critical analysis of literature, it's essential to look for limitations in the studies you review. This involves identifying any weaknesses in the research design, sample size, or data collection methods. By noting these limitations, you can uncover areas that need further investigation.

Identifying Unanswered Questions

As you review the literature, pay attention to the questions that remain unanswered. These gaps can provide valuable opportunities for your own research. Ask yourself what issues or questions the authors have not addressed and consider how you might explore these areas in your work.

Assessing Research Impact

Assessing the impact of existing research is crucial for understanding its significance to your field. Consider how the findings have influenced subsequent studies and whether they have led to new lines of inquiry. By evaluating the impact, you can identify key areas where further research is needed to advance knowledge.

In the process of crafting a bachelor thesis, research rebels often include identifying research gaps, organizing literature review, and developing theoretical framework for valuable academic contribution.

Synthesizing Literature Findings

When you synthesize literature findings, you bring together insights from multiple studies to create a comprehensive understanding of a topic. This process involves describing how sources converse with each other and organizing similar ideas together so readers can see how they overlap. Synthesis helps readers see where you can contribute new knowledge and identify gaps in the existing research.

Integrating Diverse Perspectives

To integrate diverse perspectives, you need to look at the different angles from which researchers have approached the topic. This means considering various methodologies, theoretical frameworks, and findings. By doing so, you can provide a more rounded view of the subject.

Highlighting Knowledge Gaps

While synthesizing, it's crucial to highlight knowledge gaps. These are areas where the research is either lacking or contradictory. Identifying these gaps can help you formulate a strong research question and guide future studies.

Proposing Future Research Directions

Finally, propose future research directions based on your synthesis. Use the discussion and future research sections of the articles to understand what researchers have found and where they suggest additional research is needed. This will help you pinpoint areas that require further investigation and contribute to the field.

Bringing together insights from various studies can be a game-changer for your thesis. Our guides simplify this process, making it easy for you to understand and apply the findings. Ready to make your thesis journey smoother? Visit our website to learn more and get started today!

In conclusion, identifying gaps in literature is a crucial step for advancing knowledge in any field. By conducting thorough literature reviews, recognizing inconsistencies, and pinpointing unanswered questions, researchers can uncover areas that need further exploration. This process not only highlights opportunities for new research but also helps in refining existing theories and practices. Utilizing various strategies and tools, such as comprehensive reviews and critical analyses, ensures that researchers can systematically identify and address these gaps. Ultimately, the continuous effort to find and fill these gaps drives innovation and contributes to the overall growth of academic and scientific communities.

Frequently Asked Questions

What is a literature gap.

A literature gap is an area in a field of study that hasn't been fully explored or addressed in existing research. It represents opportunities for new studies and discoveries.

Why is it important to identify gaps in literature?

Identifying gaps helps researchers find new areas to explore. It ensures that new studies add value and contribute to the existing body of knowledge.

How do I start a comprehensive literature review?

Begin by reading recent publications related to your topic. Look for patterns, recurring themes, and areas that need more investigation.

What are some effective search strategies for finding literature gaps?

Use specific keywords related to your topic along with terms like "literature gap" or "future research." Research databases can also help pinpoint these gaps.

How can I recognize contradictions in research?

Look for studies with conflicting results or different conclusions. Pay attention to differences in methods and theories used in these studies.

Why should I engage with scholarly communities?

Engaging with scholarly communities, like attending conferences and networking with experts, can provide insights into current research trends and help identify gaps in the literature.

Discovering Statistics Using IBM SPSS Statistics: A Fun and Informative Guide

Unlocking the Power of Data: A Review of 'Essentials of Modern Business Statistics with Microsoft Excel'

Discovering Statistics Using SAS: A Comprehensive Review

How to Research for Your Thesis Paper Without Feeling Overwhelmed

Collaborative Genius: 6 Hacks for Seamless Group Thesis Writing

Understanding the Difference Between Research Objectives and Research Questions

Thesis Action Plan

• Blog Articles
• Affiliate Program
• Terms and Conditions
• Payment and Shipping Terms
• Privacy Policy
• Return Policy

© 2024 Research Rebels, All rights reserved.

Your cart is currently empty.

How to find and fill gaps in the literature [Research Gaps Made Easy]

As we dive deeper into the realm of research, one term repeatedly echoes in the corridors of academia: “gap in literature.”

But what does it mean to find a gap in the literature, and why is it so crucial for your research project?

A gap in the literature refers to an area that hasn’t been studied or lacks substantial inquiry in your field of study. Identifying such gaps allows you to contribute fresh insights and innovation, thereby extending the existing body of knowledge.

It’s the cornerstone for every dissertation or research paper, setting the stage for an introduction that explicitly outlines the scope and aim of your investigation.

This gap review isn’t limited to what has been published in peer-reviewed journals; it may also include conference papers, dissertations, or technical reports, i.e., types of papers that provide an overview of ongoing research. 

This step is where your detective work comes in—by spotting trends, common methodologies, and unanswered questions, you can unearth an opportunity to explore an unexplored domain, thereby finding a research gap. 

Why Looking for Research Gaps is Essential

Looking for research gaps is essential as it enables the discovery of novel and unique contributions to a particular field.

By identifying these gaps, found through methods such as analyzing concluding remarks of recent papers, literature reviews, examining research groups’ non-peer-reviewed outputs, and utilizing specific search terms on Google Scholar, one can discern the trajectory of ongoing research and unearth opportunities for original inquiry.

These gaps highlight areas of potential innovation, unexplored paths, and disputed concepts, serving as the catalyst for valuable contributions and progression in the field. Hence, finding research gaps forms the basis of substantial and impactful scientific exploration.

Then your research can contribute by finding and filling the gap in knowledge. 

Method 1: Utilizing Concluding Remarks of Recent Research

When embarking on a quest to find research gaps, the concluding remarks of recent research papers can serve as an unexpected treasure map.

This section of a paper often contains insightful comments on the limitations of the work and speculates on future research directions.

These comments, although not directly pointing to a research gap, can hint at where the research is heading and what areas require further exploration.

Consider these remarks as signposts, pointing you towards uncharted territories in your field of interest.

For example, you may come across a conclusion in a recent paper on artificial intelligence that indicates a need for more research on ethical considerations. This gives you a direction to explore – the ethical implications of AI. 

However, it’s important to bear in mind that while these statements provide valuable leads, they aren’t definitive indicators of research gaps. They provide a starting point, a clue to the vast research puzzle.

Your task is to take these hints, explore further, and discern the most promising areas for your investigation. It’s a bit like being a detective, except your clues come from scholarly papers instead of crime scenes!

Method 2: Examining Research Groups and Non-peer Reviewed Outputs

If concluding remarks are signposts to potential research gaps, non-peer reviewed outputs such as preprints, conference presentations, and dissertations are detailed maps guiding you towards the frontier of research.

These resources reflect the real-time development in the field, giving you a sense of the “buzz” that surrounds hot topics.

These materials, presented but not formally published, offer a sneak peek into ongoing studies, providing you with a rich source of information to identify emerging trends and potential research gaps.

For instance, a presentation on the impact of climate change on mental health might reveal a new line of research that’s in its early stages.

One word of caution: while these resources can be enlightening, they have not undergone the rigorous peer review process that published articles have.

This means the quality of research may vary and the findings should be interpreted with a critical eye. Remember, the key is to pinpoint where the research is heading and then carve out your niche within that sphere.

Exploring non-peer reviewed outputs allows you to stay ahead of the curve, harnessing the opportunity to investigate and contribute to a burgeoning area of study before it becomes mainstream.

Method 3: Searching for ‘Promising’ and ‘Preliminary’ Results on Google Scholar

With a plethora of research at your fingertips, Google Scholar can serve as a remarkable tool in your quest to discover research gaps. The magic lies in a simple trick: search for the phrases “promising results” or “preliminary results” within your research area. Why these specific phrases? Scientists often use them when they have encouraging but not yet fully verified findings.

To illustrate, consider an example. Type “promising results and solar cell” into Google Scholar, and filter by recent publications.

The search results will show you recent studies where researchers have achieved promising outcomes but may not have fully developed their ideas or resolved all challenges.

These “promising” or “preliminary” results often represent areas ripe for further exploration.

They hint at a research question that has been opened but not fully answered. However, tread carefully.

While these findings can indeed point to potential research gaps, they can also lead to dead ends. It’s crucial to examine these leads with a critical eye and further corroborate them with a comprehensive review of related research.

Nevertheless, this approach provides a simple, effective starting point for identifying research gaps, serving as a launchpad for your explorations.

Method 4: Reading Around the Subject

Comprehensive reading forms the bedrock of effective research. When hunting for research gaps, you need to move beyond just the preliminary findings and delve deeper into the context surrounding these results.

This involves broadening your view and reading extensively around your topic of interest.

In the course of your reading, you will start identifying common themes, reoccurring questions, and shared challenges in the research.

Over time, patterns will emerge, helping you recognize areas where research is thin or missing.

For instance, in studying autonomous vehicles, you might find recurring questions about regulatory frameworks, pointing to a potential gap in the legal aspects of this technology.

However, this method is not about scanning through a huge volume of literature aimlessly. It involves strategic and critical reading, looking for patterns, inconsistencies, and areas where the existing literature falls short.

It’s akin to painting a picture where some parts are vividly detailed while others remain sketchy. Your goal is to identify these sketchy areas and fill in the details.

So grab your academic reading list, and start diving into the ocean of knowledge. Remember, it’s not just about the depth, but also the breadth of your reading, that will lead you to a meaningful research gap.

Method 5: Consulting with Current Researchers

Few methods are as effective in uncovering research gaps as engaging in conversations with active researchers in your field of interest.

Current researchers, whether they are PhD students, postdoctoral researchers, or supervisors, are often deeply engaged in ongoing studies and understand the current challenges in their respective fields.

Start by expressing genuine interest in their work. Rather than directly asking for research gaps, inquire about the challenges they are currently facing in their projects.

You can ask, “What are the current challenges in your research?”

Their responses can highlight potential areas of exploration, setting you on the path to identifying meaningful research gaps.

Moreover, supervisors, particularly those overseeing PhD and Master’s students, often have ideas for potential research topics. By asking the right questions, you can tap into their wealth of knowledge and identify fruitful areas of study.

While the act of discovering research gaps can feel like a solitary journey, it doesn’t have to be.

Engaging with others who are grappling with similar challenges can provide valuable insights and guide your path. After all, the world of research thrives on collaboration and shared intellectual curiosity.

Method 6: Utilizing Online Tools

The digital age has made uncovering research gaps easier, thanks to a plethora of online tools that help visualize the interconnectedness of research literature.

Platforms such as:

• Connected Papers,
• ResearchRabbit, and

allow you to see how different papers in your field relate to one another, thereby creating a web of knowledge.

Upon creating this visual web, you may notice that many papers point towards a certain area, but then abruptly stop. This could indicate a potential research gap, suggesting that the topic hasn’t been adequately addressed or has been sidelined for some reason.

By further reading around this apparent gap, you can understand if it’s a genuine knowledge deficit or merely a research path that was abandoned due to inherent challenges or a dead end.

These online tools provide a bird’s eye view of the literature, helping you understand the broader landscape of research in your area of interest.

By examining patterns and relationships among studies, you can effectively zero in on unexplored areas, making these tools a valuable asset in your quest for research gaps.

Method 7: Seeking Conflicting Ideas in the Literature

In scientific research, areas of conflict can often be fertile ground for finding research gaps. These are areas where there’s a considerable amount of disagreement or ongoing debate among researchers.

If you can bring a fresh perspective, a new technique, or a novel hypothesis to such a contentious issue, you may well be on your way to uncovering a significant research gap.

Take, for instance, an area in psychology where there is a heated debate about the influence of nature versus nurture.

If you can introduce a new dimension to the debate or a method to test a novel hypothesis, you could potentially fill a significant gap in the literature.

Investigating areas of conflict not only opens avenues for exploring research gaps, but it also provides opportunities for you to make substantial contributions to your field. The key is to be able to see the potential for a new angle and to muster the courage to dive into contentious waters.

However, engaging with conflicts in research requires careful navigation.

Striking the right balance between acknowledging existing research and championing new ideas is crucial.

In the end, resolving these conflicts or adding significant depth to the debate can be incredibly rewarding and contribute greatly to your field.

The Right Perspective Towards Research Gaps

The traditional understanding of research gaps often involves seeking out a ‘bubble’ of missing knowledge in the sea of existing research, a niche yet to be explored. However, in today’s fast-paced research environment, these bubbles are becoming increasingly rare.

The paradigm of finding research gaps is shifting. It’s no longer just about seeking out holes in existing knowledge, but about understanding the leading edge of research and the directions it could take. It involves not just filling in the gaps but extending the boundaries of knowledge.

To identify such opportunities, develop a comprehensive understanding of the research landscape, identify emerging trends, and keep a close eye on recent advancements.

Look for the tendrils of knowledge extending out into the unknown and think about how you can push them further. It might be a challenging task, but it offers the potential for making substantial, impactful contributions to your field. 

Remember, every great innovation begins at the edge of what is known. That’s where your research gap might be hiding.

Wrapping up – Literature and research gaps

Finding and filling a gap in the literature is a task crucial to every research project. It begins with a systematic review of existing literature – a quest to identify what has been studied and more importantly, what hasn’t.

You must delve into the rich terrain of literature in their field, from the seminal, citation-heavy research articles to the fresh perspective of conference papers. Identifying the gap in the literature necessitates a thorough evaluation of existing studies to refine your area of interest and map the scope and aim of your future research.

The purpose is to explicitly identify the gap that exists, so you can contribute to the body of knowledge by providing fresh insights. The process involves a series of steps, from consulting with faculty and experts in the field to identify potential trends and outdated methodologies, to being methodological in your approach to identify gaps that have emerged.

Upon finding a gap in the literature, we’ll ideally have a clearer picture of the research need and an opportunity to explore this unexplored domain.

It is important to remember that the task does not end with identifying the gap. The real challenge lies in drafting a research proposal that’s objective, answerable, and can quantify the impact of filling this gap. 

It’s important to consult with your advisor, and also look at commonly used parameters and preliminary evidence. Only then can we complete the task of turning an identified gap in the literature into a valuable contribution to your field, a contribution that’s peer-reviewed and adds to the body of knowledge.

To find a research gap is to stand on the shoulders of giants, looking beyond the existing research to further expand our understanding of the world.

Dr Andrew Stapleton has a Masters and PhD in Chemistry from the UK and Australia. He has many years of research experience and has worked as a Postdoctoral Fellow and Associate at a number of Universities. Although having secured funding for his own research, he left academia to help others with his YouTube channel all about the inner workings of academia and how to make it work for you.

Thank you for visiting Academia Insider.

We are here to help you navigate Academia as painlessly as possible. We are supported by our readers and by visiting you are helping us earn a small amount through ads and affiliate revenue - Thank you!

2024 © Academia Insider

info This is a space for the teal alert bar.

notifications This is a space for the yellow alert bar.

Research Process

• Brainstorming
• Explore Google This link opens in a new window
• Explore Web Resources
• Explore Background Information
• Explore Books
• Explore Scholarly Articles
• Narrowing a Topic
• Primary and Secondary Resources
• Academic, Popular & Trade Publications
• Scholarly and Peer-Reviewed Journals
• Grey Literature
• Clinical Trials
• Evidence Based Treatment
• Scholarly Research
• Database Research Log
• Search Limits
• Keyword Searching
• Boolean Operators
• Phrase Searching
• Truncation & Wildcard Symbols
• Proximity Searching
• Field Codes
• Subject Terms and Database Thesauri
• Reading a Scientific Article
• Website Evaluation
• Article Keywords and Subject Terms
• Cited References
• Citing Articles
• Related Results
• Search Within Publication
• Database Alerts & RSS Feeds
• Personal Database Accounts
• Persistent URLs
• Literature Gap and Future Research
• Web of Knowledge
• Annual Reviews
• Systematic Reviews & Meta-Analyses
• Finding Seminal Works
• Exhausting the Literature
• Finding Dissertations
• Researching Theoretical Frameworks
• Research Methodology & Design
• Tests and Measurements
• Organizing Research & Citations This link opens in a new window
• Picking Where to Publish
• Bibliometrics
• Learn the Library This link opens in a new window

Research Articles

These examples below illustrate how researchers from different disciplines identified gaps in existing literature. For additional examples, try a NavigatorSearch using this search string: ("Literature review") AND (gap*)

• Addressing the Recent Developments and Potential Gaps in the Literature of Corporate Sustainability
• Applications of Psychological Science to Teaching and Learning: Gaps in the Literature
• Attitudes, Risk Factors, and Behaviours of Gambling Among Adolescents and Young People: A Literature Review and Gap Analysis
• Do Psychological Diversity Climate, HRM Practices, and Personality Traits (Big Five) Influence Multicultural Workforce Job Satisfaction and Performance? Current Scenario, Literature Gap, and Future Research Directions
• Entrepreneurship Education: A Systematic Literature Review and Identification of an Existing Gap in the Field
• Evidence and Gaps in the Literature on HIV/STI Prevention Interventions Targeting Migrants in Receiving Countries: A Scoping Review
• Homeless Indigenous Veterans and the Current Gaps in Knowledge: The State of the Literature
• A Literature Review and Gap Analysis of Emerging Technologies and New Trends in Gambling
• A Review of Higher Education Image and Reputation Literature: Knowledge Gaps and a Research Agenda
• Trends and Gaps in Empirical Research on Open Educational Resources (OER): A Systematic Mapping of the Literature from 2015 to 2019
• Where Should We Go From Here? Identified Gaps in the Literature in Psychosocial Interventions for Youth With Autism Spectrum Disorder and Comorbid Anxiety

What is a ‘gap in the literature’?

The gap, also considered the missing piece or pieces in the research literature, is the area that has not yet been explored or is under-explored. This could be a population or sample (size, type, location, etc.), research method, data collection and/or analysis, or other research variables or conditions.

It is important to keep in mind, however, that just because you identify a gap in the research, it doesn't necessarily mean that your research question is worthy of exploration. You will want to make sure that your research will have valuable practical and/or theoretical implications. In other words, answering the research question could either improve existing practice and/or inform professional decision-making (Applied Degree), or it could revise, build upon, or create theoretical frameworks informing research design and practice (Ph.D Degree). See the Dissertation Center  for additional information about dissertation criteria at NU.

For a additional information on gap statements, see the following:

• How to Find a Gap in the Literature
• Write Like a Scientist: Gap Statements

How do you identify the gaps?

Conducting an exhaustive literature review is your first step. As you search for journal articles, you will need to read critically across the breadth of the literature to identify these gaps. You goal should be to find a ‘space’ or opening for contributing new research. The first step is gathering a broad range of research articles on your topic. You may want to look for research that approaches the topic from a variety of methods – qualitative, quantitative, or mixed methods. 

See the videos below for further instruction on identifying a gap in the literature.

Identifying a Gap in the Literature - Dr. Laurie Bedford

How Do You Identify Gaps in Literature? - SAGE Research Methods

Literature Gap & Future Research - Library Workshop

This workshop presents effective search techniques for identifying a gap in the literature and recommendations for future research.

Where can you locate research gaps?

As you begin to gather the literature, you will want to critically read for what has, and has not, been learned from the research. Use the Discussion and Future Research sections of the articles to understand what the researchers have found and where they point out future or additional research areas. This is similar to identifying a gap in the literature, however, future research statements come from a single study rather than an exhaustive search. You will want to check the literature to see if those research questions have already been answered.

Roadrunner Search

Identifying the gap in the research relies on an exhaustive review of the literature. Remember, researchers may not explicitly state that a gap in the literature exists; you may need to thoroughly review and assess the research to make that determination yourself.

However, there are techniques that you can use when searching in NavigatorSearch to help identify gaps in the literature. You may use search terms such as "literature gap " or "future research" "along with your subject keywords to pinpoint articles that include these types of statements.

Was this resource helpful?

• << Previous: Resources for a Literature Review
• Next: Web of Knowledge >>
• Last Updated: Aug 12, 2024 7:35 AM
• URL: https://resources.nu.edu/researchprocess

© Copyright 2024 National University. All Rights Reserved.

Privacy Policy | Consumer Information

Educational resources and simple solutions for your research journey

How to Identify Gaps in Research: Tips to Speed Up the Process

If you have ever wondered how to identify research gaps, well, you’re not alone. All researchers looking to make a solid contribution to their field need to start by identifying a topic or issue that hasn’t been tackled before and coming up with possible solutions for it. This is where learning what is a research gap, knowing about some research gap examples, and knowing how to identify research gaps becomes important. Through this article, we will try answering these questions for you.

Table of Contents

What is a research gap ?  

Research gaps are areas requiring more studies or research. 1  They can be:   

• an unsolved question or problem within your field.   
• a case where inconclusive or contradictive results exist.   
• a new concept or idea that hasn’t been studied.   
• a new/updated research to replace the outdated existing research.   
• a specific demographic or location that has not been well studied.   

Why is it important to identify research gaps ?  

Identifying and prioritizing research gaps  is an essential part of any research for the following reasons. 2  This can help you:  

• ensure the rapid generation of subsequent research that is informed by input from previous research studies.    
• understand areas of uncertainty within the research problem.   
• establish the research problem and scope of the study.   
• determine the scope of funding opportunities.   

Identifying research gaps : A challenge for early researchers  

Coming up with original, innovative ideas in your chosen area of research can be tricky, especially if you are an early career researcher, for the following reasons: 3,4

• Enormous information available : The introduction, discussion, and future research sections in published research articles provide information about gaps in the research field. It is easy to get overwhelmed and feel confused about which one to address. Using digital tools can help you seek out popular topics or the most cited research papers.   
• Difficulty in organizing the data : One can quickly lose ideas if not appropriately noted. Mapping the question to the resource and maintaining a record can help narrow research gap s.  
• Fear of challenging the existing knowledge : Beginner researchers may not feel confident to question established norms in their field. A good plan of action would be discussing such ideas with your advisor and proceeding according to their feedback or suggestions.   
• Lack of direction and motivation : Early researchers have reported negative emotions regarding academic research, including feeling directionless or frustrated with the effort required in identifying research topics. Again a good advisor can help you stay focused. Mentors can help novice researchers avoid cases with a high risk of failure, from misunderstanding the literature, weak design, or too many unknowns. Talking with other fellow researchers can also help overcome some of the anxiety.

How to identify research gaps  in the literature  

More than 7 million papers get published annually. 5  Considering the volume of existing research, identifying research gaps  from existing literature may seem a daunting task. While there are no hard rules for identifying research gaps, the literature has provided some guidelines for identifying problems worth investigating.   

1. Observe : Personal interests and experiences can provide insight into possible research problems. For example, a researcher interested in teaching may start with a simple observation of students’ classroom behavior and observe the link with learning theories. Developing the habit of reading literature using smart apps like  R Discovery   can keep you updated with the latest trends and developments in the field.   

2. Search : Exploring existing literature will help to identify if the observed problem is documented. One approach is identifying the independent variables used to solve the researcher’s topic of interest (i.e., the dependent variable). Databases such as Emerald, ProQuest, EbscoHost, PubMed, and ScienceDirect can help potential researchers explore existing research gaps. The following steps can help with optimizing the search process once you decide on the key research question based on your interests.

-Identify key terms.

-Identify relevant articles based on the keywords.

-Review selected articles to identify gaps in the literature.  

3. Map : This involves mapping key issues or aspects across the literature. The map should be updated whenever a researcher comes across an article of interest.   

4. Synthesize : Synthesis involves integrating the insights of multiple but related studies. A research gap is identified by combining results and findings across several interrelated studies. 6

5. Consult:  Seeking expert feedback will help you understand if the  research gaps identified are adequate and feasible or if improvements are required.  

6. Prioritize : It is possible that you have identified multiple questions requiring answers. Prioritize the question that can be addressed first, considering their relevance, resource availability, and your research strengths.  

7. Enroll : Research Skills Development Programs, including workshops and discussion groups within or outside the research institution, can help develop research skills, such as framing the research problem. Networking and corroborating in such events with colleagues and experts might help you know more about current issues and problems in your research domain.   

While there is no well-defined process to identify gaps in knowledge, curiosity, judgment, and creativity can help you in identifying these research gaps . Regardless of whether the  research gaps identified are large or small, the study design must be sufficient to contribute toward advancing your field of research.    

References  

• Dissanayake, D. M. N. S. W. (2013). Research, research gap and the research problem.  
• Nyanchoka, L., Tudur-Smith, C., Porcher, R., & Hren, D. Key stakeholders’ perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study.  BMJ open ,  10 (11), e039932 (2020).  
• Müller-Bloch, C., & Kranz, J. (2015). A framework for rigorously identifying research gaps in qualitative literature reviews.  
• Creswell, J. W., & Clark, V. L. P. (2017).  Designing and conducting mixed methods research . Sage publications.  
• Fire, M., & Guestrin, C. Over-optimization of academic publishing metrics: observing Goodhart’s Law in action.  GigaScience ,  8 (6), giz053 (2019).  
• Ellis, T. J., & Levy, Y. Framework of problem-based research: A guide for novice researchers on the development of a research-worthy problem.  Informing Science: the International Journal of an Emerging Transdiscipline Volume 11, 2008 ). 

Frequently Asked Questions (FAQs)

Question: How can research gaps be addressed?

Research gaps can be addressed by conducting further studies, experiments, or investigations that specifically target the areas where knowledge is lacking or incomplete. This involves conducting a thorough literature review to identify existing gaps, designing research methodologies to address these gaps, and collecting new data or analyzing existing data to fill the void. Collaboration among researchers, interdisciplinary approaches, and innovative research designs can also help bridge research gaps and contribute to the advancement of knowledge in a particular field.

Question: Can research gaps change over time?

Yes, research gaps can change over time. As new studies are conducted, technologies advance, and societal needs evolve, gaps in knowledge may be identified or existing gaps may become more pronounced. Research gaps are dynamic and subject to shifts as new discoveries are made, new questions arise, and priorities change. It is crucial for researchers to continuously assess and update their understanding of the field to identify emerging research gaps and adapt their research efforts accordingly.

Question: Are research gaps specific to a particular discipline or field?

Research gaps can exist within any discipline or field. Each discipline has its own unique body of knowledge and areas where understanding may be limited. Research gaps can arise from unanswered questions, unexplored phenomena, conflicting findings, practical challenges, or new frontiers of knowledge. They are not limited to a specific discipline or field, as gaps can exist in natural sciences, social sciences, humanities, engineering, or any other area of study.

Question: How can research gaps contribute to the research proposal?

Research gaps play a significant role in the development of research proposals. They help researchers identify a clear rationale and justification for their study. By addressing identified gaps in knowledge, researchers can demonstrate the significance and relevance of their proposed research. Research proposals often include a literature review section that highlights existing gaps and positions the proposed study as a contribution to the field. By explicitly addressing research gaps, researchers can strengthen the credibility and importance of their research proposal, as well as its potential impact on advancing knowledge and addressing critical questions or challenges.

R Discovery is a literature search and research reading platform that accelerates your research discovery journey by keeping you updated on the latest, most relevant scholarly content. With 250M+ research articles sourced from trusted aggregators like CrossRef, Unpaywall, PubMed, PubMed Central, Open Alex and top publishing houses like Springer Nature, JAMA, IOP, Taylor & Francis, NEJM, BMJ, Karger, SAGE, Emerald Publishing and more, R Discovery puts a world of research at your fingertips.  

Try R Discovery Prime FREE for 1 week or upgrade at just US$72 a year to access premium features that let you listen to research on the go, read in your language, collaborate with peers, auto sync with reference managers, and much more. Choose a simpler, smarter way to find and read research – Download the app and start your free 7-day trial today !  

Related Posts

What is Research Impact: Types and Tips for Academics

Research in Shorts: R Discovery’s New Feature Helps Academics Assess Relevant Papers in 2mins 

The Best Method In Identifying Research Gap: An In-depth Analysis

What is research gap.

A research gap refers to an area or topic that has not been sufficiently explored or studied, leaving unanswered questions or unresolved issues. This article will provide an overview of the research gaps concept and their significance in the research process. It will also discuss the importance of identifying research gaps and how they can be used to formulate research objectives and problem statements. Additionally, this section will explore various techniques and strategies for conducting research gap analysis and bridging the gap between existing knowledge and future research endeavors.

Background of Research Gap

In the world of research, identifying and addressing research gaps is a crucial step towards advancing knowledge and understanding in a particular field. A research gap refers to an area in the existing body of knowledge where there is a lack of research or unanswered questions. In other words, it is a gap in the literature that needs to be addressed through further research.

Research gaps can occur for various reasons, such as a lack of studies on a specific topic, contradictory findings in existing research, or the emergence of new ideas or concepts that have not been explored. Identifying research gaps is crucial for the advancement of knowledge and the development of new research questions . By filling these gaps, researchers can contribute to the existing body of knowledge and address unanswered questions.

Furthermore, research gaps provide opportunities for researchers to make significant contributions to their field by conducting innovative and impactful studies. Understanding the background of research gaps is essential for researchers to identify areas where their research can make a meaningful impact.

Significance of Research Gap

The research gap plays a crucial role in the field of academia and scientific research. It holds significant importance for researchers, scholars, and the overall advancement of knowledge.

Contributing to Knowledge and Developing New Theories

One of the primary reasons why the research gap is significant is that it identifies areas where there is a lack of knowledge or understanding. It highlights the gaps in existing research, indicating the need for further investigation and exploration. By identifying research gaps, researchers can contribute to the existing body of knowledge by filling in the missing pieces. This leads to the development of new theories, concepts, and insights that can enhance our understanding of a particular subject or field.

Fostering Innovation and Progress through Unexplored Areas

Furthermore, the significance of research gaps lies in their potential to drive innovation and progress. When researchers identify areas that have not been extensively studied, they have the opportunity to explore new ideas, methodologies, and approaches. This can lead to groundbreaking discoveries and advancements in various disciplines.

Efficient Utilization of Resources to Avoid Duplication

Moreover, research gaps also help in avoiding duplication of efforts. By identifying what has already been studied and what areas are yet to be explored, researchers can focus their efforts on addressing the gaps rather than repeating existing research. This ensures that resources are utilized effectively and efficiently.

Impacting Practical Applications and Real-World Solutions

Additionally, the significance of research gaps extends to the practical application of research findings. By addressing the gaps in existing knowledge, researchers can provide valuable insights and solutions to real-world problems. This can have a direct impact on industries, policy-making, and decision-making processes. In conclusion, the significance of research gaps cannot be overstated. They serve as catalysts for knowledge advancement, innovation, and practical application. By identifying and addressing these gaps, researchers contribute to the growth and development of their respective fields, ultimately benefiting society as a whole.

Research Gap Examples

Identifying research gaps is crucial for pursuing innovative research. There are various types of research gaps that can be found in existing literature.

Knowledge gaps

Sometimes, a research gap exists when there is a concept or new idea that hasn’t been studied at all. For example, in the field of psychology, there might be a lack of research on the effects of social media on mental health in adolescents.

Conceptual gaps

Conceptual gaps occur when there is a lack of understanding or clarity about a particular concept or theory. For instance, in the field of economics, there might be a research gap in understanding the relationship between income inequality and economic growth.

Methodological gaps

Methodological gaps refer to the absence of appropriate research methods or techniques to study a specific phenomenon. For example, in the field of biology, there might be a research gap in developing a reliable method to detect a certain type of genetic mutation.

Data gaps occur when there is a lack of available data or insufficient data to address a research question. For instance, in the field of climate science, there might be a research gap in obtaining long-term temperature data for a specific region.

Practical gaps

Practical gaps exist when there is a discrepancy between theoretical knowledge and practical application. For example, in the field of education, there might be a research gap in implementing effective teaching strategies for students with learning disabilities.

Research Gap Analysis Techniques

Carry out a comprehensive literature review.

There are several techniques that can be used to identify research gaps. One common technique is conducting a comprehensive literature review, where researchers examine existing research papers, articles, books, and other relevant sources. By analyzing these materials, researchers can pinpoint what has already been explored and identify areas that require further investigation.

Examining Limitations and Contradictions in Existing Studies

During the literature review, researchers should pay attention to the limitations and gaps in the existing studies. These limitations can include unanswered research questions, contradictory findings, methodological issues, or areas that have not been explored in depth. Researchers can also gain insights by comparing and contrasting the findings, methodologies, and conclusions of different studies within their field, which helps in building a more complete understanding of the topic.

Exploring Interdisciplinary Insights to Identify Gaps

Additionally, researchers can seek inspiration from interdisciplinary fields or related disciplines to identify research gaps. Sometimes, a research gap in one field may have been addressed in another field, and researchers can draw upon these insights to identify areas that have not been explored within their own field. It is important to note that identifying research gaps is not a one-time process. As new studies are published and the field evolves, new gaps may emerge. Therefore, researchers should continuously update their knowledge and review the literature to stay informed about the latest developments and identify new research gaps.

Utilizing Surveys and Interviews for Direct Insights

Another technique is conducting surveys or interviews . This allows researchers to gather information directly from individuals who are knowledgeable in the field. Surveys can be distributed to a large number of participants, while interviews provide more in-depth insights from a smaller group of experts. By collecting data through surveys or interviews, researchers can identify gaps in knowledge or areas where more research is needed. Focus groups are another effective technique for conducting a research gap analysis. In a focus group, a small group of individuals with relevant expertise or experience is brought together to discuss a specific topic. Through group discussions and interactions, researchers can gain valuable insights and identify gaps in knowledge or areas that require further investigation.

Employing Quantitative Analysis to Discover Data Gaps

Quantitative analysis techniques, such as statistical analysis, can also be used to identify research gaps. By analyzing existing data sets, researchers can identify patterns, trends, or gaps in the data that may indicate areas where further research is needed. This type of analysis can provide valuable insights into the gaps in existing knowledge and guide future research directions.

Applying Gap Analysis Frameworks for Structured Assessment

In addition to these techniques, researchers can also use gap analysis frameworks or models to systematically identify and analyze research gaps. These frameworks provide a structured approach to assess the current state of knowledge, determine the desired future state, and identify the gaps that need to be addressed. By using a framework, researchers can ensure a comprehensive analysis of research gaps and develop strategies to bridge those gaps.

Research Gap and Problem Statement

A research problem is a specific issue or question that a researcher wants to investigate. It is the foundation of a research study and provides a clear direction for the research process. The identification of a research gap often leads to the formulation of a research problem.

The problem statement is a constructed sentence that defines the research problem and guides the research question. It helps to clarify the purpose of the study and provides a framework for the research design and research methodology. By addressing the research gap through the problem statement, researchers can contribute to the existing body of knowledge and fill the void in the literature. The research problem becomes the focal point of the study, and the research gap serves as the motivation for conducting the research.

Identifying a research gap and formulating a problem statement are crucial steps in the research process. They ensure that the research study is relevant, meaningful, and contributes to the advancement of knowledge in the field. As a key component of the research framework , the problem statement integrates directly into the overall structure that guides the entire research process, ensuring that all aspects of the investigation are aligned with the identified gaps and research questions.

Bridging the Research Gap

Bridging the research gap is crucial for the advancement of knowledge and the improvement of various fields. It involves closing the divide between research findings and their practical application in real-world settings. By bridging this gap, researchers can ensure that their work has a meaningful impact on society and that it is effectively utilized by practitioners and policymakers.

Effective Collaboration between Researchers and Practitioners

There are several strategies and approaches that can be employed to bridge the research gap. One practical way is to establish collaborations and partnerships between researchers and practitioners. By working together, researchers can gain valuable insights from practitioners’ experiences and expertise, while practitioners can benefit from the latest research findings and evidence-based practices. This collaboration can lead to the development of more relevant and effective solutions to real-world problems.

For facilitating such connections and collaborations, platforms like Researchmate.net are invaluable resources, providing the tools and community needed to bring together researchers and practitioners from diverse fields.

Intermediary Organizations in Facilitating Research Application

Another approach to bridging the research gap is through the use of intermediary organizations. These organizations act as a bridge between researchers and practitioners, facilitating the translation and dissemination of research findings into practical applications. They can provide training, resources, and support to practitioners, helping them to implement evidence-based practices in their work. Intermediary organizations also play a crucial role in promoting knowledge exchange and collaboration between researchers and practitioners.

Enhancing Communication and Knowledge Transfer in Research

Furthermore, bridging the research gap requires effective communication and knowledge transfer. Researchers need to communicate their findings in a clear and accessible manner, using language that is understandable to practitioners and policymakers. This can be achieved through the use of plain language summaries, policy briefs, and other forms of knowledge translation.

Engaging with Practitioners and Policymakers

Additionally, researchers should actively engage with practitioners and policymakers, seeking their input and feedback to ensure that research findings are relevant and applicable to real-world contexts.

In conclusion, exploring the research gap is a critical step in the research process. It helps researchers identify areas where further investigation is needed, contributes to the advancement of knowledge, and drives innovation. By understanding the research gap, researchers can make meaningful contributions to their field and address unanswered questions. Bridging the research gap requires collaboration and commitment from all stakeholders, but the potential benefits are immense.

Leave a Comment Cancel reply

Save my name, email, and website in this browser for the next time I comment.

Related articles

How to Formulate Research Questions in a Research Proposal? Discover The No. 1 Easiest Template Here!

7 Easy Step-By-Step Guide of Using ChatGPT: The Best Literature Review Generator for Time-Saving Academic Research

Writing Engaging Introduction in Research Papers : 7 Tips and Tricks!

Understanding Comparative Frameworks: Their Importance, Components, Examples and 8 Best Practices

Revolutionizing Effective Thesis Writing for PhD Students Using Artificial Intelligence!

3 Types of Interviews in Qualitative Research: An Essential Research Instrument and Handy Tips to Conduct Them

Highlight Abstracts: An Ultimate Guide For Researchers!

Crafting Critical Abstracts: 11 Expert Strategies for Summarizing Research

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

How to Identify a Research Gap

5-minute read

• 10th January 2024

If you’ve been tasked with producing a thesis or dissertation, one of your first steps will be identifying a research gap. Although finding a research gap may sound daunting, don’t fret! In this post, we will define a research gap, discuss its importance, and offer a step-by-step guide that will provide you with the essential know-how to complete this critical step and move on to the rest of your research project.

What Is a Research Gap?

Simply put, a research gap is an area that hasn’t been explored in the existing literature. This could be an unexplored population, an untested method, or a condition that hasn’t been investigated yet. 

Why Is Identifying a Research Gap Important?

Identifying a research gap is a foundational step in the research process. It ensures that your research is significant and has the ability to advance knowledge within a specific area. It also helps you align your work with the current needs and challenges of your field. Identifying a research gap has many potential benefits.

1. Avoid Redundancy in Your Research

Understanding the existing literature helps researchers avoid duplication. This means you can steer clear of topics that have already been extensively studied. This ensures your work is novel and contributes something new to the field.

2. Guide the Research Design

Identifying a research gap helps shape your research design and questions. You can tailor your studies to specifically address the identified gap. This ensures that your work directly contributes to filling the void in knowledge.

3. Practical Applications

Research that addresses a gap is more likely to have practical applications and contributions. Whether in academia, industry, or policymaking, research that fills a gap in knowledge is often more applicable and can inform decision-making and practices in real-world contexts.

4. Field Advancements

Addressing a research gap can lead to advancements in the field . It may result in the development of new theories, methodologies, or technologies that push the boundaries of current understanding.

5. Strategic Research Planning

Identifying a research gap is crucial for strategic planning . It helps researchers and institutions prioritize areas that need attention so they can allocate resources effectively. This ensures that efforts are directed toward the most critical gaps in knowledge.

6. Academic and Professional Recognition

Researchers who successfully address significant research gaps often receive peer recognition within their academic and professional communities. This recognition can lead to opportunities for collaboration, funding, and career advancement.

How Do I Identify a Research Gap?

1. clearly define your research topic .

Begin by clearly defining your research topic. A well-scoped topic serves as the foundation for your studies. Make sure it’s not too broad or too narrow; striking the right balance will make it easier to identify gaps in existing literature.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

2. Conduct a Thorough Literature Review

A comprehensive literature review is a vital step in any research. Dive deep into the existing research related to your topic. Look for patterns, recurring themes, and consensus among scholars. Pay attention to areas where conflicting opinions or gaps in understanding emerge.

3. Evaluate Existing Studies

Critically evaluate the studies you encounter during your literature review. Assess the paradigms , methodologies, findings, and limitations of each. Note any discrepancies, unanswered questions, or areas where further investigation is warranted. These are potential indicators of research gaps.

4. Identify Unexplored Perspectives

Consider the perspectives presented in the existing literature. Are there alternative viewpoints or marginalized voices that haven’t been adequately explored? Identifying and incorporating diverse perspectives can often lead to uncharted territory and help you pinpoint a unique research gap.

Additional Tips

Stay up to date with emerging trends.

The field of research is dynamic, with new developments and emerging trends constantly shaping the landscape. Stay up to date with the latest publications, conferences, and discussions in your field and make sure to regularly check relevant academic search engines . Often, identifying a research gap involves being at the forefront of current debates and discussions.

Seek Guidance From Experts

Don’t hesitate to reach out to experts in your field for guidance. Attend conferences, workshops, or seminars where you can interact with seasoned researchers. Their insights and experience can provide valuable perspectives on potential research gaps that you may have overlooked. You can also seek advice from your academic advisor .

Use Research Tools and Analytics

Leverage tech tools to analyze patterns and trends in the existing literature. Tools like citation analysis, keyword mapping, and data visualization can help you identify gaps and areas with limited exploration.

Identifying a research gap is a skill that evolves with experience and dedication. By defining your research topic, meticulously navigating the existing literature, critically evaluating studies, and recognizing unexplored perspectives, you’ll be on your way to identifying a research gap that will serve as the foundation for your paper, thesis, or dissertation topic .

If you need any help with proofreading your research paper , we can help with our research paper editing services . You can even try a sample of our services for free . Good luck with all your research!

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

Free email newsletter template.

Promoting a brand means sharing valuable insights to connect more deeply with your audience, and...

6-minute read

How to Write a Nonprofit Grant Proposal

If you’re seeking funding to support your charitable endeavors as a nonprofit organization, you’ll need...

9-minute read

How to Use Infographics to Boost Your Presentation

Is your content getting noticed? Capturing and maintaining an audience’s attention is a challenge when...

8-minute read

Why Interactive PDFs Are Better for Engagement

Are you looking to enhance engagement and captivate your audience through your professional documents? Interactive...

7-minute read

Seven Key Strategies for Voice Search Optimization

Voice search optimization is rapidly shaping the digital landscape, requiring content professionals to adapt their...

4-minute read

Five Creative Ways to Showcase Your Digital Portfolio

Are you a creative freelancer looking to make a lasting impression on potential clients or...

Make sure your writing is the best it can be with our expert English proofreading and editing.

How to identify research gaps

Anthony Newman

About this video

Researching is an ongoing task, as it requires you to think of something nobody else has thought of before. This is where the research gap comes into play.

We will explain what a research gap is, provide you with steps on how to identify these research gaps, as well as provide you several tools that can help you identify them.

About the presenter

Senior Publisher, Life Sciences, Elsevier

Anthony Newman is a Senior Publisher with Elsevier and is based in Amsterdam. Each year he presents numerous Author Workshops and other similar trainings worldwide. He is currently responsible for fifteen biochemistry and laboratory medicine journals, he joined Elsevier over thirty years ago and has been Publisher for more than twenty of those years. Before then he was the marketing communications manager for the biochemistry journals of Elsevier.  By training he is a polymer chemist and was active in the surface coating industry before leaving London and moving to Amsterdam in 1987 to join Elsevier.

Researcher Academy on Twitter

UNE Library Services

Research Help

Gaps in the literature.

Gaps in the literature are missing pieces or insufficient information in the published research on a topic. These are areas that have opportunities for further research because they are unexplored, under-explored, or outdated. 

Finding Gaps

Gaps can be missing or incomplete:

• Population or sample: size, type, location etc…
• Research methods: qualitative, quantitative, or mixed
• Data collection or analysis
• Research variables or conditions

Conduct a thorough literature search to find a broad range of research articles on your topic. Search research databases ;  you can find recommended databases for your subject area in  research by subject  for your course or program.

Identifying Gaps

If you do not find articles in your literature search, this may indicate a gap.

If you do find articles, the goal is to find a gap for contributing new research. Authors signal that there is a gap using phrases such as:

• Has not been clarified, studied, reported, or elucidated
• Further research is required or needed
• Is not well reported
• Suggestions for further research
• Key question is or remains
• It is important to address
• Poorly understood or known
• Lack of studies
• These findings provide valuable insights into the potential benefits of mindfulness-based interventions for stress management,however, further study is needed to address several limitations and extend our understanding in this area .
• While this study provides preliminary evidence of the potential efficacy of VRET in reducing PTSD symptoms, several aspects related to its implementation and specific treatment outcomes  remain inadequately clarified, highlighting the need for further research .
• Although the studies reviewed provide valuable insights into the potential effects of climate change on species composition and ecosystem functioning.  The question of how climate change will interact with other anthropogenic stressors to influence the resilience and adaptive capacity of tropical rainforest ecosystems remains unanswered, highlighting the need for further research .

Questions & Help

If you have questions on this, or another, topic, contact a librarian for help!

Here's how to discover potential research gaps using Litmaps.

Finding research gaps is an essential part of novel research. Identifying gaps can connect disparate fields of research and advance knowledge, as well as enable opportunities for academic growth and success.

Understanding and reviewing scientific literature is essential to spotting these gaps. In this guide, we'll go over how to use Litmaps to find research gaps faster. You'll learn how to:

Comprehensively review existing literature

See where research does (and doesn't!) connect

Find inconsistencies and missing references

What is a research gap?

A research gap refers to any unexplored or unresolved area of research. It can also refer to a disconnections within the research, such as when researchers in different fields don't realize they work is connected, or a paper fails to reference important, related literature.

In the case of the former, research gaps are akin to open questions. Researchers are often eager to find unexplored areas which they can't make a novel contribution to by enhancing our understanding of that unknown topic.

In the case of the latter, many gaps happen simply because there is a huge amount of scientific literature, and even scientists themselves aren't aware of how it is all connected. One example is in how different scientific domains use different terminologies for the same concept. This results in unintentional research gaps. These gaps continue even today, because many researchers rely on keyword search, thus missing out on connections that don't match those "keywords". For example, in religious studies, scholar use the term "moral injury", which in medicine is referred to as "moral distress". Focusing on keywords alone means researchers from one field may miss important discoveries in the other.

Using Litmaps to find research gaps

Litmaps analyzes scientific literature and how it connects by using the citation network. By seeing the actual connections among articles, Litmaps let you observe how fields interact and find research gaps you'd otherwise miss.

How to find Research Gaps with Litmaps

Here's how to use Litmaps to find research gaps, by seeing where articles are and aren't connected.

1. Prepare your literature library

First, identify your topic and relevant articles. Save these articles to a Tag in Litmaps. Here's how to import papers you have into a Tag in Litmaps. If you have different subjects, save them as different Tags to stay organized.

Want to use papers you have already saved in Zotero, EndNote, Mendeley, etc? Here's how.

2 . Create a Map

Now that you have your articles in Litmaps, we'll create a Map to visualize how your articles connect. See the image below, or click here to read step-by-step instructions on how to create a Map from your Tag.

Your Litmap shows how your articles are connected via citations and references. At this point, you may already spot some inconsistencies or missing references. If you want toe expand your Map and find even more articles and potential research gaps, continue to the next step.

3 . Expand your Map

Now that you've made your Map, you can use Litmaps to discover even more related literature. Litmaps will use all the articles on your Map as an input to the search algorithm, and find other related articles.

By going through your Map, and adding additional articles, you can find related literature on your topic. For each suggested article, you can see how it connects to the existing papers you already know, so you can keep track of what parts of your field are more or less connected.

4 . Find disconnected literature

At this point, you've explored your topic and created a collection of connected articles. You may have already discovered new connections and sub-topics you didn't yet know about. However, this search has been largely limited based on connected papers. If you want to find articles you may have missed because they aren't connected at all to your existing literature, you'll need to change your search strategy.

You'll see a very different Litmap, one that likely shows many entirely disconnected articles. This is a great way to spot papers that fail to cite or reference any you already know.

5 . Future-proof your process

Now you've curated a comprehensive set of papers on your topic and spotted any potential gaps in your field. The last step is to ensure your work doesn't go out-of-date.

You can re-run this search process manually at regular intervals, to see what new literature has come out. Or, automate this process by enabling " Monitor " for your Litmap. Litmaps will automatically run your search for you as new papers are published, and let you know about any new articles on your topic.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Lippincott Open Access

Identifying Research Gaps and Prioritizing Psychological Health Evidence Synthesis Needs

Susanne hempel.

* RAND Corporation, Evidence-based Practice Center (EPC), Santa Monica

† University of Southern California, Keck School of Medicine, Los Angeles, CA

Kristie Gore

‡ RAND, National Security Research Division, Arlington

Bradley Belsher

§ Defense Health Agency, Psychological Health Center of Excellence (PHCoE), Falls Church, VA

Associated Data

Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's website, www.lww-medicalcare.com .

Supplemental Digital Content is available in the text.

Background:

Evidence synthesis is key in promoting evidence-based health care, but it is resource-intense. Methods are needed to identify and prioritize evidence synthesis needs within health care systems. We describe a collaboration between an agency charged with facilitating the implementation of evidence-based research and practices across the Military Health System and a research center specializing in evidence synthesis.

Scoping searches targeted 15 sources, including the Veterans Affairs/Department of Defense Guidelines and National Defense Authorization Acts. We screened for evidence gaps in psychological health management approaches relevant to the target population. We translated gaps into potential topics for evidence maps and/or systematic reviews. Gaps amenable to evidence synthesis format provided the basis for stakeholder input. Stakeholders rated topics for their potential to inform psychological health care in the military health system. Feasibility scans determined whether topics were ready to be pursued, that is, sufficient literature exists, and duplicative efforts are avoided.

We identified 58 intervention, 9 diagnostics, 12 outcome, 19 population, and 24 health services evidence synthesis gaps. Areas included: posttraumatic stress disorder (PTSD) (19), suicide prevention (14), depression (9), bipolar disorder (9), substance use (24), traumatic brain injury (20), anxiety (1), and cross-cutting (14) synthesis topics. Stakeholder input helped prioritize 19 potential PTSD topics and 22 other psychological health topics. To date, 46 topics have undergone feasibility scans. We document lessons learned across clinical topics and research methods.

Conclusion:

We describe a transparent and structured approach to evidence synthesis topic selection for a health care system using scoping searches, translation into evidence synthesis format, stakeholder input, and feasibility scans.

Evidence synthesis is an essential step in promoting evidence-based medicine across health systems; it facilitates the translation of research to practice. A systematic review of the research literature on focused review questions is a key evidence synthesis approach that can inform practice and policy decisions. 1 However, systematic reviews are resource-intense undertakings. In a resource-constrained environment, before an evidence review is commissioned, the need and the feasibility of the review must be established.

Establishing the need for the review can be achieved through a research gap analysis or needs assessment. Identification of a gap serves as the first step in developing a new research question. 2 Research gaps in health care do not necessarily align directly with research needs. Research gaps are only critical where knowledge gaps substantially inhibit the decision-making ability of stakeholders such as patients, health care providers, and policymakers, thus creating a need to fill the knowledge gap. Evidence synthesis enables the assessment of whether a research gap continues to exist or whether there is adequate evidence to close the knowledge gap.

Furthermore, a gap analysis often identifies multiple, competing gaps that are worthwhile to be pursued. Given the resource requirements of formal evidence reviews, topic prioritization is needed to best allocate resources to those areas deemed the most relevant for the health system. Regardless of the topic, the prioritization process is likely to be stakeholder-dependent. Priorities for evidence synthesis will vary depending on the mission of the health care system and the local needs of the health care stakeholders. A process of stakeholder input is an important mechanism to ensure that the evidence review will meet local needs as well to identify a receptive audience of the review findings.

In addition to establishing the need for an evidence review, the feasibility of conducting the review must also be established. In conducting primary research, feasibility is often mainly a question of available resources. For evidence reviews, the resources, the availability of primary research, and the presence of existing evidence reviews on the topic need to be explored. Not all topics are amenable for a systematic review which focus on a specific range of research questions and rely heavily on published literature. Furthermore, evidence review synthesizes the existing evidence; hence, if there is insufficient evidence in the primary research literature, an evidence review is not useful. Establishing a lack of evidence is a worthwhile exercise since it identifies the need for further research. However, most health care delivery organizations will be keen to prioritize areas that can be synthesized, that is, investing in synthesizing a body of research sizable enough to derive meaningful results. For evidence reviews, the presence of existing evidence syntheses is also an important consideration, in particular, to determine the incremental validity of a new review. Although primary research benefits profoundly by replication, secondary literature, in particular in the context of existing high-quality reviews and/or limited evidence, may not add anything to our knowledge base. 3

This work describes a structured and transparent approach to identify and prioritize areas of psychological health that are important and that can be feasibly addressed by a synthesis of the research literature. It describes a collaboration between an agency charged with facilitating the implementation of evidence-based research and practices across the Military Health System (MHS) and a research center specializing in evidence synthesis.

This project is anchored in the relationship between the Defense Health Agency Psychological Health Center of Excellence (PHCoE) and the RAND Corporation’s National Defense Research Institute (NDRI), one of the Federally Funded Research and Development Centers (FFRDC) dedicated to providing long-term analytic support to the Defense Health Agency. PHCoE, an agency charged with facilitating the implementation of evidence-based research and practices across the Military Health System funded a series of systematic reviews and evidence maps synthesizing psychological research. The project draws on the expertise of the Southern California Evidence-based Practice Center (EPC) located at RAND, a center specializing in evidence synthesis. The project included scoping searches, stakeholder input, and feasibility scans. The project is ongoing; this manuscript describes methods and results from June 2016 to September 2018. The project was assessed by our Human Subject Protection staff and determined to be exempt (date July 7, 2016, ID ND3621; August 6, 2017, ID ND3714).

The following describes the process, Figure ​ Figure1 1 provides a visual overview.

Process of identifying research gaps and prioritizing psychological health evidence synthesis needs.

Scoping Searches to Identify Evidence Synthesis Gaps

Scoping searches targeted pertinent sources for evidence gaps. The searches focused on clinical conditions and interventions relevant to psychological health, including biological psychiatry, health care services research, and mental health comorbidity. Proposed topics and study populations were not limited by deployment status or deployment eligibility, but the topic section considered the prevalence of clinical conditions among Department of Defense active duty military personnel managed by the MHS. The scoping searches excluded evidence gaps addressing children and adolescents and clinical conditions exclusively relevant to veterans managed by the Department of Veterans Affairs.

Scoping Search Sources

We screened 15 sources in total for evidence synthesis gaps.

Veterans Affairs/Department of Defense clinical practice guidelines were a key source for documented evidence gaps. 4 – 9 Recently updated guidelines were screened only for evidence gaps that indicated a lack of synthesis of existing research or content areas that were outside the scope of that guideline (guidelines rely primarily on published systematic reviews and can only review a limited number of topic areas).

We consulted the current report of the committee on armed services of the House of Representatives regarding the proposed National Defense Authorization Act (NDAA) and the report for the upcoming fiscal year. 10 , 11 We specifically screened the report for research priorities identified for psychological health. We also screened the published National Research Action Plan designed to improve access to mental health services for veterans, service members, and military families. 12

We conducted a literature search for publications dedicated to identifying evidence gaps and research needs for psychological health and traumatic brain injury. We searched for publications published since 2000–2016 in the most relevant databases, PubMed and PsycINFO, that had the words research gap, knowledge gap, or research priority in the title and addressed psychological health (Supplemental Digital Content, http://links.lww.com/MLR/B836 ). The search retrieved 203 citations. Six publications were considered potentially relevant and obtained as full text, 1 source was subsequently excluded because the authors conducted a literature search <3 years ago and it was deemed unlikely that a new review would identify substantially more eligible studies. 13 – 19

We also used an analysis of the utilization of complementary and alternative medicine in the MHS 20 to identify interventions that were popular with patients but for which potentially little evidence-based guidance exists. We focused our scoping efforts on complementary approaches such as stress management, hypnotherapy, massage, biofeedback, chiropractic, and music therapy to align with the funding scope. In the next step, we reviewed the existing clinical practice guidelines to determine whether clinicians have guidance regarding these approaches. The Department of Defense Health Related Behaviors Survey of Active Duty Military Personnel 21 is an anonymous survey conducted every 3 years on service members with the aim of identifying interventions or health behaviors patients currently use. To address evidence gaps most relevant to patients, we screened the survey results, and then matched the more prevalent needs identified with guidance provided in relevant clinical practice guidelines.

We consulted the priority review list assembled by the Cochrane group to identify research needs for systematic reviews. We screened the 2015–2017 lists for mental health topics that are open to new authors, that is, those that do not have an author team currently dedicated to the topic. None of the currently available topics appeared relevant to psychological health and no topics were added to the table. We also consulted with ongoing federally funded projects to identify evidence gaps that were beyond the scope of the other projects. In addition, we screened a list of psychological health research priorities developed at PHCoE for knowledge gaps that could be addressed in systematic reviews or evidence maps. Finally, we screened resources available on MHS web sites for evidence gaps.

Gap Analysis Procedure and Approach to Translating Gaps into Evidence Review Format

We first screened these sources for knowledge gaps, regardless of considerations of whether the gap is amenable to evidence review. However, we did not include research gaps where the source explicitly indicated that the knowledge gap is due to the lack of primary research. We distinguished 5 evidence gap domains and abstracted gaps across pertinent areas: interventions or diagnostic questions, treatment outcomes or specific populations, and health services research and health care delivery models.

We then translated the evidence gaps into potential topics for evidence maps and/or systematic reviews. Evidence maps provide a broad overview of large research areas using data visualizations to document the presence and absence of evidence. 22 Similar to scoping reviews, evidence maps do not necessarily address the effects of interventions but can be broader in scope. Systematic reviews are a standardized research methodology designed to answer clinical and policy questions with published research using meta-analysis to estimate effect sizes and formal grading of the quality of evidence. We considered systematic reviews for effectiveness and comparative effectiveness questions regarding specific intervention and diagnostic approaches.

Stakeholder Input

Evidence synthesis gaps that were determined to be amenable to systematic review or evidence map methods provided the basis for stakeholder input. Although all topics were reviewed by project personnel, we also identified psychological health service leads for Army, Navy, Air Force, and Marines within the Defense Health Agency as key stakeholders to be included in the topic selection process. To date, 2 rounds of formal ratings by stakeholders have been undertaken.

The first round focused on the need for systematic review covering issues related to posttraumatic stress disorder (PTSD). The second round focused on other potential psychological health topics determined to be compatible with the MHS mission. Represented clinical areas were suicide prevention and aftercare, depressive disorders, anxiety disorders, traumatic brain injury, substance use disorder including alcohol and opioid use disorder, and chronic pain. All of the potential topics addressed either the effects of clinical interventions or health service research questions.

Stakeholders rated the topics based on their potential to inform psychological health care in the military health system. The raters used a scale 5-point rating scale ranging from “No impact” to “Very high impact.” In addition, stakeholders were able to add additional suggestions for evidence review. We analyzed the mean, the mode, and individual stakeholder rating indicating “high impact” for individual topics.

Feasibility Scans

Feasibility scans provided an estimate of the volume and the type of existing research literature which is informative for 3 reasons. First, this process determined whether sufficient research was available to inform a systematic review or an evidence map. Second, feasibility scans can provide an estimate of the required resources for an evidence review by establishing whether only a small literature base or a large number of research studies exists. Finally, feasibility scans identify existing high-profile evidence synthesis reports that could make a new synthesis obsolete.

Feasibility scans for potential evidence maps concentrated on the size of the body of research that would need to be screened and the relevant synthesis questions that can inform how this research should be organized in the evidence map. Feasibility scans for systematic reviews aimed to determine the number of relevant studies, existing high-quality reviews, and the number of studies not covered in existing reviews. Randomized controlled trials (RCTs) are the focus of most of the systematic review topics, that is, strong research evidence that could inform clinical practice guideline committees to recommend either for or against interventions. An experienced systematic reviewer used PubMed, a very well-maintained and user-friendly database for biomedical literature, developed preliminary search strategies, and applied database search filters (eg, for RCTs or systematic reviews) in preliminary literature searches to estimate the research volume for each topic.

Scans also identified any existing high-quality evidence review published by agencies specializing in unbiased evidence syntheses such as the Agency for Healthcare Research and Quality (AHRQ)’s Evidence-based Practice Center program, the Cochrane Collaboration, the Campbell Collaboration, the Evidence Synthesis Program of the Department of Veterans Affairs, and the Federal Health Technology Assessment program. We used the databases PubMed and PubMed Health to identify reports. We appraised the scope, relevance and publication year of the existing high-profile evidence reviews. The research base for psychological health develops rapidly and evidence syntheses need to ensure that current clinical policies reflect the best available evidence. When determining the feasibility and appropriateness of a new systematic review, we took the results of the original review and any new studies that had been published subsequent to the systematic review on the same topic into account.

The following results are described: the results of the scoping searches and gap analysis, the translation of gaps into evidence synthesis format, the stakeholder input ratings, and the feasibility scans.

Scoping Searches and Gap Analysis Results

The scoping search and gap analysis identified a large number of evidence gaps as documented in the gap analysis table in the Appendix (Supplemental Digital Content, http://links.lww.com/MLR/B836 ). Across sources, we identified 58 intervention, 9 diagnostics, 12 outcome, 19 population, and 24 health services evidence synthesis gaps. The evidence gaps varied considerably with regard to scope and specificity, for example, highlighting knowledge gaps in recommendations for medications for specific clinical indications or treatment combinations 4 to pointing out to gaps in supporting caregivers. 11 The largest group of evidence gaps were documented for interventions. This included open questions for individual interventions (eg, ketamine) 12 as well as the best format and modality within an intervention domain (eg, use of telehealth). 6 Diagnostic evidence gaps included open questions regarding predictive risk factors that could be used in suicide prevention 8 and the need for personalized treatments. 12 Outcome evidence gaps often pointed to the lack of measured outcomes to include cost-effectiveness as well as the lack of knowledge on hypothesized effects, such as increased access or decreased stigma associated with technology-based modalities. 23 Population evidence gaps addressed specific patient populations such as complex patients 5 and family members of service members. 11 The health services evidence gaps addressed care support through technology (eg, videoconferencing 23 ) as well as treatment coordination within health care organizations such as how treatment for substance use disorder should be coordinated with treatment for co-occurring conditions. 4

Potential Evidence Synthesis Topics

The gaps were translated into potential evidence map or systematic review topics. This translation process took into account that some topics cannot easily be operationalized as an evidence review. For example, knowledge gaps regarding prevalence or utilization estimates were hindered by the lack of publicly available data. In addition, we noted that some review questions may require an exhaustive search and a full-text review of the literature because the information cannot be searched for directly, and hence were outside the budget restraints.

The clinical areas and number of topics were: PTSD (n=19), suicide prevention (n=14), depression (n=9), bipolar disorder (n=9), substance use (n=24), traumatic brain injury (n=20), anxiety (n=1), and cross-cutting (n=14) evidence synthesis topics. All topic areas are documented in the Appendix (Supplemental Digital Content, http://links.lww.com/MLR/B836 ).

Stakeholder Input Results

Stakeholders rated 19 PTSD-related research gaps and suggested an additional 5 topics for evidence review, addressing both preventions as well as treatment topics. Mean ratings for topics ranged from 1.75 to 3.5 on a scale from 0 (no impact potential) to 4 (high potential for impact). Thus, although identified as research gaps, the potential of an evidence review to have an important impact on the MHS varied across the topics. Only 2 topics received a mean score of ≥3 (high potential), including predictors of PTSD treatment retention and response and PTSD treatment dosing, duration, and sequencing . In addition, raters’ opinions varied considerably across some topics with SDs ranging from 0.5 to 1.5 across all topics.

The stakeholders rated 22 other psychological health topics, suggested 2 additional topics for evidence review, and revised 2 original topics indicating which aspect of the research gap would be most important to address. Mean scores for the rated topics ranged from 0.25 to 3.75, with the SDs for each item ranging from 0 to 1.4. Six topics received an average score of ≥3, primarily focused on the topics of suicide prevention, substance use disorders, and telehealth interventions. Opinions on other topics varied widely across service leads.

Feasibility Scan Results

Evidence review topics that were rated by stakeholders as having some potential for impact (using a rating cutoff score>1) within the MHS were selected for formal feasibility scans. To date, 46 topics have been subjected to feasibility scans. Of these, 11 were evaluated as potential evidence map, 17 as a systematic review, and 18 as either at the time of the topic suggestion. The results of the feasibility scans are documented in the table in the Appendix (Supplemental Digital Content, http://links.lww.com/MLR/B836 ).

The feasibility scan result table shows the topic, topic modification suggestions based on literature reviews, and the mean stakeholder impact rating. The table shows the employed search strategy to determine the feasibility; the estimated number of RCTs in the database PubMed; the number and citation of Cochrane, Evidence Synthesis Program, and Health Technology Assessment reviews, that is, high-quality syntheses; and the estimated number of RCTs published after the latest existing systematic review that had been published on the topic.

Each potential evidence review topic was discussed in a narrative review report that documented the reason for determining the topic to be feasible or not feasible. Reasons for determining the topic to be not feasible included the lack of primary research for an evidence map or systematic review, the presence of an ongoing research project that may influence the evidence review scope, and the presence of an existing high-quality evidence review. Some topics were shown to be feasible upon further modification; this included topics that were partially addressed in existing reviews or topics where the review scope would need to be substantially changed to result in a high-impact evidence review. Topics to be judged feasible met all outlined criteria, that is, the topic could be addressed in a systematic review or evidence map, there were sufficient studies to justify a review, and the review would not merely replicate an existing review but make a novel contribution to the evidence base.

The project describes a transparent and structured approach to identify and prioritize evidence synthesis topics using scoping reviews, stakeholder input, and feasibility scans.

The work demonstrates an approach to establishing and evaluating evidence synthesis gaps. It has been repeatedly noted that research gap analyses often lack transparency with little information on analytic criteria and selection processes. 24 , 25 In addition, research need identification may not be informed by systematic literature searches documenting gaps but primarily rely on often unstructured content expert input. 26 , 27 Evidence synthesis needs assessment is a new field that to date has received very little attention. However, as health care delivery organizations move towards providing evidence-based treatments and the existing research continue to grow, both evidence reviews and evidence review gap identification and prioritization will become more prominent.

One of the lessons learned is that the topic selection process added to the timeline and required additional resources. The scoping searches, translation into evidence synthesis topics, stakeholder input, and feasibility scans each added time and the project required a longer period of performance compared to previous evidence synthesis projects. The project components were undertaken sequentially and had to be divided into topic areas. For example, it was deemed too much to ask for stakeholder input for all 122 topics identified as potential evidence review topics. Furthermore, we needed to be flexible to be able to respond to unanticipated congressional requests for evidence reviews. However, our process of identifying synthesis gaps, checking whether topics can be translated into syntheses, obtaining stakeholder input to ensure that the gaps are meaningful and need filling, and estimating the feasibility and avoiding duplicative efforts, has merit considering the alternative. More targeted funding of evidence syntheses ensures relevance and while resources need to be spent on the steps we are describing, these are small investments compared to the resources required for a full systematic review or evidence map.

The documented stakeholder engagement approach was useful for many reasons, not just for ensuring that the selection of evidence synthesis topics was transparent and structured. The stakeholders were alerted to the evidence synthesis project and provided input for further topic refinement. This process also supported the identification of a ‘customer’ after the review was completed, that is, a stakeholder who is keen on using the evidence review is likely to take action on its results and ready to translate the findings into clinical practice. The research to practice gap is substantial and the challenges of translating research to practice are widely documented. 28 – 30 Inefficient research translation delays delivery of proven clinical practices and can lead to wasteful research and practice investments.

The project had several strengths and limitations. The project describes a successful, transparent, and structured process to engage stakeholders and identifies important and feasible evidence review topics. However, the approach was developed to address the specific military psychological health care system needs, and therefore the process may not be generalizable to all other health care delivery organizations. Source selection was tailored to psychological health synthesis needs and process modifications (ie, sources to identify gaps) are needed for organizations aiming to establish a similar procedure. To keep the approach manageable, feasibility scans used only 1 database and we developed only preliminary, not comprehensive searches. Hence, some uncertainty about the true evidence base for the different topics remained; feasibility scans can only estimate the available research. Furthermore, the selected stakeholders were limited to a small number of service leads. A broader panel of stakeholders would have likely provided additional input. In addition, all evaluations of the literature relied on the expertise of experienced systematic reviewers; any replication of the process will require some staff with expertise in the evidence review. Finally, as outlined, all described processes added to the project timeline compounding the challenges of providing timely systematic reviews for practitioners and policymakers. 31 , 32

We have described a transparent and structured approach to identify and prioritize areas of evidence synthesis for a health care system. Scoping searches and feasibility scans identified gaps in the literature that would benefit from evidence review. Stakeholder input helped ensure the relevance of review topics and created a receptive audience for targeted evidence synthesis. The approach aims to advance the field of evidence synthesis needs assessment.

Supplementary Material

Acknowledgments.

The authors thank Laura Raaen, Margaret Maglione, Gulrez Azhar, Margie Danz, and Thomas Concannon for content input and Aneesa Motala and Naemma Golshan for administrative assistance.

Supported by the Office of the Secretary of Defense, Psychological Health Center of Excellence. The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the views of the Psychological Health Center of Excellence, the Office of the Secretary of Defense, or the United States government.

The authors declare no conflict of interest.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Journal Proposal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Online pedagogies and the middle grades: a scoping review of the literature.

1. Introduction

2. rationale, 3. objectives, 4. materials and methods, 5. the current review boundaries.

• Published in a peer-reviewed journal;
• Published between January 2013 and June 2024;
• Specific focus on research and/or pedagogically based articles pertaining to middle-grade (5–8) online teaching and learning;
• Research explicit to middle-grade adolescent participants (grades 5–8) or middle-grade teachers;
• Pedagogically focused articles that speak specifically to pedagogical suggestions, strategies, or resources for middle-grade (5–8) online teaching (i.e., fully virtual, blended, emergency remote teaching);
• Research and pedagogical articles with a national and/or international focus;
• Schools could be public, private, charter, religious, or alternative settings;
• Written in English;
• Inclusive of theoretical, empirical, conceptual, and pedagogical articles.
• Editorials and book reviews;
• Articles that center on technological pedagogical knowledge (i.e., a pedagogical focus on ways that particular technological tools or applications can be utilized within teaching [ 25 ]) or digital pedagogy (i.e., a pedagogical focus on the strategic incorporation of contemporary digital technologies in education to enhance teaching, assessment, and curriculum [ 26 ]) rather than online pedagogy (emphasis on technological-based pedagogies accomplished at least partially through asynchronous or synchronous virtual instruction [ 13 ]).
• Books and book chapters;
• Gray literature;
• Articles that included participants or data from participants outside the middle grades (i.e., inclusive of high school, elementary, or higher education teachers and/or students);
• Dissertations and thesis papers.

6. Search Process

• The Association for Middle Level Education (AMLE) ( n = 2);
• The Online Teaching and Learning SIG (OTL) of the American Education Research Association (AERA) ( n = 2);
• International Society for Technology in Education (ISTE) ( n = 0);
• European League for Middle Level Education (ELMLE) ( n = 0).
• Voices from the Middle ( n = 10);
• Middle School Journal ( n = 5);
• Journal of Online Learning Research ( n = 4);
• Middle Grades Review ( n = 4);
• Journal of Research on Technology in Education ( n = 2);
• Research in Middle Level Education Online ( n = 1);
• Journal of Digital Learning in Teacher Education ( n = 0);
• Quarterly Review of Distance Education ( n = 0);
• Educational Considerations ( n = 0);
• Educational Media International ( n = 0);
• Journal of Interactive Online Learning ( n = 0);
• Online Journal of Distance Learning Administration ( n = 0);
• Online Learning Journal ( n = 0);
• Journal of Online Learning Research ( n = 0);
• Education and Information Technologies ( n = 0);
• Education Technology and Society ( n = 0);
• MLS Educational Research ( n = 0);
• International Journal of Online Pedagogy and Course Design ( n = 0);
• Journal of Distance Education ( n = 0).

7.1. Trends in Online Pedagogical Literature

7.2. research methodologies, 7.3. study contexts, 7.4. online learning models, 7.5. study focus, 8. current gaps in the literature, 9. amle essential attributes and middle-grade online pedagogical research, 10. conclusions, 11. limitations of the scoping review, 12. recommendations.

• What is the state of the digital divide in online learning in the middle grades?
• To what extent are the AMLE essential attributes—responsive, challenging, empowering, equitable, engaging—explicitly employed in the design of online learning for young adolescents?
• What is the potential of augmented and immersive realities in middle-grade education?
• What accountability and performance measures are in place for virtual schools?
• How are teacher-preparation programs preparing middle-grade educators for effective online teaching?
• How has the use of online, blended, and emergency remote pedagogies impacted student engagement, collaboration, and learning in the middle grades?
• What are the long-term impacts of online teaching and learning on student development and academic success?

Author Contributions

Institutional review board statement, data availability statement, conflicts of interest.

• Digital Learning Collaborative [DLC]. Snapshot 2019: A Review of K-12 Online, Blended, and Digital Learning. 2019. Available online: https://www.digitallearningcollab.com (accessed on 18 June 2024).
• Barbour, M.K. The Shift to Distance Learning: Tracing the Roots of 100+ Years of Practice and Opportunity. Tech Trends 2021 , 65 , 919–922. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Barbour, M.K.; Brown, R.; Hasler Waters, L.; Hoey, R.; Hunt, J.; Kennedy, K.; Ounsworth, C.; Powell, A.; Trimm, T. Online and Blended Learning: A Survey of Policy and Practice from K-12 Schools around the World . International Association for K-12 Online Learning. 2011. Available online: https://aurora-institute.org/resource/online-and-blended-learning-a-survey-of-policy-and-practice-from-k-12-schools-around-the-world/ (accessed on 26 June 2024).
• Clark, T. The Evolution of K-12 Distance Education and Virtual Schools. In Handbook of Distance Education ; Moore, M.G., Ed.; Routledge: Abingdon, UK, 2013; Volume 3, pp. 555–573. [ Google Scholar ]
• Toppin, I.N.; Toppin, S.M. Virtual Schools: The Changing Landscape of K12 Education in the U.S. Educ. Inf. Technol. 2016 , 21 , 1571–1581. [ Google Scholar ] [ CrossRef ]
• Darrow, R. A Comparative Study between Online Charter High Schools and Traditional High Schools in California. Unpublished Doctoral Dissertation, California State University, Fresno, CA, USA, 2010. [ Google Scholar ]
• Clark, T. Virtual Schools: Trends and Issues—A Study of Virtual Schools in the United States ; Western Regional Educational Laboratories: San Francisco, CA, USA, 2001. [ Google Scholar ]
• Molnar, A.; Miron, G.; Hagle, S.; Gulosino, C.; Mann, B.; Huerta, L.A.; Rice, J.K.; Glover, A.; Bill, K. Virtual Schools in the U.S. 2023. National Education Policy Center. 2023. Available online: http://nepc.colorado.edu/publication/virtual-schools-annual-2023 (accessed on 26 June 2024).
• U.S. Department of Education, National Center for Education Statistics (NCES). Characteristics of Public and Private Elementary and Secondary Schools in the United States: Results from the 2017–18 National Teacher and Principal Survey First Look. 2019; Table 3. Available online: https://nces.ed.gov/fastfacts/display.asp?id=79 (accessed on 18 June 2024).
• Digital Learning Collaborative [DLC]. Snapshot 2024: A Review of K-12 Online, Blended, and Digital Learning. 2024. Available online: https://www.digitallearningcollab.com/snapshot-2024 (accessed on 15 July 2024).
• Hodges, C.; Moore, S.; Lockee, B.; Trust, T.; Bond, A. The Difference between Emergency Remote Teaching and Online Learning. Educause Review . 2020. Available online: https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning (accessed on 18 June 2024).
• Johnson, C.C.; Walton, J.B.; Strickler, L.; Elliott, J.B. Online Teaching in K12 Education in the United States: A Systematic Review. Rev. Educ. Res. 2023 , 93 , 353–411. [ Google Scholar ] [ CrossRef ]
• Vawter, D.H.; McMurtrie, D.H. Meeting the Academic, Social, and Emotional Needs of Our Middle Level Students in the Online Environment. Middle Sch. J. 2022 , 53 , 26–35. [ Google Scholar ] [ CrossRef ]
• Kennedy, K.; Tomaselli, K.; Stimson, R. National Standards for Quality Online Courses (K-12) and QM K-12 Secondary and K-12 Publisher Rubric Revision: Literature Review. Quality Matters . 2018. Available online: https://www.qualitymatters.org/sites/default/files/research-docs-pdfs/National-Standards-for-Quality-Online-Courses-Lit-Review-122818.pdf (accessed on 12 July 2024).
• Association for Middle Level Education. 2024. AMLE.org. Available online: https://www.amle.org/ (accessed on 8 July 2024).
• Bishop, P.; Harrison, L. The Successful Middle School: This We Believe ; AMLE: Westerville, OH, USA, 2021. [ Google Scholar ]
• Barbour, M.K. Looking Back to see Ahead: An Analysis of K-12 Distance, Online, and Remote Learning during the Pandemic. J. Digit. Soc. Res. 2022 , 4 , 7–25. [ Google Scholar ] [ CrossRef ]
• Evergreen Education Group; Gemin, B.; Pape, L. Keeping Pace with K-12 Online Learning 2016 ; Evergreen Education Group: Durango, CO, USA, 2017. [ Google Scholar ]
• Barbour, M. The Promise and the Reality: Exploring Virtual Schooling in Rural Jurisdictions. Educ. Rural Aust. 2011 , 21 , 1–19. [ Google Scholar ] [ CrossRef ]
• Barbour, M.; McLaren, A.; Zhang, L. It’s Not That Tough: Students Speak about Their Online Learning Experiences. Turk. Online J. Distance Educ. 2012 , 13 , 226–241. [ Google Scholar ]
• DiPietro, M.; Ferdig, R.; Black, E.; Presto, M. Best Practices in Teaching K-12 Online: Lessons Learned from Michigan Virtual School Teachers. J. Interact. Online Learn. 2010 , 9 , 10–35. [ Google Scholar ]
• Arksey, H.; O’Malley, L. Scoping Studies: Towards a Methodological Framework. Int. J. Soc. Res. Methodol. 2005 , 8 , 19–32. [ Google Scholar ] [ CrossRef ]
• Frydman, J.; Mayor, C. A Scoping Review on the Use and Potential of School-based Drama Therapy to Enhance Socio-emotional Skills in Early Childhood. Early Child. Educ. J. 2023 , 52 , 1–12. [ Google Scholar ] [ CrossRef ]
• Mays, N.; Roberts, E.; Popay, J. Synthesising research evidence. In Studying the Organisation and Delivery of Health Services: Research Methods ; Routledge: Abingdon, UK, 2001. [ Google Scholar ]
• Schmidt, D.A.; Baran, E.; Thompson, A.E.; Mishra, P.; Koehler, M.J.; Shin, T.S. Technological Pedagogical Content Knowledge (TPACK): The Development and Validation of an Assessment Instrument for Preservice Teachers. J. Res. Technol. Educ. 2009 , 42 , 123–149. [ Google Scholar ] [ CrossRef ]
• Kivunja, C. Embedding Digital Pedagogy in Pre-service Higher Education to Better Prepare Teachers for the Digital Generation. Int. J. Sustain. High. Educ. 2013 , 2 , 131–142. [ Google Scholar ] [ CrossRef ]
• Brinegar, K.M.; Moulton, M.; Falbe, K.N.; Rintamaa, M.; Ellerbrock, C.R. Navigating opportunities for middle level education research: The MLER SIG Research Agenda. Res. Middle Level Educ. Online 2024 , 47 . [ Google Scholar ]
• Miyashiro, C. The Impact of a Global Pandemic on Teaching: A Narrative of Two Veteran Middle School Teachers in Hawaii. Delta Kappa Gamma Bull. 2023 , 89 , 52–63. [ Google Scholar ]
• Eisenbach, B.B.; Greathouse, P. Stage-environment Fit and Middle Level Virtual Learners: A Phenomenological Case Study. RMLE Online 2020 , 43 , 1–12. [ Google Scholar ] [ CrossRef ]
• Rice, M.F.; Ortiz, K.R.; Curry, T.M.; Petropoulos, R. A Case Study of a Foster Parent Working to Support a Child with Multiple Disabilities in a Full-time Virtual School. J. Online Learn. Res. 2019 , 5 , 145–168. Available online: https://www.learntechlib.org/primary/p/184933/ (accessed on 12 May 2024).
• Deng, F. Literature Review of the Flipped Classroom. Theory Pract. Lang. Stud. 2019 , 9 , 1350–1356. [ Google Scholar ] [ CrossRef ]
• Nida, N.K.; Usodo, B.; Saputro, D.R.S. The Blended Learning with WhatsApp Media on Mathematics Creative Thinking Skills and Math Anxiety. J. Educ. Learn. 2020 , 14 , 307–314. [ Google Scholar ] [ CrossRef ]
• Swan, B.; Coulombe-Quach, X.-L.; Huang, A.; Godek, J.; Becker, D.; Zhou, Y. Meeting the Needs of Gifted and Talented Students: Case Study of a Virtual Learning Lab in a Rural Middle School. J. Adv. Acad. 2015 , 26 , 294–319. [ Google Scholar ] [ CrossRef ]
• Longo, C.M. Changing the instructional model: Utilizing blended learning as a tool of inquiry instruction in middle school science. Middle Sch. J. 2006 , 47 , 33–40. [ Google Scholar ] [ CrossRef ]
• Yalavaç, G.; Samur, Y. Students’ and Teachers’ Perceptions of After School Online Course. Eur. J. Contemp. Educ. 2016 , 15 , 147–162. [ Google Scholar ]
• Cui, Y.; Zhao, G.; Zhang, D. Improving Students’ Inquiry Learning in Web-based Environments by Providing Structure: Does the Teacher Matter or Platform Matter? Br. J. Educ. Technol. 2022 , 53 , 1049–1068. [ Google Scholar ] [ CrossRef ]
• Renick, J.; Reich, S.M. The Mismatch of Virtual Instruction Practices with Young Adolescents’ Developmental Needs. Middle Sch. J. 2023 , 54 , 28–40. [ Google Scholar ] [ CrossRef ]
• Selvakumar, V.; Venkata, T.P.; Venkata, T.P.; Singh, S. Predicting Primary and Middle-school Students’ Preferences for Online Learning with Machine Learning. S. Afr. J. Child. Educ. 2023 , 13 , a1324. [ Google Scholar ] [ CrossRef ]
• Bishop, P.A. Middle Grades Teacher Practices During the COVID-19 Pandemic. RMLE Online 2021 , 44 , 1–18. [ Google Scholar ] [ CrossRef ]
• Beeman, K.L. The Quiet Girl in a Virtual World: Learning from the Virtual Classroom to Better Support Quiet Girls in the Middle Grades. RMLE Online 2022 , 45 , 1–19. [ Google Scholar ] [ CrossRef ]
• Griendling, L.M.; VanUitert, V.J.; McDonald, S.D. Are Students’ Basic Psychological Needs Fulfilled in Remote Learning Environments?: A Mixed Methods Study. Middle Grades Rev. 2022 , 8 , 1–15. [ Google Scholar ]
• Khlaif, Z.N.; Salha, S.; Kouraichi, B. Emergency Remote Learning during COVID-19 Crisis: Students’ Engagement. Educ. Inf. Technol. 2021 , 26 , 7033–7055. [ Google Scholar ] [ CrossRef ]
• Marcum-Dietrich, N.; Stunkard, C.; Krauss, Z.; Kerlin, S.; Staudt, C.; Muenz, T.; Kline, D. Stormy WATERS: COVID-19 Transition to Online Learning for an Environmental Education Middle School Curriculum. Sci. Educ. 2022 , 28 , 97–106. [ Google Scholar ]
• Pace, J.R.; Mellard, D.F. Reading Achievement and Reading Efficacy Changes for Middle School Students with Disabilities Through Blended Learning Instruction. J. Spec. Educ. Technol. 2016 , 31 , 156–169. [ Google Scholar ] [ CrossRef ]
• Hashemifardnia, A.; Namaziandost, E.; Shafiee, S. The Effect of Implementing Flipped Classrooms on Iranian Junior High School Students’ Reading Comprehension. Theory Pract. Lang. Stud. 2018 , 8 , 665–673. [ Google Scholar ] [ CrossRef ]
• Nafiâ, E.R.; Purwanti, E.; Permana, F.H.; Fauzi, A. Metacognitive Skills of Junior High School Students in a Pandemic Period Based on the Enriched Virtual Model of PjBL. J. Educ. Technol. 2022 , 6 , 29–37. [ Google Scholar ] [ CrossRef ]
• Daley, S.G.; Hillaire, G.; Sutherland, L.M. Beyond Performance Data: Improving Student Help Seeking by Collecting and Displaying Influential Data in an Online Middle-School Science Curriculum. Br. J. Educ. Technol. 2016 , 47 , 121–134. [ Google Scholar ] [ CrossRef ]
• Zhong, C.; Lyu, K. Scaffolding Junior Middle School Students’ Engagement in Online Project-based Learning During the COVID-19 Pandemic: A Case Study from East China. Sage Open 2022 , 12 , 21582440221131815. [ Google Scholar ] [ CrossRef ]
• Mariati, N.K.S.; Artini, L.P.; Marsakawati, N.P.E. Students’ Perception of Blended Learning at Junior High School (Combination of Synchronous and Asynchronous Learning). IDEAS J. Engl. Lang. Teach. Learn. Linguist. Lit. 2022 , 10 , 1235–1245. [ Google Scholar ] [ CrossRef ]
• Simanjorang, E.; Simatupang, J.; Butarbutar, W.; Purba, K.; Tampubolon, S.; Lestari Lumbantoruan, F.D. The Blended Learning Method in Improving Speaking Skills of Middle School Students. Edusaintek J. Pendidik. Sains Dan Teknol. 2023 , 10 , 694–703. [ Google Scholar ] [ CrossRef ]
• Stratton, E.; Chitiyo, G.; Mathende, A.M.; Davis, K.M. Evaluating Flipped Versus Face-to-face Classrooms in Middle School on Science Achievement and Student Perceptions. Contemp. Educ. Technol. 2020 , 11 , 131–142. [ Google Scholar ] [ CrossRef ]
• Aslan, S.A.; Duruhan, K. The Effect of Virtual Learning Environments Designed According to Problem-based Learning Approach to Students’ Success, Problem-solving Skills, and Motivations. Educ. Inf. Technol. 2021 , 26 , 2253–2283. [ Google Scholar ] [ CrossRef ]
• Ateş, H. Designing a Self-regulated Flipped Learning Approach to Promote Students’ Science Learning Performance. Educ. Technol. Soc. 2024 , 27 , 65–83. [ Google Scholar ] [ CrossRef ]
• Winter, J.W. Performance and Motivation in a Middle School Flipped Learning Course. TechTrends 2018 , 62 , 176–183. [ Google Scholar ] [ CrossRef ]
• Asim, S.; Ponners, P.J.; Bartlett, C.; Parker, M.A.; Star, R. Differentiating Instruction: For Middle School Students in Virtual Learning Environments. Delta Kappa Gamma Bull. 2020 , 86 , 19–31. [ Google Scholar ]
• Fassbender, W.; Lucier, J. Equalizing the Teacher-to-Student Ratio through Technology: A New Perspective on the Role of Blended Learning. Voices Middle 2014 , 22 , 21–28. [ Google Scholar ] [ CrossRef ]
• Gonzalez, A. Strategies to get Started with Blended Learning. Voices Middle 2014 , 22 , 34–38. [ Google Scholar ] [ CrossRef ]
• Benavides, D.M. A Flipped Classroom Experience to Analyze Language Interaction in a Group of Students of English as a Foreign Language. MLS Educ. Res. 2023 , 7 , 152–169. [ Google Scholar ] [ CrossRef ]
• Mabrur, I.A.M.; Suwartono, T.; Lutfiana. Junior High School Students’ Readiness to Participate in e-Learning and Online EFL Classes During the COVID-19 Pandemic. Int. Soc. Sci. J. 2021 , 71 , 153–161. [ Google Scholar ] [ CrossRef ]
• Louwrens, N.; Hartnett, M. Student and Teacher Perceptions of Online Student Engagement in an Online Middle School. J. Open Flex. Distance Learn. 2015 , 19 , 27–44. [ Google Scholar ] [ CrossRef ]
• Nissinen, S.; Vartiainen, H.; Vanninen, P.; Pöllänen, S. Connected Learning in International Learning Projects: Emergence of a Hybrid Learning System. Int. J. Inf. Learn. Technol. 2019 , 36 , 381–394. [ Google Scholar ] [ CrossRef ]
• Wendt, J.L.; Rockinson-Szapkiw, A.J. The Effect of Online Collaboration on Adolescent Sense of Community in Eighth-grade Physical Science. J. Sci. Educ. Technol. 2015 , 24 , 671–683. [ Google Scholar ] [ CrossRef ]
• Wahyuni, S.; Sanjaya, I.G.M.; Erman, E.; Jatmiko, B. Edmodo-based Blended Learning Model as an Alternative of Science Learning to Motivate and Improve Junior High School Students’ Scientific Critical Thinking Skills. Int. J. Emerg. Technol. Learn. 2019 , 14 , 98–110. [ Google Scholar ] [ CrossRef ]
• Çakiroǧlu, N.; Pişkin Tunç, M. Middle School Students’ Experiences and Perceptions about Flipped Classroom Applications Used in Geometry Lessons. Acta Didact. Napoc. 2022 , 15 , 26–42. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Eisenbach, B.; Coleman, B. Online Pedagogies and the Middle Grades: A Scoping Review of the Literature. Educ. Sci. 2024 , 14 , 1017. https://doi.org/10.3390/educsci14091017

Eisenbach B, Coleman B. Online Pedagogies and the Middle Grades: A Scoping Review of the Literature. Education Sciences . 2024; 14(9):1017. https://doi.org/10.3390/educsci14091017

Eisenbach, Brooke, and Bridget Coleman. 2024. "Online Pedagogies and the Middle Grades: A Scoping Review of the Literature" Education Sciences 14, no. 9: 1017. https://doi.org/10.3390/educsci14091017

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Advertisement

Explainable deep learning approach for advanced persistent threats (APTs) detection in cybersecurity: a review

• Open access
• Published: 18 September 2024
• Volume 57 , article number  297 , ( 2024 )

Cite this article

You have full access to this open access article

• Noor Hazlina Abdul Mutalib 1 ,
• Aznul Qalid Md Sabri 1 ,
• Ainuddin Wahid Abdul Wahab 2 ,
• Erma Rahayu Mohd Faizal Abdullah 1 &
• Nouar AlDahoul 3  

In recent years, Advanced Persistent Threat (APT) attacks on network systems have increased through sophisticated fraud tactics. Traditional Intrusion Detection Systems (IDSs) suffer from low detection accuracy, high false-positive rates, and difficulty identifying unknown attacks such as remote-to-local (R2L) and user-to-root (U2R) attacks. This paper addresses these challenges by providing a foundational discussion of APTs and the limitations of existing detection methods. It then pivots to explore the novel integration of deep learning techniques and Explainable Artificial Intelligence (XAI) to improve APT detection. This paper aims to fill the gaps in the current research by providing a thorough analysis of how XAI methods, such as Shapley Additive Explanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME), can make black-box models more transparent and interpretable. The objective is to demonstrate the necessity of explainability in APT detection and propose solutions that enhance the trustworthiness and effectiveness of these models. It offers a critical analysis of existing approaches, highlights their strengths and limitations, and identifies open issues that require further research. This paper also suggests future research directions to combat evolving threats, paving the way for more effective and reliable cybersecurity solutions. Overall, this paper emphasizes the importance of explainability in enhancing the performance and trustworthiness of cybersecurity systems.

Explore related subjects

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

Advanced persistent threats (APTs) represent a significant cybersecurity challenge in the digital era. (Hasan et al. 2023 ). In this study, we explore the integration of Explainable Artificial Intelligence (XAI) with deep learning models to enhance the detection of Advanced Persistent Threats (APTs). APTs are prolonged and targeted cyberattacks in which an intruder gains access to a network and remains undetected for an extended period. Unlike traditional attacks, APTs aim to steal sensitive data over time rather than causing immediate damage. APT attacks are conducted by highly skilled adversaries with vast resources and target organizations through IT network systems for long-term access, without being discovered. APT attacks target critical infrastructure, such as financial institutions, government agencies, energy companies, shipping and transportation companies, industrial companies, and food services, which can disrupt critical infrastructure and cause massive damage. APT attacks have major impacts on countries, organizations, and individuals, leading to severe financial losses, reputational damage, legal liability, lost productivity, business continuity issues, and disruption of critical operations (Salim et al. 2023 ). For example , the Stuxnet worm, attributed to state-sponsored actors, targeted Iran’s nuclear facilities and caused significant operational disruptions (Ahmad et al. 2024 ). Another notable example is the SolarWinds attack in 2020, which involved compromising the Orion software platform (Alkhadra et al. 2021 ). This attack affected numerous government agencies and private sector companies, leading to widespread data breaches and significant financial losses. The estimated cost of the SolarWinds breach is still being assessed, but the vulnerability of even the most secure networks to sophisticated APTs has been highlighted.

The National Institute of Standards and Technology (NIST) defines APTs as sophisticated, resourceful adversaries that use multiple attack methods, such as cyber-attacks, physical intrusions, and deception to infiltrate an organization’s infrastructure​​ (de Abreu et al. 2020 ). APTs are particularly challenging to detect due to their stealth and persistence, blending with legitimate network traffic and activities (Salim et al. 2023 ). Unlike typical malware, which executes its payload quickly, APTs focus on long-term espionage or data theft, making detection more difficult​​ (Jabar and Mahinderjit Singh 2022 ).

(Source: Symantec)

APT attack distribution by Leafminer group.

In August 2018, Symantec Footnote 1 reported APT attacks by the Leafminer group, known as RASPITE, in the Middle Eastern region since 2017 Footnote 2 . Figure  1 illustrates the distribution of APT attacks by the Leafminer group, across various industrial sectors. Government agencies were one of the primary targets, representing 17% of all attacks. Financial institutions were equally targeted, accounting for 17% of all attacks. The Ponemon Institute, as analyzed by IBM Security, reported that the average cost of an APT attack and data breach in 2023 was $4.35 million Footnote 3 . Large organizations experienced an average loss of $6.93 million per incident. This distribution emphasizes the need for robust cybersecurity measures across diverse industries to protect against sophisticated threats, such as those posed by the Leafminer group. In 2020, $945 billion was lost due to cyber incidents, and another $145 billion was spent on cybersecurity. These costs have surged by more than 50% since 2018, when approximately $600 billion was allocated to mitigate cybercrime (B. Ballard, 2021).

In summary, APT attack detection requires specialized techniques that go beyond the traditional detection methods. These specialized techniques must address the unique challenges posed by APTs, including their stealth, persistence, targeted nature, sophistication, advanced evasion techniques, and focus on data exfiltration. Detecting APTs often involves leveraging advanced deep-learning methods, continuous monitoring, and the use of XAI to ensure transparency and trust in the detection process (Schwalbe and Finzel 2023 ).

1.1 Paper organization

The organization structure of the review

Figure  2 provides the organization structure for this review. It outlines the main sections and subsections of the paper and provides a clear roadmap of the content and topics covered. In Sect. 2, we introduce the background of Advanced Persistent Threats (APTs), including the lifecycle and characteristics of these cybersecurity threats. Section 3 outlines the research methods, detailing the search strategy and eligibility criteria for selecting relevant literature. Section 4 delves into related work, discussing the classification of APT detection, various deep learning models used, and the limitations of existing detection methods. Section 5 presents the methodology, including a comparative analysis of the deep learning and XAI approaches, the need for XAI, and relevant case studies. Section 6 focuses on explainable AI in cybersecurity, explores explanation methods, integrates XAI techniques in deep learning, and addresses the current issues surrounding black box models. Section 7 discusses the findings, providing key considerations of XAI, challenges, and recommendations for future research. Finally, Sect. 8 concludes the review and suggests future research directions at the intersection of XAI and APT detection.

1.2 Motivation

The increasing sophistication of APT attacks and the limitations of existing IDS drive the key motivation for this review. Traditional IDS methods struggle with low detection accuracy, high false-positive rates, and the inability to detect unknown or early-stage attacks. For example, signature-based IDS can detect only known threats, making them ineffective against novel APTs (Sarker et al. 2024 ). Furthermore, these methods have difficulty detecting unknown or early-stage attacks because of their reliance on signature-based detection mechanisms, which fail to recognize novel threats.

Additionally, comprehensive studies detailing the current state of interpretability in published research and the application of state-of-the-art XAI models in the cybersecurity domain are lacking (Saeed and Omlin 2023 ). Without interpretability, cybersecurity experts cannot fully trust or understand the decisions made by AI models, which is crucial for accurately identifying false positives and negatives and for adapting to new attack vectors (Brown et al. 2022 ).

This review examines a critical gap in the literature on the implementation of deep learning techniques with XAI for APT detection. Table  1 provides a summary of various studies on APT detection methods, highlighting the objectives, motivations, and key findings of each study. This comparative analysis helps to identify gaps and challenges in existing methods, providing a foundation for enhancing state-of-the-art APT detection.

By addressing these gaps, our study aims to enhance the effectiveness and transparency of APT detection systems, thereby advancing the state-of-the-art in cybersecurity.

1.3 Contribution

In this review, we address the challenges of APT detection by integrating deep learning techniques with XAI methods. The following section discusses the key contributions of our research.

Our research expands the state-of-the-art in cybersecurity by providing robust, scalable, and interpretable detection systems that can effectively combat sophisticated APT attacks.

We evaluated robust APT detection frameworks that improve detection accuracy and scalability while providing clear, interpretable insights into the model’s decision-making process. This approach ensures that cybersecurity experts can respond more effectively to threats and understand how and why specific decisions are made.

We highlight the limitations of existing detection systems, such as low detection accuracy and high false-positive rates. Our study demonstrates how integrating deep learning with XAI can address these challenges, paving the way for more robust and real-time detection capabilities.

We include case studies to demonstrate the application of XAI in combating APTs. These scenarios illustrate the potential impact of implementing XAI techniques in real-world settings, providing a comprehensive view of how these advanced methods can transform APT detection and response.

We examine various issues in the cybersecurity area and propose an improved method of APT detection on the basis of the specific attributes of such attacks. This tailored approach enhances detection capabilities and addresses the unique challenges posed by APTs.

In summary, our research provides significant contributions to the field of cybersecurity by integrating XAI APT detection systems. By addressing the current limitations and demonstrating the practical application of our methods, we pave the way for more robust, scalable, and transparent cybersecurity solutions. Our work enhances the overall resilience of network systems to APTs, ensuring better protection and response capabilities against sophisticated APT attacks.

The scope of this paper includes a comprehensive review of various deep learning techniques utilized for APT detection, the application of XAI methods to improve model interpretability and trustworthiness, and a critical analysis of existing approaches. By examining the intersection of deep learning and XAI, this paper aims to provide valuable insights and pave the way for more effective and reliable cybersecurity solutions. Our approach aligns with the need for explainability in AI models to ensure their practical application and trustworthiness.

Having established the importance of APT detection and the potential of XAI, we delve into the background study to provide a foundation for our research.

2 Background

In this section, we provide an overview of the background studies relevant to APT detection. Our focus is on the various methods that have been explored in the literature, such as deep learning and XAI approaches. This review aims to place our research within the wider scope of existing research and highlight the contributions we make to advancing the state-of-the-art in APT detection.

Deep learning approaches have gained significant attention for their ability to automatically learn complex patterns from large datasets. Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTM) networks have shown promising results when applied to APT detection. These models capture temporal and spatial dependencies in data, making them suitable for detecting sophisticated attack patterns. However, the black-box nature of deep learning models poses challenges in interpretability and transparency, which are critical for cybersecurity applications. To address the interpretability challenge, XAI techniques have been developed. XAI aims to make AI models’ decision-making processes transparent and understandable to humans. Techniques such as SHAP and LIME help elucidate the contributions of individual features to a model’s predictions, enhancing trust and facilitating collaboration between human experts and AI systems.

APTs pose a unique characteristic cyber-attack problem. They target specific victims and use tricks to hide. Skilled, well-funded attackers use various TTPs to evade detection (Sharma et al. 2023 ). APTs are managed by highly skilled and resourceful adversaries, often backed by nation-states or organized crime groups (Lemay et al. 2018 ). These threat actors have clear goals, such as espionage, sabotage, and damage, and spend much of their time succeeding (Ahmad et al. 2019 ). For example, the Lazarus Group established a method for compromising common software on the Internet with Trojans. These advanced attackers use social engineering techniques to inject malicious software into a target system. This allows them to attack the supply chain via untrustworthy sources (Villalón-Huerta et al. 2022 ).

The detection of APT attacks presents several distinct challenges compared with malware. The ways in which APT attack detection differs include the following:

Stealth and persistence: APTs remain undetected for extended periods by blending with legitimate network traffic and activities (Bierwirth et al. 2024 ) (Moustafa & Slay, 2016 ). Unlike typical malware, which may execute its payload quickly, APTs focus on long-term espionage or data theft, making detection more difficult.

Targeted nature: APTs are highly targeted and tailored to specific organizations or individuals, rendering generic security solutions less effective. Additionally, APT tactics, techniques, and procedures (TTPs) are continuously evolving (Ahmad et al. 2019 ) requiring deep learning models to be frequently retrained to detect new threats.

Sophistication and complexity: APT attacks involve multiple stages, such as initial access, lateral movement (Fang et al., 2022 ), data exfiltration, and persistence. This multistep approach necessitates sophisticated detection mechanisms to identify and correlate various stages.

Advanced evasion techniques: APTs use advanced evasion techniques,  such as encryption, obfuscation, and polymorphism, to avoid detection by traditional methods (Shenderovitz & Nissim, 2024 ). They may also use legitimate system tools for malicious activities, complicating detection.

Focus on data exfiltration: While malware might aim to cause immediate damage or disruption, APTs primarily aim to steal sensitive information, posing significant detection challenges (Sharma et al. 2023 a).

Nation-State Involvement: Many APTs are linked to nation-state actors conducting espionage or intelligence gathering, leveraging vast resources and advanced capabilities (Holt et al., 2023) (Bierwirth et al. 2024 ).

Mitigation Strategies: Addressing the threat of APTs requires a multilayered approach (Mohamed, 2023 ) that includes proactive network monitoring, continuous vulnerability assessments, user awareness training, and robust incident response plans.

2.1 APT attack

APTs follow a well-defined lifecycle that consists of several stages. Initially, attackers infiltrate the target system using methods such as phishing, spear-phishing, or exploiting vulnerabilities in web applications. Once inside, they establish a foothold by installing malware to maintain persistence within the system. Subsequently, they escalate privileges by exploiting system vulnerabilities to gain deeper access. The attackers then perform internal reconnaissance, moving laterally within the network to map infrastructure and identify critical assets. In the next stage, attackers extract sensitive data and send it to the external servers they control. Finally, attackers maintain persistence using advanced evasion techniques to remain undetected and retain long-term access to compromised systems (Bierwirth et al. 2024 ).

APTs are characterized by their sophisticated and prolonged attack campaigns, which follow a well-defined lifecycle. The APT attack life cycle can be divided into six phases, intelligence gathering, point of entry, command and control (C&C), lateral movement, data of interest and the external server. The initial phase of an APT attack is reconnaissance or intelligence gathering, in which the attackers gather information about their target organization such as network infrastructure and identify potential vulnerabilities. This phase involves passive information gathering techniques and social media analysis (Bodstrom & Hamalainen, 2019 ). Once the attackers have identified potential entry points, they move to the establish phase, where they prepare customer malware to exploit the target environment. If successful, the attackers establish a foothold in the compromised systems, installing backdoors and remote access tools (RATs) to maintain persistence. From this point, the APT actors proceed to the command and control (C&C) phase, establishing secure communication channels with their malicious infrastructure to receive updates and exfiltrate data. With a persistent presence in the target network, the attackers move to the lateral movement phase, where they seek to escalate privileges and gain access to additional systems and resources. The final stages of the APT lifecycle involve the actual achievement of the attackers’ objectives, such as data of interest, data exfiltration, sabotage, or establishing long-term access for future attacks (Chen et al., 2018 ).

In summary, APTs represent a sophisticated and persistent threat to organizations worldwide. Their ability to remain undetected and adapt to new defenses makes them particularly challenging to address. Recent high-profile cases, such as the SolarWinds (Alkhadra et al. 2021 ) and Hafnium attacks, highlight the critical need for advanced detection methods. This paper aims to explore the integration of deep learning techniques with Explainable AI (XAI) to enhance APT detection and provide more transparent, interpretable solutions for cybersecurity experts.

3 Research methods

In this section, we outline the research questions and search strategy that guided our study. Next, we explain the research process.

3.1 Research questions

We established a set of research questions to address important aspects of APT detection. These questions will guide the development of an effective detection system. They are designed to explore the performance, limitations, and potential enhancements of APT detection systems. Table  2 provides description of the research questions as follows:

This section outlines the key research questions that drive our investigation into APT detection. By addressing these questions, we aim to uncover the strengths and weaknesses of current systems, explore the benefits of integrating XAI, and identify the unique challenges in applying these advanced techniques.

3.2 Search strategy and eligibility criteria

In this review, we conducted a rigorous search strategy and defined clear eligibility criteria for the selection of relevant papers. We searched multiple academic databases, including IEEE Xplore, ACM Digital Library, ScienceDirect, Springer Link, Web of Science, and Google Scholar. These databases were chosen for their relevance to research in computer science, cybersecurity, and artificial intelligence. We used specific search terms to find studies related to APT detection mechanisms. Boolean expressions such as (“Advanced Persistent Threat” OR “APT”) AND (“Deep Learning AND “Cybersecurity”) AND (“Explainable Artificial Intelligence” OR “XAI”) were used to combine search terms.

Our review aimed to include papers presenting novel methods, particularly those involving deep learning-based APT detection, the application of deep learning, and XAI techniques, along with empirical results. The inclusion criteria required papers to be published in English in peer-reviewed journals, conference proceedings, or book chapters from 2018 to 2024. We excluded papers without abstracts, without access to full content, or not written in English.

Overview of the search strategy and eligibility criteria

We reviewed selected articles that met our inclusion criteria, focusing on those that provided insights into the integration of XAI with APT detection. Through this process, we collected 100 articles deemed the most relevant to our study. Figure  3 provides an overview of our search strategy and eligibility criteria. To refine the search results and relevant papers, we applied the following inclusion criteria:

The paper must be written in English and published in peer-reviewed journals, conference proceedings, or book chapters.

The study must focus on applying deep learning techniques specifically for detecting APT attacks.

The study must involve applying XAI methods in cybersecurity, specifically for APT detection.

The study must provide empirical results, such as experiments or case studies, which demonstrate the effectiveness of the proposed approach in real-world or simulated scenarios.

The papers must have been published between 2018 to 2024, reflecting recent advancements in deep learning and XAI research.

This section outlines the rigorous search strategy and well-defined eligibility criteria used to select relevant papers for this review. The search was conducted across several major academic databases using specific terms related to APT detection, deep learning, and XAI.

4 Related work

In this section, we review the existing literature on APT detection methods, focusing on both traditional and advanced approaches. We organize the discussion to provide a clearer understanding of the strengths and weaknesses of each approach relative to our study. Current IDS and security measures face significant limitations when dealing with APTs.

4.1 The classification of APT detection approaches

Classifying APT detection approaches on the basis of data sources such as Network Traffic Analysis, Host-based Analysis, and Log and Event Analysis allows for a more targeted and efficient detection strategy. A study by (Do Xuan et al. 2020 ) analyzed network traffic into IP-based network flows, reconstructed IP information from these flows, and used deep learning models to extract features to distinguish APT attack IPs from other IPs. They introduced a combined deep learning model using Bidirectional Long Short-Term Memory (BiLSTM) and Graph Convolutional Networks (GCN). Meanwhile, host-based analysis focuses on the data and activities occurring on individual hosts or endpoints within a network.(Chen et al. 2024 ) proposed a hybrid Network Intrusion Detection System (NIDS) that combines host-based intrusion detection (HIDS) and network-based detection to enhance the detection of APTs and other network intrusions. Log and Event Analysis involves collecting and analyzing logs and events generated by various systems, applications, and devices within an organization. (Wang et al. 2022 ) presented a novel method for reconstructing APTs in large-scale networks to improve attack forensics and traceability. The proposed method addresses these issues with a low transmission cost and does not require raw data from terminal devices, making it suitable for extensive networks with numerous terminal devices. Classifying APT detection approaches based on explainability involves distinguishing between black-box models and XAI models. The black-box models will be explained in Sect. 5.4. XAI models offer transparent and interpretable insights into the detection process. These models aim to maintain a high level of accuracy while ensuring that the reasoning behind their predictions is clear and understandable to cybersecurity experts. This transparency increases trust in the detection system and facilitates a faster and more effective response to identified threats.

4.2 Deep learning models

A taxonomy of AI/XAI based methods for cybersecurity modeling. Adapted this figure from (Sarker et al. 2024 )

Machine learning (ML) is a key component of data science, and is important in terms of its flexibility, scalability, and adaptability to new challenges. This article explores ML applications in cybersecurity, including phishing detection, network intrusion detection, spam detection in social networks, smart meter energy consumption profiling, and security concerns inherent in ML techniques. To understand this, we have added Fig.  4 , which shows taxonomies based on specific themes and deep learning (DL) methods. This structured approach will not only add more organization to our review but also increase its reference value by providing a clearer framework. This study emphasizes the methodology of collecting large datasets, extracting relevant features, and training ML models using supervised learning algorithms to achieve high accuracy and low false positive rates.

(Manoharan et al. 2023 ) addressed the challenge of detecting insider threats, which pose significant cybersecurity risks. This study evaluated various supervised machine learning algorithms on a balanced dataset using the same feature extraction method and investigated the impact of hyperparameter tuning and different conditions on imbalanced datasets. Using the publicly available CERT r4.2 dataset, the results showed that Random Forest achieved the best accuracy and F1-score of 95.9%, outperforming existing methods such as the DNN, LSTM Autoencoder, and User Behavior Analysis. (Raju et al. 2021 ) explored various ML models, including decision trees and support vector machines, to detect APT activities. Although these models offered improved detection capabilities over traditional methods, they still struggled with interpretability. Cybersecurity experts often find it difficult to understand the reasoning behind ML predictions, which hampers their trust and usability. (Stojanović et al. 2020 ) proposed the use of ensemble methods to combine multiple ML models, thereby enhancing detection accuracy. However, this approach increases computational complexity and may not be suitable for real-time applications. Our study leverages the strengths of ML while addressing its weaknesses by using deep learning and XAI to provide more interpretable and scalable solutions.

Deep learning is a widely used technique for APT detection, offering several advantages over traditional machine learning techniques. These advantages include the ability to process and analyze unstructured data, such as text, images, and network traffic, as well as high-dimensional data, which are common in cybersecurity domains. Unlike traditional methods, which often rely on predefined rules and signatures, deep learning models can automatically learn complex patterns and features from large datasets. This capability allows for more accurate and robust detection of sophisticated cyber threats, making deep learning an essential tool in the evolving landscape of cybersecurity (Alzubaidi et al. 2021 ).

(Mittal et al. 2023 ) conducted a systematic review on deep learning for detecting Distributed Denial of Service (DDoS) attacks, analyzing literature from multiple sources. They categorize findings into five key areas: deep learning approaches, methodologies, datasets, preprocessing strategies, and research gaps. This study evaluates existing methods, highlights strengths and weaknesses, and identifies gaps in current research, suggesting future directions. This review offers a comprehensive overview, is organized for clarity, and emphasizes the relevance of deep learning in addressing the evolving threat of DDoS attacks. Despite its thoroughness, the study’s reliance on existing literature and the rapid evolution of attack strategies may limit its timeliness and practical application.

4.2.1 Convolutional neural networks (CNNs)

Among the various types of deep learning techniques, Convolutional Neural Networks (CNNs) excel at handling high-dimensional data and automatically learn and extract relevant spatial features from raw data. This capability makes them particularly effective in identifying complex patterns associated with APT activities. In computer vision, researchers use CNNs to recognize APTs by extracting features from APT data (Teuwen and Moriakov 2020 ). The CNN architecture includes five block layers: convolutional layers, pooling layers, channel max-pooling layers, rectified linear unit (ReLU) layers, fully connected layers, and a SoftMax loss function (Alzubaidi et al. 2021 ). However, CNNs have limitations in converting one-dimensional traffic flows into two-dimensional flows without considering the spatial correlations between the traffic flows. (Alzubaidi et al. 2021 )

(Jayapradha et al. 2024 ) proposed an innovative intrusion detection system (IDS) that leverages deep learning algorithms to enhance phishing detection. Specifically, CNNs automatically extract sophisticated features from raw input data, whereas RNNs effectively model sequential data to recognize phishing patterns over time. This IDS adapts in real-time to new phishing variants through back propagation-based model optimization, significantly improving detection accuracy compared with traditional rule-based methods. By utilizing the KDD-CUP99 dataset for training, the study demonstrates a robust defense against evolving phishing threats. Consequently, the system enables a proactive incident response and strengthens overall network security. Despite potential challenges in terms of computational complexity and data dependency, the proposed system offers substantial improvements in protecting networks and users from sophisticated phishing attacks.

(Patel et al. 2024 ) proposed an advanced malware detection system that leverages RNNs and CNNs to enhance cybersecurity vigilance. The study highlights a pioneering capability by extending detection to unconventional formats such as GIFs and images, addressing emerging threats that exploit multimedia channels. This approach demonstrates the adaptability and comprehensiveness of the deep learning-based system. Traditional cybersecurity methods struggle to keep pace with dynamic cyber adversaries, but this research leverages deep learning’s ability to recognize complex patterns within vast datasets. By integrating RNNs and CNNs, this study aims to improve the accuracy of threat identification and the system’s resilience against emerging and previously unseen cyber threats. This study focuses on broadening the detection scope to multimedia formats and enhancing the system’s adaptability and reliability. The strengths of this study lie in its innovative approach and comprehensive coverage of diverse data formats, whereas its weaknesses include potential computational complexity and reliance on high-quality datasets for training.

(Tadesse and Choi 2024 ) proposed a novel intrusion detection system (IDS) that converts raw datasets into image datasets using the Short-Term Fourier Transform (STFT) to enhance pattern recognition. This system uses a lightweight convolutional neural network (CNN) to classify denial of service (DoS) and distributed denial of service (DDoS) attacks. Evaluated on both modern (CSE-CIC-IDS2018) and legacy (NSLKDD) datasets, the proposed methods achieve high accuracy and low false alarm rates, demonstrating high specificity and sensitivity. The study highlights the model’s excellent generalizability and avoidance of overfitting across different datasets, emphasizing the effectiveness of the dataset conversion methodology.

(Najar & S., 2024 ) proposed an innovative feature selection approach to develop a robust and reliable intrusion detection system (IDS) for detecting and classifying Distributed Denial-of-Service (DDoS) attack types. Using the CICDDoS2019 benchmark dataset, the model demonstrates high performance, achieving 96.82% accuracy, 96.82% recall, 96.76% precision, and a 96.50% F1 score. It also provides rapid prediction times, identifies attacks in just 0.189 milliseconds, and outperforms existing methods and baseline models. The contributions include a novel feature selection approach, effective preprocessing techniques, and comprehensive evaluation using benchmark datasets. The strengths of this study are its high detection accuracy, rapid response, balanced performance metrics, and innovative methodology. However, weaknesses include challenges with imbalanced data handling, computational complexity, dependency on dataset quality, generalizability to other attack types, and implementation challenges.

(Korium et al. 2024 ) proposed an advanced IDS tailored for the Internet of Vehicles environment, leveraging CNNs to detect both traditional and new cyber-attacks effectively. The novel network architecture of the model at the data processing layer and the use of the synthetic minority oversampling technique significantly enhanced the detection accuracy and speed, achieving a notable 94% detection rate using the AWID dataset. The strengths of this study are its high detection accuracy, improved data processing through synthetic oversampling, innovative network architecture, and comprehensive evaluation with the AWID dataset. The model’s performance is heavily dependent on the AWID dataset, and its scalability may be limited by the computational complexity introduced by CNNs and data processing techniques.

(Sun 2024 ) proposed a novel intrusion detection system (IDS) that combines data preprocessing with four deep neural network models: Convolutional Neural Networks (CNN), Bi-directional Long Short-Term Memory (BiLSTM), Bidirectional Gate Recurrent Unit (BiGRU), and Attention mechanism to identify network attacks accurately. Evaluated using the NSL-KDD dataset, the models utilize preprocessing techniques and particle swarm optimization for feature selection, with hyperparameter tuning via the BO-TPE algorithm. The study’s contributions include a comprehensive IDS framework, the introduction of multiple DNN architectures, and effective feature selection and class imbalance handling, resulting in high detection accuracy rates of 0.999158 in binary classification and 0.999091 in multiclass classification. The strengths of this study include its thorough approach, high accuracy, and advanced optimization techniques, whereas its weaknesses include dataset dependency, computational complexity, generalizability concerns, implementation challenges, and the need for further real-world application exploration.

(Yin et al. 2024 ) introduced a novel multi-scale Convolutional Neural Networks (CNN) and bidirectional Long-short Term Memory (bi-LSTM) arbitration dense network model (MSCBL-ADN) for detecting LDDoS attacks effectively with limited datasets and reduced time consumption. The MSCBL-ADN model integrates a CNN for spatial feature extraction, bi-LSTM for temporal relationship extraction, an arbitration network to re-weigh feature importance, and a 2-block dense connection network for final classification. The experimental results on the ISCX-2016-SlowDos dataset demonstrate that the MSCBL-ADN model significantly improves detection accuracy and time performance compared to state-of-the-art models.

(Ersavas et al. 2024 ) explored the potential of Convolutional Neural Networks (CNNs) beyond traditional image processing, highlighting their ability to analyze high-dimensional datasets by transforming them into pseudo-images. Despite the rise of newer architectures such as Transformers, CNNs remain crucial in various applications, including Generative AI. The authors introduce DeepMapper, a pipeline that enables the analysis of complex datasets without intermediate filtering or dimension reduction, thus preserving data integrity and detecting small variations typically considered noise. They demonstrated that DeepMapper can efficiently and accurately identify subtle perturbations in large datasets with numerous features, highlighting its superiority in speed and comparable accuracy to existing methods.

Mendonça et al. (2023) proposed a new IDS using a hierarchical tree-CNN algorithm with soft-root-sign (SRS) activation (Daoud et al. 2023 ) to detect attacks in infiltrations, Distributed Denial-of-Service (DDoS), brute force, and web attacks. The authors reported that their proposed system could detect DDoS attacks with high accuracy and a reduced execution time of approximately 36%. Furthermore, the results showed a significant increase in the average detection accuracy of 98% when all the analyzed attacks were considered. This indicates that Tree-CNN performs better because it is less complex, requires less processing time, and consumes fewer computing resources than other current machine-learning-based IDSs do.

On the other hand, Zhu & Zu, 2022 ) implemented a fully convolutional neural network (FCNN) classifier architecture that substitutes the linear layers and correlated activation functions from prevalent CNN classifiers. They trained the FCNN classifier using SoftMax loss. The main benefits of this architecture are its simplicity and flexibility. However, it does not consider channel information. They can use SoftMax loss directly to train the network by reconstructing the number of output channels. The predefined best distribution (POD loss) of the latent features was used to improve the recognition rate performance with SoftMax loss.

Network Intrusion Detection Systems (NIDS) are essential for detecting malicious activities in modern networks. However, class imbalance in intrusion detection datasets hinders the performance of classifiers in minority classes. To address this, (Zhang et al. 2020 ) proposed a novel class imbalance processing technique called the SGM, which combines the Synthetic Minority Over-Sampling Technique (SMOTE) and under-sampling using Gaussian Mixture Model (GMM). They developed a flow-based intrusion detection model, SGM-CNN, which integrates imbalance processing with a convolutional neural network (CNN). They evaluated the performance on the UNSW-NB15 and CICIDS2017 datasets, and reported high detection rates of 99.74% for binary classification, 96.54% for multiclass classification for the UNSW-NB15 dataset, and 99.85% for 15-class classification for the CICIDS2017 dataset. The authors claimed that SGM-CNN effectively addresses the challenge of imbalanced intrusion detection and outperforms the existing state-of-the-art methods.

(Tian, 2020) proposed a combination approach, the CNN algorithm, to learn the deep features of an image using CNNs and RNNs in parallel. The authors subsequently created a ShortCut3-ResNet residual module. In their study, they demonstrated that the convolutional neural network algorithm can identify various features of images, optimize the accuracy of feature extraction, and improve the ability of the convolutional neural network to recognize images.

CNNs have shown significant promise in enhancing cybersecurity measures through various innovative approaches. These studies highlight the adaptability of CNNs in recognizing complex patterns, handling diverse data formats, and integrating with other deep learning models to improve detection accuracy and resilience against sophisticated cyber threats. Despite challenges such as computational complexity and data dependency, CNN-based systems offer substantial improvements in protecting networks and users from evolving threats.

4.2.2 Recurrent neural networks (RNNs)

Recurrent Neural Networks (RNNs) are specifically designed to handle sequential data, making them particularly suitable for analyzing time-series data in network traffic. By incorporating internal memory states, RNNs can effectively process input sequences of varying lengths, capturing temporal dependencies within the data (DiPietro and Hager 2020 ). This capability allows RNNs and their advanced variants, such as Long Short-Term Memory (LSTM) networks, to model the sequential nature of APT attack campaigns (Yuan et al. 2017 ). RNNs have proven effective for APT detection because they can learn from event sequences over time and predict future actions based on past observations. This makes RNNs an invaluable tool for identifying and mitigating sophisticated cyber threats (Galli et al. 2024 ).

In the current landscape of complex cyber threats, network security is of utmost importance. (Kumaresan et al. 2024 ) evaluated the efficacy of Recurrent Neural Networks (RNNs) in network anomaly detection, comparing them with traditional methods such as statistical techniques and simple neural networks. Using an extensive dataset of normal and malicious network traffic, the authors demonstrate the potential of RNNs to detect anomalies by utilizing sequential dependencies in the data. They investigated various RNN architectures, hyperparameter settings, and feature representations to improve detection performance. The paper also addresses challenges such as model interpretability, scalability, and computational resource demands, and proposes ways to increase the resilience of RNN-based systems against malicious interference.

(Yang et al. 2024 ) introduced the Hypergraph Recurrent Neural Network (HRNN), a novel intrusion detection method that leverages hypergraph structures and recurrent networks. The HRNN represents flow data as hypergraph structures to enhance information representation and incorporates a recurrent module to extract temporal features. This design integrates rich spatial and temporal semantics, significantly improving anomaly detection capabilities. Evaluations on several publicly available datasets demonstrate that the HRNN outperforms other state-of-the-art methods, demonstrating its superior performance in detecting network anomalies. However, the increased complexity due to the use of hypergraph structures and recurrent networks may impact computational efficiency and scalability. Furthermore, the performance of the HRNN may heavily depend on the quality and characteristics of the datasets used for training and evaluation, potentially requiring fine-tuning for different data forms and limiting its transferability. Integrating HRNN with existing network infrastructure might also present technical challenges.

(Saravanan et al. 2023 ) proposed a Blockchain-based African Buffalo (BbAB) scheme with a Recurrent Neural Network (RNN) to enhance an IDS. The method encrypts normal and malware user datasets using Identity Based Encryption (IBE) and securely stores them in a blockchain within a cloud environment. The RNN detects intrusions, using African buffalo optimization for continuous monitoring. The approach achieves 99.87% accuracy and 99.92% recall, demonstrating robust detection capabilities. While the method improves security and monitoring, it presents challenges such as high computational complexity and significant resource requirements. Overall, the model enhances IDS effectiveness in cloud environments.

(Pahuja and Ojha 2024 ) addressed the growing threat of network attacks by deploying deep learning techniques, including Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU), to detect and mitigate Denial-of-Service (DoS) attacks. These models analyze sequential time-series data to identify patterns linked to DoS attacks. Among the techniques evaluated, LSTM achieved the highest accuracy of 92.3%, demonstrating superior capability in classifying attack traffic. This approach aims to reduce system unavailability and potential losses caused by DoS attacks, although it faces challenges in computational complexity and scalability.

(Sakthipriya et al. 2024 ) proposed enhancing IoT network security by reducing data dimensionality for efficient attack classification on memory-constrained devices. They used a Conditional Adversarial Auto Encoder (CAAE) to generate realistic botnet traffic and extract deep features, combined with a Dilated and Cascaded Recurrent Neural Network (DC-RNN) for accurate classification. The model achieves 96% accuracy, 98% precision, 97% F1-score, and 96% recall. This approach addresses deep learning implementation challenges in IoT environments, outperforming conventional methods. However, it faces issues with computational complexity, data dependency, scalability, generalizability, and implementation challenges in existing IoT infrastructures.

(Hasan et al. 2023 ) proposed an effective identification model for APT attacks by boosting-based machine learning methods combined with XAI to enhance prediction and provide actionable insights. The model, which achieves a high weighted F1 score of 0.97 with XGBoost, effectively predicts APT attacks and utilizes SHAP to make predictions understandable and actionable for cybersecurity stakeholders. While the approach demonstrates high detection accuracy and offers a promising framework for future research, it faces challenges such as computational complexity, data dependency, generalizability, implementation difficulties, and resource intensity, potentially impacting scalability and practical application in resource-constrained environments.

Furthermore, Lo et al. ( 2023 ) introduced the XG-BoT GNN and used GNNExplainer on a botnet graph dataset. Their model demonstrated impressive precision between 99.23% and 99.63%. The effectiveness of this approach relies heavily on the quality of the input data.

AlDahoul et al. ( 2021 ) developed a fusion model that combined two DNNs, trained using class-weight optimization. The main idea is to learn complex patterns from rare anomalies in the traffic data. The authors used Adam optimization algorithms with classes to train the DNNs. Their results showed that their method could achieve a higher accuracy in terms of the Fβ score and the false alarm rate when their proposed model compared to conventional single DNNs. Furthermore, their experiment used the ZYELL real-world datasetand yielded promising results.

These studies demonstrate the versatility and effectiveness of RNNs and their variants in enhancing cybersecurity measures, particularly in detecting and mitigating sophisticated cyber threats. Despite challenges such as computational complexity and data dependency, RNN-based systems offer substantial improvements in network security.

4.2.3 Autoencoders

Autoencoders (AEs) are used for anomaly detection to represent normal behavior and subsequently identify deviations from this learned behavior. These neural network models are trained on datasets comprising normal activity, enabling them to compress and reconstruct input data accurately. When presented with anomalous data, autoencoders produce higher reconstruction errors, thus flagging the anomalies. They have shown significant promise in detecting APTs by highlighting unusual activities within the network traffic, such as unexpected patterns that differ from established norms, thereby providing a robust mechanism for identifying potential security breaches.

(Yashwanth et al. 2024 ) proposed a novel approach by combining Auto-encoders with Multi-Layer Perceptron (MLP). The study evaluates three algorithms: Auto-encoders, Auto-encoders with MLP, and CNNs, and the results demonstrate their effectiveness in detecting network intrusions. This study highlights the significance of using diverse machine learning techniques for effective anomaly detection, pattern recognition in complex network traffic, and handling imbalanced data, although challenges in computational complexity, resource requirements, and implementation in existing IDS infrastructure exist. Table  3 summarizes the deep learning-based methods used in cybersecurity.

4.3 Limitations of existing detection methods

The application of deep learning for APT detection presents several challenges. Traditional IDSs typically employ predefined signatures of known threats, which fail to detect new, unknown, or evolving APT tactics that do not match existing signatures. Anomalybased detection systems generate numerous false positives by flagging benign activities as potential threats, leading to alert fatigue and overwhelming cybersecurity experts, thereby reducing the overall effectiveness of the security operations center (SOC). Detecting early-stage attacks, such as initial compromise and establishing a foothold, is difficult because these activities often involve subtle actions that are difficult to distinguish from normal behavior. Traditional methods lack the ability to detect early indicators of compromise. APTs employ sophisticated evasion techniques, including encryption, polymorphism, and the use of legitimate system tools for malicious purposes, making traditional IDSs challenging. Furthermore, conventional methods often lack the ability to contextualize alerts within a broader network environment, failing to correlate seemingly unrelated events that together indicate an ongoing APT attack.

The limitations of the existing detection methods emphasize the need for advanced techniques that can effectively identify and mitigate APTs. Deep learning is a promising approach because of its ability to analyze large volumes of data and identify complex patterns indicative of APT activities. For example, using Convolutional Neural Networks (CNNs) to detect anomalies in network traffic data can uncover subtle changes indicative of a potential threat that traditional methods might miss. XAI can highlight the specific features of the data that led to this detection. However, the black-box nature of deep-learning models poses challenges in terms of interpretability and trust, making it difficult for cybersecurity experts to understand and act upon the model’s predictions (Hassija et al., 2024 ). XAI enhances the transparency and interpretability of deep learning models, enabling cybersecurity experts to gain insight into the model’s decision-making process. This not only improves the effectiveness of APT detection, but also fosters trust and reliability in automated security systems.

By focusing on the unique characteristics of APTs and addressing the specific limitations of current methods, this review aims to improve state-of-the-art APT detection and provide practical solutions for enhancing cybersecurity defenses. Cybersecurity experts must understand the explanations behind the model’s predictions.

4.4 Datasets

Overview of datasets used in APT detection

In this section, we delve into datasets curated for APT detection. Figure  5 shows an overview of the most widely used methods in this field. In cybersecurity, benchmark datasets play a critical role in APT attacks (Agrawal et al. 2024 ).

We categorize the datasets based on their characteristics, such as the type of data, network traffic, system logs, presence of labeled attack scenarios, and extent of real-world applicability. This analysis helps to understand the strengths and limitations of each dataset, guiding researchers in selecting the most appropriate data for their studies. However, publicly available datasets that capture the behavior of APT attacks are lacking (Khraisat et al. 2019 ). This limitation hinders the development of effective APT detection models. Some existing datasets for analyzing APT attacks, such as DARPA1998, NSK-KDD 2009, UNSW-NB15, CICIDS2017, and ZYELL, focus on general network intrusion detection. However, they do not specifically target APT attacks.

DARPA1998 is the first dataset collection of attack traces from the internet. MIT Lincoln compiled and distributed it under DARPA and ARFL to evaluate network intrusion detection. It has seven weeks of training data and two weeks of testing data containing 38 attacks from four diverse groups: DoS, U2R, R2L, and probe (Homoliak et al. 2020 ). Another widely used dataset is the NSL-KDD (2009), which is a revised version of the KDD99 dataset. This revision reduces classifier bias and provides better detection rates (V C et al., 2023 ). Researchers commonly use the NSL-KDD for evaluation. This is an improved version of the original KDD-Cup99 benchmark. It considers 148,514 network traffic items across 41 features and five main attack types (Yang et al. 2021 ). The dataset includes denial-of-service attacks, remote access breaches, R2L, U2R, and probes from 77,054 benign samples (Barnard et al. 2022 ). The ISCX-IDS2012 dataset contains 2,381,532 data samples and is based on profiles that include intrusion details.

The UNSW-NB15 dataset was created at the Australian Center for Cyber Security (ACCS) Cyber RangeLab, using the IXIA Perfect Storm tool. It exhibits both benign and malicious attacks. The dataset included 49 features, with a single-class label indicating the connection property of each data instance (Moustafa and Slay 2016 ). The dataset contains nine types of attacks: analysis, fuzzers, generic, exploits, DoS, backdoors, reconnaissance, worms, and shellcodes. Another dataset, CICIDS2017, was created based on a large-scale cybersecurity research project that collected information from more than three million computers globally beginning in April 2016. The CICIDS2017 comprises 78 features, 168,186 normal samples, and 2,180 attack samples (Patil et al. 2022 ).

The ZYELL dataset was generated to detect network anomalies using real-world network traffic data obtained from the ZYELL security system (L. Chen et al., 2021 ). The dataset comprises both benign and malicious network traffic with a proportion of anomalies of approximately 1%. It also targets two main types of attacks: probing and DoS (AlDahoul et al. 2021 ). It has 22 features, including connection duration and inbound/outbound traffic counts in bytes, and is stored as csv files (L. Chen et al., 2021 ). The dataset includes 9,241,463 training samples and 13,290,530 testing samples. However, creating an effective APT detection system using network datasets from different sources and organizations can be challenging because of privacy concerns and the lack of a universal dataset format.

In conclusion, a lack of comprehensive datasets remains a major challenge in the research and development of deep-learning models for APT detection (Karim et al. 2024 ). Collaborative efforts among researchers, industry partners, and government agencies are needed to create large-scale, diverse, and labeled datasets that can support the advancement of deep learning models in this domain. This will help the deep learning techniques in this field move forward. While previous studies have demonstrated the promise of deep learning and XAI in cybersecurity, there remains a need for integrated approaches that enhance both detection accuracy and interpretability.

5 Methodology

We outline the methodology used in our study on how XAI techniques are integrated with deep learning models. The methodology included the following key steps; data collection, data preprocessing, deep learning models, black box models, XAI models, and model explanations. Figure  6 provides an overview of the APT detection enhancement process, illustrating the various stages from data collection to model explanation. This framework aims to provide a clear understanding of the integration of different models and techniques for effective and concise APT detection.

Proposed XAI pipeline for APT detection

5.1 Data collection

We gathered data from various sources, including network logs, system events, and user activities. The latest dataset from the Symantec Virtual Conference (SVC) 2021, SCVIC-APT-2021 (Liu et al. 2022 ) were utilized due to their comprehensive logs of simulated APT attacks and normal network traffic.

5.2 Data preprocessing

To ensure the quality and consistency of the data, the following preprocessing steps were performed:

Data cleaning involves identifying and removing irrelevant or redundant data points to improve the quality of the dataset (Sakthipriya et al. 2024 ) (Ridzuan and Zainon 2019 ). Irrelevant data points, such as incomplete logs or noise generated by benign activities, do not contribute to APT detection. Redundant data points are duplicate records that can skew the analysis.

Normalization ensures that data values are standardized to a common scale, which is crucial for the performance of machine learning models (Davis et al. 2020 ). Different features in the dataset may have varying scales, and normalization helps bring them to a comparable range (Siddiqi and Pak 2021 ).

Feature engineering involves identifying and extracting relevant features that can distinguish between normal and malicious activities (Abbas et al. 2023 ). This process enhances the model’s ability to detect APTs by providing it with informative and discriminative attributes (Abu Bakar et al. 2023 ).

5.3 Deep learning models

Deep learning models have significantly advanced the field of cybersecurity, particularly in detecting advanced persistent threats (APTs). We utilized several deep learning models, each suited to different types of data:

Convolutional Neural Networks (CNNs) are widely employed for their ability to automatically learn and extract spatial features from raw data, making them effective for high-dimensional data (Taye 2023 ). CNNs apply convolutional layers to input data, use filters to detect spatial hierarchies and identify various patterns and structures, which is particularly useful for datasets such as images or network traffic logs (Ersavas et al. 2024 ). CNNs automatically extract spatial features through layers of convolution and pooling. This hierarchical feature extraction process enables CNNs to construct increasingly abstract representations of the input data, from low-level details to high-level concepts (Geng and Niu 2024 ). Leveraging their strengths in feature extraction and pattern recognition, CNNs effectively analyze patterns in network traffic, such as abnormal packet flows or unusual access behaviors, allowing them to detect complex and subtle anomalies that signify APTs. By using their powerful feature extraction and pattern recognition capabilities, CNNs detect these intricate patterns and enhance the detection of sophisticated cyber threats.

Recurrent Neural Networks (RNNs) are another crucial type of deep learning model, that are adept at capturing temporal patterns in network traffic, such as time-series data, thereby enhancing the detection of APTs (Hewamalage et al. 2021 ). RNNs are specialized for sequential data, making them suitable for analyzing time-series data such as network traffic logs (Das et al. 2023 ). RNNs have internal memory states that enable them to retain information about previous inputs while processing current ones, effectively capturing temporal dependencies. Long Short-Term Memory (LSTM) networks, a type of RNN, are designed to handle long-term dependencies by using gates to control the flow of information, preventing the vanishing gradient problem commonly encountered in standard RNNs (Smagulova and James 2020 ). RNNs and LSTMs can model the sequential nature of network activities, such as user login sessions or data transfer patterns over time (Al-Selwi et al. 2024 ). This helps in identifying unusual sequences of events that could indicate an ongoing APT attack. For example, RNN-based IDSs can detect APTs by identifying actions that deviate from typical behavior, such as accessing sensitive files at odd hours, which could indicate a compromised account (Keshk et al. 2023 ).

Autoencoders (AEs) are also extensively utilized for anomaly detection by learning to represent normal behavior and subsequently identifying deviations from this learned behavior, thus enabling the detection of potential threats (Schneider et al. 2022 ). Autoencoders are effective for anomaly detection in network traffic, as they can identify unusual patterns that deviate from the learned normal behavior (Hdaib et al. 2024 ). This makes them useful for detecting subtle and rare event characteristic of APTs (Salim et al. 2023 ). For example, an autoencoder trained on normal network traffic can flag unusual login times or data access rates as anomalies, helping security analysts identify and respond to potential threats.

5.3.1 Transformer-based approach

Transformer architectures, which were originally developed for natural language processing, have been utilized for APT detection. This model can focus on important parts of the input sequence, thus enhancing the detection of relevant events and activities within large volumes of data. In recent advancements, (Zhang et al. 2023 ) proposed an intrusion detection method that leverages the strengths of both the Transformer and LSTM models. This combination results in robust intrusion detection with high accuracy and efficient processing capabilities. However, the complexity of integrating both models may pose challenges for real-time processing. (Ullah et al. 2023 ) developed IDS-INT, a system utilizing transformer-based transfer learning specifically designed for imbalanced network traffic. This method significantly improves the detection rates for minority classes, thereby addressing a common issue in network security. Nonetheless, the requirement for large volumes of labeled data presents a notable limitation, particularly in scenarios with imbalanced datasets. (Y.Liu and Wu 2023 ) focused on enhancing the standard transformer model to increase the intrusion detection performance. Their improvements resulted in significant accuracy gains; however, the computational intensity of the enhanced model may hinder real-time detection applications.(Y. Wang and Li 2023 ) proposed an anomaly-detection method for time-series data based on transformer reconstruction. This approach offers accurate and timely anomaly detection. However, managing high-dimensional time-series data remains challenging, which may impact overall performance.

(Ullah et al. 2022 ) introduced an explainable system using transformer-based transfer learning combined with multi-model visual representation. This approach enhances interpretability and accuracy; conversely, reliance on visual representation can result in increased computational overhead. (Z. Zhang and Wang 2022 ) proposed an efficient intrusion detection model that integrates CNNs with transformer models. This hybrid approach achieved high accuracy with reduced computational costs. Nevertheless, the complexity introduced by combining the CNN and Transformer models can complicate training and maintenance processes. (Huang et al., n.d. )applied a Transformer with TSGL for Named Entity Recognition (NER) in the cyber threat intelligence domain. This method improved the accuracy of the information extraction. The limitation is that this application may be restricted to the cyber-threat intelligence domain and may not be generalizable to other areas.

5.3.2 Training and validation

The Data were split into training and validation sets. The models were trained on the training set, and their performance was validated on the validation set. Techniques such as cross-validation were used to ensure robust evaluation. These studies demonstrate the potential of transformer-based models for enhancing the effectiveness and interpretability of IDSs. Each method has unique strengths, while also presenting specific challenges that need to be addressed for broader application in real-world scenarios. ​.

5.4 Black box model

Understanding the decision-making processes of AI is important for organizations. Monitoring AI models and ensuring accountability is essential rather than blindly trusting them. XAI aids in machine learning (ML), deep learning, and neural network algorithms. Many ML models are often viewed as black boxes (Chennam et al., 2023 ), as neural networks are extremely difficult to interpret. Additionally, biases in AI models and performance drift due to different production data pose significant risks. Therefore, organizations need to continuously monitor and manage the model to promote AI explainability, build trust; and reduce compliance, legal, security, and reputational risk.

5.5 XAI models

XAI models are crucial for interpreting the predictions of deep learning models used in APT detection. Techniques such as LIME, SHAP, and LRP provide insights into the model’s decision-making process, enhancing transparency and trustworthiness. The details of this process are presented below.

5.5.1 Local interpretable model-agnostic explanations (LIME)

LIME creates local approximations of the model by perturbing the input data and fitting a simple, interpretable model to these perturbed instances, helping explain individual predictions by highlighting the specific features that influence the decision. LIME is widely used for interpreting local predictions (Hasan et al. 2023 ) by learning local linear approximations (Ibrahim et al. 2023 ). This algorithm provides local explanations of individual predictions (Volkov and Averkin 2023 ).

5.5.2 Shapley additive explanations (SHAP)

SHAP uses Shapley values from cooperative game theory to indicate the contribution of each feature to the model’s predictions, providing a consistent measure of feature importance (Hasan et al. 2023 ). This technique determines which features are most influential in detecting APT activities and offers global explanations, thus helping cybersecurity experts understand the model’s decision-making process. For example, SHAP can highlight unusual login times or data transfer volumes that might indicate an APT attack (Lundberg and Lee 2017 ). By assigning contribution values to each feature, their importance in decision making can be demonstrated (Band et al. 2023 ). This method, which relies on Shapley’s principles from cooperative game theory, has been effective in various areas, including IDSs.

Specifically, SHAP proposes the following three methods:

kernelSHAP: A model-agnostic version of the algorithm that operates only with the inputs and outputs of the function (Remman et al. 2021 ).

treeSHAP: Computes the Shapley values on tree-based classification models (Wallsberger et al. 2022 ).

deepSHAP: Optimizes the computation for neural network architectures (Meister et al. 2021 ).

By utilizing these methods, SHAP provides a comprehensive and interpretable framework for understanding and explaining model predictions.

5.5.3 Layer-wise relevance propagation (LRP)

The LRP computes relevance scores for individual input features by backpropagating the relevance of the class output node in a layer-wise fashion down to the input layer (Bach et al. 2015 ). This propagation adheres to a strict conservation property, ensuring that the relevance received by any neuron is redistributed equally. In CNNs, the LRP carries information about the relevance of the output class from higher layers back to the input pixel space layer by layer (Gu et al. 2019 ). By assigning relevance scores to input features, the LRP aims to interpret their contributions to model decisions. This technique requires access to the neural network’s structure because it performs a backward pass through the network to compute relevance scores(Montavon et al. 2019 ). This method enhances the interpretability of neural networks by providing insight into the importance of individual features in the decision-making process of the model.

By integrating deep learning models with XAI techniques, we aimed to improve the detection accuracy and interpretability of APT detection systems. This approach ensured that cybersecurity experts could respond effectively to threats and understand the rationale behind specific decisions made by the models.

6 Explainable AI in cybersecurity

XAI is a type of artificial intelligence (AI) that focuses on the study and development of techniques to enable machines to behave in smart ways that humans can understand (Gunning, 2019). The goal is to create models that are easier to understand and more accurate. This allows people to trust and control their next-generation AI partners (Capuano et al. 2022a ). The use of XAI techniques has gained significant attention in cybersecurity, especially for APT detection (Pawlicki et al. 2024 ).

XAI aims to provide transparency and explainability (Saeed and Omlin 2023 ). This shows how APT detection models make decisions so that cybersecurity experts can understand and trust the model’s prediction (Haque et al. 2023 ). Moreover, XAI systems should respond to feedback from cybersecurity experts or decision-makers and adjust their actions accordingly. By incorporating XAI methods in APT detection systems, organizations can improve their defense strategies, enhance operational efficiency, and increase transparency and accountability. However, current XAI-related work in cybersecurity still lacks sufficient comparisons of different XAI algorithms in terms of metrics such as model specificity, scope, and methodology (Yang et al. 2023 ). Improving the coverage of detection system studies that leverage modern interpretability techniques will reinforce the technical depth of APT detection.

Timeline of XAI review papers published from 2018 - 2024

Figure  7 illustrates the timeline of the significant XAI review papers published from 2018 to 2024. This timeline provides a historical perspective on the development and evolution of XAI techniques, highlighting key contributions and advancements in the field. The exploration of XAI has garnered significant interest across various domains, particularly in enhancing the interpretability and transparency of complex machine learning models. This review delves into the contributions of researchers in advancing the field of XAI.

(Reis et al. 2019 ) developed a framework for explainable machine learning aimed at detecting fake news, focusing on both interpretability and accuracy. Their approach highlighted the necessity of transparent AI models to mitigate misinformation. (Xu et al. 2019 ) provided a brief survey of the history, research areas, approaches, and challenges of XAI, offering a foundational understanding of the field’s evolution.

Insider threats pose significant cybersecurity challenges that common security solutions do not adequately address. (Homoliak et al. 2020 ) proposed a structural taxonomy and novel categorization to organize and clarify insider threat incidents and defense solutions. They utilized a grounded theory method for rigorous literature review, categorizing incidents, datasets, analysis, simulations, and defense solutions. Their taxonomy builds on existing frameworks and the 5W1H information gathering method. This survey aimed to enhance insider threat research by providing a structured taxonomy, an overview of publicly available datasets, references to existing case studies and frameworks, and a discussion on trends and future research directions.

(Speith 2022 ) explored deep learning-based XAI concepts, proposed a taxonomy method, and created a timeline of key XAI studies. They demonstrated the need for structured approaches to navigate the rapidly growing field of explainable artificial intelligence (XAI). The recent surge in publications related to XAI has created a daunting challenge for those seeking to start or stay current with the latest developments. (Giudici and Raffinetti 2021 )introduced the Shapley-Lorenz method for XAI, which focuses on fairness and transparency in AI decisions. Their work emphasized ethical considerations in AI model explanations. (Fouladgar and Främling 2020 ) reviewed both the practical and theoretical aspects of XAI and proposed a framework for integrating XAI into various applications. They bridge the gap between theory and practice, highlighting the versatility of XAI methods. (Kuhn et al. 2020 ) discussed combinatorial methods for enhancing the interpretability of AI models and provided insights into advanced techniques for model interpretation. (Angelov et al. 2021 ) presented an analytical review of XAI methods, critically evaluating their strengths and weaknesses. They offered a nuanced perspective on the effectiveness of different XAI techniques in various contexts. (Linardatos et al. 2021 ) reviewed machine learning interpretability methods, categorized them and discussed their applications across different domains. Their work emphasized the importance of methodological categorization in understanding XAI techniques.

(Sharma et al. 2022 ) explored the application of XAI techniques in cybersecurity, and proposed methods to enhance transparency and trustworthiness. They address the critical need for interpretable AI models in securing digital infrastructure. (Zhang et al. 2022 )surveyed current literature on XAI techniques in cybersecurity, emphasizing the need for transparent and accountable models. They propose a clear roadmap for future XAI research in this domain. provided an overview of various XAI methods, summarizing their applications and effectiveness. This concise overview serves as a valuable reference for researchers and practitioners. (Capuano et al. 2022b )surveyed XAI techniques in cybersecurity, and proposed approaches to enhance interpretability and effectiveness. They highlighted the evolving landscape of XAI applications for securing digital environments.

(Rjoub et al. 2023 ) reviewed XAI techniques in cybersecurity, and proposed approaches for enhancing the interpretability of AI models. Their work focused on making AI systems more transparent and understandable. (Band et al. 2023 ) conducted a systematic review of interpretability methods in medical health applications, and proposed a framework for XAI in healthcare. They demonstrated the critical role of interpretability in clinical decision making. (Hassija et al. 2024b ) provided a meticulous review and comprehensive analysis of state-of-the-art XAI models to address their complexity and lack of interpretability.. They offer insight into overcoming the challenges associated with complex AI systems. (Shams Khoozani et al. 2024 ) discussed the challenges, innovations, and future directions of concept-supported XAI, navigating the landscape of XAI research and application.

These contributions collectively advance the field of XAI by addressing key challenges, proposing innovative methods, and highlighting practical applications of XAI across various domains. This review highlights the importance of continued research and development on XAI to ensure the deployment of trustworthy and interpretable AI systems.

6.1 Explanation methods

Before delving deeper into the potential use of XAI for APT detection, we first briefly review XAI techniques. We investigated the most prominent, state-of-the-art XAI techniques. We analyzed the explanations provided by LIME, SHAP (Galli et al. 2024 ) LRP (Bach et al. 2015 ), and the attention mechanism. XAI techniques represent a diverse range of approaches. LIME focuses on local model explanations through perturbations of input data,  whereas SHAP uses a game-theoretical approach to allocate contributions to each feature. The LRP propagates relevance through neural network layers, and attention mechanisms highlight the input features attended to by the model.

LIME and SHAP offer model-agnostic explainability, making them applicable to a wide range of models, including neural networks. Decision trees and rule-based models are inherently interpretable. (Sarker et al. 2024 ) proposed a decision tree-based model for intrusion detection, which allows for easy interpretation of detection rules.

The LRP specifically addresses layer-wise relevance in neural networks, and attention mechanisms are commonly used in neural networks for sequence data. LIME and SHAP have gained widespread acceptance in the research community due to their versatility and effectiveness in describing complex models. The LRP is renowned for its application in neural networks, and attention mechanisms have become standard in natural language processing and computer vision tasks.

6.2 Integration of XAI techniques in deep learning models

This paper thoroughly explores how XAI techniques, such as feature attribution and saliency maps, enhance deep learning models for APT detection (Lo et al. 2023 ). Specifically, it demonstrates how these approaches identify critical factors in the model’s decision-making process. By incorporating SHAP, LIME, and LRP into the deep learning framework, this paper offers a comprehensive approach to improve interpretability. This integration clarifies influential factors and helps identify potential threats, thereby increasing the efficacy and transparency of cybersecurity measures.

This paper uses feature attribution methods to determine the importance of each factor in the model’s predictions and employs saliency maps to visualize critical regions in network traffic data (Gevaert et al. 2024 ). By utilizing these XAI approaches, the interpretability of deep learning models can be enhanced. This transparency allows cybersecurity experts to understand and trust the model’s predictions, making it easier to validate and fine-tune the detection systems (Sharma et al. 2022 ). Furthermore, integrating XAI helps pinpoint the factors most indicative of APT activity, improving detection accuracy and enabling quicker response times by focusing on the most relevant data aspects.

The paper includes practical examples and case studies demonstrating the application of XAI in real-world scenarios, illustrating how its use leads to more effective and transparent APT detection systems.

6.3 Comparative analysis of XAI- based APT detection

In recent years, the integration of XAI techniques with deep learning models has gained significant attention in the field of APT detection. This subsection provides a comprehensive overview of recent advancements from 2020 to 2024, highlighting various approaches and their effectiveness in enhancing cybersecurity measures.

(Singh et al. 2024 ) introduced SFC-NIDS, a sustainable and explainable network intrusion detection approach that analyzes VM traffic at the hypervisor level. It uses a gradient descent-based flow filtering mechanism, auto-encoders to reconstruct traffic features, and a 1D-CNN to detect malicious flows. When validated with hypervisor traffic artifacts and the KDD99 dataset, it achieves 98.9% and 99.97% accuracy, respectively. The strengths of these methods include high accuracy, adaptability, and explainability, but they are difficult to implement and specific to hypervisor environments, requiring further validation with various datasets.

(Khan et al. 2024 ) presented a novel security model for biomedical data collection and transmission, addressing privacy, security, and reputation concerns in medical networks. They introduced a threat-vector database based on the dynamic behaviors of smart healthcare systems and designed an improved SRU network to mitigate fading gradient issues and enhance the learning process by reducing computational costs. This approach is parallelizable and computationally efficient, dynamically adjusting the number of participating clients to reduce the communication overhead. Additionally, the model enhances the understanding of security experts by visualizing the decision process and explaining feature relevance. Compared with existing methods, this security model thoroughly analyzes and detects severe security threats with high accuracy, reduces overhead, lowers computational costs, and enhances the privacy of biomedical data. However, the potential complexity in implementing the novel security model across diverse healthcare networks and the need for further validation and testing in real-world healthcare environments to ensure robustness and effectiveness are noted as weaknesses.

Building on the idea of integrating explainability into AI-based cybersecurity solutions. (Galli et al. 2024 ) proposed a framework that integrates XAI methodologies within AI-based malware detection processes to address the lack of interpretability in traditional machine learning (ML) and deep learning (DL) approaches. The framework incorporates four XAI methods, such as SHAP, LIME, LRP, and Attention mechanisms. The key strengths of the proposed system include the integration of multiple XAI methods to improve model interpretability, thorough evaluation across diverse datasets, insightful comparisons of the LSTM and GRU models, and enhanced real-world applicability due to improved explainability. However, the study also faces challenges such as increased computational complexity, the need for further validation across different types of malware and cybersecurity contexts, and potential trade-offs between model explainability and performance.

Similarly, (Hasan et al. 2023 ) developed a highly effective model for identifying APTs using boosting-based machine learning methods, with XGBoost achieving an impressive weighted F1 score of 0.97. They enhance model interpretability by integrating SHAP, providing actionable insights for stakeholders. The key strengths of their work include exceptional predictive accuracy and enhanced transparency through XAI. However, potential challenges include computational complexity and the need for broader validation across various cyber threats. This approach underscores the promise of boosting-based XAI models in cybersecurity.

Expanding on the use of XAI in cybersecurity, (Zolanvari et al. 2023 ) employed TRUST XAI using multimodal Gaussian distributions and the TRUST Explainer technique. They evaluated their model on multiple datasets, including WUSTL-IIoT, NSL-KDD, and UNSW, and achieved 98% accuracy. A limitation of this study is that they did not address the computational resources necessary to implement their models.

(Khan et al. 2022 ) proposed a novel explainable deep learning framework for cyber threat discovery in Industrial IoT (IIoT) networks, addressing the critical need for data integrity and accuracy. They used an autoencoder-based detection system with convolutional and recurrent networks, employing a two-step sliding window technique to enhance feature extraction from raw time series data. Fully connected networks classify and explain attack events using temporal and spatial features. Empirical results demonstrated robust performance, with the framework outperforming contemporary methods. However, its complexity might limit scalability, and further exploration is needed for generalizability, real-time adaptation, and explainability depth.

(Khan et al. 2024 ) focused on bidirectional simple recurrent units (SRUs) and developed the XSRU-IoMT method, which was tested on the ToN_IoT dataset. Their model achieved 98% accuracy, but was validated on only a single dataset, limiting its generalizability. (Zhou et al. 2022 ) used an M&M Decision Tree model with prime implicant explanation techniques on various DDoS attack datasets, and achieved perfect recall and F1 scores. This method, which is based on an artificial immune system, may not be suitable for all intrusion-detection scenarios.

Another approach by (Patil et al. 2022 ) implemented an ensemble method using voting classifiers and LIME for interpretability on the CICIDS2017 dataset, achieving 96.25% accuracy. Reliance on a black box model may limit its transparency. (Barnard et al. 2022 ) combined extreme gradient boosting (XGBoost) with autoencoders (AEs) and utilized SHAP for interpretability, achieving 93% accuracy on the NSL-KDD dataset. However, this framework was assessed using only this dataset.

(Houda et al. 2022 ) applied deep neural networks (DNNs) with RuleFit, LIME, and SHAP on the NSL-KDD and UNSW NB-15 datasets, achieving 88% accuracy. However, their framework evaluation on only two datasets may not be generalizable to other datasets. Le et al. (2022) used ensemble trees (DT and RF classifiers) and SHAP on NF-BoT-IoT v2 and NF-ToN-IoT-v2 datasets, and achieved  100% accuracy. However, the method’s long training time might render it unsuitable for real-time IDS systems.

(Kuppa and Le-Khac 2021 ) explored autoencoders with counterfactual explanations on the Leaked Password Dataset, achieving 96.7% accuracy. Their study did not cover the broader aspects of machine learning or AI beyond cybersecurity. (Liu et al. 2021 ) proposed FAIXID using Boolean Rule Column Generation (BRCG) with data cleaning methods on real-world datasets, which achieved 87% accuracy. In this study, they did not provide detailed information on the size and diversity of the datasets used for the evaluation. Wali et al. ( 2021 ) utilized a primary Random Forest Classifier (RFC) with SHAP, achieving 98.5–100% accuracy on the Hop Skip Jump Attack and CICIDS datasets. The proposed framework requires significant computational resources.

(Antwarg et al. 2021 ) employed autoencoders with kernel SHAP on the KDD Cup 1999 and Credit Card datasets, focusing on explaining anomalies with a mean MSE of 0.0006. However, this method may not be applicable to APTs. Wang et al. ( 2020 ) used one-vs-all and multiclass classifiers with SHAP on the NSL-KDD dataset, achieving accuracies between 80.3% and 80.6%. The proposed framework was assessed using an outdated NSL-KDD dataset, which limited its relevance. Table  4 summarizes the XAI-based methods employed in cybersecurity.

This review of XAI-based APT detection methods highlights the diversity and advancements in the field while also identifying specific shortcomings that need to be addressed. These insights pave the way for future research to develop more robust, interpretable, and effective APT detection systems.

6.4 Case studies

This section presents various case studies that demonstrate the application of XAI techniques for combating APTs. These scenarios provide a comprehensive overview of how XAI can enhance the detection and mitigation of APTs.

Figure  8 compares different attack scenarios with the corresponding XAI techniques applied to improve detection and interpretation. The figure shows how techniques such as SHAP, LIME, LRP, and attention mechanisms enhance the interpretability of models in various attack scenarios such as phishing, insider threats, DoS attacks, and malware.

Attack scenarios vs. XAI techniques

To address various attack scenarios, recent studies have applied XAI techniques to improve the detection and interpretation of cyber threats. (Adebowale et al. 2023 ) proposed integrating CNN and LSTM models with LIME to enhance the email classification model for phishing attacks. This hybrid XAI approach effectively addresses the challenges posed by large datasets and significantly improves classifier prediction performance.

(Homoliak et al. 2020 ) utilized SHAP to explain the global behavior of a detection model. Their study emphasized the diverse nature of insider threats and demonstrated how SHAP can identify common patterns and unique contributing factors across different clusters, such as fraud, IP theft, and sabotage. For DoS attacks, (Hariharan et al. 2021 ) applied Permutation Importance, SHAP, LIME, and Contextual Importance and Utility algorithms to provide a unified measure of feature importance and contribution to model predictions. Their case study revealed key insights into the impact of various features on IDS prediction performance, highlighting the most influential features for detecting DoS attacks.

A downloader, a type of Trojan horse, downloads and installs malicious software without user consent. The explanation provided by the LRP highlights the Image 4 call, which attempts to copy files from one part of the file system to another, as discussed by (Galli et al. 2024 ). Finally, for malware attacks, (Galli et al. 2024 ) integrated the SHAP, LIME, LRP, and Attention mechanisms with robust deep learning models (e.g., LSTM and GRU) trained on labeled datasets. Their findings demonstrated that these XAI techniques significantly enhance the interpretability of malware detection models, ensuring that they remain effective and understandable in real-world applications.

Beyond cybersecurity, XAI has significant applications in other domains, such as the following:

Healthcare providers have observed an increase in phishing attempts and suspicious activity in their networks. Given the sensitivity of patient data, they sought to improve their security measures by using an advanced detection system that combines deep learning and XAI techniques (Kalutharage et al. 2024 ).The data used include email traffic data collected from the providers’ email servers, encompassing metadata and content analysis, as well as network logs from firewalls and intrusion detection systems (IDSs) that capture network activity and possible intrusions. CNNs can analyze email traffic and detect phishing attempts (McGinley and Monroy 2021 ), whereas RNNs can analyze sequential network logs for signs of APT activity. XAI techniques, such as SHAP, explain the phishing detection of CNN models, while LIME provides interpretability for network activity anomalies. The explanations provided by SHAP and LIME helped security teams identify specific email campaigns targeting employees and unusual data streams that indicated an APT in progress. This facilitated immediate corrective actions, including email filtering and network segmentation (Jia et al. 2021 ).

A major financial institution experienced unusual network activity, suggesting a potential APT attack. The institution’s security team decided to use an integrated approach that combines deep learning models and XAI techniques to identify and mitigate the threat. The data used included network traffic data collected from the institution’s internal network, encompassing packet headers, payloads, and flow data over a six-month period. Additionally, system and application logs from critical servers and endpoints were analyzed, providing insights into user activities and system changes. SHAP values indicated that unusual login times and connections to known malicious IP addresses were significant factors in these detections. The LIME explanations for specific alerts revealed that certain endpoints repeatedly communicated with suspicious external servers, suggesting a potential APT foothold. These insights allowed the security team to prioritize these alerts and initiate a targeted investigation. The integrated approach not only detected the APT early but also provided clear explanations that helped the security team understand the attack vector and take swift action. As a result, the financial institution was able to isolate the affected endpoints, mitigate the threat, and prevent data exfiltration.

These use cases demonstrate how XAI techniques can be applied to various aspects of APT detection, from email classification and insider threat detection to network anomaly detection and malware identification. By examining these case studies, cybersecurity experts can better understand the practical applications of XAI and see how they can enhance their ability to investigate and mitigate APT attacks. These examples also highlight the potential benefits of XAI in real-world APT detection scenarios.

6.5 Role of XAI in APT detection

To effectively integrate XAI with deep learning for APT detection, several enhancements are necessary to address the unique challenges posed by sophisticated threats. (1) Handling sparse data is critical because traditional methods struggle with sparse datasets. Integrating XAI with anomaly-detection techniques, such as autoencoders, can help identify and explain anomalies in sparse data. (2) The high-dimensional nature of APT data requires dimensionality reduction techniques, such as Principal Component Analysis (PCA) or t-distributed Stochastic Neighbor Embedding (t-SNE), to simplify data complexity while retaining essential features, thereby improving interpretability. Moreover, APTs are characterized by evolving tactics, techniques, and procedures (TTPs). To adapt to these changes, XAI techniques must be dynamic and capable of real-time updating through continuous learning frameworks. This allows models to refine their understanding of new attack patterns and provide explanations that highlight these adaptations. (3) In terms of high-dimensional space, advanced feature importance techniques, such as Integrated Gradients, should be used to attribute importance across different layers of neural networks, offering a deeper understanding of model decisions. Finally, the implementation of a real-time feedback loop, in which cybersecurity experts can interact with the XAI system to refine explanations is crucial. This can be achieved using reinforcement learning to adapt models based on analyst feedback, ensuring that explanations remain accurate and relevant (Kute et al. 2021 ). To address sparse data, high-dimensional space, evolving tactics, and the need for real-time adaptation, this review aims to provide a more robust framework for explaining AI models used in APT detection, thereby advancing state-of-the-art models in cybersecurity.

In the discussion section, we explore the implications of these findings and how they contribute to advancing APT detection. Specifically, we discuss how the enhanced interpretability provided by XAI techniques can lead to more effective threat mitigation and greater trust in AI-driven security solutions.

7 Discussion and recommendations

This section delves into the need for explainability, challenges, and future directions in APT detection using deep learning and XAI. Traditional anomaly detection techniques are effective for known threats but struggle with new, evolving threats and large-scale data. Recurrent Neural Networks (RNNs) show promise in detecting complex and sequential patterns in network traffic, but they face challenges in interpretability and computational demands. XAI offers potential solutions by making model decisions transparent and understandable, adopting trust and collaboration between human experts and AI systems. This integration aims to increase the effectiveness and robustness of APT detection systems.

7.1 Key considerations for XAI

7.1.1 the need for explainability.

XAI is crucial in making deep learning models interpretable and transparent, especially in cybersecurity where trust and reasoning are essential. Various techniques have been developed to enhance model interpretability. Feature importance methods, such as permutation importance and SHAP, determine the significance of each feature in the decision-making process (Hariharan et al. 2021 ). Partial Dependence Plots (PDPs) visualize the marginal effect of a feature on the predicted outcome (Ding et al. 2021 ). Activation Maximization identifies inputs that maximize the activation of specific neurons, revealing what each neuron “wants to see” (Zeltner et al. 2021 ). Saliency Maps highlight parts of the input most relevant to the model’s decision (Brunke et al. 2020 ). LIME explains the predictions of any classifier by learning a simple model locally around the prediction (Houda et al. 2022 ). Layer-wise Relevance Propagation (LRP) backpropagates the output through the network to assign a “relevance score” to each input feature (Seibold et al. 2021 ). These techniques collectively enhance the transparency and trustworthiness of AI models, facilitating their adoption in critical cybersecurity applications.

7.1.2 Need for trust

Trust is essential in cybersecurity, and XAI methods enhance this by providing clear, understandable explanations of AI model predictions. The SHAP and LIME techniques help experts determine which features influence decisions, fostering confidence in AI systems [1]. Transparent models reduce perceived risk and ensure fair, accurate operations by identifying and correcting biases (Saeed and Omlin 2023 ). This increased trust leads to better decision-making and faster response times, which are crucial in mitigating APTs. By integrating XAI, cybersecurity experts can confidently use AI systems, resulting in more robust and effective threat detection and response strategies (Sourati et al. 2023 ).

7.1.3 Need for reasoning

XAI enables cybersecurity experts to understand and validate model reasoning. This leads to better human-AI collaboration. This makes the detection of APT threats more effective. When security teams understand how the model works, they can better mitigate the potential vulnerabilities to adversarial attacks. This can enhance the robustness of the APT detection system more effectively. By seeing how the model makes decisions, security teams can identify and fix errors more efficiently. This helps improve the APT detection system.

XAI enables cybersecurity experts to understand and validate AI model reasoning, enhancing human-AI collaboration and APT detection (Sarker et al. 2024 ). By revealing how models arrive at their conclusions, XAI helps identify and mitigate vulnerabilities to adversarial attacks, improving system robustness. Understanding model decisions allows security teams to correct errors efficiently, maintaining detection accuracy (Došilović et al. 2018 ). XAI also fosters a shared understanding between AI systems and human operators, promoting the trust and effective use of AI in cybersecurity, ultimately leading to better detection and response strategies. The next subsection addresses the current issues associated with black-box models.

7.2 Current issues surrounding black-box models

One problem with using deep learning models for APT detection is their black box behavior, which makes it difficult to interpret and explain their decisions. (Kute et al. 2021 ). For example, when dealing with APT incidents, cybersecurity experts must understand why a particular event is deemed malicious. The lack of transparency in black-box models can lead to several issues. Cybersecurity experts may not trust a model’s predictions without understanding the reasoning, especially when false positives or false negatives can have severe consequences. (Galli et al. 2024 ). When a model makes an incorrect prediction, identifying the root cause and making the necessary adjustments becomes challenging without insight into the model’s decision-making process (Samek et al., 2017). During a security incident, organizations may face difficulties in explaining and justifying actions based on the model’s predictions, potentially leading to legal and regulatory issues.

Additionally, black-box models are vulnerable to adversarial examples (Jia et al. 2024 ), which are carefully crafted inputs that deceive a model and lead to incorrect predictions. Without a clear understanding of how the model works internally, detecting and mitigating such attacks is more difficult (Ali et al. 2023 ).Therefore, promoting AI explainability is imperative to ensure the reliability and robustness of AI systems in critical applications such as cybersecurity.

Black box models pose significant challenges because of their opaque nature, making it difficult to interpret and explain their decisions. This lack of transparency can lead to issues such as biases, performance drifting, and vulnerability to adversarial attacks. In cybersecurity, the inability to understand and justify model predictions can undermine trust, complicate incident response, and lead to legal and regulatory challenges. XAI is essential for enhancing model transparency, building trust, and mitigating various risks, ensuring the reliability and robustness of AI systems in critical applications.

7.3 Research challenges and recommendations

Research challenges for APT detection

Figure  9 shows that APT detection is technically challenging for several reasons. To address the challenges identified in APT detection using deep learning and XAI, several practical solutions are recommended.

Highly imbalanced datasets can hinder the performance of AI models in detecting minority class events. We address this issue using data augmentation techniques, such as the Synthetic Minority Oversampling Technique (SMOTE), to generate synthetic samples for the minority class(Y. Liu and Wu 2023 ). Employing unsupervised or semi-supervised learning techniques specifically designed to handle imbalanced data can also be effective (Zhang et al. 2020 ), (Kute et al. 2021 )

The rapidly evolving threat landscape requires the continuous adaptation of detection models. To address this problem, continual learning approaches allow models to adapt to new data without forgetting previously learned information. Automated pipelines for data collection, model retraining, and deployment ensure that the detection system remains updated with the latest threat information (Khoozani et al., 2024 ). APT tactics, techniques, and procedures (TTPs) are continuously changing, making it difficult for static models to remain effective. By integrating XAI with deep learning models, we enhance understanding and adapt to new attack patterns. This integration improves the robustness of the system against evolving threats, ensuring that it remains effective in detecting and mitigating APTs.

Comprehensive datasets for effective APT detection are lacking. This problem can be addressed through collaborative efforts among academia, industry, and government agencies to create large-scale, diverse, and labeled datasets. Promoting data-sharing initiatives encourages organizations to share anonymized threat data for research purposes, thereby enriching the pool of available datasets (Ferrag et al. 2020 ).

Integrating XAI techniques into APT detection systems is challenging because of the complexity and scale of operational environments. (Kuhn et al. 2020 ) developed modular XAI frameworks that can be easily integrated into existing cybersecurity infrastructures. In addition, providing training and resources for cybersecurity experts to use XAI tools effectively enhances their practical applicability.

Deep-learning models are often criticized for their difficulty in interpretation and lack of transparency (Liang et al. 2021 ). XAI techniques, such as SHAP and LIME, highlight the features that most influence model predictions and provide visual explanations that are easy to interpret. AI models must provide clear, understandable explanations for their decisions to build trust and enable cybersecurity experts to make informed decisions (Galli et al. 2024 ).

Black-box models can be tricked by adversarial examples, which can lead to incorrect predictions. Understanding the internal operation of the model is crucial for detecting and mitigating such attacks. The development of XAI systems that reveal the decision-making process of the model can help identify and counter adversarial inputs effectively (Kenny et al. 2021 ).

Our discussion highlights the strengths and potential of integrating XAI with deep learning for APT detection. The enhanced interpretability provided by XAI techniques allows cybersecurity experts to understand and trust model predictions, leading to more effective threat mitigation. In conclusion, we summarize our contributions and suggest directions for future research, emphasizing the need for continued innovation in this critical area of cybersecurity.

8 Conclusions and future research directions

Future research directions for APT research

Our research addresses the challenges in APT detection by integrating XAI with deep learning models. As illustrated in Fig.  10 , future research directions for APT detection focus on several key areas. First, examining the potential of deep feature extraction methods is essential for enhancing detection capabilities. Second, integrating XAI techniques with deep learning models is crucial, as it offers transparent and effective explanations for APT detection. Additionally, optimizing automatically extracted features can enhance detection accuracy, reduce human bias, and improve resource efficiency. Furthermore, augmenting existing datasets by generating synthetic examples of APT activities and benign behaviors can enrich the data available for training. Moreover, it is important to evaluate detection models using comprehensive evaluation metrics and scenario-based security testing to ensure robust performance. Finally, maintaining privacy and security is essential when robust detection models that safeguard data privacy are constructed, thus ensuring the overall integrity of the system.

This review provides a comprehensive overview of state-of-the-art models for dvanced persistent threats (APTs) detection. Through extensive analysis, we found that deep learning-based techniques, especially those incorporating XAI methods are increasingly prevalent. These approaches address significant challenges in traditional IDSs, such as low detection accuracy, high false-positive rates, and difficulties in detecting unknown or early-stage attacks. By incorporating XAI techniques such as SHAP, cybersecurity experts can understand the contribution of each feature to the model’s prediction, such as unusual login times or unexpected IP addresses, allowing for quicker verification of alerts and reduced false positives. Moreover, the transparency provided by XAI models enhances trust and accountability in detection systems. Users and stakeholders can audit and validate the decisions made by the models, ensuring that the detection process is transparent and reliable. Empirical evidence from recent studies further supports the effectiveness of integrating XAI with deep learning, demonstrating improvements in detection accuracy and a significant reduction in false positives. This not only validates our approach but also underscores its practical benefits in real-world applications.

Our review also highlights the strengths and weaknesses of various APT detection methods across different datasets, techniques, and experimental results. By identifying gaps and challenges in current methodologies, we propose integrating XAI with deep learning models to advance the state-of-the-art. This integration not only improves detection accuracy and scalability but also provides clear, interpretable insights into the model’s decision-making process, ensuring effective threat response. The impact of our findings is substantial for cybersecurity. By enhancing the transparency and interpretability of APT detection systems, we can enable more effective and timely responses to sophisticated threats. This research contributes to the development of advanced security measures that adapt to evolving cyber attacker tactics.

In conclusion, our research paves the way for more robust, scalable, and interpretable APT detection systems that effectively combat sophisticated APT attacks. By addressing both technical and interpretability challenges, we advance the state-of-the-art in cybersecurity and enhance network resilience to APTs.

Data availability

No datasets were generated or analysed during the current study.

https://symantec-enterprise-blogs.security.com/blogs/threat-intelligence/leafminer-espionage-middle-east .

https://attack.mitre.org/groups/G0099/ .

https://www.ibm.com/reports/data-breach .

Abbas G, Farooq U, Singh P, Khurana SS, Singh P (2023) Feature Engineering and Ensemble Learning-based classification of VPN and Non-VPN-Based Network traffic over temporal features. SN Comput Sci 4(5):546. https://doi.org/10.1007/s42979-023-01944-5

Article   Google Scholar  

Abu Bakar R, Huang X, Javed MS, Hussain S, Majeed MF (2023) An Intelligent Agent-based detection system for DDoS attacks using automatic feature extraction and selection. SENSORS 23(6). https://doi.org/10.3390/s23063333

Adebowale MA, Lwin KT, Hossain MA (2023) Intelligent phishing detection scheme using deep learning algorithms. J Enterp Inform Manage 36(3):747–766. https://doi.org/10.1108/JEIM-01-2020-0036

Agrawal, G.; Kaur, A.; Myneni, S. A Review of Generative Models in Generating Synthetic Attack Data for Cybersecurity. Electronics 2024, 13, 322. https://doi.org/10.3390/electronics13020322

Ahmad A, Webb J, Desouza KC, Boorman J (2019) Strategically-motivated advanced persistent threat: definition, process, tactics and a disinformation model of counterattack. COMPUTERS Secur 86:402–418. https://doi.org/10.1016/j.cose.2019.07.001

Ahmad HB, Gao H, Latif N, Aziiz A, Auraangzeb M, Khan MT (2024) Adversarial Machine Learning for Detecting Advanced Threats Inspired by StuxNet in Critical Infrastructure Networks. 2024 12th International Symposium on Digital Forensics and Security (ISDFS) , 1–7. https://doi.org/10.1109/ISDFS60797.2024.10527326

AlDahoul N, Karim A, H., Ba Wazir AS (2021) Model fusion of deep neural networks for anomaly detection. J Big Data 8(1):1–18

Ali S, Abuhmed T, El-Sappagh S, Muhammad K, Alonso-Moral JM, Confalonieri R, Guidotti R, Del Ser J, Díaz-Rodríguez N, Herrera F (2023) Explainable Artificial Intelligence (XAI): what we know and what is left to attain Trustworthy Artificial Intelligence. Inform Fusion 99:101805. https://doi.org/10.1016/j.inffus.2023.101805

Alkhadra R, Abuzaid J, AlShammari M, Mohammad N (2021) Solar Winds Hack: In-Depth Analysis and Countermeasures. 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT) , 1–7. https://doi.org/10.1109/ICCCNT51525.2021.9579611

Al-Selwi SM, Hassan MF, Abdulkadir SJ, Muneer A, Sumiea EH, Alqushaibi A, Ragab MG (2024) RNN-LSTM: from applications to modeling techniques and beyond—systematic review. J King Saud Univ - Comput Inform Sci 36(5):102068. https://doi.org/10.1016/j.jksuci.2024.102068

Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, Santamaría J, Fadhel MA, Al-Amidie M, Farhan L (2021) Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data 8(1):53. https://doi.org/10.1186/s40537-021-00444-8

Angelov PP, Soares EA, Jiang R, Arnold NI, Atkinson PM (2021) Explainable artificial intelligence: an analytical review. Wiley Interdisciplinary Reviews: Data Min Knowl Discovery 11(5). https://doi.org/10.1002/widm.1424

Antwarg L, Miller RM, Shapira B, Rokach L (2021) Explaining anomalies detected by autoencoders using Shapley Additive explanations. Expert Syst Appl 186:115736. https://doi.org/10.1016/j.eswa.2021.115736

Bach S, Binder A, Montavon G, Klauschen F, Müller K-R, Samek W (2015) On pixel-wise explanations for non-linear classifier decisions by Layer-wise Relevance Propagation. PLoS ONE 10(7):e0130140. https://doi.org/10.1371/journal.pone.0130140

Ballard (2021) Cybercrime apparently cost the world over $1 trillion in 2020. https://www.techradar.com/news/cybercrime-cost-the-world-over-dollar1-trillion-in-2020

Band S, Yarahmadi S, Hsu A, Biyari C-C, Sookhak M, Ameri M, Dehzangi R, Chronopoulos I, A. T., Liang H-W (2023) Application of explainable artificial intelligence in medical health: a systematic review of interpretability methods. Inf Med Unlocked 40:101286. https://doi.org/10.1016/j.imu.2023.101286

Barnard P, Marchetti N, DaSilva LA (2022) Robust Network Intrusion Detection through Explainable Artificial Intelligence (XAI). IEEE Netw Lett 4(3):167–171. https://doi.org/10.1109/LNET.2022.3186589

Bierwirth T, Pfützner S, Schopp M, Steininger C (2024) Design and evaluation of Advanced Persistent threat scenarios for Cyber ranges. IEEE Access 12:72458–72472. https://doi.org/10.1109/ACCESS.2024.3402744

Bodström T, Hämäläinen T (2019) A novel deep learning stack for APT detection. Appl Sci 9:1055. https://doi.org/10.3390/app9061055

Brown D, Cianfarani G, Vlajic N (2022) Real World snapshot of trends in IoT device and Protocol Deployment: IEEE CNS 22 poster. 2022 IEEE Conf Commun Netw Secur (CNS) 1(2). https://doi.org/10.1109/CNS56114.2022.9947257

Brunke L, Agrawal P, George N (2020) Evaluating input perturbation methods for interpreting CNNs and Saliency Map Comparison. In: Bartoli A, Fusiello A (eds) Computer vision – ECCV 2020 Workshops. Springer International Publishing, pp 120–134

Capuano N, Fenza G, Loia V, Stanzione C (2022a) Explainable Artificial Intelligence in CyberSecurity: a Survey. IEEE Access 10:93575–93600. https://doi.org/10.1109/ACCESS.2022.3204171

Capuano N, Fenza G, Loia V, Stanzione C (2022b) Explainable Artificial Intelligence in CyberSecurity: a Survey. IEEE ACCESS 10:93575–93600. https://doi.org/10.1109/ACCESS.2022.3204171

Chen J, Su C, Yeh KH, Yung M (2018) Special issue on advanced persistent threat. Future Gener Comput Syst 79(Part 1):243–246. https://doi.org/10.1016/j.future.2017.11.005

Chen L, Weng S, Peng C, Shuai H, Cheng W (2021) ZYELL-NCTU NetTraffic-1.0: a large-scale dataset for real-world network anomaly detection. In: 2021 IEEE international conference on consumer electronics-Taiwan (ICCE-TW), pp 1-2

Chennam KK, Mudrakola S, Maheswari VU, Aluvalu R, Rao KG (2023) Black box models for eXplainable artificial intelligence. In: Mehta M, Palade V, Chatterjee I (eds) Explainable AI: foundations, methodologies and applications. Intelligent systems reference library, vol 232. Springer, Cham. https://doi.org/10.1007/978-3-031-12807-3_1

Chen Z, Simsek M, Kantarci B, Bagheri M, Djukic P (2024) Machine learning-enabled hybrid intrusion detection system with host data transformation and an advanced two-stage classifier. Comput Netw 250:110576. https://doi.org/10.1016/j.comnet.2024.110576

Daoud M, Dahmani Y, Bendaoud M, Ouared A, Ahmed H (2023) Convolutional neural network-based high-precision and speed detection system on CIDDS-001. Data Knowl Eng 144:102130. https://doi.org/10.1016/j.datak.2022.102130

Das S, Tariq A, Santos T, Kantareddy SS, Banerjee I (2023) Recurrent Neural Networks (RNNs): Architectures, Training Tricks, and Introduction to Influential Research. In O. Colliot (Ed.), Machine Learning for Brain Disorders (pp. 117–138). Springer US. https://doi.org/10.1007/978-1-0716-3195-9_4

Davis A, Gill S, Wong R, Tayeb S (2020) Feature Selection for Deep Neural Networks in Cyber Security Applications. 2020 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS) , 1–7. https://doi.org/10.1109/IEMTRONICS51293.2020.9216403

de Abreu SF, Kendzierskyj S, Jahankhani H (2020) Attack Vectors and Advanced Persistent Threats. In H. Jahankhani, S. Kendzierskyj, N. Chelvachandran, & J. Ibarra (Eds.), Cyber Defence in the Age of AI, Smart Societies and Augmented Humanity (pp. 267–288). Springer International Publishing. https://doi.org/10.1007/978-3-030-35746-7_13

Ding R, Yin W, Cheng G, Chen Y, Wang J, Wang R, Rui Z, Li J, Liu J (2021) Boosting the optimization of membrane electrode assembly in proton exchange membrane fuel cells guided by explainable artificial intelligence. Energy AI 5:100098. https://doi.org/10.1016/j.egyai.2021.100098

DiPietro R, Hager GD (2020) Chapter 21 - Deep learning: RNNs and LSTM. In S. K. Zhou, D. Rueckert, & G. Fichtinger (Eds.), Handbook of Medical Image Computing and Computer Assisted Intervention (pp. 503–519). Academic Press. https://doi.org/10.1016/B978-0-12-816176-0.00026-0

Došilović FK, Brčić M, Hlupić N (2018) Explainable artificial intelligence: A survey. 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) , 210–215. https://doi.org/10.23919/MIPRO.2018.8400040

Do Xuan C, Dao MH, Nguyen HD (2020) APT attack detection is based on flow network analysis techniques using deep learning. J Intell Fuzzy Syst 39(3):4785–4801. https://doi.org/10.3233/JIFS-200694

Ersavas T, Smith MA, Mattick JS (2024) Novel applications of Convolutional Neural Networks in the age of transformers. Sci Rep 14(1):10000. https://doi.org/10.1038/s41598-024-60709-z

Fang Y, Wang C, Fang Z, Huang C (2022) LMTracker: Lateral movement path detection based on heterogeneous graph embedding. Neurocomputing 474:37-47. https://doi.org/10.1016/j.neucom.2021.12.026

Ferrag MA, Maglaras L, Moschoyiannis S, Janicke H (2020) Deep learning for cyber security intrusion detection: approaches, datasets, and comparative study. J Inform Secur Appl 50:102419. https://doi.org/10.1016/j.jisa.2019.102419

Fouladgar N, Främling K (2020) XAI-P-T: A Brief Review of Explainable Artificial Intelligence from Practice to Theory

Galli A, La Gatta V, Moscato V, Postiglione M, Sperlì G (2024) Explainability in AI-based behavioral Malware Detection systems. Computers Secur 103842. https://doi.org/10.1016/j.cose.2024.103842

Geng L, Niu B (2024) APSSF: adaptive CNN pruning based on structural similarity of filters. Int J Comput Intell Syst 17(1):129. https://doi.org/10.1007/s44196-024-00518-4

Gevaert A, Rousseau A-J, Becker T, Valkenborg D, De Bie T, Saeys Y (2024) Evaluating feature attribution methods in the image domain. Mach Learn. https://doi.org/10.1007/s10994-024-06550-x

Article   MathSciNet   Google Scholar  

Giudici P, Raffinetti E (2021) Shapley-Lorenz eXplainable Artificial Intelligence. Expert Syst Appl 167(October 2020):114104. https://doi.org/10.1016/j.eswa.2020.114104

Gu J, Yang Y, Tresp V (2019) Understanding individual decisions of CNNs via Contrastive Backpropagation. In: Jawahar CV, Li H, Mori G, Schindler K (eds) Computer vision – ACCV 2018. Springer International Publishing, pp 119–134

Haque AKMB, Islam AKMN, Mikalef P (2023) Explainable Artificial Intelligence (XAI) from a user perspective: a synthesis of prior literature and problematizing avenues for future research. Technol Forecast Soc Chang 186:122120. https://doi.org/10.1016/j.techfore.2022.122120

Hariharan S, Velicheti A, Anagha AS, Thomas C, Balakrishnan N (2021) Explainable Artificial Intelligence in Cybersecurity: A Brief Review. 2021 4th International Conference on Security and Privacy (ISEA-ISAP) , 1–12. https://doi.org/10.1109/ISEA-ISAP54304.2021.9689765

Hasan M, Islam MU, Uddin J (2023) Advanced persistent threat identification with boosting and explainable AI. SN Comput Sci 4:1–9

Hassija V, Chamola V, Mahapatra A, Singal A, Goel D, Huang K, Scardapane S, Spinelli I, Mahmud M, Hussain A (2024a) Interpreting Black-Box models: a review on explainable Artificial Intelligence. Cogn Comput 16(1):45–74. https://doi.org/10.1007/s12559-023-10179-8

Hassija V, Chamola V, Mahapatra A, Singal A, Goel D, Huang K, Scardapane S, Spinelli I, Mahmud M, Hussain A (2024b) Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence. In Cognitive Computation (Vol. 16, Issue 1, pp. 45–74). Springer. https://doi.org/10.1007/s12559-023-10179-8

Hdaib M, Rajasegarar S, Pan L (2024) Quantum deep learning-based anomaly detection for enhanced network security. Quantum Mach Intell 6(1):26. https://doi.org/10.1007/s42484-024-00163-2

Hewamalage H, Bergmeir C, Bandara K (2021) Recurrent neural networks for Time Series forecasting: current status and future directions. Int J Forecast 37(1):388–427. https://doi.org/10.1016/j.ijforecast.2020.06.008

Holt, T. J., Griffith, M., Turner, N., Greene-Colozzi, E., Chermak, S., & Freilich, J. D. (2023). Assessing nation-state-sponsored cyberattacks using aspects of Situational Crime Prevention. Criminology & Public Policy, 22, 825–848. https://doi.org/10.1111/1745-9133.12646

Homoliak I, Toffalini F, Guarnizo J, Elovici Y, Ochoa M (2020) Insight into insiders and IT. ACM-CSUR 52(2):1–40. https://doi.org/10.1145/3303771

Houda ZA, El, Brik B, Khoukhi L (2022) Why should I trust your IDS? An Explainable Deep Learning Framework for Intrusion Detection systems in Internet of things networks. IEEE Open J Commun Soc 3:1164–1176. https://doi.org/10.1109/OJCOMS.2022.3188750

Huang YH, Su M, Xu YT, Liu T (n.d.) NER in Cyber threat intelligence domain using transformer with TSGL. J CIRCUITS Syst COMPUTERS. https://doi.org/10.1142/S0218126623502018

Ibrahim SM, Ansari SS, Hasan SD (2023) Towards white box modeling of compressive strength of sustainable ternary cement concrete using explainable artificial intelligence (XAI). Appl Soft Comput 149:110997. https://doi.org/10.1016/j.asoc.2023.110997

Jabar T, Mahinderjit Singh M (2022) Exploration of Mobile device behavior for Mitigating Advanced Persistent threats (APT): a systematic literature review and conceptual Framework. Sensors 22(13):4662. https://doi.org/10.3390/s22134662

Jayapradha J, Vineethkumar S, Vigneshwaran R, Ramprasath A (2024) Intrusion detection system for Phising Detection Using Convolution Neural Network. Educational Administration: Theory Pract 30(5):5565–5575. https://doi.org/10.53555/kuey.v30i5.3823

Jia W, Liu Z, Zhang H, Yu R, Li L (2024) Towards Score-Based Black-Box Adversarial Examples Attack in Real World. In Y. Pei, H. S. Ma, Y.-W. Chan, & H.-Y. Jeong (Eds.), Proceedings of Innovative Computing 2024, Vol. 4 (pp. 211–216). Springer Nature Singapore

Jia Y, McDermid J, Lawton T, Habli I (2021) The role of Explainability in Assuring Safety of Machine Learning in Healthcare. May , 1–30. http://arxiv.org/abs/2109.00520

Kalutharage CS, Liu X, Chrysoulas C, Bamgboye O (2024) Utilizing the Ensemble Learning and XAI for Performance Improvements in IoT Network Attack Detection. In S. Katsikas, H. Abie, S. Ranise, L. Verderame, E. Cambiaso, R. Ugarelli, I. Praça, W. Li, W. Meng, S. Furnell, B. Katt, S. Pirbhulal, A. Shukla, M. Ianni, M. Dalla Preda, K.-K. R. Choo, M. Pupo Correia, A. Abhishta, G. Sileno, … N. Yanai (Eds.), Computer Security. ESORICS 2023 International Workshops (pp. 125–139). Springer Nature Switzerland

Karim SS, Afzal M, Iqbal W, Abri D, Al (2024) Advanced Persistent threat (APT) and intrusion detection evaluation dataset for linux systems 2024. Data Brief 54:110290. https://doi.org/10.1016/j.dib.2024.110290

Kenny EM, Ford C, Quinn M, Keane MT (2021) Explaining black-box classifiers using post-hoc explanations-by-example: the effect of explanations and error-rates in XAI user studies. Artif Intell 294:103459. https://doi.org/10.1016/j.artint.2021.103459

Keshk M, Koroniotis N, Pham N, Moustafa N, Turnbull B, Zomaya AY (2023) An explainable deep learning-enabled intrusion detection framework in IoT networks. Inf Sci 639:119000. https://doi.org/10.1016/j.ins.2023.119000

Khan IA, Moustafa N, Pi D, Sallam KM, Zomaya AY, Li B (2022) A New Explainable Deep Learning Framework for Cyber threat Discovery in Industrial IoT Networks. IEEE Internet Things J 9(13):11604–11613. https://doi.org/10.1109/JIOT.2021.3130156

Khan IA, Moustafa N, Razzak I, Tanveer M, Pi D, Pan Y, Ali BS (2022b) XSRU-IoMT: explainable simple recurrent units for threat detection in internet of medical things networks. Future Generation Comput Syst 127:181–193. https://doi.org/10.1016/j.future.2021.09.010

Khan IA, Razzak I, Pi D, Zia U, Kamal S, Hussain Y (2024) A Novel Collaborative SRU Network with dynamic behaviour aggregation, reduced communication overhead and explainable features. IEEE J Biomedical Health Inf 28(6):3228–3235. https://doi.org/10.1109/JBHI.2024.3352013

Khraisat A, Gondal I, Vamplew P, Kamruzzaman J (2019) Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecurity 2(1):20. https://doi.org/10.1186/s42400-019-0038-7

Korium MS, Saber M, Beattie A, Narayanan A, Sahoo S, Nardelli PHJ (2024) Intrusion detection system for cyberattacks in the internet of vehicles environment. Ad Hoc Netw 153:103330. https://doi.org/10.1016/j.adhoc.2023.103330

Kuhn DR, Kacker RN, Lei Y, Simos DE (2020) Combinatorial Methods for Explainable AI. 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) , 167–170. https://doi.org/10.1109/ICSTW50294.2020.00037

Kumaresan SJ, Senthilkumar C, Kongkham D, B, B. B., Nirmala P (2024) Investigating the Effectiveness of Recurrent Neural Networks for Network Anomaly Detection. 2024 International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE) , 1–5. https://doi.org/10.1109/IITCEE59897.2024.10467790

Kuppa A, Le-Khac NA (2021) Adversarial XAI methods in Cybersecurity. IEEE Trans Inf Forensics Secur 16:4924–4938. https://doi.org/10.1109/TIFS.2021.3117075

Kute DV, Pradhan B, Shukla N, Alamri A (2021) Deep learning and explainable Artificial Intelligence techniques Applied for detecting money Laundering-A critical review. IEEE Access 9:82300–82317. https://doi.org/10.1109/ACCESS.2021.3086230

Lee JS, Chen YC, Chew CJ, Chen CL, Huynh TN, Kuo CW (2022) CoNN-IDS: Intrusion detection system based on collaborative neural networks and agile training. Comput Secur 122:102908. https://doi.org/10.1016/j.cose.2022.102908

Lemay A, Calvet J, Menet F, Fernandez JM (2018) Survey of publicly available reports on advanced persistent threat actors. Computers Secur 72:26–59. https://doi.org/10.1016/j.cose.2017.08.005

Le T-T-H, Kim H, Kang H, Kim H (2022) Classification and explanation for intrusion detection system based on ensemble trees and SHAP method. Sensors 22:1154. https://doi.org/10.3390/s22031154

Liang Y, Li S, Yan C, Li M, Jiang C (2021) Explaining the black-box model: a survey of local interpretation methods for deep neural networks. Neurocomputing 419:168–182. https://doi.org/10.1016/j.neucom.2020.08.011

Linardatos P, Papastefanopoulos V, Kotsiantis S (2021) Explainable AI: a review of machine learning interpretability methods. Entropy 23(1). https://doi.org/10.3390/e23010018

Liu H, Zhong C, Alnusair A, Islam SR (2021) FAIXID: a Framework for enhancing AI explainability of intrusion detection results using data cleaning techniques. J Netw Syst Manage 29(4):1–30. https://doi.org/10.1007/s10922-021-09606-8

Liu J, Shen Y, Simsek M, Kantarci B, Mouftah HT, Bagheri M, Djukic P (2022) A new realistic benchmark for Advanced Persistent threats in Network Traffic. IEEE Netw Lett 4:1. https://doi.org/10.1109/LNET.2022.3185553

Liu Y, Wu L (2023) Intrusion detection model based on Improved Transformer. Appl Sci 13(10). https://doi.org/10.3390/app13106251

Lo WW, Kulatilleke G, Sarhan M, Layeghy S, Portmann M (2023) XG-BoT: an explainable deep graph neural network for botnet detection and forensics. Internet Things 22:100747. https://doi.org/10.1016/j.iot.2023.100747

Lundberg S, Lee S-I (2017) A Unified Approach to Interpreting Model Predictions . http://arxiv.org/abs/1705.07874

Manoharan P, Yin J, Wang H, Zhang Y, Ye W (2023) Insider threat detection using supervised machine learning algorithms. Telecommunication Syst. https://doi.org/10.1007/s11235-023-01085-3

McGinley C, Monroy SAS (2021) Convolutional Neural Network Optimization for Phishing email classification. 2021 IEEE Int Conf Big Data (Big Data) 5609–5613. https://doi.org/10.1109/BigData52589.2021.9671531

Meister S, Wermes M, Stüve J, Groves RM (2021) Investigations on explainable Artificial Intelligence methods for the deep learning classification of fibre layup defect in the automated composite manufacturing. Compos Part B: Eng 224(May):109160. https://doi.org/10.1016/j.compositesb.2021.109160

Mendonça RV, Teodoro AA, Rosa RL, Saadi M, Melgarejo DC, Nardelli PH, Rodríguez DZ (2021) Intrusion detection system based on fast hierarchical deep convolutional neural network. IEEE Access 96:1024–61034 https://doi.org/10.1109/ACCESS.2021.3074664

Mittal M, Kumar K, Behal S (2023) Deep learning approaches for detecting DDoS attacks: a systematic review. Soft Comput 27(18):13039–13075. https://doi.org/10.1007/s00500-021-06608-1

Mohamed, N. (2023). Current trends in AI and ML for cybersecurity: a state-of-the-art survey. Cogent Eng 10(2). https://doi.org/10.1080/23311916.2023.2272358

Montavon G, Binder A, Lapuschkin S, Samek W, Müller K-R (2019) Layer-Wise Relevance Propagation: An Overview. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (pp. 193–209). https://doi.org/10.1007/978-3-030-28954-6_10

Moustafa N, Slay J (2016) The evaluation of Network Anomaly Detection systems: statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set. Inform Secur Journal: Global Perspective 25(1–3):18–31. https://doi.org/10.1080/19393555.2015.1125974

Najar AA,S., M. N (2024) A robust DDoS intrusion detection system using convolutional neural network. Comput Electr Eng 117:109277. https://doi.org/10.1016/j.compeleceng.2024.109277

Pahuja V, Ojha SS (2024) DeepDeter: Strengthening Cybersecurity Against DoS Attacks with Deep Learning. 2024 2nd International Conference on Device Intelligence, Computing and Communication Technologies (DICCT) , 1–6. https://doi.org/10.1109/DICCT61038.2024.10533167

Patel D, Rajesh T, Balamurugan G (2024) Enhancing Cybersecurity Vigilance with Deep Learning for Malware Detection. 2024 10th International Conference on Communication and Signal Processing (ICCSP) , 1005–1010. https://doi.org/10.1109/ICCSP60870.2024.10544228

Patil S, Varadarajan V, Mazhar SM, Sahibzada A, Ahmed N, Sinha O, Kumar S, Shaw K, Kotecha K (2022) Explainable Artificial Intelligence for Intrusion Detection System. Electronics 11(19). https://doi.org/10.3390/electronics11193079

Pawlicki M, Pawlicka A, Kozik R, Choraś M (2024) Advanced insights through systematic analysis: mapping future research directions and opportunities for xAI in deep learning and artificial intelligence used in cybersecurity. Neurocomputing 590:127759. https://doi.org/10.1016/j.neucom.2024.127759

Raju AD, Abualhaol IY, Giagone RS, Zhou Y, Huang S (2021) A Survey on Cross-architectural IoT Malware threat Hunting. IEEE Access 9:91686–91709. https://doi.org/10.1109/access.2021.3091427

Reis JCS, Correia A, Murai F, Veloso A, Benevenuto F (2019) Explainable machine learning for fake news detection. WebSci 2019 - Proc 11th ACM Conf Web Sci 17–26. https://doi.org/10.1145/3292522.3326027

Remman SB, Strümke I, Lekkas AM (2021) Causal versus Marginal Shapley Values for Robotic Lever Manipulation Controlled using Deep Reinforcement Learning . http://arxiv.org/abs/2111.02936

Ridzuan F, Zainon WMN (2019) A review on data cleansing methods for Big Data. Procedia Comput Sci 161:731–738. https://doi.org/10.1016/j.procs.2019.11.177

Rjoub G, Bentahar J, Abdel Wahab O, Mizouni R, Song A, Cohen R, Otrok H, Mourad A (2023) A Survey on Explainable Artificial Intelligence for Cybersecurity. IEEE Trans Netw Serv Manage 20(4):5115–5140. https://doi.org/10.1109/TNSM.2023.3282740

Saeed W, Omlin C (2023) Explainable AI (XAI): a systematic meta-survey of current challenges and future opportunities. Knowl Based Syst 263:110273. https://doi.org/10.1016/j.knosys.2023.110273

Sakthipriya N, Govindasamy V, Akila V (2024) Security-aware IoT botnet attack detection framework using dilated and cascaded deep learning mechanism with conditional adversarial autoencoder-based features. Peer-to-Peer Netw Appl 17(3):1467–1485. https://doi.org/10.1007/s12083-024-01657-3

Salim DT, Singh MM, Keikhosrokiani P (2023) A systematic literature review for APT detection and effective Cyber situational awareness (ECSA) conceptual model. Heliyon 9(7):e17156. https://doi.org/10.1016/j.heliyon.2023.e17156

Samek W, Wiegand T, Müller KR (2017) Explainable artificial intelligence: understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296

Saravanan V, Madiajagan M, Shaik M, Sanju P, Rehman T, Pattanaik B (2023) IoT-based blockchain intrusion detection using optimized recurrent neural network. Multimedia Tools Appl 83:1–22. https://doi.org/10.1007/s11042-023-16662-6

Sarker IH, Janicke H, Mohsin A, Gill A, Maglaras L (2024) Explainable AI for cybersecurity automation, intelligence and trustworthiness in digital twin: methods, taxonomy, challenges and prospects. ICT Express. https://doi.org/10.1016/j.icte.2024.05.007

Schneider S, Antensteiner D, Soukup D, Scheutz M (2022) Autoencoders - A Comparative Analysis in the Realm of Anomaly Detection. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) , 1985–1991. https://doi.org/10.1109/CVPRW56347.2022.00216

Schwalbe G, Finzel B (2023) A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts. Data Mining and Knowledge Discovery . https://doi.org/10.1007/s10618-022-00867-8

Seibold C, Hilsmann A, Eisert P (2021) Focused LRP: Explainable AI for Face Morphing Attack Detection. Proceedings – 2021 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2021 , 88–96. https://doi.org/10.1109/WACVW52041.2021.00014

Shams Khoozani Z, Sabri AQM, Seng WC, Seera M, Eg KY (2024) Navigating the landscape of concept-supported XAI: challenges, innovations, and future directions. Multimedia Tools Appl. https://doi.org/10.1007/s11042-023-17666-y

Sharma A, Gupta BB, Singh AK, Saraswat VK (2023) Advanced Persistent threats (APT): evolution, anatomy, attribution and countermeasures. J Ambient Intell Humaniz Comput 14(7):9355–9381. https://doi.org/10.1007/s12652-023-04603-y

Sharma DK, Mishra J, Singh A, Govil R, Srivastava G, Lin JC-W (2022) Explainable Artificial Intelligence for Cybersecurity. Comput Electr Eng 103:108356. https://doi.org/10.1016/j.compeleceng.2022.108356

Shenderovitz G, Nissim N (2024) Bon-APT: Detection, attribution, and explainability of APT malware using temporal segmentation of API calls. Comput Secur 142:103862 https://doi.org/10.1016/j.cose.2024.103862

Siddiqi MA, Pak W (2021) An Agile Approach to identify single and hybrid normalization for Enhancing Machine Learning-Based Network Intrusion Detection. IEEE Access 9:137494–137513. https://doi.org/10.1109/ACCESS.2021.3118361

Singh A, Mishra P, Vinod P, Gaur A, Conti M (2024) SFC-NIDS: a sustainable and explainable flow filtering based concept drift-driven security approach for network introspection. Cluster Comput. https://doi.org/10.1007/s10586-024-04444-0

Smagulova K, James AP (2020) Overview of Long Short-Term Memory Neural Networks. In A. P. James (Ed.), Deep Learning Classifiers with Memristive Networks: Theory and Applications (pp. 139–153). Springer International Publishing. https://doi.org/10.1007/978-3-030-14524-8_11

Sourati Z, Prasanna Venkatesh VP, Deshpande D, Rawlani H, Ilievski F, Sandlin H-Â, Mermoud A (2023) Robust and explainable identification of logical fallacies in natural language arguments. Knowl Based Syst 266:110418. https://doi.org/10.1016/j.knosys.2023.110418

Speith T (2022) A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , 2239–2250. https://doi.org/10.1145/3531146.3534639

Stojanović B, Hofer-Schmitz K, Kleb U (2020) APT datasets and attack modeling for automated detection methods: a review. Computers Secur 92:19. https://doi.org/10.1016/j.cose.2020.101734

Stojanovic B, Hofer-Schmitz K, Kleb U (2020) APT datasets and attack modeling for automated detection methods: A review. COMPUTERS & SECURITY , 92 . https://doi.org/10.1016/j.cose.2020.101734

Sun BXL, X. M. C. L. Z. D (2024) Strengthening Network Security: deep learning models for intrusion detection with optimized feature subset and effective Imbalance Handling. Computers Mater Continua 78(2):1995–2022. https://doi.org/10.32604/cmc.2023.046478

Tadesse YE, Choi Y-J (2024) Pattern augmented lightweight convolutional neural network for intrusion detection system. Electronics 13(5). https://doi.org/10.3390/electronics13050932

Taye MM (2023) Theoretical understanding of convolutional neural network: concepts, architectures, applications, future directions. Computation 11(3). https://doi.org/10.3390/computation11030052

Teuwen J, Moriakov N (2020) Chapter 20 - Convolutional neural networks. In S. K. Zhou, D. Rueckert, & G. Fichtinger (Eds.), Handbook of Medical Image Computing and Computer Assisted Intervention (pp. 481–501). Academic Press. https://doi.org/10.1016/B978-0-12-816176-0.00025-9

Tian Y (2020) Artificial intelligence image recognition method based on convolutional neural network algorithm. IEEE Access 81:25731–125744. https://doi.org/10.1109/Access.6287639

Ullah F, Alsirhani A, Alshahrani MM, Alomari A, Naeem H, Shah SA (2022) Explainable Malware Detection System using transformers-based transfer learning and Multi-model Visual representation. SENSORS 22(18). https://doi.org/10.3390/s22186766

Ullah F, Ullah S, Srivastava G, Lin JC-W (2023) IDS-INT: intrusion detection system using transformer-based transfer learning for imbalanced network traffic. Digit Commun Networks. https://doi.org/10.1016/j.dcan.2023.03.008

Vajipayajula S (2023) Comparative Analysis of Deep Learning and Machine Learning models for Network Intrusion Detection. 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT) , 1–13. https://doi.org/10.1109/ICCCNT56998.2023.10308108

Villalón-Huerta A, Marco-Gisbert H, Ripoll-Ripoll I (2022) A taxonomy for threat actors’ persistence techniques. Computers Secur 121:102855. https://doi.org/10.1016/j.cose.2022.102855

Volkov EN, Averkin AN (2023) Possibilities of Explainable Artificial Intelligence for Glaucoma Detection Using the LIME Method as an Example. 2023 XXVI International Conference on Soft Computing and Measurements (SCM) , 130–133. https://doi.org/10.1109/SCM58628.2023.10159038

Wali S, Khan I (2021) Explainable AI and random forest based reliable intrusion detection system

Wallsberger R, Eberhardt TD, Bartlau P-A, Dörnte ML, Schröter TL, Matzka S (2022) Explainable Artificial Intelligence for a high dimensional condition monitoring application using the SHAP Method. 2022 5th International Conference on Artificial Intelligence for Industries (AI4I) , 68–72. https://doi.org/10.1109/AI4I54798.2022.00024

Wang M, Zheng K, Yang Y, Wang X (2020) An Explainable Machine Learning Framework for Intrusion Detection systems. IEEE Access 8:73127–73141. https://doi.org/10.1109/ACCESS.2020.2988359

Wang YF, Guo YB, Fang C (2022) An end-to-end method for advanced persistent threats reconstruction in large-scale networks based on alert and log correlation. J Inform Secur Appl 71. https://doi.org/10.1016/j.jisa.2022.103373

Wang Y, Li J (2023) Anomaly Detection Method for Time Series Data Based on Transformer Reconstruction. Proceedings of the 2023 12th International Conference on Informatics, Environment, Energy and Applications , 58–63. https://doi.org/10.1145/3594692.3594702

Xu F, Uszkoreit H, Du Y, Fan W, Zhao D, Zhu J (2019). Explainable AI: A brief survey on history, research areas, approaches and challenges. In: Tang J, Kan MY, Zhao D, Li S, Zan H (eds) Natural language processing and Chinese computing. NLPCC 2019. Lecture notes in computer science, vol 11839. Springer, Cham. https://doi.org/10.1007/978-3-030-32236-6_51

Yang H, Zeng R, Xu G, Zhang L (2021) A network security situation assessment method based on adversarial deep learning. Appl Soft Comput 102:107096. https://doi.org/10.1016/j.asoc.2021.107096

Yang W, Wei Y, Wei H, Chen Y, Huang G, Li X, Li R, Yao N, Wang X, Gu X, Amin MB, Kang B (2023) Survey on explainable AI: from approaches, limitations and Applications aspects. Human-Centric Intell Syst 3(3):161–188. https://doi.org/10.1007/s44230-023-00038-y

Yang Z, Ma Z, Zhao W, Li L, Gu F (2024) HRNN: Hypergraph Recurrent Neural Network for Network Intrusion Detection. J Grid Comput 22(2):52. https://doi.org/10.1007/s10723-024-09767-1

Yashwanth T, Ashwini K, Chaithanya GS, Tabassum A (2024) Network Intrusion Detection using Auto-encoder Neural Networks and MLP. 2024 Third International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE) , 1–6. https://doi.org/10.1109/ICDCECE60827.2024.10548660

Yin X, Fang W, Liu Z, Liu D (2024) A novel multi-scale CNN and Bi-LSTM arbitration dense network model for low-rate DDoS attack detection. Sci Rep 14(1):5111. https://doi.org/10.1038/s41598-024-55814-y

Yuan X, Li C, Li X (2017) DeepDefense: Identifying DDoS Attack via Deep Learning. 2017 IEEE International Conference on Smart Computing (SMARTCOMP) , 1–8. https://doi.org/10.1109/SMARTCOMP.2017.7946998

Zeltner D, Schmid B, Csiszár G, Csiszár O (2021) Squashing activation functions in benchmark tests: towards a more eXplainable Artificial Intelligence using continuous-valued logic. Knowl Based Syst 218:106779. https://doi.org/10.1016/j.knosys.2021.106779

Zhang H, Huang L, Wu CQ, Li Z (2020) An effective convolutional neural network based on SMOTE and Gaussian mixture model for intrusion detection in imbalanced dataset. Comput Netw 177:107315. https://doi.org/10.1016/j.comnet.2020.107315

Zhang Z, Hamadi H, Al, Damiani E, Yeun CY, Taher F (2022) Explainable Artificial Intelligence Applications in Cyber Security: state-of-the-art in Research. IEEE Access 10:93104–93139. https://doi.org/10.1109/ACCESS.2022.3204051

Zhang Z, Si X, Li L, Gao Y, Li X, Yuan J, Xing G (2023) An Intrusion Detection Method Based on Transformer-LSTM Model. 2023 3rd International Conference on Neural Networks, Information and Communication Engineering (NNICE) , 352–355. https://doi.org/10.1109/NNICE58320.2023.10105733

Zhang Z, Wang L (2022) An Efficient Intrusion Detection Model Based on Convolutional Neural Network and Transformer. 2021 Ninth International Conference on Advanced Cloud and Big Data (CBD) , 248–254. https://doi.org/10.1109/CBD54617.2021.00050

Zhou Q, Li R, Xu L, Nallanathan A, Yang J, Fu A (2022) Towards Explainable Meta-Learning for DDoS Detection . http://arxiv.org/abs/2204.02255

Zhu Q, Zu X (2022) Fully convolutional neural network structure and its loss function for image classification. IEEE Access 10:35541–35549. https://doi.org/10.1109/ACCESS.2022.3163849

Zolanvari M, Yang Z, Khan K, Jain R, Meskin N (2023) TRUST XAI: model-agnostic explanations for AI with a case study on IIoT Security. IEEE Internet Things J 10(4):2967–2978. https://doi.org/10.1109/JIOT.2021.3122019

Download references

This work is funded by the Ministry of Higher Education, Malaysia (JPT(BKPI)1000/016/018/25(58)) through Malaysia Big Data Research Excellence Consortium (BiDaREC), via the research grant managed by Universiti Malaya (Grant No.: KKP002-2021).

Author information

Authors and affiliations.

Department of Artificial Intelligence, Faculty of Computer Science and Information Technology, University Malaya, Kuala Lumpur, Wilayah Persekutuan Kuala Lumpur, 50603, Malaysia

Noor Hazlina Abdul Mutalib, Aznul Qalid Md Sabri & Erma Rahayu Mohd Faizal Abdullah

Department of Computer System & Technology, Faculty of Computer Science and Information Technology, University Malaya, Wilayah Persekutuan Kuala Lumpur, Kuala Lumpur, 50603, Malaysia

Ainuddin Wahid Abdul Wahab

Department of Computer Science, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates

Nouar AlDahoul

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, N.H.A.M, A.Q.M. and N.A.; methodology, N.H.A.M and A.Q.M; original draft preparation and writing of the article, N.H.A.M; review, mentoring and proofreading, A.Q.M, A.W.A.W, E.R.M.F.A and N.A. All authors have read and agreed to submit this article.

Corresponding author

Correspondence to Aznul Qalid Md Sabri .

Ethics declarations

Conflict of interest.

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Mutalib, N.H.A., Sabri, A.Q.M., Wahab, A.W.A. et al. Explainable deep learning approach for advanced persistent threats (APTs) detection in cybersecurity: a review. Artif Intell Rev 57 , 297 (2024). https://doi.org/10.1007/s10462-024-10890-4

Download citation

Accepted : 25 July 2024

Published : 18 September 2024

DOI : https://doi.org/10.1007/s10462-024-10890-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Advanced persistent threats (APTs)
• Explainable artificial intelligence (XAI)
• Interpretability
• Deep learning
• Black-box models
• Cybersecurity
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 18 September 2024

Clinicians’ perspectives of immersive tools in clinical mental health settings: a systematic scoping review

• Jessica Cushnan 1 ,
• Paul McCafferty 1 &
• Paul Best 1  

BMC Health Services Research volume  24 , Article number:  1091 ( 2024 ) Cite this article

Metrics details

Virtual Reality in mental health treatment has potential to address a wide spectrum of psychological and neurocognitive disorders. Despite the proven benefits, integration into clinical practice faces significant challenges. There is a critical need for research into clinicians’ perceptions of virtual reality due to the gap between rapid technological advancements and their adoption in mental health services.

A scoping review was conducted to comprehensively understand clinicians’ perspectives on the application of immersive virtual reality technologies within mental health settings. 4 data bases were searched, from inception, with the search areas of clinicians’, technology, perspectives and mental health. The scoping review followed the PRISMA-ScR checklist. All results were thematically analysed to identify and categorise themes with a focus on qualitative analyses of clinicians’ experiences and perceptions of VR applications in therapeutic contexts.

17 articles were selected, encompassing a range of mental health settings. The findings indicate that the integration of VR in clinical environments is heavily influenced by clinicians’ knowledge and experience, with unfamiliarity often leading to scepticism. Positive attitudes towards VR, bolstered by direct experience and training, were found to drive acceptance, as clinicians’ acknowledged its potential to complement traditional therapies. However, there are still gaps in understanding VR’s therapeutic applications, particularly concerning its impact on human interaction and its suitability for specific patient groups. Balancing VR’s clinical benefits with ethical and safety concerns is crucial, especially when working with vulnerable populations. Furthermore, structural and administrative support is essential to overcoming the financial and logistical challenges of VR implementation, ensuring its safe and effective integration into mental health care.

While VR holds significant potential for enhancing mental health care, its successful integration into clinical practice necessitates addressing existing gaps in knowledge, training, and structural support. By carefully balancing its clinical benefits with ethical, practical, and safety considerations, VR can be effectively utilised as a valuable tool in mental health treatment, providing innovative solutions while ensuring that patient care remains paramount.

Peer Review reports

Immersive technologies, such as Virtual Reality (VR) and game-based interventions, have made significant strides in the field of mental health, demonstrating their potential and promise in therapeutic care [ 1 , 2 ]. The research surrounding the use of VR, which is defined as a technology that creates a simulated, immersive environment where users can interact with computer-generated sensory experiences, often through visual, auditory, and sometimes tactile feedback. Using headsets or other devices that track movements, users can explore and engage with these virtual spaces in real-time, giving them the sensation of being physically present within the digital world [ 3 ]. This capability to design and manipulate these environments is fuelling the growing interest in using VR for both the treatment, and assessment of psychological and neurocognitive conditions, such as anxiety [ 4 ], post-traumatic stress disorder (PTSD) [ 5 ] obsessive compulsive disorder [ 6 ], psychosis [ 7 ], children with autism [ 8 ], and a range of developmental and learning disabilities [ 9 ]. The application of VR offers several key benefits. Firstly, it offers a safe and controlled environment for many of the aforementioned mental health conditions, allowing patients to face their fears within the varied scenarios that VR can provide, whether these are simulated real-life situations or entirely fictional and imaginative environments. This exposure component serves as a tool that has comparable effects, if not more so, than traditional exposure therapies [ 10 ]. Due to the immersive and interactive nature, it offers new possibilities to enhance areas of traditional therapy that were previously unattainable [ 11 ], for example, the element of personalisation which allows clinicians’ to tailor contextually relevant environments to meet the specific needs of their patients safely [ 12 ]. Furthermore, the research has indicated that VR has the potential to open a more engaging and motivational therapeutic platform for patients, which could be especially beneficial for those who are sceptical about therapy or find it challenging and stigmatising [ 3 ].

This is particularly relevant given the current state of affairs within the remit of mental health services. In 2019, mental health disorders ranked in the top ten leading causes of burden worldwide, a situation that has shown no reduction since the 1990’s [ 13 ]. Moreover, there is a growing treatment gap within mental health care services, seeing an estimated 70% of people who are in need not getting access to them [ 14 , 15 ].

In addition to the benefits, there is a significant cultural shift in healthcare, reflective of global recognition of the potential of digital solutions. For instance, the World Health Organisation’s Global Strategy on Digital Health [ 16 ] sets a foundational tone for this transition, highlighting the need for better training and infrastructure to harness digital technologies to their full potential to enhance healthcare delivery. Furthermore, commitment to enhancing digital competencies are underscored within nationwide government plans such as the Department of Health’s All Ireland Digital Capability Framework [ 17 ] and the UK governments Plan for Digital Health and Social Care [ 18 ]. This paradigm shift aligns not only with the increasing evidence base of research and technological advancements, but the evolving needs and preferences of service providers and service users [ 19 ] opening the doors for a new era for mental health treatment.

To date however, there is an evident gap between the rate in which technology is advancing and the adoption in mental health services [ 20 ]. In a recent systematic review, Best and colleagues [ 21 ] highlight the implementation difficulties of implementing VR within clinical mental health settings which include perceived costs, the lack of technical standardisations, and low acceptance amongst clinicians’. Additionally, there is broader belief that the technology may impede patient engagement or replace the role of a mental health professional [ 3 ]. Currently, there is a gap in the research concerning the detailed qualitative insights of clinicians’ on the use of VR within clinical settings.

By adopting the Population, Concept, Context (PCC) Framework to define the research goal and guide the study protocol [ 22 ], this review targets clinicians’ (population), the use of immersive VR tools (concept), within the setting of clinical mental health care (context). Therefore, this review aims to combine and analyse evidence from clinicians’ qualitative feedback to answer the question ‘What are clinicians’ views on using immersive VR tools in mental health clinical settings?’ The specific aims were (i) Conduct a scoping review: Following the Joanna Briggs Institute Manual for Evidence Synthesis [ 22 ] for Scoping Reviews, research was gathered meticulously to review the clinicians’ viewpoint on the application of VR in mental health settings. This includes studies across various mental health disciplines and clinical environments; (ii) Carry out a thematic analysis and synthesis: A thematic analysis approach was applied to the qualitative data collected using the Braun and Clarke [ 23 ] six stage model to allow for the charting of key themes and insights. This method will also allow for the nuanced understanding of the clinicians’ experiences and attitudes towards VR in mental health care; (iii) Report findings and identify research gaps: To identify and report any gaps in the current body of literature. This will include aspects of the VR application in mental health care that may need further investigation or if the clinicians’ perspectives have not been understood or reflected.

Study design

We utilised an enhanced scoping review methodology, guided by the JBI Manual for Evidence Synthesis [ 22 ], which explicitly details each stage of the review process: (1) defining and aligning the objectives and question, (2) developing and aligning the inclusion criteria with the objectives and question, (3) describing the planned approach to evidence searching, selection, data extraction, and presentation of the evidence, (4) searching for the evidence, (5) selecting the evidence, (6) extracting the evidence, (7) analysis of the evidence, (8) presentation of the results, (9) summarising the evidence in relation to the purpose of the review, making conclusions and noting any implications of the findings.

Search strategy and data sources

After consulting with a subject expert and the subject librarian at Queen’s University, Belfast, we undertook a search across PTSD Pubs, PUB Med, Scopus, and APA PsychInfo databases from October to November 2023, selected for their topic relevance. Initial searches in October 2023 led to further refinement, culminating in a final search in November 2023. We employed Boolean operators and truncation to accommodate different variations of the term “mental,” integrating these terms with ‘AND’ to focus the search further.:

Clinician OR Therapist.

VR OR ‘Virtual Reality’ OR VRET OR ‘Virtual Reality Exposure Therapy’.

Perspective OR View OR Attitude OR experience.

Mental* OR psychological OR disorder.

All results were imported to EndNote 21 for screening.

Study selection

To be included in this review, studies needed to meet specific criteria: (1) they must capture the qualitative perspectives of clinicians’ (population), (2) concentrate on the use of immersive VR technology (concept), and (3) be pertinent to mental health care settings (context). The emphasis on VR technology within mental health care aims to directly address the review’s specific interest area. Including studies that offer clinicians’ qualitative insights was deliberately chosen to gather practical and professional perspectives on VR’s application in clinical settings, valuing clinicians’ experiences as key to understanding VR’s real-world impact on mental health care. We opted exclusively for peer-reviewed articles to ensure the reliability and credibility of our findings. Peer review introduces an essential layer of quality control, as these articles undergo rigorous scrutiny before publication [ 24 ]. We limited our search to articles in English for practicality, acknowledging potential limitations in scope. While focusing on English-language articles might not encompass all available evidence, this constraint is unlikely to significantly affect our comprehensive findings [ 25 ]. We applied no date restrictions, recognising the importance of capturing the full research spectrum on VR in mental health. This approach ensures a broad overview of both historical and recent studies, enriching the review with diverse insights.

In the final phase, the search yield was 297 studies. After removing 41 duplicate records, the remaining 256 articles then underwent an independent screening based on their titles and abstracts in alignment with the predefined inclusion criteria. Each article was then colour coded and categorised as ‘yes’, ‘no’, and ‘maybe’ for further consideration with ‘maybe’ indicating the need for a full text review to determine if the qualitative perspectives of the clinicians’ were included in the data. To uphold the integrity and consistency of the review process a re-evaluation of 100% of articles was conducted by an independent author to ensure inter-rater reliability [ 22 ]. This approach led to the preliminary selection of 30 articles for detailed full text review. During the full text phase, any discrepancies between authors were extensively discussed until a unanimous agreement was reached. As a result, a further 13 articles were excluded for not meeting the established criteria, leaving 17 articles deemed relevant and suitable for inclusion in the review. The final step in the search process involved an independent hand search conducted by the primary author among the final 17 articles to identify any additional studies. However, this careful examination did not reveal any new studies for inclusion, solidifying the selection of the final 17 articles. The review selection process is shown in Fig.  1 , using the PRISMA diagram [ 26 ].

PRISMA 2021 Flow Diagram

Quality appraisal

The critical appraisal of the seventeen selected articles, a step beyond the usual scope of scoping reviews, was undertaken to improve the quality of the review. While scoping reviews typically map out research areas without assessing study quality [ 22 ], appraising these articles helped to clarify evidence levels and bolster the review’s credibility [ 27 ]. This mix of 9 mixed method and 8 qualitative studies underwent appraisal using the JBI Checklist for Qualitative Research [ 28 ], following Bryman’s [ 29 ] advice for a separate criteria approach for mixed methods. Despite evaluating studies on a ten-point scale, no cutoff was used to exclude studies, aiming for inclusivity and a rich qualitative exploration [ 30 ]. To ensure inter-rater reliability, all articles were re-evaluated by an independent author, with discrepancies discussed until consensus, ensuring all seventeen articles were thoroughly and consistently appraised.

Characteristics and thematic analysis

The characteristics of the included studies are charted in Table  1 . Among the 17 studies, 3 were conducted in Australia, United Kingdom, the Netherlands, and Canada. 2 in the United States, and 1 study each in Spain, Germany, and Norway. All of the studies had trained professionals as their participants, however included in this, 2 of the studies also incorporated stakeholders and service managers as participants. The study design included 9 mixed method studies and 8 qualitative studies. This study employed Reflexive Thematic Analysis (RTA) as outlined by Braun and Clarke [ 23 , 31 ] to analyse the qualitative data collected to explore the perspectives of clinicians’ on the use of immersive VR tools in clinical mental health settings. RTA emphasises the researcher’s active role in identifying and interpreting patterns of meaning within the data, acknowledging that the researcher’s subjective position influences the analysis. This approach is suitable for an inductive analysis, which allowed flexibility in exploring themes within the data for this study. Qualitative data retrieved from the 17 studies was transferred to NVivo 12 Plus analysis software, and the process began with familiarisation, where data was read and re-read to gain a deep understanding. Initial codes were then generated directly from the data without a pre-set framework. These codes were grouped into potential themes that reflected shared meanings. Themes were reviewed and refined to ensure accuracy and coherence. Each theme was then clearly defined and named to capture the core meaning. Finally, the themes were synthesised into a coherent narrative to provide a rich understanding of the findings. These themes and sub-themes are detailed in Fig.  2 .

Themes and Subthemes

Theme one: managing negative perceptions through experience and knowledge

The first, smallest of the three themes, demonstrates how the integration of VR into clinical settings is significantly influenced by how knowledge and experience shape perceptions. A lack of familiarity with VR often fuels negative attitudes and scepticism, creating barriers to its adoption as clinicians’ cited their insufficient skills and experience with VR as a major hurdle [ 32 ]. Studies indicate that many clinicians’ have never used VR or are unaware of its therapeutic potential [ 33 ], viewing it primarily as a tool for gaming and entertainment [ 34 ] rather than a clinical asset. This encompasses an understanding of VR’s applications, objectives, benefits, potential side effects, scenario variety, and follow-up care procedures [ 35 ]. This gap in knowledge has the potential to lead to resistance, especially with clinicians’ expressing concerns that VR might be used by private health providers to replace service provision, potentially diminishing human interactions and connections [ 33 ]. These worries were founded by the belief that people respond better to the human element, highlighting potential negative workforce attitudes towards technological advances and changes in the work place in general [ 34 , 35 ]. Furthermore, some clinicians’ have the belief that older veterans may be less inclined to engage in VR-based interventions due to unfamiliarity with the technology [ 32 ] and additionally concerns were raised that novice therapists, especially those inexperienced in trauma therapy, may face difficulties in applying complex VR-based treatments [ 36 ].

However, positive attitudes towards technology can serve as powerful drivers of acceptance, with clinicians’ recognising the importance of both staff and patient attitudes towards VR [ 34 , 35 ]. Clinicians’ were generally optimistic about the role of technology in therapy, valuing the new approaches that VR can offer [ 37 ], especially given its potential to reach patients who are less engaged with conventional treatments [ 33 ]. These positive views are further supported by the cultural popularity of technology and shifts in clinicians’ attitudes [ 32 ], perhaps influenced by media portrayals of VR’s potential [ 38 ]. Those with VR experience gained valuable insights into its therapeutic applications, as illustrated by their personal experiences with the technology [ 33 ]. Training also contributed to more positive attitudes, with clinicians’ recognising VR’s potential in skill training and safe exposure, which helped overcome their initial hesitance [ 38 , 39 ]. Familiarity with VR’s clinical applications, such as its use for PTSD treatment by Barbara Rothbaum in the USA, further deepened appreciation for its potential [ 34 ]. Once clinicians’ saw the benefits of VR as a tool for patient treatment, it increased their willingness to support and promote VR, as well as their understanding of the intervention, leading to more targeted referrals [ 40 ].

Theme two: balancing clinical benefits with practical, ethical and safety concerns

The second, more prominent theme highlighted the significant clinical benefits that VR can offer within mental health services but emphasised the need to carefully balance these advantages against various practical, ethical, and safety concerns. The results affirmed VR’s clinical benefits, with many recognising its potential flexibility to be applied across a wide array of therapeutic areas. VR is recognised as an effective addition to conventional tools, offering realistic practice settings that can expedite therapeutic interventions [ 33 , 34 , 35 ]. VR’s role as a complementary tool in enhancing traditional treatments and broadening clinicians’ skills and opportunities was emphasised [ 37 , 41 ]. Reports indicate that VR interventions are promising and well-received by both clinicians’ and end users [ 32 , 33 , 34 , 35 ]. Additionally, the effectiveness and user satisfaction of VR, particularly in interactive sessions that include therapist support, were observed [ 42 , 43 ]. The broad application and potential flexibility of VR were highlighted across various contexts, including social and daily skill training for socially isolated individuals [ 38 ], interventions for older adults with hoarding disorder [ 44 ], relaxation sessions within psychiatric services [ 40 ], and for diverse trauma-affected groups, not just the military [ 45 ]. VR has also been effectively tailored for specific uses, such as aiding individuals with fire-setting behaviours [ 46 ], and, supporting those recovering from substance abuse by providing recovery insights for family work and psychoeducation [ 47 ].

An additional benefit to using VR in therapy is driven by its perceived safety and the ability to provide controlled exposure to various scenarios [ 33 , 38 , 41 ]. VR enhances access and control over therapeutic stimuli [ 35 ], allowing therapists to monitor sessions and intervene as needed, thereby ensuring client safety [ 39 ]. This safety contributes to clients feeling more secure [ 37 ] and the structured environment of VR empowers clients by providing a safer alternative to real-life exposure [ 36 ], which is particularly beneficial in managing high-risk behaviours [ 47 , 48 ].

However, despite the benefits, this perception of safety is counterbalanced by significant concerns about the potential clinical risks and ethical dilemmas posed by VR, particularly the fear that it could exacerbate symptoms or lead to avoidance of real-world interactions if not used correctly. Concerns were raised about using VR without proper training or thorough client assessments, which could increase the risk of negative outcomes [ 33 ]. While some clinicians’ view these risks as comparable to traditional methods, others advocate for stringent protocols to ensure safe and ethical use, particularly to avoid triggering content [ 35 ]. Ethical considerations also include maintaining professional boundaries to prevent injury [ 34 ]. Additional concerns involve the potential for VR to worsen symptoms in vulnerable populations, such as veterans, and the risk of re-traumatisation or triggering trauma, which stresses the need for careful patient selection and management [ 32 , 46 , 47 ].

Concerns about the compatibility of VR with certain therapeutic philosophies also present challenges to its adoption. Some professionals are sceptical about VR’s fit with therapeutic approaches that emphasise creativity and strength-based methods, such as art therapy [ 33 ]. Clinicians’ from non-cognitive-behavioural backgrounds, particularly those practicing psychodynamic therapy, question VR’s effectiveness in addressing the underlying causes of anxiety, as it tends to focus on behavioural symptoms instead [ 42 ]. Concerns have also been raised about VR’s interaction with psychotropic medications and its potential to alter patients’ perceptions or exacerbate detachment [ 35 ]. Additionally, the appropriateness of VR for severe conditions like schizophrenia, and its challenges for individuals sensitive to VR’s sensory inputs, such as those with vision impairments, are significant concerns [ 32 , 34 , 39 , 45 ]. While VR offers substantial clinical benefits and therapeutic potential, these advantages must be carefully balanced with the legitimate, ethical and safety concerns associated with its use in mental health services.

Finally, clinicians’ raised practical concerns about equipment reliability, the need for software customisation, and the user-friendliness of the VR technology [ 33 , 35 , 42 ]. Issues such as the bulkiness of headsets, the lack of validated scenarios, and limitations in the realism and personalisation of VR environments stand out as main issues [ 32 , 34 , 47 ]. Additionally, the complexity of using VR for certain patient groups, particularly the elderly who often require additional training [ 43 , 44 ], has left clinicians’ at times struggling to effectively deliver interventions to patients who lack technological proficiency [ 40 ].

Theme three: the role of structural and administrative support in VR feasibility

The final theme examines the broader logistical elements and the role of structural and administrative support, highlighting how these factors are crucial in determining the feasibility of implementing VR in clinical settings. The time required for VR interventions often exceeded initial estimates [ 43 ], which poses a challenge in busy psychiatric environments where clinicians’ primarily work in groups, VR was perceived as an isolating activity, creating practical difficulties due to workload and time constraints [ 33 ], with the additional consideration that a separate room might even be needed [ 42 ]. Clinicians’ also raised concerns about access to appropriate resources and technology to address the disconnect and relationship between clinicians’ and patients during VR sessions, such as reliable Wi-Fi, and the necessity for preparatory measures like additional TV screens or mobile devices to stream and monitor client activities [ 32 , 39 , 40 , 46 ]. It was evident, however, that there was shared confidence among clinicians’ that with the right support and forward planning, challenges such as funding, space, time, and resources could be overcome [ 33 , 39 ]. Support from managers was considered crucial, as they played a significant role due to their influence over other staff members and general service operations [ 34 ]. In studies where VR interventions were successfully implemented, the quality and availability of technical support were highly praised [ 36 , 43 ], with clinicians’ at all sites feeling they had adequate support to manage in-session situations. Furthermore, clinicians’ generally felt they were able to receive support from other professionals using the technologies globally, highlighting the wider support network available [ 48 ].

Cost and the facilitation of additional training were also flagged as prominent concerns, due to the notable lack of expertise to provide adequate training for clinicians’ [ 35 ]. Managers reported that they need expert advice on the evidence base, available hardware and software, training resources, and implementation strategies [ 35 , 45 ]. Clinicians’ unanimously questioned the impact of previous investment in quality improvement activities, which could constrain resources or affect organisational stability. This concern was compounded by the additional costs associated with purchasing and maintaining VR, providing staff training, and ongoing technical assistance [ 32 , 35 , 46 ]. The financial viability of VR was also questioned, with participants perceiving that VR might not be viewed as financially lucrative enough by private stakeholders [ 33 , 34 ]. However, it was argued that this fiscal barrier might be lessened if the VR program was suitable for other clinical applications, making it more cost-effective and useful for a broader population [ 32 , 42 , 45 , 46 ].

As mentioned in the second theme, there are identified gaps in procedural knowledge, including how to select appropriate clients for VR, apply VR clinically, and manage safety risks and procedures [ 34 ], with the need for technology to be simplified for easy use [ 41 ]. This highlights the importance of access to treatment manuals, in-service training days, the development of clinical governance processes, and consultation opportunities with VR developers and early adopter services [ 33 ]. There were mixed responses to the manualised nature of a VR protocol, some clinicians’ found it helpful in guiding treatment implementation and considered it intuitive and easy to use, while others felt it limited therapy time, with deviations from the protocol being reported [ 40 , 43 , 44 ]. Additionally, similar to the first theme of managing negative perceptions through experience and knowledge, after applying guided VR interventions, clinicians’ were willing to participate in future studies or adapt their clinical practice to include this novel intervention [ 36 ]. Clinicians’ became more confident about using VR in mental health care after testing it themselves and recognised opportunities for using VR in different situations [ 38 ]. Interestingly, not all clinicians’ deemed formal training and protocols necessary, suggesting that simply being given a platform within a clinical setting to explore VR with the patient could enhance engagement [ 37 ].

Managing negative perceptions through experience and knowledge

While this study focused specifically on the use of VR in clinical settings, it is important to acknowledge the growing global trends in research, that explore the use of diverse technologies in the field of mental health [ 49 ]. This is perhaps reflected in the results of the first theme, which revealed considerable positivity and openness among clinicians’ towards integrating technology into their practice. Positive attitudes and perceptions highlight optimism about the future role of VR in therapy, particularly its potential to complement conventional therapies by offering novel approaches when other methods are less effective [ 50 , 51 ]. This move towards more technology-centric approaches is supported by research, showing that attitudes towards VR exposure therapy are no longer a barrier to its implementation [ 52 ] reflecting a cultural shift towards embracing VR within clinical settings.

There were, however, definite gaps in understanding VR’s therapeutic applications which generated certain negative attitudes and scepticisms on behalf of some clinicians’. These attitudes were likely not due to indifference but rather a lack of knowledge about the clinical applications of VR. Prior experience and familiarity with VR play a crucial role in shaping its perceived therapeutic potential as clinicians’ with direct experience in using VR in clinical settings tend to have a deeper appreciation of its benefits [ 53 , 54 ]. Research consistently shows that education and training are key to enhancing clinicians’ competence and willingness to integrate VR into their practice, helping to overcome initial hesitance [ 55 , 56 ].

Apprehensions that VR could replace traditional service provision and reduce human interaction were raised. Research emphasises that the clinician-patient relationship remains paramount when using VR interventions, which should complement rather than replace human empathy and support in therapy [ 57 ]. While there is a perception that vulnerable patients, particularly those seeking mental health treatments, may prefer human connections [ 58 ], some studies have found that some patients actually prefer using VR over traditional methods [ 11 , 59 ]. Additionally, recent studies like THRIVE—a four-session automated cognitive VR intervention—demonstrate that automated VR can be effective and offer advantages in cost efficiency and treatment accessibility [ 60 ]. However, due to the automation, it also carries risks of reduced human interaction. These findings highlight the need for further research to investigate service users’ preferences through patient and public involvement, addressing the apprehensions surrounding reduced human interaction [ 61 ].

As clinicians’ gain more experience and knowledge, the growing acceptance and optimism towards integrating VR into clinical practice also bring to light concerns about the practical, ethical, and safety implications of this technology. These concerns form the basis of the second discussion section, which explores the need to balance the undeniable clinical benefits of VR with the challenges posed by its implementation. As enthusiasm for VR continues to rise, it becomes increasingly important to critically evaluate how these tools can be safely and effectively integrated into practice, ensuring they enhance, rather than detract from, patient care.

Balancing clinical benefits with practical, ethical and safety concerns

The primary therapeutic component of VR is widely recognised to be its exposure element [ 1 ]. While VR technology offers the potential to enhance patient safety, control, and adaptability across various interventions, these features also raise concerns. Firstly, even with potentially triggering imagery, [ 3 , 5 ]. VR exposure has proven effective in increasing engagement. The controlled environment of VR provides a unique opportunity for safe exposure to various situations, allowing clinicians’ to monitor and intervene as needed [ 62 , 63 , 64 , 65 ]. This not only offers a safety net for patients but also empowers them with a sense of control and security that may not be achievable through traditional therapeutic methods. Additionally, the capacity for VR to simulate relevant stimuli for high-risk behaviours without exposing clients to actual dangers highlights its value as a therapeutic tool [ 46 , 47 ]. However, clinicians’ express concerns about using VR with vulnerable patient groups, fearing the potential for re-traumatisation, symptom exacerbation, or avoidance of real-world interactions. Despite these concerns, research has shown positive outcomes in using VR with populations such as those with dementia [ 66 ], schizophrenia and psychosis [ 53 ], Autism Spectrum Disorder [ 67 , 68 ], and Attention Deficit Hyperactivity Disorder [ 69 ]. Additionally, there were apprehensions around the suitability of VR for older patients. However, this concern may be based on outdated stereotypes or as highlighted in the first theme, negative perceptions stemming from a lack of knowledge and experience with VR. Recent research specifically focusing on older adults with mental health issues challenges the assumption that older individuals cannot benefit from or are intimidated by VR technology. Studies have reported promising outcomes, demonstrating that older adults can indeed engage with and benefit from VR interventions [ 70 , 71 , 72 ].

Given the unique approach that VR offers through its immersive environments, the realism and customisation capabilities of VR equipment and software are sometimes perceived as limitations, particularly when they are not identical to the real world. However, evidence suggests that VR does not need to perfectly mimic reality to be effective for therapeutic purposes. Studies have shown that VR can successfully elicit the therapeutic responses necessary for psychological treatment, even without perfect realism [ 73 , 74 , 75 ]. In addition, one effective study demonstrated that VR was simply used to set the scene, enabling patients to engage in their own imagery exercises within the VR environment thereafter [ 76 ]. This is particularly significant considering that highly customisable VR experiences might not always be accessible for standard practice or may not yet be developed for specific types of traumas. Furthermore, a meta-analysis of attrition rates in VR exposure therapy for anxiety-related disorders found that drop-out rates were comparable to those for traditional in vivo exposure [ 77 ]. This highlights VR’s effectiveness, suggesting that if it were not a viable treatment option, we would likely see a higher rate of dropouts in comparison to traditional methods.

The juxtaposition between the potential benefits and practical challenges of using VR is a recurring theme in the literature [ 73 , 78 , 79 , 80 ]. This raises the question: do the clear clinical benefits of VR outweigh, or at least balance, the practical, ethical, and safety concerns raised by clinicians’? Reflecting on the sentiments by the pioneers of technology and mental health, Bouchard and Rizzo [ 81 ], it is crucial for clinicians’ to approach the use of VR with discernment, rather than adopting the technology simply because it is available. Essentially then, giving clinicians’ the autonomy to use VR as an additional tool to expedite interventions or as an adjunct to traditional therapies, particularly when other treatment options have been exhausted [ 50 , 51 ], would enable the development of flexible, tailored treatment plans. This approach would allow for clinicians’ to make informed decisions based on practical, ethical, and safety concerns, ensuring that the treatment aligns with the specific needs of their patients.

Nonetheless, to allow clinicians’ the flexibility to use VR within their practice, the ongoing concerns reiterated in the literature regarding the scarcity of empirically supported scenarios and evidence-backed VR programs, as highlighted by Bell et al. [ 3 ] must be addressed. Providing clinicians’ with robust evidence and guidelines will enable careful screening and preparation before integrating VR into therapy. The need for specific protocols to ensure the safe and ethical use of VR is further emphasised by Best et al. [ 21 ], who, in a systematic review, noted the lack of detailed clinical guidelines for VR applications. Moreover, integrating VR into existing treatment frameworks presents challenges, particularly outside of Cognitive Behavioural Therapy (CBT). While VR aligns well with CBT’s focus on behavioural modification, its compatibility with psychodynamic approaches, which explore deeper psychological processes, remains underexplored. Addressing these concerns requires the development of detailed clinical protocols, ethical guidelines, and comprehensive training programs, which will be further explored in the third and final section.

The role of structural and administrative support in VR feasibility

The role of structural and administrative support is closely intertwined with the previous discussion sections, particularly when it comes to the cost and training challenges associated with VR implementation in mental health settings. Historically, the high costs of acquiring and maintaining VR equipment have been a significant barrier [ 54 , 82 , 83 ]. Moreover, the need for comprehensive staff training, which incurs additional expenses, creates a cycle that hinders broader implementation, as noted in theme one, the lack of training and education around VR significantly limits the uptake of VR.

Clinicians’ and managers widely acknowledge that hesitancy towards adopting VR often stems from concerns over costs and the necessary training [ 84 ]. This creates a paradox, as integrating VR into practice could lead to more efficient use of clinician time and enhanced treatment outcomes, potentially resulting in cost savings. Importantly, the costs of VR equipment have reduced significantly, making it more accessible and affordable for healthcare providers [ 21 , 83 ], however, despite these reductions, many studies still highlight cost as a significant barrier [ 85 , 86 ]. These costs, however, could be mitigated if VR is utilised for multiple applications within therapeutic settings, enhancing its overall cost-effectiveness. Additionally, recent research is offering low-cost, viable solutions [ 87 , 88 ], and demonstrating the potential of using automated VR in psychiatric care to reduce the strain on already stretched services [ 60 , 89 ].

Health and Social Care (HSC) systems are increasingly adopting digital skills strategies, like the Topol Review [ 90 ] and the All-Ireland Digital Capability Framework [ 17 ], to enhance service delivery and workforce digital literacy. While training may suffice for small, incremental changes in digital tool adoption, more significant shifts require substantial structural adjustments [ 91 ]. The implementation of VR in clinical settings, though promising, has received mixed feedback, particularly regarding its manualised protocols. Ensuring patient safety is crucial, but flexibility is also needed for patient-centred care. Addressing challenges related to funding, space, and resources through strategic planning is essential. Expert guidance on evidence-based practices, technology options, and implementation strategies is vital [ 92 ]. Successful VR interventions are supported by strong technical support and collaborative efforts, highlighting the importance of global networks [ 93 ]. Ultimately, the feasibility of VR in clinical settings hinges on overcoming technological, logistical, and financial barriers while maximising the benefits through coordinated training, education, and planning.

Limitations

This scoping review has limitations, notably its exclusion of grey literature and restriction to English-language articles, potentially overlooking significant research on VR in mental health conducted worldwide. The search across four databases might have missed studies in other databases such as CINAHL and APA PsycNet, and despite a rigorous review protocol, bias cannot be completely eliminated. Additionally, not imposing date restrictions to the search strategy provided an extensive literature overview, offering a broad view of the evidence.

The integration of VR into clinical settings represents a promising yet complex evolution in mental health care. While there is growing acceptance and optimism among clinicians’ about incorporating VR into practice, significant challenges remain. These challenges are deeply interconnected with the need for robust structural and administrative support, as well as the practical, ethical, and safety considerations that must be addressed to ensure VR’s effective and safe use. Empowering clinicians’ to use VR as an additional tool in therapy involves addressing these challenges through comprehensive training, strategic planning, and the development of evidence-based guidelines. Additionally, the findings highlight the need for further research to investigate service users’ preferences, particularly through patient and public involvement. This research is essential to address apprehensions surrounding reduced human interaction, the suitability of VR for vulnerable user groups, and the overall adaptability of VR across various conditions and demographics. As advancements in VR technology continue to unfold, these investigations will be crucial in ensuring that VR is used in a manner that maximises its benefits while safeguarding patient welfare and enhancing therapeutic outcomes.

Data availability

No datasets were generated or analysed during the current study.

Abbreviations

Virtual Reality

Post Traumatic Stress Disorder

Cognitive Behavioural Therapy

Population, Concept, Context

Joanna Briggs Institute

Health and Social Care

Hollis C, Sampson S, Simons L, Davies EB, Churchill R, Betton V, et al. Identifying research priorities for digital technology in Mental Health Care: results of the james lind alliance priority setting partnership. Lancet Psychiatry. 2018;5(10):845–54. https://doi.org/10.1016/s2215-0366(18)30296-7 .

Article   PubMed   Google Scholar  

Fleming TM, Bavin L, Stasiak K, Hermansson-Webb E, Merry SN, Cheek C, et al. Serious games and gamification for Mental Health: current status and promising directions. Front Psychiatry. 2017;7. https://doi.org/10.3389/fpsyt.2016.00215 .

Bell IH, Nicholas J, Alvarez-Jimenez M, Thompson A, Valmaggia L. Virtual reality as a clinical tool in Mental Health Research and Practice. Dialog Clin Neurosci. 2020;22(2):169–77. https://doi.org/10.31887/dcns.2020.22.2/lvalmaggia .

Article   Google Scholar  

Serrano B, Botella C, Wiederhold BK, Baños RM. Virtual reality and anxiety disorders treatment: evolution and future perspectives. Virtual Real Psychol Neurocognitive Interventions. 2019;47–84. https://doi.org/10.1007/978-1-4939-9482-3_3 .

Peskin M, Wyka K, Cukor J, Olden M, Altemus M, Lee FS, et al. The relationship between posttraumatic and depressive symptoms during virtual reality exposure therapy with a cognitive enhancer. J Anxiety Disord. 2019;61:82–8. https://doi.org/10.1016/j.janxdis.2018.03.001 .

Bouchard S, Laforest M, Gamito P, Cardenas-Lopez G. Using VR for obsessive-compulsive and related disorders. Virtual Real Psychol Neurocognitive Interventions. 2019;103–30. https://doi.org/10.1007/978-1-4939-9482-3_5 .

Freeman D, Rosebrock L, Waite F, Loe BS, Kabir T, Petit A, et al. Virtual reality (VR) therapy for patients with psychosis: satisfaction and side effects. Psychol Med. 2022;53(10):4373–84. https://doi.org/10.1017/s0033291722001167 .

Article   PubMed   PubMed Central   Google Scholar  

Gal E, Weiss PL, Zancanaro M. Using innovative technologies as therapeutic and educational tools for children with autism spectrum disorder. Virtual Real Psychol Neurocognitive Interventions. 2019;227–46. https://doi.org/10.1007/978-1-4939-9482-3_10 .

Standen PJ, Brown DJ. Developmental and learning disabilities. Virtual Real Psychol Neurocognitive Interventions. 2019;267–87. https://doi.org/10.1007/978-1-4939-9482-3_12 .

Garcia-Palacios A, Hoffman HG, Kwong See S, Tsai A, Botella C. Redefining therapeutic success with virtual reality exposure therapy. CyberPsychology Behav. 2001;4(3):341–8. https://doi.org/10.1089/109493101300210231 .

Article   CAS   Google Scholar  

Garcia-Palacios A, Botella C, Hoffman H, Fabregat S. Comparing acceptance and refusal rates of virtual reality exposure vs. in vivo exposure by patients with specific phobias. CyberPsychology & Behavior. 2007;10(5):722–4. doi:10.1089/cpb.2007.9962.

Parsons TD, Rizzo AA. Affective outcomes of virtual reality exposure therapy for anxiety and specific phobias: a meta-analysis. J Behav Therapy Experimental Psychiatry 2008 Sept;39(3):250–61. https://doi.org/10.1016/j.jbtep.2007.07.007

Global Burden of Disease Study. 2019. https://ghdx.healthdata.org/gbd-2019 Accessed: 18 March 2024.

Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ. 2004;82(11):858–66.

PubMed   PubMed Central   Google Scholar  

Wainberg ML, Scorza P, Shultz JM, Helpman L, Mootz JJ, Johnson KA, et al. Challenges and opportunities in Global Mental Health: a research-to-practice perspective. Curr Psychiatry Rep. 2017;19(5). https://doi.org/10.1007/s11920-017-0780-z .

World Health Organization. Global strategy on Digital Health 2020–2025. World Health Organization. 2021. https://www.who.int/publications-detail-redirect/9789240020924 . Accessed 18 March 2024.

Health Service Executive and Digital Health and Social Care Northern Ireland. All-Ireland Digital Capability Framework for Health and Social Care. 2022. https://dhcni.hscni.net/wp-content/uploads/2022/11/All-Ireland-Digital-Capability-Framework.pdf . Accessed 09 August 2024.

GOV UK. A plan for Digital Health and Social Care. In: GOV.UK. 2022. https://www.gov.uk/government/publications/a-plan-for-digital-health-and-social-care/a-plan-for-digital-health-and-social-care . Accessed 19 Mar 2024.

ORCHA Health orcha-reports-digital-health-products. 2021. https://www.digitalhealth.net/2021/10/orcha-reports-digital-health-products/ . Accessed: 18 March 2024.

Ionescu A, Van Daele T, Rizzo A, Blair C, Best P. 360° videos for Immersive Mental Health Interventions: a systematic review. J Technol Behav Sci 2021 Sept 29;6(4):631–51. https://doi.org/10.1007/s41347-021-00221-7

Best P, McKenna A, Quinn P, Duffy M, Van Daele T. Can virtual reality ever be implemented in routine clinical settings? A systematic narrative review of clinical procedures contained within case reports for the treatment of PTSD. Front Virtual Real. 2020;1. https://doi.org/10.3389/frvir.2020.563739 .

Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: Scoping Reviews (2020 version). Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-12Chapter : Scoping reviews.

Braun V, Clarke V. Thematic analysis: a practical guide. London: SAGE; 2022.

Book   Google Scholar  

McCafferty P, Mercado García E. Children’s participation in child welfare: a systematic review of systematic reviews. Br J Social Work. 2023. https://doi.org/10.1093/bjsw/bcad167 .

Morrison A, Polisena J, Husereau D, Moulton K, Clark M, Fiander M, et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int J Technol Assess Health Care. 2012;28(2):138–44. https://doi.org/10.1017/s0266462312000086 .

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The Prisma 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3). https://doi.org/10.1371/journal.pmed.1003583 .

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010 Sept 20;5(1). https://doi.org/10.1186/1748-5908-5-69

Lockwood C, Munn Z, Porritt K. Qualitative research synthesis. Int J Evid Based Healthc. 2015 Sept;13(3):179–87. https://doi.org/10.1097/xeb.0000000000000062 .

Bryman A. Integrating quantitative and qualitative research: how is it done? Qualitative Res. 2006;6(1):97–113. https://doi.org/10.1177/1468794106058877 .

Carroll C, Booth A. Quality Assessment of qualitative evidence for systematic review and synthesis: is it meaningful, and if so, how should it be performed? Res Synthesis Methods. 2014;6(2):149–54. https://doi.org/10.1002/jrsm.1128 .

Braun V, Clarke V. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Res Psychol. 2020;18(3):328–52. https://doi.org/10.1080/14780887.2020.1769238 .

Kramer TL, Pyne JM, Kimbrell TA, Savary PE, Smith JL, Jegley SM. Clinician perceptions of virtual reality to assess and treat returning veterans. Psychiatric Serv. 2010;61(11):1153–6. https://doi.org/10.1176/appi.ps.61.11.1153 .

Chung OS, Robinson T, Johnson AM, Dowling NL, Ng CH, Yücel M, et al. Implementation of therapeutic virtual reality into psychiatric care: clinicians’ and service managers’ perspectives. Front Psychiatry. 2022;12. https://doi.org/10.3389/fpsyt.2021.791123 .

Chung OS, Dowling NL, Brown C, Robinson T, Johnson AM, Ng CH, et al. Using the theoretical domains framework to inform the implementation of therapeutic virtual reality into Mental Healthcare. Adm Policy Mental Health Mental Health Serv Res. 2022;50(2):237–68. https://doi.org/10.1007/s10488-022-01235-w .

Chung OS, Johnson AM, Dowling NL, Robinson T, Ng CH, Yücel M, et al. Are Australian mental health services ready for therapeutic virtual reality? An investigation of knowledge, attitudes, implementation barriers and enablers. Front Psychiatry. 2022;13. https://doi.org/10.3389/fpsyt.2022b.792663 .

Jones C, Smith-MacDonald L, Van Veelen N, VanderLaan A, Kaneva Z, Dunleavy RS et al. Therapist and operator experiences utilizing multi-modal motion-assisted memory desensitization and reconsolidation (3MDR) for treatment of combat related posttraumatic stress disorder amongst military and veteran populations. Eur J Psychotraumatology 2022b May 16;13(1). https://doi.org/10.1080/20008198.2022.2062996

Haeyen S, Jans N, Heijman J. The use of VR Tilt Brush in art and psychomotor therapy: an innovative perspective. Arts Psychother 2021b Nov;76:101855. https://doi.org/10.1016/j.aip.2021.101855

Ose SO, Færevik H, Kaasbøll J, Lindgren M, Thaulow K, Antonsen S, et al. Exploring the potential for use of virtual reality technology in the treatment of severe mental illness among adults in mid-norway: collaborative research between clinicians and researchers. JMIR Formative Res. 2019;3(2). https://doi.org/10.2196/13633 .

Haeyen S, Jans N, Glas M, Kolijn J. VR health experience: A virtual space for Arts and psychomotor therapy. Frontiers in Psychology. 2021a Sept 14;12. https://doi.org/10.3389/fpsyg.2021.704613

Riches S, Nicholson SL, Fialho C, Little J, Ahmed L, McIntosh H, et al. Integrating a virtual reality relaxation clinic within acute psychiatric services: a pilot study. Psychiatry Res. 2023;329:115477. https://doi.org/10.1016/j.psychres.2023.115477 .

Guillén V, Baños RM, Botella C. Users’ opinion about a virtual reality system as an adjunct to psychological treatment for stress-related disorders: a quantitative and qualitative mixed-methods study. Front Psychol. 2018;9. https://doi.org/10.3389/fpsyg.2018.01038 .

Mayer G, Gronewold N, Polte K, Hummel S, Barniske J, Korbel JJ, et al. Experiences of patients and therapists testing a virtual reality exposure app for symptoms of claustrophobia: mixed methods study. JMIR Mental Health. 2022;9(12). https://doi.org/10.2196/40056 .

Nijman SA, Veling W, Greaves-Lord K, Vos M, Zandee CE, Aan het Rot M, et al. Dynamic interactive social cognition training in virtual reality (DiSCoVR) for people with a psychotic disorder: single-group feasibility and acceptability study. JMIR Mental Health. 2020;7(8). https://doi.org/10.2196/17808 .

Raila H, Avanesyan T, Valentine KE, Koo B, Huang C, Tsutsumi Y, et al. Augmenting group hoarding disorder treatment with virtual reality discarding: a pilot study in older adults. J Psychiatr Res. 2023;166:25–31. https://doi.org/10.1016/j.jpsychires.2023.08.002 .

Jones C, Miguel Cruz A, Smith-MacDonald L, Brown MR, Vermetten E, Brémault-Phillips S. Technology acceptance and usability of a virtual reality intervention for military members and veterans with posttraumatic stress disorder: Mixed methods unified theory of acceptance and use of Technology study. JMIR Formative Research. 2022c Apr 21;6(4). https://doi.org/10.2196/33681

Sambrooks K, Lockerbie L, Majid S, Gannon T. Clinicians’ perceptions of virtual reality for firesetting. J Forensic Pract. 2022;24(4):404–19. https://doi.org/10.1108/jfp-05-2022-0027 .

Skeva R, Gregg L, Jay C, Pettifer S. Views of practitioners and researchers on the use of virtual reality in treatments for substance use disorders. Front Psychol. 2021;12. https://doi.org/10.3389/fpsyg.2021.606761 .

Jones C, Smith-MacDonald L, Brown MR, VanDehy J, Grunnet-Jepsen R, Ordek VP, et al. The redesign and validation of multimodal motion-assisted memory desensitization and reconsolidation hardware and software: mixed methods, modified Delphi–based Validation Study. JMIR Hum Factors. 2022;9(3). https://doi.org/10.2196/33682 .

De Witte NA, Joris S, Van Assche E, Van Daele T. Technological and digital interventions for Mental Health and Wellbeing: an overview of systematic reviews. Front Digit Health. 2021;3. https://doi.org/10.3389/fdgth.2021.754337 .

Difede J, Hoffman HG. Virtual reality exposure therapy for world trade center post-traumatic stress disorder: a case report. CyberPsychology Behav. 2002;5(6):529–35. https://doi.org/10.1089/109493102321018169 .

Tworus R, Szymanska S, Ilnicki S. A soldier suffering from PTSD, treated by controlled stress exposition using virtual reality and behavioral training. Cyberpsychology Behav Social Netw. 2010;13(1):103–7. https://doi.org/10.1089/cyber.2009.0329 .

Lindner P, Miloff A, Zetterlund E, Reuterskiöld L, Andersson G, Carlbring P. Attitudes toward and familiarity with virtual reality therapy among practicing cognitive behavior therapists: a cross-sectional survey study in the era of consumer VR platforms. Front Psychol. 2019;10:do1.

Maples-Keller JL, Bunnell BE, Kim S-J, Rothbaum BO. The use of virtual reality technology in the treatment of anxiety and other psychiatric disorders. Harv Rev Psychiatry. 2017;25(3):103–13. https://doi.org/10.1097/hrp.0000000000000138 .

Freeman D, Reeve S, Robinson A, Ehlers A, Clark D, Spanlang B, et al. Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol Med. 2017;47(14):2393–400. https://doi.org/10.1017/s003329171700040x .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Kouijzer MM, Kip H, Bouman YH, Kelders SM. Implementation of virtual reality in Healthcare: a scoping review on the implementation process of virtual reality in various healthcare settings. Implement Sci Commun. 2023;4(1). https://doi.org/10.1186/s43058-023-00442-2 .

Anderson PL, Molloy A. Maximizing the impact of virtual reality exposure therapy for anxiety disorders. Curr Opin Psychol. 2020;36:153–7. https://doi.org/10.1016/j.copsyc.2020.10.001 .

Carlson CG. Virtual and augmented simulations in Mental Health. Curr Psychiatry Rep. 2023;25(9):365–71. https://doi.org/10.1007/s11920-023-01438-4 .

Kellmeyer P, Biller-Andorno N, Meynen G. Ethical tensions of virtual reality treatment in vulnerable patients. Nat Med. 2019;25(8):1185–8. https://doi.org/10.1038/s41591-019-0543-y .

Article   CAS   PubMed   Google Scholar  

Carl E, Stein AT, Levihn-Coon A, Pogue JR, Rothbaum B, Emmelkamp P, et al. Virtual reality exposure therapy for anxiety and related disorders: a meta-analysis of randomized controlled trials. J Anxiety Disord. 2019;61:27–36. https://doi.org/10.1016/j.janxdis.2018.08.003 .

Freeman D, Lister R, Waite F, Galal U, Yu L-M, Lambe S, et al. Automated virtual reality cognitive therapy versus virtual reality mental relaxation therapy for the treatment of persistent persecutory delusions in patients with psychosis (thrive): a parallel-group, single-blind, randomised controlled trial in England with mediation analyses. Lancet Psychiatry. 2023;10(11):836–47. https://doi.org/10.1016/s2215-0366(23)00257-2 .

Selaskowski B, Wiebe A, Kannen K, Asché L, Pakos J, Philipsen A, et al. Clinical adoption of virtual reality in mental health is challenged by lack of high-quality research. Npj Mental Health Res. 2024;3(1). https://doi.org/10.1038/s44184-024-00069-8 .

McLay RN, Baird A, Webb-Murphy J, Deal W, Tran L, Anson H, et al. A randomized, head-to-head study of virtual reality exposure therapy for posttraumatic stress disorder. Cyberpsychology Behav Social Netw. 2017;20(4):218–24. https://doi.org/10.1089/cyber.2016.0554 .

Miyahira SD, Folen RA, Hoffman HG, García-Palacios A, Spira JL, Kawasaki MM. The Effectiveness of VR Exposure Therapy for PTSD in Returning Warfighters. 2012. Studies in health technology and informatics, 181, 128 – 32.

Ready DJ, Gerardi RJ, Backscheider AG, Mascaro N, Rothbaum BO, Cyberpsychology. Behav Social Netw. 2010;13(1):49–54. https://doi.org/10.1089/cyber.2009.0239 .

Reger GM, Koenen-Woods P, Zetocha K, Smolenski DJ, Holloway KM, Rothbaum BO, et al. Randomized controlled trial of prolonged exposure using imaginal exposure vs. virtual reality exposure in active duty soldiers with deployment-related posttraumatic stress disorder (PTSD). J Consult Clin Psychol. 2016;84(11):946–59. https://doi.org/10.1037/ccp0000134 .

Appel L, Kisonas E, Appel E, Klein J, Bartlett D, Rosenberg J, et al. Administering virtual reality therapy to manage behavioral and psychological symptoms in patients with dementia admitted to an acute care hospital: results of a pilot study. JMIR Formative Res. 2021;5(2). https://doi.org/10.2196/22406 .

Didehbani N, Allen T, Kandalaft M, Krawczyk D, Chapman S. Virtual reality social cognition training for children with high functioning autism. Computers Hum Behav 2016 Sept;62:703–11. https://doi.org/10.1016/j.chb.2016.04.033

Kandalaft MR, Didehbani N, Krawczyk DC, Allen TT, Chapman SB. Virtual reality social cognition training for young adults with high-functioning autism. J Autism Dev Disord. 2012;43(1):34–44. https://doi.org/10.1007/s10803-012-1544-6 .

Article   PubMed Central   Google Scholar  

Daley D, van der Oord S, Ferrin M, Danckaerts M, Doepfner M, Cortese S, et al. Behavioral interventions in attention-deficit/hyperactivity disorder: a meta-analysis of randomized controlled trials across multiple outcome domains. J Am Acad Child Adolesc Psychiatry. 2014;53(8). https://doi.org/10.1016/j.jaac.2014.05.013 .

Finnegan DJ, Campbell S. Tackling loneliness and isolation in older adults with virtual reality: how do we move forward? Gerontol Geriatric Med. 2023;9. https://doi.org/10.1177/23337214231186204 .

Hu Au E, Lee JJ. Virtual reality in education: a tool for learning in the experience age. Int J Innov Educ. 2017;4(4):215. https://doi.org/10.1504/ijiie.2017.10012691 .

Randall N, Šabanović S, Chang W. Engaging older adults with depression as co-designers of assistive in-home robots. Proc 12th EAI Int Conf Pervasive Comput Technol Healthc. 2018. https://doi.org/10.1145/3240925.3240946 .

Bouchard S, Dumoulin S, Robillard G, Guitard T, Klinger É, Forget H, et al. Virtual reality compared within vivo exposure in the treatment of social anxiety disorder: a three-arm randomised controlled trial. Br J Psychiatry. 2017;210(4):276–83. https://doi.org/10.1192/bjp.bp.116.184234 .

Ling Y, Nefs HT, Morina N, Heynderickx I, Brinkman W-P. A meta-analysis on the relationship between self-reported presence and anxiety in virtual reality exposure therapy for anxiety disorders. PLoS ONE. 2014;9(5). https://doi.org/10.1371/journal.pone.0096144 .

Wiederhold BK, Bouchard S. Fear of heights (acrophobia): efficacy and lessons learned from Psychophysiological Data. Adv Virtual Real Anxiety Disorders. 2014;119–44. https://doi.org/10.1007/978-1-4899-8023-6_6 .

Freedman SA, Hoffman HG, Garcia-Palacios A, (Tamar), Weiss PL, Avitzour S, Josman N. Prolonged exposure and virtual reality–enhanced imaginal exposure for PTSD following a terrorist Bulldozer attack: A case study. Cyberpsychology, Behavior, and Social Networking. 2010;13(1):95–101. https://doi.org/10.1089/cyber.2009.0271

Benbow AA, Anderson PL. A meta-analytic examination of attrition in virtual reality exposure therapy for anxiety disorders. J Anxiety Disord. 2019;61:18–26. https://doi.org/10.1016/j.janxdis.2018.06.006 .

Rizzo A, ‘Skip’, Shilling R. Clinical virtual reality tools to advance the prevention, assessment, and treatment of PTSD. Eur J Psychotraumatology. 2017;8(sup5). https://doi.org/10.1080/20008198.2017.1414560 .

Freeman D, Bradley J, Antley A, Bourke E, DeWeever N, Evans N, et al. Virtual reality in the treatment of persecutory delusions: randomised controlled experimental study testing how to reduce delusional conviction. Br J Psychiatry. 2016;209(1):62–7. https://doi.org/10.1192/bjp.bp.115.176438 .

Krijn M, Emmelkamp PM, Olafsson RP, Biemond R. Virtual reality exposure therapy of anxiety disorders: a review. Clin Psychol Rev. 2004;24(3):259–81. https://doi.org/10.1016/j.cpr.2004.04.001 .

Bouchard S, Rizzo A. Skip. Applications of virtual reality in clinical psychology and clinical cognitive neuroscience–an introduction. Virtual Real Psychol Neurocognitive Interventions. 2019;1–13. https://doi.org/10.1007/978-1-4939-9482-3_1 .

Rothbaum BO, Rizzo A, Skip, Difede J. Virtual reality exposure therapy for combat-related posttraumatic stress disorder. Ann N Y Acad Sci. 2010;1208(1):126–32. https://doi.org/10.1111/j.1749-6632.2010.05691.x .

Maples-Keller JL, Yasinski C, Manjin N, Rothbaum BO. Virtual reality-enhanced extinction of phobias and post-traumatic stress. Neurotherapeutics. 2017b Jul;14(3):554–63. https://doi.org/10.1007/s13311-017-0534-y .

Schwartzman D, Segal R, Drapeau M. Perceptions of virtual reality among therapists who do not apply this technology in clinical practice. Psychol Serv. 2012;9(3):310–5. https://doi.org/10.1037/a0026801 .

Vincent C, Eberts M, Naik T, Gulick V, O’Hayer CV. Provider experiences of virtual reality in clinical treatment. PLoS ONE. 2021;16(10). https://doi.org/10.1371/journal.pone.0259364 .

Ma L, Mor S, Anderson PL, Baños RM, Botella C, Bouchard S, et al. Integrating virtual realities and psychotherapy: SWOT analysis on VR and MR based treatments of anxiety and stress-related disorders. Cogn Behav Ther. 2021;50(6):509–26. https://doi.org/10.1080/16506073.2021.1939410 .

Langlet BS, Odegi D, Zandian M, Nolstam J, Södersten P, Bergh C. Virtual reality app for treating eating behavior in eating disorders: development and usability study. JMIR Serious Games. 2021;9(2). https://doi.org/10.2196/24998 .

Best P, Kupeli-Holt S, D’Arcy J, Elliot A, Duffy M, Van Daele T. Low-cost virtual reality to support imaginal exposure within PTSD treatment: a case report study within a community mental healthcare setting. Cogn Behav Pract. 2023. https://doi.org/10.1016/j.cbpra.2023.03.003 .

Brown P, Waite F, Lambe S, Rosebrock L, Freeman D. Virtual reality cognitive therapy in inpatient psychiatric wards: protocol for a qualitative investigation of staff and patient views across multiple National Health Service sites. JMIR Res Protocols. 2020;9(8). https://doi.org/10.2196/20300 .

Topol E. The Topol Review. Preparing the healthcare workforce to deliver the digital future. An independent report on behalf of the Secretary of State for Health and Social Care. 2019. https://topol.hee.nhs.uk/wp-content/uploads/HEE-Topol-Review-2019.pdf . Accessed 09 August 2024.

Lukka L, Karhulahti V-M, Palva JM. Factors affecting digital tool use in client interaction according to Mental Health professionals: interview study. JMIR Hum Factors. 2023;10. https://doi.org/10.2196/44681 .

Cullen AJ, Dowling NL, Segrave R, Carter A, Yücel M. Exposure therapy in a virtual environment: validation in obsessive compulsive disorder. J Anxiety Disord. 2021;80:102404. https://doi.org/10.1016/j.janxdis.2021.102404 .

Kip H, Kelders SM, Weerink K, Kuiper A, Brüninghoff I, Bouman YH, et al. Identifying the added value of virtual reality for treatment in forensic Mental Health: a scenario-based, qualitative approach. Front Psychol. 2019;10. https://doi.org/10.3389/fpsyg.2019.00406 .

Download references

Acknowledgements

Not applicable.

Author information

Authors and affiliations.

School of Social Sciences, Education and Social Work, Queen’s University, Belfast, Northern, Northern Ireland

Jessica Cushnan, Paul McCafferty & Paul Best

You can also search for this author in PubMed   Google Scholar

Contributions

J.C and P.MCC refined the search strategy and defined the study protocol. J.C wrote the main manuscript and prepared Fig. 1; Tables 1 and 2. P.B acted as independent author to ensure inter-rater reliability. All authors reviewed the manuscript.

Corresponding author

Correspondence to Jessica Cushnan .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Authors’ information (optional)

Additional information, publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Cushnan, J., McCafferty, P. & Best, P. Clinicians’ perspectives of immersive tools in clinical mental health settings: a systematic scoping review. BMC Health Serv Res 24 , 1091 (2024). https://doi.org/10.1186/s12913-024-11481-3

Download citation

Received : 30 April 2024

Accepted : 22 August 2024

Published : 18 September 2024

DOI : https://doi.org/10.1186/s12913-024-11481-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Virtual reality
• Mental health
• Clinicians’
• Scoping review
• Qualitative

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 18 September 2024

Recommended procedures for managing carious lesions in primary teeth with pulp involvement—a scoping review

• Ilze Maldupa 1 ,
• Waraf Al-Yaseen   ORCID: orcid.org/0000-0002-7071-5074 2 ,
• Julius Giese 1 ,
• Rokaia Ahmed Elagami 3 &
• Daniela Prócida Raggio   ORCID: orcid.org/0000-0002-0048-2068 2 , 3  

BDJ Open volume  10 , Article number:  74 ( 2024 ) Cite this article

Metrics details

• Paediatric dentistry

Managing dental caries in primary teeth with pulp involvement is a significant challenge. Clinical guidelines offer recommendations for effective management.

To identify and analyze policies, guidelines, and recommendations for treating primary teeth with pulp-involved carious lesions, highlighting existing research gaps and setting the foundation for future research.

A comprehensive search was conducted across databases (PubMed, Scopus, Embase, GIN, and LILACS) and grey literature sources (Trip and ProQuest) to identify guidelines, consensus, policy, and position statements on primary teeth pulp therapy and extraction thresholds. Two independent reviewers screened the abstracts and titles, followed by full-text screening.

After removing duplication, of the 1098 records, 14 were selected for analysis. This review examined various treatments for deep caries lesions in primary teeth, including indirect/direct pulp capping, pulpotomy, pulpectomy, lesion sterilization/tissue restoration, and extraction. Time search was restricted to documents published from 30th January 2008 to 30th January 2024, offering insights into evolving clinical practices.

Treatment for carious lesions in primary teeth involving the pulp depends on clinical indications and may involve minimally invasive techniques. Recommended options are indirect pulp capping, pulpotomy, and pulpectomy, while direct capping and tooth removal are discouraged. Further research is needed to address gaps, improve guideline development, and enhance consistency of recommendations.

Similar content being viewed by others

Are different pulp treatment techniques and associated medicaments effective for the treatment of extensive decay in primary teeth?

What is the best caries removal strategy for primary molars?

Is partial pulpotomy in cariously exposed posterior permanent teeth a viable treatment option?

Introduction.

Dental caries remains a significant oral health issue affecting approximately half of the global child population, with little change in prevalence over the past few decades (Uribe, Innes, & Maldupa, 2021). Despite efforts to address the problem, complications arising from carries, including pulp damage, continue to impact a considerable number of children (Lin et al., 2019). Pulp damage resulting from caries lesions accounts for nearly 90% of cases and requires effective management strategies [ 1 ].

Primary teeth present unique challenges in the treatment of pulp pathology due to their distinct anatomical and physiological characteristics, as well as the psycho-emotional development of young children [ 2 ]. While the preservation of pulp for as long as possible is advocated as the primary approach, minimally intervention dental (MID) techniques have emerged as successful preventive measures for pulp pathologies [ 3 ]. However, there are clinical scenarios where MID cannot be applied, and immediate treatment becomes necessary for pulp-related complications.

Management of carious lesion with pulp involvement in primary teeth include pulpotomy, pulpectomy, or extraction. The choice of treatment and medications is influenced by various factors. However, pulp therapy treatments often require a child’s cooperation and may involve multiple sessions or general anaesthesia. These factors can heighten anxiety for the child and increase the time and resource burden for their family and dental staff [ 4 ]. Besides, the success rate of endodontic treatment for primary teeth remains a contentious issue [ 5 ]. Consequently, extractions are increasingly favoured as the preferred treatment for pulpal pathology [ 4 ]. Nevertheless, concerns arise regarding space loss, potential orthodontic needs, and the overall impact on the child’s quality of life following premature loss of primary teeth [ 4 ].

To guide healthcare providers in making well-informed decisions regarding the management of caries lesions involving the pulp in primary teeth, various authoritative sources have developed clinical practice guidelines (CPGs), consensus statements, policy documents, and position statements. These resources aim to provide evidence-based recommendations and standardised clinical practices. However, there is considerable variability in the recommendations due to differences in available evidence, contextual factors, and variations in healthcare systems across different countries.

Numerous systematic reviews have synthesized the efficacy of pulp therapy for primary teeth, including Cochrane reviews and other comprehensive analyses [ 6 , 7 ]. These reviews have significantly contributed to our understanding of the outcomes associated with different treatment approaches. Moreover, research has explored the factors influencing clinicians’ decision-making process when choosing between endodontic treatment and extraction for primary teeth with pulp involvement [ 8 ].

Despite these existing systematic reviews and research efforts, the optimal course of action for the preservation or extraction of primary teeth with pulp involvement remains a subject of ongoing debate. CPGs and recommendations have been developed to offer clear and evidence-based guidance. However, the variability in recommendations, based on the best available evidence and contextual factors, underscores the need for further exploration and examination of the existing documents related to the management of caries lesions reaching the pulp in primary teeth.

Hence, this scoping review aims to identify and describe documents (current CPGs, consensus, policies, clinical recommendations and position statements) for managing caries lesions that reached the pulp in primary teeth.

Protocol registration

The research protocol for this scoping review was registered on the Open Science Framework (OSF) platform to ensure transparency and adherence to the planned methodology (OSF registration: ( https://doi.org/10.17605/OSF.IO/APCKG )). The review followed the established methodology for scoping reviews outlined by the Joanna Briggs Institute (JBI) and the report followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Guidelines for Scoping Reviews [ 9 ].

Identification of relevant studies

A comprehensive search strategy was developed in collaboration with experienced researchers and information specialists to identify relevant studies and sources of evidence. Electronic databases, including MEDLINE, Scopus, Embase, GIN, and LILACS, were systematically searched up to 30 th January 2024. Grey literature sources, such as Trip and ProQuest, were also consulted to capture unpublished documents, reports, and guidelines. The search terms and keywords were carefully selected to cover the key concepts of paediatric dentistry, dental caries, clinical practice guidelines, extraction, and pulp therapy (See Appendix  1 ).

Inclusion criteria

This study included recent clinical practice guidelines (CPGs), consensus statements, policy documents, and position statements addressing the threshold for pulp therapy and extraction in primary teeth with pulp involvement. Eligible documents had to be officially endorsed or produced by reputable organisations like government agencies, professional associations, or expert panels, using a systematic consensus approach. In the case of multiple versions, only the latest one was considered. Time search was restricted to the last 20 years (30th January 2004) as older guidelines are likely to be updated or irrelevant. No language restrictions were applied in the search strategy.

Exclusion criteria

Primary and secondary research articles, expert opinions, editorials, and letters to editors, as well as studies involving adult participants or special needs children.

Study selection process

A two-step screening process was conducted by two independent reviewers. In the initial step, titles and abstracts of the identified records were screened to assess their eligibility based on the predefined inclusion and exclusion criteria. Full-text articles were obtained for potentially relevant sources, and two reviewers independently evaluated them for final inclusion in the review. Discrepancies or disagreements between the reviewers were resolved through discussion or consultation with a third reviewer to ensure consensus.

Data extraction

Data extraction was performed using an ad hoc standardised form developed specifically for this scoping review. Two reviewers independently extracted relevant information from the included studies and documents, including publication year, authorship, study design, participant characteristics, methodology, key recommendations, and additional findings deemed relevant. Consistency and accuracy of the extracted data were ensured through cross-checking, and any disagreements were resolved through discussion or involvement of a third reviewer.

Data analysis and presentation

Data analysis aimed to capture the range of recommendations and approaches related to the threshold for pulp therapy and extraction in primary teeth with pulp involvement. A narrative synthesis approach was employed to analyse and present the extracted data. Key themes, concepts, and recommendations from the included studies and documents were systematically organised and summarised. The extracted data were presented using table and graphs to enhance clarity and facilitate understanding. Patterns and variations among the recommendations were identified to provide a comprehensive overview of the existing literature.

Included studies and documents

The initial search process yielded a total of 1098 records from multiple databases and organisations [American Academy of Paediatric Dentistry (AAPD), European Academy of Paediatric Dentistry (EAPD), Scottish Dental Clinical Effectiveness Programme (SDCEP), Qatari, Saudi, Health Ministry of Chile, Brazilian and Iraqi Dental Association] (See Appendix  3 ). After removing duplicates, the titles and abstracts of 417 unique records were screened for eligibility. From this initial screening, 30 papers were selected for a more detailed evaluation. Finally, 14 papers met the inclusion criteria and were included in the in-depth analysis. Figure  1 provides a visual representation of the study selection process following PRISMA-ScR guidelines (See Appendix  2 ) [ 10 ].

The process consists of four stages: Identification, Screening, Eligibility, and Inclusion. Identification: a total of 1098 records were identified from various databases, including PubMed/Medline ( n  = 120), Embase ( n  = 135), Scopus ( n  = 176), Lilacs ( n  = 84), Trip ( n  = 170), ProQuest ( n  = 26), and other sources ( n  = 387). After removing 681 duplicate records, 417 records were screened. Screening: out of the 417 records screened, 387 were excluded based on title and abstract screening, and 30 reports were sought for retrieval. Eligibility: among the 30 reports, 8 were not retrieved. The remaining 22 reports were assessed for eligibility, with 8 reports being excluded due to updated versions being available. Inclusion: ultimately, 14 studies were included in the final review.

Characteristics of included studies and documents

The included studies were primarily sourced from reputable databases, with the highest number of records obtained from the American Academy of Paediatric Dentistry (AAPD) (345), Scopus (176), Trip (170), Embase (135), and PubMed/MEDLINE (120). Additional sources, such as Lilacs (84), ProQuest (26), Scottish Dental Clinical Effectiveness Programme (SDCEP) (16), European Academy of Paediatric Dentistry (EAPD) (14), and GIN (6), also contributed to the final selection of papers. The selected papers focused on the treatment of primary teeth with deep caries lesions and provided insights into extraction, pulpectomy, pulpotomy, and variations within these treatment options.

Treatment modalities

Most CPGs discussed both Direct Pulp Capping (DPC) ( n  = 6) and Indirect Pulp Capping (IPC) ( n  = 8), pulpotomy ( n  = 10) and pulpectomy ( n  = 9) as potential treatment options for specific cases involving deep caries lesions in primary teeth. IPC and DPC techniques were outlined concerning their indications, recommended protocols, and supporting evidence. Additionally, some papers explored the concept of lesion sterilisation/tissue repair (LSTR), albeit being mentioned in only two documents. The use of LSTR for managing carious lesions reaching the pulp in primary teeth was discussed in terms of its effectiveness, limitations, and potential application.

Variations and gaps in recommendations

While the included studies and documents shared commonalities in their recommendations, some variations were observed. These variations were often influenced by contextual factors, such as the healthcare system, resources, and available treatment options in different countries or regions. Additionally, certain aspects of the management of caries lesions in primary teeth with pulp involvement lacked clear consensus or had limited evidence, indicating gaps in the existing literature.

Summary of key findings

The reviewed literature explored various treatment options for primary teeth with deep caries lesions, including extraction, pulpectomy, pulpotomy, and their subdivisions.

Indirect pulp capping

Indirect Pulp Capping was recommended as a successful treatment for vital deciduous teeth affected by deep caries, as it is a MID approach that does not interfere with the natural exfoliation process [ 11 , 12 ]. This treatment is indicated when there is no pulp involvement [ 13 ] and is considered a standard treatment option [ 14 ]. To ensure successful IPC, it is crucial to achieve an excellent seal of the coronal part of the tooth [ 12 ]. The procedure involves selective removal of soft caries tissue, particularly from the dentin-enamel junction, using hand instruments [ 11 , 15 , 16 ]. Subsequently, materials such as zinc oxide eugenol (ZOE), hard-setting calcium hydroxide (Ca(OH)₂), or resin-modified glass ionomer cement (RMGIC) are placed and covered with a preformed crown or adhesive restoration. These procedures have received a grade B recommendation, and level III evidence, and have shown success rates of over 90% after three years [ 16 ]. Alternative approaches, including the use of slow rotary instruments and other biocompatible materials like mineral trioxide aggregate (MTA), were also mentioned (See Fig.  2 ). Compared to pulpotomy treatments, IPC has demonstrated higher long-term success rates [ 11 , 15 ]. Meta-analyses did not find significant differences between bonding agent liners and Ca(OH)₂, with moderate and low evidence after 24–48 months [ 17 ]. In contrast, recommendations from 2005 suggested complete removal of caries and direct pulp capping (DPC) or pulpotomy in cases of iatrogenic pulp exposure due to higher symptom occurrence and uncertain outcomes. Before filling the cavity, Ca(OH)₂ is placed to promote secondary dentin formation (See Fig.  2 ) [ 18 ].

This timeline summarises the evolution of IPC guidelines across various countries and regions from 2005 to 2022. It outlines the indications, recommendations, and levels of evidence in countries such as the UK, Chile, USA, New Zealand, and others. Specific recommendations range from total caries removal to selective caries removal with materials like calcium hydroxide and glass ionomer cement. Evidence levels vary from low to high across different years and regions.

Direct pulp capping (DPC)

The use of DPC as a treatment option is restricted to spot-like pulpal exposure areas due to trauma or mechanical opening during caries removal in cases of non-symptomatic and non-infectious circumstances (See Fig.  3 ), to facilitate dentine structure development [ 16 , 18 , 19 ].

The figure displays DPC guidelines across regions from 2005 to 2022, including the UK, USA, and international recommendations. Indications include traumatic pulp exposures and materials recommended include calcium hydroxide and MTA. Evidence levels range from low to not stated, reflecting the evolving clinical recommendations over the years.

For the purpose of teeth protection from microleakage, Ca(OH)₂ and mineral trioxide aggregate (MTA) have been suggested [ 11 ]. Meta-analyses show no significant difference in success between Ca(OH)₂ and MTA, formocresol (FC) and dentin bonding agents after 24 months. Due to the missing discrepancy of included studies, the quality of evidence was rated as very low [ 17 ]. Prior haemorrhage control by a piece of cotton damped with saline or water has been recommended with grade C and evidence quality level IV [ 16 ].

In general, DPC is not recommended as a regular treatment option for primary teeth [ 15 ]. This might be connected to the elevated cellular density in the pulp tissue of deciduous teeth and poor prognosis [ 20 , 21 ] However, close to the physiological exfoliation time, DPC can be indicated due to less severe consequences (See Fig.  3 ) [ 16 ].

Pulpotomy is a recommended treatment option for primary teeth with profound carious lesions, boasting a 24-month success rate of 82.6% [ 22 ]. However, due to limited direct comparisons, no definitive evidence-supported recommendation can be made regarding the choice between pulpotomy, DPC, and IPC (See Fig.  4 ) [ 14 , 22 ].

This figure highlights pulpotomy guidelines in the UK, Chile, Italy, and other regions from 2005 to 2022. Indications include symptoms of irreversible pulpitis, and materials recommended include formocresol (FC), MTA, and ferric sulfate. Evidence is rated from low to high depending on the year and location. The timeline also covers regional variations in managing pulpotomy procedures.

Pulpotomy is generally indicated for primary teeth with exposed vital pulp or irreversible pulpitis of the coronal pulp, if the underlying tissue is healthy or shows reversible inflammation [ 16 , 17 ]. It can be performed on deciduous teeth at any developmental stage [ 13 ]. Contraindications include severe root resorption, facial cellulitis, abscess history, or specific patient conditions necessitating general anaesthesia [ 23 ]. Pulpotomy for vital pulp in primary molars is a recommended treatment, while non-vital pulpotomy, which differs in procedure and indication, is considered obsolete in most current guidelines.

Some guidelines discourage the use of Ca(OH)₂ during pulpotomy due to compromised results and lower success rates compared to ferric sulphate (FS), mineral trioxide aggregate (MTA), and formocresol [ 12 , 16 , 17 , 19 ].

MTA (87.8%) and formocresol (85%) have shown the highest success rates among recommended treatment choices, leading to a strong recommendation for their use [ 17 ]. Other options are conditionally recommended, and the use of formocresol may raise concerns among parents [ 15 , 16 , 17 ].

MTA, despite higher initial costs, proves to be more cost-effective in the long run due to its greater success rates and reduced need for secondary treatments compared to Ca(OH)₂ [ 17 ]. MTA preserves pulp integrity, reduces inflammation, and promotes tissue formation, while Portland cement is considered a low-cost alternative [ 24 ].

Additional research is needed to determine specific recommendations for lining materials, caution is advised regarding the combination of FS and eugenol, and control of haemorrhage is essential during treatment [ 16 , 18 , 24 ].

Stainless steel crowns are recommended as a permanent restoration after pulpotomy, while composite resin and amalgam can be used for deciduous teeth with minor structural damage [ 11 ].

Pulpectomy is a recommended treatment for restorable primary teeth with necrosis, irreversible pulpitis, root resorption, and other pathologies [ 11 ]. It is preferred over LSTR in the absence of root resorption. Pulpectomy is generally not recommended as a first-line treatment for deep caries in vital primary molars due to the effectiveness of more conservative alternatives like indirect pulp capping or pulpotomy. However, it may be used instead of extraction when tooth loss could harm dental health and long-term occlusion, or if there is no permanent successor [ 12 ].

Prior to treatment, a periapical radiograph is taken for diagnosis, and anaesthesia is administered [ 4 ]. Root canal shaping can be done with rotary or hand files, followed by irrigation using sodium hypochlorite or alternative solutions [ 11 , 15 ]. Canals are dried before using zinc oxide eugenol (ZOE) cement or calcium hydroxide (Ca(OH)₂) with iodoform paste for obturation [ 11 , 15 ].

Different approaches exist for pulpectomy depending on the condition, such as two-stage or one-stage procedures [ 18 ]. The Italian Ministry of Health recommends pulpectomy for non-vital primary teeth in specific developmental stages and with clinical signs like abscesses, fistula, and pain [ 2 ]. The use of Ca(OH)₂ combined with iodoform paste is advantageous, although ZOE is also suggested [ 2 ]. Irrigation should be performed using hypochlorite, saline, or chlorhexidine [ 16 ].

The Federal University of Rio de Janeiro (UFRJ) recommends specific irrigation techniques and materials for obturation, such as ZOE, glass ionomer cement (GIC), or heated gutta-percha. The heated gutta-percha is used specifically to seal the canal orifice, not to fill the canals [ 25 ]. Preformed crowns are suggested for excellent coronal seal [ 16 ]. Pulpectomy success rates range from 59% to 69% for teeth with root resorption and 84% to 90% for those without [ 25 ]. Extraction may be necessary if fistula or abscess persists after Ca(OH)₂ [ 25 ]. The Dubai Health Authority limits pulpectomy to primary teeth with less than one-third root resorption and without facial cellulitis or abscess [ 23 ]. Considerations for pulpectomy include long-term retention of second deciduous molars and stable occlusion, with conservative treatments preferred for profound carious lesions (See Fig.  5 ).

The pulpectomy timeline from 2005 to 2022 illustrates recommendations from the UK, Chile, USA, and other countries. It covers the management of irreversible pulpitis and related pathology, with recommendations including the use of zinc oxideeugenol, MTA, and root canal instrumentation. Evidence levels range from low to moderate across different regions.

Lesion sterilisation/tissue repair (LSTR)

LSTR is a possible treatment option for primary teeth experiencing clinical symptoms of irreversible pulpitis, fistula formation, and other pathologies (see Fig.  6 ) [ 11 , 15 ]. It is considered preferable over pulpectomy in cases of root resorption and teeth expected to exfoliate within one year. The treatment involves establishing access to the pulp chamber and augmenting the orifices. Phosphoric acid is used to cleanse the chamber, followed by rinsing and drying. Subsequently, a paste containing ciprofloxacin, metronidazole, and clindamycin, along with macrogol and polyethylene, is placed in the affected areas. It is important to avoid the incorporation of tetracycline into the antibiotic mix. Finally, glass ionomer cement (GIC) and a stainless-steel crown are placed [ 11 , 15 ].

This figure outlines LSTR treatment guidelines between 2005 and 2022 in regions like the USA and international contexts. The timeline reflects recommendations for disinfecting root canals using antibiotics like ciprofloxacin and metronidazole for cases of irreversible pulpitis and root resorption. Evidence is generally not stated.

Extractions

Extraction is indicated for primary teeth in the following situations: teeth approaching exfoliation, teeth that are non-restorable due to extensive caries or uncontrolled pulp haemorrhage [ 16 , 18 , 20 , 26 ]. In addition, pulpectomy with repeated medication application without symptom relief or continuous exudation is also a reason for extraction [ 25 ]. (See Fig.  7 ).

From 2005 to 2022, this figure tracks extraction guidelines in the UK, Chile, USA, and other regions. Indications include nonrestorable teeth with extensive decay or advanced root resorption. Recommendations include balanced extractions and use of chlorhexidine (CHX) irrigation. Evidence levels vary from low to not stated.

Balanced bilateral extractions may be considered for primary canines, and in cases where there is absence of the contralateral tooth, extraction may be indicated for the first deciduous molars, provided that the jaw space is not excessively crowded [ 16 , 18 ]. However, primary incisors are less frequently subjected to extraction [ 18 ]. It is important to consider the need for space maintainers when the development of permanent root formation does not exceed one-third of its completion [ 25 ].

In addition to clinical factors, such as tooth condition and stage of eruption, other factors including patient cooperation, social factors, and medical conditions should be considered when deciding on extraction [ 20 ]. Furthermore, the attitude of the patient and parents, as well as the number and complexity of required treatments, should be also considered [ 16 , 18 ]. It is generally recommended to avoid extractions during initial dental visits [ 13 , 20 ]. Whenever possible, extraction should be avoided in cases of crowding, absence of underlying permanent teeth, and situations that may cause increased stress for the patient [ 18 ].

The management of caries lesions that reach the pulp in primary teeth presents a complex challenge for dental professionals. CPGs play a crucial role in providing evidence-based recommendations for the treatment of such cases. This scoping review aimed to identify and describe documents, including CPGs, consensus statements, policies, and clinical recommendations, pertaining to the management of caries lesions that reached the pulp in primary teeth. Hence, this review provides valuable insights into the variations in thresholds and recommendations for different treatment procedures. Although our focus was on published documents, it is important to note that we could not verify whether these documents utilised the best available evidence to formulate their recommendations or if they had low risk of bias.

The analysis of the included documents revealed variations in thresholds and recommendations for different treatment procedures. These variations stem from differences in the interpretation of the available evidence, clinical judgement, and priorities of different dental organizations and professional societies.

The comparison of indications for each procedure among different CPGs provides valuable insights into the diverse perspectives and considerations when managing caries lesions that reach the pulp in primary teeth. These variations in recommendations reflect the complexities of clinical decision-making and the diverse approaches taken by different guidelines.

For instance, the AAPD guidelines recommend Indirect Pulp Capping as a standard treatment option for vital primary teeth with deep caries lesions but without pulp involvement [ 16 ]. IPC is widely practiced in the United States and has shown favourable outcomes. In contrast, the EAPD guidelines also support IPC but provide more specific indications, such as minimal pulp involvement and limited symptoms [ 12 ].

When considering direct pulp capping and pulpotomy, guidelines offer varying recommendations. The AAPD guidelines suggest DPC for spot-like pulpal exposures resulting from trauma or mechanical opening during caries removal [ 19 ]. On the other hand, the SDCEP guidelines discourage the routine use of DPC and instead recommend pulpotomy as a treatment option [ 13 ].

The indications for pulpotomy also show variations among guidelines. The IAPD guidelines recommend pulpotomy for primary teeth with exposed vital pulp or irreversible pulpitis of the coronal pulp [ 15 ]. However, specific indications provided by different guidelines may vary, taking into account factors such as clinical signs, developmental stages, and the prevalence of dental conditions.

Similarly, the indications for pulpectomy vary among guidelines. The AAPD guidelines recommend pulpectomy for restorable primary teeth with necrosis, irreversible pulpitis, root resorption, or other pathologies [ 11 ]. In contrast, guidelines from other sources may provide more specific indications based on their own research and clinical experience.

These variations in recommendations highlights the influence of context on treatment recommendations. Guidelines developed in different regions may reflect the specific needs and resources available in those areas. Furthermore, the availability of materials and resources can significantly impact the treatment options recommended in the guidelines. Different regions may have varying access to materials such as Ca(OH)₂, MTA, or formocresol. These variations in material availability can lead to differences in the recommended treatment modalities. Cost-effectiveness of materials play a key role in the treatment choices outlined in the guidelines. The review evidence highlights the departure from historical practices, with the more recent CPGs no longer endorsing complete caries removal [ 21 ]. Similarly, the reconsideration of calcium hydroxide (Ca(OH)₂) usage, previously employed for secondary dentin formation before cavity filling, reflects this evolving perspective. This shift aligns with our growing understanding of cariology, emphasising minimally invasive procedures to preserve teeth tissue whenever possible [ 27 ]. Despite these advancements, the adoption of these guideline points among clinicians remains limited [ 28 ]. Addressing this gap may necessitate broader dissemination of updated guidelines, targeted educational initiatives, and ongoing efforts to bridge the translation gap between evidence-based recommendations and clinical implementation.

Another important consideration is the publication and development process of the guidelines. While guidelines aim to provide evidence-based recommendations, the level of detail and transparency in their development can vary. Some guidelines may provide extensive information on the underlying evidence, the grading of recommendations, and the consensus process followed. An example of such comprehensive guidelines is the Guidelines for the Management of Dental Emergencies by the SDCEP, which provide clear explanations of the evidence base, and the consensus process used [ 13 ]. On the other hand, some guidelines may lack sufficient information on the level of evidence, or the specific studies considered during their development.The lack of transparency and detail in guideline development makes it challenging to understand the rationale behind certain recommendations and hampers the ability to compare and reconcile differences between guidelines. To enhance the transparency and quality of guidelines, future efforts should prioritize the adherence to established guideline development methodologies, such as those recommended by Guidelines International Network. This includes clearly outlining the process for evidence synthesis, the grading of recommendations, and the involvement of multidisciplinary experts. The reporting quality of clinical practice guidelines exhibits significant variability. A previous study assessing the reporting quality of CPGs in paediatric dentistry has indicated suboptimal adherence to quality standards, underscoring the need for future improvement in their applicability. Implementation of standardised checklists such as the Appraisal of Guidelines for Research and Evaluation (AGREE) is imperative during CPG development processes to ensure methodological rigour and transparency in newly developed guidelines, before their adoption into clinical practice.

Among the myriad recommended treatments for various pulp conditions, a fundamental principle must remain unequivocal—treating patients based on the minimum intervention approach. In navigating the therapeutic choices, the essence of the Hippocratic principle, “primum non nocere” (first, do no harm). Thus, as practitioners, we are reminded to balance the intricacies of diverse treatment modalities with a commitment to the overarching goal of delivering care that is both effective and minimally invasive.

There are significant variations in guidelines for diagnosing and managing caries lesions that reach the pulp in primary teeth, reflecting differences in evidence interpretation, clinical judgement, and organisational priorities.

For caries with pulp involvement, guidelines differ: the AAPD advocates treating small pulpal exposures due to trauma or mechanical opening with direct pulp capping, while the SDCEP recommends more conservative approaches. The IAPD focuses on specific clinical signs and developmental stages to guide treatment, such as the extent of pulp involvement and the child’s age.

Regional needs and resources influence treatment recommendations, including the availability of materials like calcium hydroxide, mineral trioxide aggregate (MTA), or formocresol, and cost-effectiveness considerations. Recent guidelines favour minimally invasive procedures and reconsider the use of traditional materials like calcium hydroxide.

Transparency and methodological rigour in guideline development vary. Comprehensive guidelines, such as those from the SDCEP, provide clear evidence bases and consensus processes. Future guidelines should adhere to established methodologies to ensure transparency and methodological rigour.

Why this paper is important to paediatric dentists:

There is significant variation in guidelines for managing caries in primary teeth, influenced by regional contexts, material availability, and opaque guideline development processes. Understanding this variation is crucial for paediatric dentists to interpret and apply guidelines appropriately.

Adhering to clear guideline development methodologies and transparency about the rationale and evidence behind recommendations is needed to promote consistency and confidence in guidelines. This allows paediatric dentists to make properly evidence-based decisions.

Enhancing the quality and usability of guidelines on managing dental caries will facilitate decision-making for paediatric dentists seeking to provide optimal patient care. Reducing ambiguity supports the provision of appropriate care tailored to specific contexts.

Data availability

The data that support the findings of this study are available in the Appendix  3 .

Popoola BO, Ayebameru OE, Olanloye OM. Endodontic treatment in children: a five-year retrospective study of cases seen at the University College Hospital, Ibadan, Nigeria. Ann Ib Postgrad Med. 2018;16:136–41.

PubMed   PubMed Central   CAS   Google Scholar  

Cleghorn BM, Boorberg NB, Christie WH. Primary human teeth and their root canal systems. Endod Top. 2010;23:6–33.

Article   Google Scholar  

Banerjee A, Splieth C, Breschi L, Fontana M, Paris S, Burrow M, et al. When to intervene in the caries process? A Delphi consensus statement. Br Dent J. 2020;229:474–82.

Article   PubMed   Google Scholar  

Abanto J, Tsakos G, Olegário IC, Paiva SM, Mendes FM, Ardenghi TM, et al. Impact of pulpectomy versus tooth extraction in children’s oral health-related quality of life: a randomized clinical trial. Community Dent Oral Epidemiol. 2023;52:13–23.

Xie Y, Wang Y, Ma Q, Li J, Chen Y, Yang R, et al. Survival analysis of pulpectomy in primary molars performed under dental general anaesthesia: a two-year retrospective study. BMC Oral Health. 2022;22:597.

Article   PubMed   PubMed Central   Google Scholar  

Smaïl-Faugeron V, Glenny AM, Courson F, Durieux P, Muller-Bolla M, Fron Chabouis H. Pulp treatment for extensive decay in primary teeth. Cochrane Database Syst Rev. 2018;5:Cd003220.

PubMed   Google Scholar  

Stratigaki E, Tong HJ, Seremidi K, Kloukos D, Duggal M, Gizani S. Contemporary management of deep caries in primary teeth: a systematic review and meta-analysis. Eur Arch Paediatr Dent. 2022;23:695–725.

Sebourn S, Yu Q, Ritwik P. Pulpectomy versus extraction for the treatment of nonvital primary second molars: a retrospective chart review. J Clin Pediatr Dent. 2020;44:302–6.

Article   PubMed   CAS   Google Scholar  

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Bmj. 2021;372:n71.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73.

American Academy of Pediatric Dentistry. Pulp Therapy for Primary and Immature Permanent Teeth. The Reference Manual Of Pediatric Dentistry . 2020. p. 343–51.

Duggal M, Gizani S, Albadri S, Krämer N, Stratigaki E, Tong HJ, et al. Best clinical practice guidance for treating deep carious lesions in primary teeth: an EAPD policy document. Eur Arch Paediatr Dent: Off J Eur Acad Paediatr Dent. 2022;23:659–66.

Article   CAS   Google Scholar  

Scottish Dental Clinical Effectiveness Programme. Prevention and Management of Dental Caries in Children [Internet] 2021 [Available from: www.sdcep.org.uk/wp-content/uploads/2018/05/SDCEP-Prevention-and-Management-of-Dental-Caries-in-Children-2nd-Edition.pdf .

Ministry of Health New Zealand. Combined Dental Agreement: Operation Guidelines for Oral Health Services for Adolescents and Special Dental Services for Children and Adolescents. New Zealand: Ministry of Health New Zealand; 2022.

Google Scholar  

Internation Association of Paediatric Dentistry. Foundational Articles and Consensus Recommendations: Pulp Therapy for Primary and Young Permanent Teeth 2021 [Available from: http://www.iapdworld.org/2021_12_pulp-therapy-for-primary-and-young-permanent-teeth .

Rodd HD, Waterhouse PJ, Fuks AB, Fayle SA, Moffat MA. Pulp therapy for primary molars. Int J Paediatr Dent. 2006;16:15–23.

Dhar V, Marghalani AA, Crystal YO, Kumar A, Ritwik P, Tulunoglu O, et al. Use of vital pulp therapies in primary teeth with deep caries lesions. Pediatr Dent. 2017;39:146–59.

Carrotte P. Endodontic treatment for children. Br Dent J. 2005;198:9–15.

Dentistry AAoP. Pulp Therapy for Primary and Immature Permanent Teeth. The Reference Manual of Pediatric Dentistry,. European Archives of Paediatric Dentistry: Official Journal of the European Academy of Paediatric Dentistry ; 2020.

Division CSHTASMD. Management Of Severe Early Childhood Caries . 2012; Medical Development Division of Malaysia.

Schwendicke F, Krois J, Robertson M, Splieth C, Santamaria R, Innes N. Cost-effectiveness of the hall technique in a randomized trial. J Dent Res. 2019;98:61–7.

Daher A, Abreu MH, Costa LR. Recognizing preschool children with primary teeth needing dental treatment because of caries-related toothache. Community Dent oral Epidemiol. 2015;43:298–307.

Dubai Health Authority. Health Policies and Standards Department, Health Regulation Sector. Guidelines for Pediatric Dentistry . 2021; Dubai: Government of Dubai.

Gherlone E, Allegrini S, Annibali S, Baggi L, Barbato E, Barone A, et al. Clinical Recommendation in Odontostomatology Ministry of Health . 2017; Italy.

Primo LG, Pintor AVB, Allegretto MJ, Sancas MC, Duarte ML, Barcelos R, et al. Protocolo de pulpectomia para dentes decíduos: um guia clínico e prático baseado em evidências científicas. Rev Cient CRO-RJ (Online). 2021;6:38–46.

Vitali FC, Santos PS, Cardoso M, Massignan C, Garcia L, Bortoluzzi EA, et al. Are electronic apex locators accurate in determining working length in primary teeth pulpectomies? A systematic review and meta-analysis of clinical studies. Int Endod J. 2022;55:989–1009.

Pitts NZD. White Paper on Dental Caries Prevention and Management: A summary of the current evidence and the key issues in controlling this preventable disease . 2016; FDI World Dental Press Ltd.

Villarosa ARMD, Ramjan LM, Srinivas R, Camilleri M, George A. The effectiveness of guideline implementation strategies in the dental setting: a systematic review. Implement Sci. 2019;14:1–16.

Download references

Acknowledgements

The study was funded by Fundamental & Applied Research Projects (FLPP), Latvian Council of Science wide no. lzp-2022/1-0047, IEVA—Implementation of the Evidence-Based Paediatric CAries Management Strategies in Latvian Clinical Practice—an Evidence Transfer Study. The Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES; Coordination for the Improvement of Higher Education Personnel)—finance code 001 granting scholarships to RAE. The funders had no role in the conduct of the study, the analysis of the data or the decision to submit the manuscript for publication.

Author information

Authors and affiliations.

Department of Conservative Dentistry and Oral Health, Riga Stradiņš University Faculty of Dentistry, Riga Stradiņš University, Rīga, Latvia

Ilze Maldupa & Julius Giese

School of Dentistry, College of Biomedical and Life Sciences, Cardiff University, Cardiff, Wales, UK

Waraf Al-Yaseen & Daniela Prócida Raggio

Department of Orthodontics and Paediatric Dentistry, School of Dentistry, University of São Paulo, São Paulo, Brazil

Rokaia Ahmed Elagami & Daniela Prócida Raggio

You can also search for this author in PubMed   Google Scholar

Contributions

IM, DPR, and WA conception and design of the study; IM, JG, RAE, and WA data acquisition; JG, IM and DPR. data analysis and interpretation; WA, IM, RAE AND DPR drafted the manuscript; IM, WA, RAE, DPR revised and gave final approval of the manuscript.

Corresponding author

Correspondence to Waraf Al-Yaseen .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

Ethics approval was not required for this study as it is a scoping review based on secondary data, all of which are publicly available. The review did not involve any interaction with human subjects, and no personal or sensitive information was obtained.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Appendix 1 and 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Maldupa, I., Al-Yaseen, W., Giese, J. et al. Recommended procedures for managing carious lesions in primary teeth with pulp involvement—a scoping review. BDJ Open 10 , 74 (2024). https://doi.org/10.1038/s41405-024-00259-8

Download citation

Received : 19 June 2024

Revised : 04 August 2024

Accepted : 07 August 2024

Published : 18 September 2024

DOI : https://doi.org/10.1038/s41405-024-00259-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies