literature review on educational research

• Library databases
• Library website

Education Literature Review: Education Literature Review

What does this guide cover.

Writing the literature review is a long, complex process that requires you to use many different tools, resources, and skills.

This page provides links to the guides, tutorials, and webinars that can help you with all aspects of completing your literature review.

The Basic Process

These resources provide overviews of the entire literature review process. Start here if you are new to the literature review process.

• Literature Reviews Overview : Writing Center
• How to do a Literature Review : Library
• Video: Common Errors Made When Conducting a Lit Review (YouTube)  

The Role of the Literature Review

Your literature review gives your readers an understanding of the evolution of scholarly research on your topic.

In your literature review you will:

• survey the scholarly landscape
• provide a synthesis of the issues, trends, and concepts
• possibly provide some historical background

Review the literature in two ways:

• Section 1: reviews the literature for the Problem
• Section 3: reviews the literature for the Project

The literature review is NOT an annotated bibliography. Nor should it simply summarize the articles you've read. Literature reviews are organized thematically and demonstrate synthesis of the literature.

For more information, view the Library's short video on searching by themes:

Short Video: Research for the Literature Review

(4 min 10 sec) Recorded August 2019 Transcript 

Search for Literature

The iterative process of research:

• Find an article.
• Read the article and build new searches using keywords and names from the article.
• Mine the bibliography for other works.
• Use “cited by” searches to find more recent works that reference the article.
• Repeat steps 2-4 with the new articles you find.

These are the main skills and resources you will need in order to effectively search for literature on your topic:

• Subject Research: Education by Jon Allinder Last Updated Aug 7, 2023 5748 views this year
• Keyword Searching: Finding Articles on Your Topic by Lynn VanLeer Last Updated Sep 12, 2023 27023 views this year
• Google Scholar by Jon Allinder Last Updated Aug 16, 2023 17588 views this year
• Quick Answer: How do I find books and articles that cite an article I already have?
• Quick Answer: How do I find a measurement, test, survey or instrument?

Video: Education Databases and Doctoral Research Resources

(6 min 04 sec) Recorded April 2019 Transcript 

Staying Organized

The literature review requires organizing a variety of information. The following resources will help you develop the organizational systems you'll need to be successful.

• Organize your research
• Citation Management Software

You can make your search log as simple or complex as you would like.  It can be a table in a word document or an excel spread sheet.  Here are two examples.  The word document is a basic table where you can keep track of databases, search terms, limiters, results and comments.  The Excel sheet is more complex and has additional sheets for notes, Google Scholar log; Journal Log, and Questions to ask the Librarian.  

• Search Log Example Sample search log in Excel
• Search Log Example Sample search log set up as a table in a word document.
• Literature Review Matrix with color coding Sample template for organizing and synthesizing your research

Writing the Literature Review

The following resources created by the Writing Center and the Academic Skills Center support the writing process for the dissertation/project study. 

• Critical Reading
• What is Synthesis 
• Walden Templates
• Quick Answer: How do I find Walden EdD (Doctor of Education) studies?
• Quick Answer: How do I find Walden PhD dissertations?

Beyond the Literature Review

The literature review isn't the only portion of a dissertation/project study that requires searching. The following resources can help you identify and utilize a theory, methodology, measurement instruments, or statistics.

• Education Theory by Jon Allinder Last Updated Aug 5, 2024 700 views this year
• Tests & Measures in Education by Kimberly Burton Last Updated Nov 18, 2021 54 views this year
• Education Statistics by Jon Allinder Last Updated Feb 22, 2022 68 views this year
• Office of Research and Doctoral Services

Books and Articles about the Lit Review

The following articles and books outline the purpose of the literature review and offer advice for successfully completing one.

• Chen, D. T. V., Wang, Y. M., & Lee, W. C. (2016). Challenges confronting beginning researchers in conducting literature reviews. Studies in Continuing Education, 38(1), 47-60. https://doi.org/10.1080/0158037X.2015.1030335 Proposes a framework to conceptualize four types of challenges students face: linguistic, methodological, conceptual, and ontological.
• Randolph, J.J. (2009). A guide to writing the dissertation literature review. Practical Assessment, Research & Evaluation 14(13), 1-13. Provides advice for writing a quantitative or qualitative literature review, by a Walden faculty member.
• Torraco, R. J. (2016). Writing integrative literature reviews: Using the past and present to explore the future. Human Resource Development Review, 15(4), 404–428. https://doi.org/10.1177/1534484316671606 This article presents the integrative review of literature as a distinctive form of research that uses existing literature to create new knowledge.
• Wee, B. V., & Banister, D. (2016). How to write a literature review paper?. Transport Reviews, 36(2), 278-288. http://doi.org/10.1080/01441647.2015.1065456 Discusses how to write a literature review with a focus on adding value rather and suggests structural and contextual aspects found in outstanding literature reviews.
• Winchester, C. L., & Salji, M. (2016). Writing a literature review. Journal of Clinical Urology, 9(5), 308-312. https://doi.org/10.1177/2051415816650133 Reviews the use of different document types to add structure and enrich your literature review and the skill sets needed in writing the literature review.
• Xiao, Y., & Watson, M. (2017). Guidance on conducting a systematic literature review. Journal of Planning Education and Research. https://doi.org/10.1177/0739456X17723971 Examines different types of literature reviews and the steps necessary to produce a systematic review in educational research.

• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 1: Introduction

Learning objectives.

At the conclusion of this chapter, you will be able to:

• Identify the purpose of the literature review in  the research process
• Distinguish between different types of literature reviews

1.1 What is a Literature Review?

Pick up nearly any book on research methods and you will find a description of a literature review.  At a basic level, the term implies a survey of factual or nonfiction books, articles, and other documents published on a particular subject.  Definitions may be similar across the disciplines, with new types and definitions continuing to emerge.  Generally speaking, a literature review is a:

• “comprehensive background of the literature within the interested topic area…” ( O’Gorman & MacIntosh, 2015, p. 31 ).
• “critical component of the research process that provides an in-depth analysis of recently published research findings in specifically identified areas of interest.” ( House, 2018, p. 109 ).
• “written document that presents a logically argued case founded on a comprehensive understanding of the current state of knowledge about a topic of study” ( Machi & McEvoy,  2012, p. 4 ).

As a foundation for knowledge advancement in every discipline, it is an important element of any research project.  At the graduate or doctoral level, the literature review is an essential feature of thesis and dissertation, as well as grant proposal writing.  That is to say, “A substantive, thorough, sophisticated literature review is a precondition for doing substantive, thorough, sophisticated research…A researcher cannot perform significant research without first understanding the literature in the field.” ( Boote & Beile, 2005, p. 3 ).  It is by this means, that a researcher demonstrates familiarity with a body of knowledge and thereby establishes credibility with a reader.  An advanced-level literature review shows how prior research is linked to a new project, summarizing and synthesizing what is known while identifying gaps in the knowledge base, facilitating theory development, closing areas where enough research already exists, and uncovering areas where more research is needed. ( Webster & Watson, 2002, p. xiii )

A graduate-level literature review is a compilation of the most significant previously published research on your topic. Unlike an annotated bibliography or a research paper you may have written as an undergraduate, your literature review will outline, evaluate and synthesize relevant research and relate those sources to your own thesis or research question. It is much more than a summary of all the related literature.

It is a type of writing that demonstrate the importance of your research by defining the main ideas and the relationship between them. A good literature review lays the foundation for the importance of your stated problem and research question.

Literature reviews:

• define a concept
• map the research terrain or scope
• systemize relationships between concepts
• identify gaps in the literature ( Rocco & Plathotnik, 2009, p. 128 )

The purpose of a literature review is to demonstrate that your research question  is meaningful. Additionally, you may review the literature of different disciplines to find deeper meaning and understanding of your topic. It is especially important to consider other disciplines when you do not find much on your topic in one discipline. You will need to search the cognate literature before claiming there is “little previous research” on your topic.

Well developed literature reviews involve numerous steps and activities. The literature review is an iterative process because you will do at least two of them: a preliminary search to learn what has been published in your area and whether there is sufficient support in the literature for moving ahead with your subject. After this first exploration, you will conduct a deeper dive into the literature to learn everything you can about the topic and its related issues.

Literature Review Tutorial

1.2 Literature Review Basics

An effective literature review must:

• Methodologically analyze and synthesize quality literature on a topic
• Provide a firm foundation to a topic or research area
• Provide a firm foundation for the selection of a research methodology
• Demonstrate that the proposed research contributes something new to the overall body of knowledge of advances the research field’s knowledge base. ( Levy & Ellis, 2006 ).

All literature reviews, whether they are qualitative, quantitative or both, will at some point:

• Introduce the topic and define its key terms
• Establish the importance of the topic
• Provide an overview of the amount of available literature and its types (for example: theoretical, statistical, speculative)
• Identify gaps in the literature
• Point out consistent finding across studies
• Arrive at a synthesis that organizes what is known about a topic
• Discusses possible implications and directions for future research

1.3 Types of Literature Reviews

There are many different types of literature reviews, however there are some shared characteristics or features.  Remember a comprehensive literature review is, at its most fundamental level, an original work based on an extensive critical examination and synthesis of the relevant literature on a topic. As a study of the research on a particular topic, it is arranged by key themes or findings, which may lead up to or link to the  research question.  In some cases, the research question will drive the type of literature review that is undertaken.

The following section includes brief descriptions of the terms used to describe different literature review types with examples of each.   The included citations are open access, Creative Commons licensed or copyright-restricted.

1.3.1 Types of Review

1.3.1.1 conceptual.

Guided by an understanding of basic issues rather than a research methodology. You are looking for key factors, concepts or variables and the presumed relationship between them. The goal of the conceptual literature review is to categorize and describe concepts relevant to your study or topic and outline a relationship between them. You will include relevant theory and empirical research.

Examples of a Conceptual Review:

• Education : The formality of learning science in everyday life: A conceptual literature review. ( Dohn, 2010 ).
• Education : Are we asking the right questions? A conceptual review of the educational development literature in higher education. ( Amundsen & Wilson, 2012 ).

1.3.1.2 Empirical

An empirical literature review collects, creates, arranges, and analyzes numeric data reflecting the frequency of themes, topics, authors and/or methods found in existing literature. Empirical literature reviews present their summaries in quantifiable terms using descriptive and inferential statistics.

Examples of an Empirical Review:

• Nursing : False-positive findings in Cochrane meta-analyses with and without application of trial sequential analysis: An empirical review. ( Imberger, Thorlund, Gluud, & Wettersley, 2016 ).
• Education : Impediments of e-learning adoption in higher learning institutions of Tanzania: An empirical review ( Mwakyusa & Mwalyagile, 2016 ).

1.3.1.3 Exploratory

Unlike a synoptic literature review, the purpose here is to provide a broad approach to the topic area. The aim is breadth rather than depth and to get a general feel for the size of the topic area. A graduate student might do an exploratory review of the literature before beginning a synoptic, or more comprehensive one.

Examples of an Exploratory Review:

• Education : University research management: An exploratory literature review. ( Schuetzenmeister, 2010 ).
• Education : An exploratory review of design principles in constructivist gaming learning environments. ( Rosario & Widmeyer, 2009 ).

1.3.1.4 Focused

A type of literature review limited to a single aspect of previous research, such as methodology. A focused literature review generally will describe the implications of choosing a particular element of past research, such as methodology in terms of data collection, analysis and interpretation.

Examples of a Focused Review:

• Nursing : Clinical inertia in the management of type 2 diabetes mellitus: A focused literature review. ( Khunti, Davies, & Khunti, 2015 ).
• Education : Language awareness: Genre awareness-a focused review of the literature. ( Stainton, 1992 ).

1.3.1.5 Integrative

Critiques past research and draws overall conclusions from the body of literature at a specified point in time. Reviews, critiques, and synthesizes representative literature on a topic in an integrated way. Most integrative reviews are intended to address mature topics or  emerging topics. May require the author to adopt a guiding theory, a set of competing models, or a point of view about a topic.  For more description of integrative reviews, see Whittemore & Knafl (2005).

Examples of an Integrative Review:

• Nursing : Interprofessional teamwork and collaboration between community health workers and healthcare teams: An integrative review. ( Franklin,  Bernhardt, Lopez, Long-Middleton, & Davis, 2015 ).
• Education : Exploring the gap between teacher certification and permanent employment in Ontario: An integrative literature review. ( Brock & Ryan, 2016 ).

1.3.1.6 Meta-analysis

A subset of a  systematic review, that takes findings from several studies on the same subject and analyzes them using standardized statistical procedures to pool together data. Integrates findings from a large body of quantitative findings to enhance understanding, draw conclusions, and detect patterns and relationships. Gather data from many different, independent studies that look at the same research question and assess similar outcome measures. Data is combined and re-analyzed, providing a greater statistical power than any single study alone. It’s important to note that not every systematic review includes a meta-analysis but a meta-analysis can’t exist without a systematic review of the literature.

Examples of a Meta-Analysis:

• Education : Efficacy of the cooperative learning method on mathematics achievement and attitude: A meta-analysis research. ( Capar & Tarim, 2015 ).
• Nursing : A meta-analysis of the effects of non-traditional teaching methods on the critical thinking abilities of nursing students. ( Lee, Lee, Gong, Bae, & Choi, 2016 ).
• Education : Gender differences in student attitudes toward science: A meta-analysis of the literature from 1970 to 1991. ( Weinburgh, 1995 ).

1.3.1.7 Narrative/Traditional

An overview of research on a particular topic that critiques and summarizes a body of literature. Typically broad in focus. Relevant past research is selected and synthesized into a coherent discussion. Methodologies, findings and limits of the existing body of knowledge are discussed in narrative form. Sometimes also referred to as a traditional literature review. Requires a sufficiently focused research question. The process may be subject to bias that supports the researcher’s own work.

Examples of a Narrative/Traditional Review:

• Nursing : Family carers providing support to a person dying in the home setting: A narrative literature review. ( Morris, King, Turner, & Payne, 2015 ).
• Education : Adventure education and Outward Bound: Out-of-class experiences that make a lasting difference. ( Hattie, Marsh, Neill, & Richards, 1997 ).
• Education : Good quality discussion is necessary but not sufficient in asynchronous tuition: A brief narrative review of the literature. ( Fear & Erikson-Brown, 2014 ).
• Nursing : Outcomes of physician job satisfaction: A narrative review, implications, and directions for future research. ( Williams & Skinner, 2003 ).

1.3.1.8 Realist

Aspecific type of literature review that is theory-driven and interpretative and is intended to explain the outcomes of a complex intervention program(s).

Examples of a Realist Review:

• Nursing : Lean thinking in healthcare: A realist review of the literature. ( Mazzacato, Savage, Brommels, 2010 ).
• Education : Unravelling quality culture in higher education: A realist review. ( Bendermacher, Egbrink, Wolfhagen, & Dolmans, 2017 ).

1.3.1.9 Scoping

Tend to be non-systematic and focus on breadth of coverage conducted on a topic rather than depth. Utilize a wide range of materials; may not evaluate the quality of the studies as much as count the number. One means of understanding existing literature. Aims to identify nature and extent of research; preliminary assessment of size and scope of available research on topic. May include research in progress.

Examples of a Scoping Review:

• Nursing : Organizational interventions improving access to community-based primary health care for vulnerable populations: A scoping review. ( Khanassov, Pluye, Descoteaux, Haggerty,  Russell, Gunn, & Levesque, 2016 ).
• Education : Interdisciplinary doctoral research supervision: A scoping review. ( Vanstone, Hibbert, Kinsella, McKenzie, Pitman, & Lingard, 2013 ).
• Nursing : A scoping review of the literature on the abolition of user fees in health care services in Africa. ( Ridde, & Morestin, 2011 ).

1.3.1.10 Synoptic

Unlike an exploratory review, the purpose is to provide a concise but accurate overview of all material that appears to be relevant to a chosen topic. Both content and methodological material is included. The review should aim to be both descriptive and evaluative. Summarizes previous studies while also showing how the body of literature could be extended and improved in terms of content and method by identifying gaps.

Examples of a Synoptic Review:

• Education : Theoretical framework for educational assessment: A synoptic review. ( Ghaicha, 2016 ).
• Education : School effects research: A synoptic review of past efforts and some suggestions for the future. ( Cuttance, 1981 ).

1.3.1.11 Systematic Review

A rigorous review that follows a strict methodology designed with a presupposed selection of literature reviewed.  Undertaken to clarify the state of existing research, the evidence, and possible implications that can be drawn from that.  Using comprehensive and exhaustive searching of the published and unpublished literature, searching various databases, reports, and grey literature.  Transparent and reproducible in reporting details of time frame, search and methods to minimize bias.  Must include a team of at least 2-3 and includes the critical appraisal of the literature.  For more description of systematic reviews, including links to protocols, checklists, workflow processes, and structure see “ A Young Researcher’s Guide to a Systematic Review “.

Examples of a Systematic Review:

• Education : The potentials of using cloud computing in schools: A systematic literature review ( Hartmann, Braae, Pedersen, & Khalid, 2017 )
• Nursing : Is butter back? A systematic review and meta-analysis of butter consumption and risk of cardiovascular disease, diabetes, and total mortality. ( Pimpin, Wu, Haskelberg, Del Gobbo, & Mozaffarian, 2016 ).
• Education : The use of research to improve professional practice: a systematic review of the literature. ( Hemsley-Brown & Sharp, 2003 ).
• Nursing : Using computers to self-manage type 2 diabetes. ( Pal, Eastwood, Michie, Farmer, Barnard, Peacock, Wood, Inniss, & Murray, 2013 ).

1.3.1.12 Umbrella/Overview of Reviews

Compiles evidence from multiple systematic reviews into one document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address those interventions and their effects. Often used in recommendations for practice.

Examples of an Umbrella/Overview Review:

• Education : Reflective practice in healthcare education: An umbrella review. ( Fragknos, 2016 ).
• Nursing : Systematic reviews of psychosocial interventions for autism: an umbrella review. ( Seida, Ospina, Karkhaneh, Hartling, Smith, & Clark, 2009 ).

For a brief discussion see “ Not all literature reviews are the same ” (Thomson, 2013).

1.4 Why do a Literature Review?

The purpose of the literature review is the same regardless of the topic or research method. It tests your own research question against what is already known about the subject.

1.4.1 First – It’s part of the whole. Omission of a literature review chapter or section in a graduate-level project represents a serious void or absence of critical element in the research process.

The outcome of your review is expected to demonstrate that you:

• can systematically explore the research in your topic area
• can read and critically analyze the literature in your discipline and then use it appropriately to advance your own work
• have sufficient knowledge in the topic to undertake further investigation

1.4.2 Second – It’s good for you!

• You improve your skills as a researcher
• You become familiar with the discourse of your discipline and learn how to be a scholar in your field
• You learn through writing your ideas and finding your voice in your subject area
• You define, redefine and clarify your research question for yourself in the process

1.4.3 Third – It’s good for your reader. Your reader expects you to have done the hard work of gathering, evaluating and synthesizes the literature.  When you do a literature review you:

• Set the context for the topic and present its significance
• Identify what’s important to know about your topic – including individual material, prior research, publications, organizations and authors.
• Demonstrate relationships among prior research
• Establish limitations of existing knowledge
• Analyze trends in the topic’s treatment and gaps in the literature

1.4.4 Why do a literature review?

• To locate gaps in the literature of your discipline
• To avoid reinventing the wheel
• To carry on where others have already been
• To identify other people working in the same field
• To increase your breadth of knowledge in your subject area
• To find the seminal works in your field
• To provide intellectual context for your own work
• To acknowledge opposing viewpoints
• To put your work in perspective
• To demonstrate you can discover and retrieve previous work in the area

1.5 Common Literature Review Errors

Graduate-level literature reviews are more than a summary of the publications you find on a topic.  As you have seen in this brief introduction, literature reviews are a very specific type of research, analysis, and writing.  We will explore these topics more in the next chapters.  Some things to keep in mind as you begin your own research and writing are ways to avoid the most common errors seen in the first attempt at a literature review.  For a quick review of some of the pitfalls and challenges a new researcher faces when he/she begins work, see “ Get Ready: Academic Writing, General Pitfalls and (oh yes) Getting Started! ”.

As you begin your own graduate-level literature review, try to avoid these common mistakes:

• Accepts another researcher’s finding as valid without evaluating methodology and data
• Contrary findings and alternative interpretations are not considered or mentioned
• Findings are not clearly related to one’s own study, or findings are too general
• Insufficient time allowed to define best search strategies and writing
• Isolated statistical results are simply reported rather than synthesizing the results
• Problems with selecting and using most relevant keywords, subject headings and descriptors
• Relies too heavily on secondary sources
• Search methods are not recorded or reported for transparency
• Summarizes rather than synthesizes articles

In conclusion, the purpose of a literature review is three-fold:

• to survey the current state of knowledge or evidence in the area of inquiry,
• to identify key authors, articles, theories, and findings in that area, and
• to identify gaps in knowledge in that research area.

A literature review is commonly done today using computerized keyword searches in online databases, often working with a trained librarian or information expert. Keywords can be combined using the Boolean operators, “and”, “or” and sometimes “not”  to narrow down or expand the search results. Once a list of articles is generated from the keyword and subject heading search, the researcher must then manually browse through each title and abstract, to determine the suitability of that article before a full-text article is obtained for the research question.

Literature reviews should be reasonably complete, and not restricted to a few journals, a few years, or a specific methodology or research design. Reviewed articles may be summarized in the form of tables, and can be further structured using organizing frameworks such as a concept matrix.

A well-conducted literature review should indicate whether the initial research questions have already been addressed in the literature, whether there are newer or more interesting research questions available, and whether the original research questions should be modified or changed in light of findings of the literature review.

The review can also provide some intuitions or potential answers to the questions of interest and/or help identify theories that have previously been used to address similar questions and may provide evidence to inform policy or decision-making. ( Bhattacherjee, 2012 ).

Read Abstract 1.  Refer to Types of Literature Reviews.  What type of literature review do you think this study is and why?  See the Answer Key for the correct response.

Nursing : To describe evidence of international literature on the safe care of the hospitalised child after the World Alliance for Patient Safety and list contributions of the general theoretical framework of patient safety for paediatric nursing.

An integrative literature review between 2004 and 2015 using the databases PubMed, Cumulative Index of Nursing and Allied Health Literature (CINAHL), Scopus, Web of Science and Wiley Online Library, and the descriptors Safety or Patient safety, Hospitalised child, Paediatric nursing, and Nursing care.

Thirty-two articles were analysed, most of which were from North American, with a descriptive approach. The quality of the recorded information in the medical records, the use of checklists, and the training of health workers contribute to safe care in paediatric nursing and improve the medication process and partnerships with parents.

General information available on patient safety should be incorporated in paediatric nursing care. ( Wegner, Silva, Peres, Bandeira, Frantz, Botene, & Predebon, 2017 ).

Read Abstract 2.  Refer to Types of Literature Reviews.  What type of lit review do you think this study is and why?  See the Answer Key for the correct response.

Education : The focus of this paper centers around timing associated with early childhood education programs and interventions using meta-analytic methods. At any given assessment age, a child’s current age equals starting age, plus duration of program, plus years since program ended. Variability in assessment ages across the studies should enable everyone to identify the separate effects of all three time-related components. The project is a meta-analysis of evaluation studies of early childhood education programs conducted in the United States and its territories between 1960 and 2007. The population of interest is children enrolled in early childhood education programs between the ages of 0 and 5 and their control-group counterparts. Since the data come from a meta-analysis, the population for this study is drawn from many different studies with diverse samples. Given the preliminary nature of their analysis, the authors cannot offer conclusions at this point. ( Duncan, Leak, Li, Magnuson, Schindler, & Yoshikawa, 2011 ).

Test Yourself

See Answer Key for the correct responses.

The purpose of a graduate-level literature review is to summarize in as many words as possible everything that is known about my topic.

A literature review is significant because in the process of doing one, the researcher learns to read and critically assess the literature of a discipline and then uses it appropriately to advance his/her own research.

Read the following abstract and choose the correct type of literature review it represents.

Nursing: E-cigarette use has become increasingly popular, especially among the young. Its long-term influence upon health is unknown. Aim of this review has been to present the current state of knowledge about the impact of e-cigarette use on health, with an emphasis on Central and Eastern Europe. During the preparation of this narrative review, the literature on e-cigarettes available within the network PubMed was retrieved and examined. In the final review, 64 research papers were included. We specifically assessed the construction and operation of the e-cigarette as well as the chemical composition of the e-liquid; the impact that vapor arising from the use of e-cigarette explored in experimental models in vitro; and short-term effects of use of e-cigarettes on users’ health. Among the substances inhaled by the e-smoker, there are several harmful products, such as: formaldehyde, acetaldehyde, acroleine, propanal, nicotine, acetone, o-methyl-benzaldehyde, carcinogenic nitrosamines. Results from experimental animal studies indicate the negative impact of e-cigarette exposure on test models, such as ascytotoxicity, oxidative stress, inflammation, airway hyper reactivity, airway remodeling, mucin production, apoptosis, and emphysematous changes. The short-term impact of e-cigarettes on human health has been studied mostly in experimental setting. Available evidence shows that the use of e-cigarettes may result in acute lung function responses (e.g., increase in impedance, peripheral airway flow resistance) and induce oxidative stress. Based on the current available evidence, e-cigarette use is associated with harmful biologic responses, although it may be less harmful than traditional cigarettes. (J ankowski, Brożek, Lawson, Skoczyński, & Zejda, 2017 ).

• Meta-analysis
• Exploratory

Education: In this review, Mary Vorsino writes that she is interested in keeping the potential influences of women pragmatists of Dewey’s day in mind while presenting modern feminist re readings of Dewey. She wishes to construct a narrowly-focused and succinct literature review of thinkers who have donned a feminist lens to analyze Dewey’s approaches to education, learning, and democracy and to employ Dewey’s works in theorizing on gender and education and on gender in society. This article first explores Dewey as both an ally and a problematic figure in feminist literature and then investigates the broader sphere of feminist pragmatism and two central themes within it: (1) valuing diversity, and diverse experiences; and (2) problematizing fixed truths. ( Vorsino, 2015 ).

Image Attributions

Literature Reviews for Education and Nursing Graduate Students Copyright © by Linda Frederiksen is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Prevent plagiarism. Run a free check.

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved August 5, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Systematic Reviews in Educational Research: Methodology, Perspectives and Application

• Open Access
• First Online: 22 November 2019

Cite this chapter

You have full access to this open access chapter

• Mark Newman 6 &
• David Gough 6  

89k Accesses

160 Citations

2 Altmetric

This chapter explores the processes of reviewing literature as a research method. The logic of the family of research approaches called systematic review is analysed and the variation in techniques used in the different approaches explored using examples from existing reviews. The key distinctions between aggregative and configurative approaches are illustrated and the chapter signposts further reading on key issues in the systematic review process.

You have full access to this open access chapter,  Download chapter PDF

Similar content being viewed by others

Reviewing Literature for and as Research

Methodological Approaches to Literature Review

The Role of Meta-analysis in Educational Research

1 what are systematic reviews.

A literature review is a scholarly paper which provides an overview of current knowledge about a topic. It will typically include substantive findings, as well as theoretical and methodological contributions to a particular topic (Hart 2018 , p. xiii). Traditionally in education ‘reviewing the literature’ and ‘doing research’ have been viewed as distinct activities. Consider the standard format of research proposals, which usually have some kind of ‘review’ of existing knowledge presented distinctly from the methods of the proposed new primary research. However, both reviews and research are undertaken in order to find things out. Reviews to find out what is already known from pre-existing research about a phenomena, subject or topic; new primary research to provide answers to questions about which existing research does not provide clear and/or complete answers.

When we use the term research in an academic sense it is widely accepted that we mean a process of asking questions and generating knowledge to answer these questions using rigorous accountable methods. As we have noted, reviews also share the same purposes of generating knowledge but historically we have not paid as much attention to the methods used for reviewing existing literature as we have to the methods used for primary research. Literature reviews can be used for making claims about what we know and do not know about a phenomenon and also about what new research we need to undertake to address questions that are unanswered. Therefore, it seems reasonable to conclude that ‘how’ we conduct a review of research is important.

The increased focus on the use of research evidence to inform policy and practice decision-making in Evidence Informed Education (Hargreaves 1996 ; Nelson and Campbell 2017 ) has increased the attention given to contextual and methodological limitations of research evidence provided by single studies. Reviews of research may help address these concerns when carried on in a systematic, rigorous and transparent manner. Thus, again emphasizing the importance of ‘how’ reviews are completed.

The logic of systematic reviews is that reviews are a form of research and thus can be improved by using appropriate and explicit methods. As the methods of systematic review have been applied to different types of research questions, there has been an increasing plurality of types of systematic review. Thus, the term ‘systematic review’ is used in this chapter to refer to a family of research approaches that are a form of secondary level analysis (secondary research) that brings together the findings of primary research to answer a research question. Systematic reviews can therefore be defined as “a review of existing research using explicit, accountable rigorous research methods” (Gough et al. 2017 , p. 4).

2 Variation in Review Methods

Reviews can address a diverse range of research questions. Consequently, as with primary research, there are many different approaches and methods that can be applied. The choices should be dictated by the review questions. These are shaped by reviewers’ assumptions about the meaning of a particular research question, the approach and methods that are best used to investigate it. Attempts to classify review approaches and methods risk making hard distinctions between methods and thereby to distract from the common defining logics that these approaches often share. A useful broad distinction is between reviews that follow a broadly configurative synthesis logic and reviews that follow a broadly aggregative synthesis logic (Sandelowski et al. 2012 ). However, it is important to keep in mind that most reviews have elements of both (Gough et al. 2012 ).

Reviews that follow a broadly configurative synthesis logic approach usually investigate research questions about meaning and interpretation to explore and develop theory. They tend to use exploratory and iterative review methods that emerge throughout the process of the review. Studies included in the review are likely to have investigated the phenomena of interest using methods such as interviews and observations, with data in the form of text. Reviewers are usually interested in purposive variety in the identification and selection of studies. Study quality is typically considered in terms of authenticity. Synthesis consists of the deliberative configuring of data by reviewers into patterns to create a richer conceptual understanding of a phenomenon. For example, meta ethnography (Noblit and Hare 1988 ) uses ethnographic data analysis methods to explore and integrate the findings of previous ethnographies in order to create higher-level conceptual explanations of phenomena. There are many other review approaches that follow a broadly configurative logic (for an overview see Barnett-Page and Thomas 2009 ); reflecting the variety of methods used in primary research in this tradition.

Reviews that follow a broadly aggregative synthesis logic usually investigate research questions about impacts and effects. For example, systematic reviews that seek to measure the impact of an educational intervention test the hypothesis that an intervention has the impact that has been predicted. Reviews following an aggregative synthesis logic do not tend to develop theory directly; though they can contribute by testing, exploring and refining theory. Reviews following an aggregative synthesis logic tend to specify their methods in advance (a priori) and then apply them without any deviation from a protocol. Reviewers are usually concerned to identify the comprehensive set of studies that address the research question. Studies included in the review will usually seek to determine whether there is a quantitative difference in outcome between groups receiving and not receiving an intervention. Study quality assessment in reviews following an aggregative synthesis logic focusses on the minimisation of bias and thus selection pays particular attention to homogeneity between studies. Synthesis aggregates, i.e. counts and adds together, the outcomes from individual studies using, for example, statistical meta-analysis to provide a pooled summary of effect.

3 The Systematic Review Process

Different types of systematic review are discussed in more detail later in this chapter. The majority of systematic review types share a common set of processes. These processes can be divided into distinct but interconnected stages as illustrated in Fig.  1 . Systematic reviews need to specify a research question and the methods that will be used to investigate the question. This is often written as a ‘protocol’ prior to undertaking the review. Writing a protocol or plan of the methods at the beginning of a review can be a very useful activity. It helps the review team to gain a shared understanding of the scope of the review and the methods that they will use to answer the review’s questions. Different types of systematic reviews will have more or less developed protocols. For example, for systematic reviews investigating research questions about the impact of educational interventions it is argued that a detailed protocol should be fully specified prior to the commencement of the review to reduce the possibility of reviewer bias (Torgerson 2003 , p. 26). For other types of systematic review, in which the research question is more exploratory, the protocol may be more flexible and/or developmental in nature.

The systematic review process

3.1 Systematic Review Questions and the Conceptual Framework

The review question gives each review its particular structure and drives key decisions about what types of studies to include; where to look for them; how to assess their quality; and how to combine their findings. Although a research question may appear to be simple, it will include many assumptions. Whether implicit or explicit, these assumptions will include: epistemological frameworks about knowledge and how we obtain it, theoretical frameworks, whether tentative or firm, about the phenomenon that is the focus of study.

Taken together, these produce a conceptual framework that shapes the research questions, choices about appropriate systematic review approach and methods. The conceptual framework may be viewed as a working hypothesis that can be developed, refined or confirmed during the course of the research. Its purpose is to explain the key issues to be studied, the constructs or variables, and the presumed relationships between them. The framework is a research tool intended to assist a researcher to develop awareness and understanding of the phenomena under scrutiny and to communicate this (Smyth 2004 ).

A review to investigate the impact of an educational intervention will have a conceptual framework that includes a hypothesis about a causal link between; who the review is about (the people), what the review is about (an intervention and what it is being compared with), and the possible consequences of intervention on the educational outcomes of these people. Such a review would follow a broadly aggregative synthesis logic. This is the shape of reviews of educational interventions carried out for the What Works Clearing House in the USA Footnote 1 and the Education Endowment Foundation in England. Footnote 2

A review to investigate meaning or understanding of a phenomenon for the purpose of building or further developing theory will still have some prior assumptions. Thus, an initial conceptual framework will contain theoretical ideas about how the phenomena of interest can be understood and some ideas justifying why a particular population and/or context is of specific interest or relevance. Such a review is likely to follow a broadly configurative logic.

3.2 Selection Criteria

Reviewers have to make decisions about which research studies to include in their review. In order to do this systematically and transparently they develop rules about which studies can be selected into the review. Selection criteria (sometimes referred to as inclusion or exclusion criteria) create restrictions on the review. All reviews, whether systematic or not, limit in some way the studies that are considered by the review. Systematic reviews simply make these restrictions transparent and therefore consistent across studies. These selection criteria are shaped by the review question and conceptual framework. For example, a review question about the impact of homework on educational attainment would have selection criteria specifying who had to do the homework; the characteristics of the homework and the outcomes that needed to be measured. Other commonly used selection criteria include study participant characteristics; the country where the study has taken place and the language in which the study is reported. The type of research method(s) may also be used as a selection criterion but this can be controversial given the lack of consensus in education research (Newman 2008 ), and the inconsistent terminology used to describe education research methods.

3.3 Developing the Search Strategy

The search strategy is the plan for how relevant research studies will be identified. The review question and conceptual framework shape the selection criteria. The selection criteria specify the studies to be included in a review and thus are a key driver of the search strategy. A key consideration will be whether the search aims to be exhaustive i.e. aims to try and find all the primary research that has addressed the review question. Where reviews address questions about effectiveness or impact of educational interventions the issue of publication bias is a concern. Publication bias is the phenomena whereby smaller and/or studies with negative findings are less likely to be published and/or be harder to find. We may therefore inadvertently overestimate the positive effects of an educational intervention because we do not find studies with negative or smaller effects (Chow and Eckholm 2018 ). Where the review question is not of this type then a more specific or purposive search strategy, that may or may not evolve as the review progresses, may be appropriate. This is similar to sampling approaches in primary research. In primary research studies using aggregative approaches, such as quasi-experiments, analysis is based on the study of complete or representative samples. In primary research studies using configurative approaches, such as ethnography, analysis is based on examining a range of instances of the phenomena in similar or different contexts.

The search strategy will detail the sources to be searched and the way in which the sources will be searched. A list of search source types is given in Box 1 below. An exhaustive search strategy would usually include all of these sources using multiple bibliographic databases. Bibliographic databases usually index academic journals and thus are an important potential source. However, in most fields, including education, relevant research is published in a range of journals which may be indexed in different bibliographic databases and thus it may be important to search multiple bibliographic databases. Furthermore, some research is published in books and an increasing amount of research is not published in academic journals or at least may not be published there first. Thus, it is important to also consider how you will find relevant research in other sources including ‘unpublished’ or ‘grey’ literature. The Internet is a valuable resource for this purpose and should be included as a source in any search strategy.

Box 1: Search Sources

The World Wide Web/Internet

Google, Specialist Websites, Google Scholar, Microsoft Academic

Bibliographic Databases

Subject specific e.g. Education—ERIC: Education Resources Information Centre

Generic e.g. ASSIA: Applied Social Sciences Index and Abstracts

Handsearching of specialist journals or books

Contacts with Experts

Citation Checking

New, federated search engines are being developed, which search multiple sources at the same time, eliminating duplicates automatically (Tsafnat et al. 2013 ). Technologies, including text mining, are being used to help develop search strategies, by suggesting topics and terms on which to search—terms that reviewers may not have thought of using. Searching is also being aided by technology through the increased use (and automation) of ‘citation chasing’, where papers that cite, or are cited by, a relevant study are checked in case they too are relevant.

A search strategy will identify the search terms that will be used to search the bibliographic databases. Bibliographic databases usually index records according to their topic using ‘keywords’ or ‘controlled terms’ (categories used by the database to classify papers). A comprehensive search strategy usually involves searching both a freetext search using keywords determined by the reviewers and controlled terms. An example of a bibliographic database search is given in Box 2. This search was used in a review that aimed to find studies that investigated the impact of Youth Work on positive youth outcomes (Dickson et al. 2013 ). The search is built using terms for the population of interest (Youth), the intervention of interest (Youth Work) and the outcomes of Interest (Positive Development). It used both keywords and controlled terms, ‘wildcards’ (the *sign in this database) and the Boolean operators ‘OR’ and ‘AND’ to combine terms. This example illustrates the potential complexity of bibliographic database search strings, which will usually require a process of iterative development to finalise.

Box 2: Search string example To identify studies that address the question What is the empirical research evidence on the impact of youth work on the lives of children and young people aged 10-24 years?: CSA ERIC Database

((TI = (adolescen* or (“young man*”) or (“young men”)) or TI = ((“young woman*”) or (“young women”) or (Young adult*”)) or TI = ((“young person*”) or (“young people*”) or teen*) or AB = (adolescen* or (“young man*”) or (“young men”)) or AB = ((“young woman*”) or (“young women”) or (Young adult*”)) or AB = ((“young person*”) or (“young people*”) or teen*)) or (DE = (“youth” or “adolescents” or “early adolescents” or “late adolescents” or “preadolescents”))) and(((TI = ((“positive youth development “) or (“youth development”) or (“youth program*”)) or TI = ((“youth club*”) or (“youth work”) or (“youth opportunit*”)) or TI = ((“extended school*”) or (“civic engagement”) or (“positive peer culture”)) or TI = ((“informal learning”) or multicomponent or (“multi-component “)) or TI = ((“multi component”) or multidimensional or (“multi-dimensional “)) or TI = ((“multi dimensional”) or empower* or asset*) or TI = (thriv* or (“positive development”) or resilienc*) or TI = ((“positive activity”) or (“positive activities”) or experiential) or TI = ((“community based”) or “community-based”)) or(AB = ((“positive youth development “) or (“youth development”) or (“youth program*”)) or AB = ((“youth club*”) or (“youth work”) or (“youth opportunit*”)) or AB = ((“extended school*”) or (“civic engagement”) or (“positive peer culture”)) or AB = ((“informal learning”) or multicomponent or (“multi-component “)) or AB = ((“multi component”) or multidimensional or (“multi-dimensional “)) or AB = ((“multi dimensional”) or empower* or asset*) or AB = (thriv* or (“positive development”) or resilienc*) or AB = ((“positive activity”) or (“positive activities”) or experiential) or AB = ((“community based”) or “community-based”))) or (DE=”community education”))

Detailed guidance for finding effectiveness studies is available from the Campbell Collaboration (Kugley et al. 2015 ). Guidance for finding a broader range of studies has been produced by the EPPI-Centre (Brunton et al. 2017a ).

3.4 The Study Selection Process

Studies identified by the search are subject to a process of checking (sometimes referred to as screening) to ensure they meet the selection criteria. This is usually done in two stages whereby titles and abstracts are checked first to determine whether the study is likely to be relevant and then a full copy of the paper is acquired to complete the screening exercise. The process of finding studies is not efficient. Searching bibliographic databases, for example, leads to many irrelevant studies being found which then have to be checked manually one by one to find the few relevant studies. There is increasing use of specialised software to support and in some cases, automate the selection process. Text mining, for example, can assist in selecting studies for a review (Brunton et al. 2017b ). A typical text mining or machine learning process might involve humans undertaking some screening, the results of which are used to train the computer software to learn the difference between included and excluded studies and thus be able to indicate which of the remaining studies are more likely to be relevant. Such automated support may result in some errors in selection, but this may be less than the human error in manual selection (O’Mara-Eves et al. 2015 ).

3.5 Coding Studies

Once relevant studies have been selected, reviewers need to systematically identify and record the information from the study that will be used to answer the review question. This information includes the characteristics of the studies, including details of the participants and contexts. The coding describes: (i) details of the studies to enable mapping of what research has been undertaken; (ii) how the research was undertaken to allow assessment of the quality and relevance of the studies in addressing the review question; (iii) the results of each study so that these can be synthesised to answer the review question.

The information is usually coded into a data collection system using some kind of technology that facilitates information storage and analysis (Brunton et al. 2017b ) such as the EPPI-Centre’s bespoke systematic review software EPPI Reviewer. Footnote 3 Decisions about which information to record will be made by the review team based on the review question and conceptual framework. For example, a systematic review about the relationship between school size and student outcomes collected data from the primary studies about each schools funding, students, teachers and school organisational structure as well as about the research methods used in the study (Newman et al. 2006 ). The information coded about the methods used in the research will vary depending on the type of research included and the approach that will be used to assess the quality and relevance of the studies (see the next section for further discussion of this point).

Similarly, the information recorded as ‘results’ of the individual studies will vary depending on the type of research that has been included and the approach to synthesis that will be used. Studies investigating the impact of educational interventions using statistical meta-analysis as a synthesis technique will require all of the data necessary to calculate effect sizes to be recorded from each study (see the section on synthesis below for further detail on this point). However, even in this type of study there will be multiple data that can be considered to be ‘results’ and so which data needs to be recorded from studies will need to be carefully specified so that recording is consistent across studies

3.6 Appraising the Quality of Studies

Methods are reinvented every time they are used to accommodate the real world of research practice (Sandelowski et al. 2012 ). The researcher undertaking a primary research study has attempted to design and execute a study that addresses the research question as rigorously as possible within the parameters of their resources, understanding, and context. Given the complexity of this task, the contested views about research methods and the inconsistency of research terminology, reviewers will need to make their own judgements about the quality of the any individual piece of research included in their review. From this perspective, it is evident that using a simple criteria, such as ‘published in a peer reviewed journal’ as a sole indicator of quality, is not likely to be an adequate basis for considering the quality and relevance of a study for a particular systematic review.

In the context of systematic reviews this assessment of quality is often referred to as Critical Appraisal (Petticrew and Roberts 2005 ). There is considerable variation in what is done during critical appraisal: which dimensions of study design and methods are considered; the particular issues that are considered under each dimension; the criteria used to make judgements about these issues and the cut off points used for these criteria (Oancea and Furlong 2007 ). There is also variation in whether the quality assessment judgement is used for excluding studies or weighting them in analysis and when in the process judgements are made.

There are broadly three elements that are considered in critical appraisal: the appropriateness of the study design in the context of the review question, the quality of the execution of the study methods and the study’s relevance to the review question (Gough 2007 ). Distinguishing study design from execution recognises that whilst a particular design may be viewed as more appropriate for a study it also needs to be well executed to achieve the rigour or trustworthiness attributed to the design. Study relevance is achieved by the review selection criteria but assessing the degree of relevance recognises that some studies may be less relevant than others due to differences in, for example, the characteristics of the settings or the ways that variables are measured.

The assessment of study quality is a contested and much debated issue in all research fields. Many published scales are available for assessing study quality. Each incorporates criteria relevant to the research design being evaluated. Quality scales for studies investigating the impact of interventions using (quasi) experimental research designs tend to emphasis establishing descriptive causality through minimising the effects of bias (for detailed discussion of issues associated with assessing study quality in this tradition see Waddington et al. 2017 ). Quality scales for appraising qualitative research tend to focus on the extent to which the study is authentic in reflecting on the meaning of the data (for detailed discussion of the issues associated with assessing study quality in this tradition see Carroll and Booth 2015 ).

3.7 Synthesis

A synthesis is more than a list of findings from the included studies. It is an attempt to integrate the information from the individual studies to produce a ‘better’ answer to the review question than is provided by the individual studies. Each stage of the review contributes toward the synthesis and so decisions made in earlier stages of the review shape the possibilities for synthesis. All types of synthesis involve some kind of data transformation that is achieved through common analytic steps: searching for patterns in data; Checking the quality of the synthesis; Integrating data to answer the review question (Thomas et al. 2012 ). The techniques used to achieve these vary for different types of synthesis and may appear more or less evident as distinct steps.

Statistical meta-analysis is an aggregative synthesis approach in which the outcome results from individual studies are transformed into a standardized, scale free, common metric and combined to produce a single pooled weighted estimate of effect size and direction. There are a number of different metrics of effect size, selection of which is principally determined by the structure of outcome data in the primary studies as either continuous or dichotomous. Outcome data with a dichotomous structure can be transformed into Odds Ratios (OR), Absolute Risk Ratios (ARR) or Relative Risk Ratios (RRR) (for detailed discussion of dichotomous outcome effect sizes see Altman 1991 ). More commonly seen in education research, outcome data with a continuous structure can be translated into Standardised Mean Differences (SMD) (Fitz-Gibbon 1984 ). At its most straightforward effect size calculation is simple arithmetic. However given the variety of analysis methods used and the inconsistency of reporting in primary studies it is also possible to calculate effect sizes using more complex transformation formulae (for detailed instructions on calculating effect sizes from a wide variety of data presentations see Lipsey and Wilson 2000 ).

The combination of individual effect sizes uses statistical procedures in which weighting is given to the effect sizes from the individual studies based on different assumptions about the causes of variance and this requires the use of statistical software. Statistical measures of heterogeneity produced as part of the meta-analysis are used to both explore patterns in the data and to assess the quality of the synthesis (Thomas et al. 2017a ).

In configurative synthesis the different kinds of text about individual studies and their results are meshed and linked to produce patterns in the data, explore different configurations of the data and to produce new synthetic accounts of the phenomena under investigation. The results from the individual studies are translated into and across each other, searching for areas of commonality and refutation. The specific techniques used are derived from the techniques used in primary research in this tradition. They include reading and re-reading, descriptive and analytical coding, the development of themes, constant comparison, negative case analysis and iteration with theory (Thomas et al. 2017b ).

4 Variation in Review Structures

All research requires time and resources and systematic reviews are no exception. There is always concern to use resources as efficiently as possible. For these reasons there is a continuing interest in how reviews can be carried out more quickly using fewer resources. A key issue is the basis for considering a review to be systematic. Any definitions are clearly open to interpretation. Any review can be argued to be insufficiently rigorous and explicit in method in any part of the review process. To assist reviewers in being rigorous, reporting standards and appraisal tools are being developed to assess what is required in different types of review (Lockwood and Geum Oh 2017 ) but these are also the subject of debate and disagreement.

In addition to the term ‘systematic review’ other terms are used to denote the outputs of systematic review processes. Some use the term ‘scoping review’ for a quick review that does not follow a fully systematic process. This term is also used by others (for example, Arksey and O’Malley 2005 ) to denote ‘systematic maps’ that describe the nature of a research field rather than synthesise findings. A ‘quick review’ type of scoping review may also be used as preliminary work to inform a fuller systematic review. Another term used is ‘rapid evidence assessment’. This term is usually used when systematic review needs to be undertaken quickly and in order to do this the methods of review are employed in a more minimal than usual way. For example, by more limited searching. Where such ‘shortcuts’ are taken there may be some loss of rigour, breadth and/or depth (Abrami et al. 2010 ; Thomas et al. 2013 ).

Another development has seen the emergence of the concept of ‘living reviews’, which do not have a fixed end point but are updated as new relevant primary studies are produced. Many review teams hope that their review will be updated over time, but what is different about living reviews is that it is built into the system from the start as an on-going developmental process. This means that the distribution of review effort is quite different to a standard systematic review, being a continuous lower-level effort spread over a longer time period, rather than the shorter bursts of intensive effort that characterise a review with periodic updates (Elliott et al. 2014 ).

4.1 Systematic Maps and Syntheses

One potentially useful aspect of reviewing the literature systematically is that it is possible to gain an understanding of the breadth, purpose and extent of research activity about a phenomenon. Reviewers can be more informed about how research on the phenomenon has been constructed and focused. This type of reviewing is known as ‘mapping’ (see for example, Peersman 1996 ; Gough et al. 2003 ). The aspects of the studies that are described in a map will depend on what is of most interest to those undertaking the review. This might include information such as topic focus, conceptual approach, method, aims, authors, location and context. The boundaries and purposes of a map are determined by decisions made regarding the breadth and depth of the review, which are informed by and reflected in the review question and selection criteria.

Maps can also be a useful stage in a systematic review where study findings are synthesised as well. Most synthesis reviews implicitly or explicitly include some sort of map in that they describe the nature of the relevant studies that they have identified. An explicit map is likely to be more detailed and can be used to inform the synthesis stage of a review. It can provide more information on the individual and grouped studies and thus also provide insights to help inform choices about the focus and strategy to be used in a subsequent synthesis.

4.2 Mixed Methods, Mixed Research Synthesis Reviews

Where studies included in a review consist of more than one type of study design, there may also be different types of data. These different types of studies and data can be analysed together in an integrated design or segregated and analysed separately (Sandelowski et al. 2012 ). In a segregated design, two or more separate sub-reviews are undertaken simultaneously to address different aspects of the same review question and are then compared with one another.

Such ‘mixed methods’ and ‘multiple component’ reviews are usually necessary when there are multiple layers of review question or when one study design alone would be insufficient to answer the question(s) adequately. The reviews are usually required, to have both breadth and depth. In doing so they can investigate a greater extent of the research problem than would be the case in a more focussed single method review. As they are major undertakings, containing what would normally be considered the work of multiple systematic reviews, they are demanding of time and resources and cannot be conducted quickly.

4.3 Reviews of Reviews

Systematic reviews of primary research are secondary levels of research analysis. A review of reviews (sometimes called ‘overviews’ or ‘umbrella’ reviews) is a tertiary level of analysis. It is a systematic map and/or synthesis of previous reviews. The ‘data’ for reviews of reviews are previous reviews rather than primary research studies (see for example Newman et al. ( 2018 ). Some review of reviews use previous reviews to combine both primary research data and synthesis data. It is also possible to have hybrid review models consisting of a review of reviews and then new systematic reviews of primary studies to fill in gaps in coverage where there is not an existing review (Caird et al. 2015 ). Reviews of reviews can be an efficient method for examining previous research. However, this approach is still comparatively novel and questions remain about the appropriate methodology. For example, care is required when assessing the way in which the source systematic reviews identified and selected data for inclusion, assessed study quality and to assess the overlap between the individual reviews (Aromataris et al. 2015 ).

5 Other Types of Research Based Review Structures

This chapter so far has presented a process or method that is shared by many different approaches within the family of systematic review approaches, notwithstanding differences in review question and types of study that are included as evidence. This is a helpful heuristic device for designing and reading systematic reviews. However, it is the case that there are some review approaches that also claim to use a research based review approach but that do not claim to be systematic reviews and or do not conform with the description of processes that we have given above at all or in part at least.

5.1 Realist Synthesis Reviews

Realist synthesis is a member of the theory-based school of evaluation (Pawson 2002 ). This means that it is underpinned by a ‘generative’ understanding of causation, which holds that, to infer a causal outcome/relationship between an intervention (e.g. a training programme) and an outcome (O) of interest (e.g. unemployment), one needs to understand the underlying mechanisms (M) that connect them and the context (C) in which the relationship occurs (e.g. the characteristics of both the subjects and the programme locality). The interest of this approach (and also of other theory driven reviews) is not simply which interventions work, but which mechanisms work in which context. Rather than identifying replications of the same intervention, the reviews adopt an investigative stance and identify different contexts within which the same underlying mechanism is operating.

Realist synthesis is concerned with hypothesising, testing and refining such context-mechanism-outcome (CMO) configurations. Based on the premise that programmes work in limited circumstances, the discovery of these conditions becomes the main task of realist synthesis. The overall intention is to first create an abstract model (based on the CMO configurations) of how and why programmes work and then to test this empirically against the research evidence. Thus, the unit of analysis in a realist synthesis is the programme mechanism, and this mechanism is the basis of the search. This means that a realist synthesis aims to identify different situations in which the same programme mechanism has been attempted. Integrative Reviewing, which is aligned to the Critical Realist tradition, follows a similar approach and methods (Jones-Devitt et al. 2017 ).

5.2 Critical Interpretive Synthesis (CIS)

Critical Interpretive Synthesis (CIS) (Dixon-Woods et al. 2006 ) takes a position that there is an explicit role for the ‘authorial’ (reviewer’s) voice in the review. The approach is derived from a distinctive tradition within qualitative enquiry and draws on some of the tenets of grounded theory in order to support explicitly the process of theory generation. In practice, this is operationalised in its inductive approach to searching and to developing the review question as part of the review process, its rejection of a ‘staged’ approach to reviewing and embracing the concept of theoretical sampling in order to select studies for inclusion. When assessing the quality of studies CIS prioritises relevance and theoretical contribution over research methods. In particular, a critical approach to reading the literature is fundamental in terms of contextualising findings within an analysis of the research traditions or theoretical assumptions of the studies included.

5.3 Meta-Narrative Reviews

Meta-narrative reviews, like critical interpretative synthesis, place centre-stage the importance of understanding the literature critically and understanding differences between research studies as possibly being due to differences between their underlying research traditions (Greenhalgh et al. 2005 ). This means that each piece of research is located (and, when appropriate, aggregated) within its own research tradition and the development of knowledge is traced (configured) through time and across paradigms. Rather than the individual study, the ‘unit of analysis’ is the unfolding ‘storyline’ of a research tradition over time’ (Greenhalgh et al. 2005 ).

6 Conclusions

This chapter has briefly described the methods, application and different perspectives in the family of systematic review approaches. We have emphasized the many ways in which systematic reviews can vary. This variation links to different research aims and review questions. But also to the different assumptions made by reviewers. These assumptions derive from different understandings of research paradigms and methods and from the personal, political perspectives they bring to their research practice. Although there are a variety of possible types of systematic reviews, a distinction in the extent that reviews follow an aggregative or configuring synthesis logic is useful for understanding variations in review approaches and methods. It can help clarify the ways in which reviews vary in the nature of their questions, concepts, procedures, inference and impact. Systematic review approaches continue to evolve alongside critical debate about the merits of various review approaches (systematic or otherwise). So there are many ways in which educational researchers can use and engage with systematic review methods to increase knowledge and understanding in the field of education.

https://ies.ed.gov/ncee/wwc/

https://educationendowmentfoundation.org.uk/evidence-summaries/teaching-learning-toolkit

https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2914

Abrami, P. C. Borokhovski, E. Bernard, R. M. Wade, CA. Tamim, R. Persson, T. Bethel, E. C. Hanz, K. & Surkes, M. A. (2010). Issues in conducting and disseminating brief reviews of evidence. Evidence & Policy , 6 (3), 371–389.

Google Scholar  

Altman, D.G. (1991) Practical statistics for medical research . London: Chapman and Hall.

Arksey, H. & O’Malley, L. (2005). Scoping studies: towards a methodological framework, International Journal of Social Research Methodology , 8 (1), 19–32,

Article   Google Scholar  

Aromataris, E. Fernandez, R. Godfrey, C. Holly, C. Khalil, H. Tungpunkom, P. (2015). Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare , 13 .

Barnett-Page, E. & Thomas, J. (2009). Methods for the synthesis of qualitative research: A critical review, BMC Medical Research Methodology, 9 (59), https://doi.org/10.1186/1471-2288-9-59 .

Brunton, G., Stansfield, C., Caird, J. & Thomas, J. (2017a). Finding relevant studies. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 93–132). London: Sage.

Brunton, J., Graziosi, S., & Thomas, J. (2017b). Tools and techniques for information management. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 154–180), London: Sage.

Carroll, C. & Booth, A. (2015). Quality assessment of qualitative evidence for systematic review and synthesis: Is it meaningful, and if so, how should it be performed? Research Synthesis Methods 6 (2), 149–154.

Caird, J. Sutcliffe, K. Kwan, I. Dickson, K. & Thomas, J. (2015). Mediating policy-relevant evidence at speed: are systematic reviews of systematic reviews a useful approach? Evidence & Policy, 11 (1), 81–97.

Chow, J. & Eckholm, E. (2018). Do published studies yield larger effect sizes than unpublished studies in education and special education? A meta-review. Educational Psychology Review 30 (3), 727–744.

Dickson, K., Vigurs, C. & Newman, M. (2013). Youth work a systematic map of the literature . Dublin: Dept of Government Affairs.

Dixon-Woods, M., Cavers, D., Agarwa, S., Annandale, E. Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., Riley R., & Sutton, A. J. (2006). Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Medical Research Methodology 6:35 https://doi.org/10.1186/1471-2288-6-35 .

Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C. & Gruen, R. L. (2014). Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Medicine, 11 (2): e1001603. http://doi.org/10.1371/journal.pmed.1001603 .

Fitz-Gibbon, C.T. (1984) Meta-analysis: an explication. British Educational Research Journal , 10 (2), 135–144.

Gough, D. (2007). Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Research Papers in Education , 22 (2), 213–228.

Gough, D., Thomas, J. & Oliver, S. (2012). Clarifying differences between review designs and methods. Systematic Reviews, 1( 28).

Gough, D., Kiwan, D., Sutcliffe, K., Simpson, D. & Houghton, N. (2003). A systematic map and synthesis review of the effectiveness of personal development planning for improving student learning. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

Gough, D., Oliver, S. & Thomas, J. (2017). Introducing systematic reviews. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 1–18). London: Sage.

Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O. & Peacock, R. (2005). Storylines of research in diffusion of innovation: A meta-narrative approach to systematic review. Social Science & Medicine, 61 (2), 417–430.

Hargreaves, D. (1996). Teaching as a research based profession: possibilities and prospects . Teacher Training Agency Annual Lecture. Retrieved from https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2082 .

Hart, C. (2018). Doing a literature review: releasing the research imagination . London. SAGE.

Jones-Devitt, S. Austen, L. & Parkin H. J. (2017). Integrative reviewing for exploring complex phenomena. Social Research Update . Issue 66.

Kugley, S., Wade, A., Thomas, J., Mahood, Q., Klint Jørgensen, A. M., Hammerstrøm, K., & Sathe, N. (2015). Searching for studies: A guide to information retrieval for Campbell Systematic Reviews . Campbell Method Guides 2016:1 ( http://www.campbellcollaboration.org/images/Campbell_Methods_Guides_Information_Retrieval.pdf ).

Lipsey, M.W., & Wilson, D. B. (2000). Practical meta-analysis. Thousand Oaks: Sage.

Lockwood, C. & Geum Oh, E. (2017). Systematic reviews: guidelines, tools and checklists for authors. Nursing & Health Sciences, 19, 273–277.

Nelson, J. & Campbell, C. (2017). Evidence-informed practice in education: meanings and applications. Educational Research, 59 (2), 127–135.

Newman, M., Garrett, Z., Elbourne, D., Bradley, S., Nodenc, P., Taylor, J. & West, A. (2006). Does secondary school size make a difference? A systematic review. Educational Research Review, 1 (1), 41–60.

Newman, M. (2008). High quality randomized experimental research evidence: Necessary but not sufficient for effective education policy. Psychology of Education Review, 32 (2), 14–16.

Newman, M., Reeves, S. & Fletcher, S. (2018). A critical analysis of evidence about the impacts of faculty development in systematic reviews: a systematic rapid evidence assessment. Journal of Continuing Education in the Health Professions, 38 (2), 137–144.

Noblit, G. W. & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park, CA: Sage.

Book   Google Scholar  

Oancea, A. & Furlong, J. (2007). Expressions of excellence and the assessment of applied and practice‐based research. Research Papers in Education 22 .

O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015). Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews 4 (1): 5. https://doi.org/10.1186/2046-4053-4-5 .

Pawson, R. (2002). Evidence-based policy: the promise of “Realist Synthesis”. Evaluation, 8 (3), 340–358.

Peersman, G. (1996). A descriptive mapping of health promotion studies in young people, EPPI Research Report. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

Petticrew, M. & Roberts, H. (2005). Systematic reviews in the social sciences: a practical guide . London: Wiley.

Sandelowski, M., Voils, C. I., Leeman, J. & Crandell, J. L. (2012). Mapping the mixed methods-mixed research synthesis terrain. Journal of Mixed Methods Research, 6 (4), 317–331.

Smyth, R. (2004). Exploring the usefulness of a conceptual framework as a research tool: A researcher’s reflections. Issues in Educational Research, 14 .

Thomas, J., Harden, A. & Newman, M. (2012). Synthesis: combining results systematically and appropriately. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (pp. 66–82). London: Sage.

Thomas, J., Newman, M. & Oliver, S. (2013). Rapid evidence assessments of research to inform social policy: taking stock and moving forward. Evidence and Policy, 9 (1), 5–27.

Thomas, J., O’Mara-Eves, A., Kneale, D. & Shemilt, I. (2017a). Synthesis methods for combining and configuring quantitative data. In D. Gough, S. Oliver & J. Thomas (Eds.), An Introduction to Systematic Reviews (2nd edition, pp. 211–250). London: Sage.

Thomas, J., O’Mara-Eves, A., Harden, A. & Newman, M. (2017b). Synthesis methods for combining and configuring textual or mixed methods data. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (2nd edition, pp. 181–211), London: Sage.

Tsafnat, G., Dunn, A. G., Glasziou, P. & Coiera, E. (2013). The automation of systematic reviews. British Medical Journal, 345 (7891), doi.org/10.1136/bmj.f139 .

Torgerson, C. (2003). Systematic reviews . London. Continuum.

Waddington, H., Aloe, A. M., Becker, B. J., Djimeu, E. W., Hombrados, J. G., Tugwell, P., Wells, G. & Reeves, B. (2017). Quasi-experimental study designs series—paper 6: risk of bias assessment. Journal of Clinical Epidemiology, 89 , 43–52.

Download references

Author information

Authors and affiliations.

Institute of Education, University College London, England, UK

Mark Newman & David Gough

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mark Newman .

Editor information

Editors and affiliations.

Oldenburg, Germany

Olaf Zawacki-Richter

Essen, Germany

Michael Kerres

Svenja Bedenlier

Melissa Bond

Katja Buntins

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2020 The Author(s)

About this chapter

Newman, M., Gough, D. (2020). Systematic Reviews in Educational Research: Methodology, Perspectives and Application. In: Zawacki-Richter, O., Kerres, M., Bedenlier, S., Bond, M., Buntins, K. (eds) Systematic Reviews in Educational Research. Springer VS, Wiesbaden. https://doi.org/10.1007/978-3-658-27602-7_1

Download citation

DOI : https://doi.org/10.1007/978-3-658-27602-7_1

Published : 22 November 2019

Publisher Name : Springer VS, Wiesbaden

Print ISBN : 978-3-658-27601-0

Online ISBN : 978-3-658-27602-7

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Subject List
• Take a Tour
• For Authors
• Subscriber Services
• Publications
• African American Studies
• African Studies
• American Literature
• Anthropology
• Architecture Planning and Preservation
• Art History
• Atlantic History
• Biblical Studies
• British and Irish Literature
• Childhood Studies
• Chinese Studies
• Cinema and Media Studies
• Communication
• Criminology
• Environmental Science
• Evolutionary Biology
• International Law
• International Relations
• Islamic Studies
• Jewish Studies
• Latin American Studies
• Latino Studies
• Linguistics
• Literary and Critical Theory
• Medieval Studies
• Military History
• Political Science
• Public Health
• Renaissance and Reformation
• Social Work
• Urban Studies
• Victorian Literature
• Browse All Subjects

How to Subscribe

• Free Trials

In This Article Expand or collapse the "in this article" section Literature Reviews

Introduction, what is a literature review.

• Literature Reviews for Thesis or Dissertation
• Stand-alone and Systemic Reviews
• Purposes of a Literature Review
• Texts on Conducting a Literature Review
• Identifying the Research Topic
• The Persuasive Argument
• Searching the Literature
• Creating a Synthesis
• Critiquing the Literature
• Building the Case for the Literature Review Document
• Presenting the Literature Review

Related Articles Expand or collapse the "related articles" section about

About related articles close popup.

Lorem Ipsum Sit Dolor Amet

Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aliquam ligula odio, euismod ut aliquam et, vestibulum nec risus. Nulla viverra, arcu et iaculis consequat, justo diam ornare tellus, semper ultrices tellus nunc eu tellus.

• Higher Education Research
• Meta-Analysis and Research Synthesis in Education
• Methodologies for Conducting Education Research
• Mixed Methods Research
• Philosophy of Education
• Politics of Education
• Qualitative Data Analysis Techniques

Other Subject Areas

Forthcoming articles expand or collapse the "forthcoming articles" section.

• Cyber Safety in Schools
• Girls' Education in the Developing World
• History of Education in Europe
• Find more forthcoming articles...
• Export Citations
• Share This Facebook LinkedIn Twitter

Literature Reviews by Lawrence A. Machi , Brenda T. McEvoy LAST REVIEWED: 27 October 2016 LAST MODIFIED: 27 October 2016 DOI: 10.1093/obo/9780199756810-0169

Literature reviews play a foundational role in the development and execution of a research project. They provide access to the academic conversation surrounding the topic of the proposed study. By engaging in this scholarly exercise, the researcher is able to learn and to share knowledge about the topic. The literature review acts as the springboard for new research, in that it lays out a logically argued case, founded on a comprehensive understanding of the current state of knowledge about the topic. The case produced provides the justification for the research question or problem of a proposed study, and the methodological scheme best suited to conduct the research. It can also be a research project in itself, arguing policy or practice implementation, based on a comprehensive analysis of the research in a field. The term literature review can refer to the output or the product of a review. It can also refer to the process of Conducting a Literature Review . Novice researchers, when attempting their first research projects, tend to ask two questions: What is a Literature Review? How do you do one? While this annotated bibliography is neither definitive nor exhaustive in its treatment of the subject, it is designed to provide a beginning researcher, who is pursuing an academic degree, an entry point for answering the two previous questions. The article is divided into two parts. The first four sections of the article provide a general overview of the topic. They address definitions, types, purposes, and processes for doing a literature review. The second part presents the process and procedures for doing a literature review. Arranged in a sequential fashion, the remaining eight sections provide references addressing each step of the literature review process. References included in this article were selected based on their ability to assist the beginning researcher. Additionally, the authors attempted to include texts from various disciplines in social science to present various points of view on the subject.

Novice researchers often have a misguided perception of how to do a literature review and what the document should contain. Literature reviews are not narrative annotated bibliographies nor book reports (see Bruce 1994 ). Their form, function, and outcomes vary, due to how they depend on the research question, the standards and criteria of the academic discipline, and the orthodoxies of the research community charged with the research. The term literature review can refer to the process of doing a review as well as the product resulting from conducting a review. The product resulting from reviewing the literature is the concern of this section. Literature reviews for research studies at the master’s and doctoral levels have various definitions. Machi and McEvoy 2016 presents a general definition of a literature review. Lambert 2012 defines a literature review as a critical analysis of what is known about the study topic, the themes related to it, and the various perspectives expressed regarding the topic. Fink 2010 defines a literature review as a systematic review of existing body of data that identifies, evaluates, and synthesizes for explicit presentation. Jesson, et al. 2011 defines the literature review as a critical description and appraisal of a topic. Hart 1998 sees the literature review as producing two products: the presentation of information, ideas, data, and evidence to express viewpoints on the nature of the topic, as well as how it is to be investigated. When considering literature reviews beyond the novice level, Ridley 2012 defines and differentiates the systematic review from literature reviews associated with primary research conducted in academic degree programs of study, including stand-alone literature reviews. Cooper 1998 states the product of literature review is dependent on the research study’s goal and focus, and defines synthesis reviews as literature reviews that seek to summarize and draw conclusions from past empirical research to determine what issues have yet to be resolved. Theoretical reviews compare and contrast the predictive ability of theories that explain the phenomenon, arguing which theory holds the most validity in describing the nature of that phenomenon. Grant and Booth 2009 identified fourteen types of reviews used in both degree granting and advanced research projects, describing their attributes and methodologies.

Bruce, Christine Susan. 1994. Research students’ early experiences of the dissertation literature review. Studies in Higher Education 19.2: 217–229.

DOI: 10.1080/03075079412331382057

A phenomenological analysis was conducted with forty-one neophyte research scholars. The responses to the questions, “What do you mean when you use the words literature review?” and “What is the meaning of a literature review for your research?” identified six concepts. The results conclude that doing a literature review is a problem area for students.

Cooper, Harris. 1998. Synthesizing research . Vol. 2. 3d ed. Thousand Oaks, CA: SAGE.

The introductory chapter of this text provides a cogent explanation of Cooper’s understanding of literature reviews. Chapter 4 presents a comprehensive discussion of the synthesis review. Chapter 5 discusses meta-analysis and depth.

Fink, Arlene. 2010. Conducting research literature reviews: From the Internet to paper . 3d ed. Los Angeles: SAGE.

The first chapter of this text (pp. 1–16) provides a short but clear discussion of what a literature review is in reference to its application to a broad range of social sciences disciplines and their related professions.

Grant, Maria J., and Andrew Booth. 2009. A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal 26.2: 91–108. Print.

DOI: 10.1111/j.1471-1842.2009.00848.x

This article reports a scoping review that was conducted using the “Search, Appraisal, Synthesis, and Analysis” (SALSA) framework. Fourteen literature review types and associated methodology make up the resulting typology. Each type is described by its key characteristics and analyzed for its strengths and weaknesses.

Hart, Chris. 1998. Doing a literature review: Releasing the social science research imagination . London: SAGE.

Chapter 1 of this text explains Hart’s definition of a literature review. Additionally, it describes the roles of the literature review, the skills of a literature reviewer, and the research context for a literature review. Of note is Hart’s discussion of the literature review requirements for master’s degree and doctoral degree work.

Jesson, Jill, Lydia Matheson, and Fiona M. Lacey. 2011. Doing your literature review: Traditional and systematic techniques . Los Angeles: SAGE.

Chapter 1: “Preliminaries” provides definitions of traditional and systematic reviews. It discusses the differences between them. Chapter 5 is dedicated to explaining the traditional review, while Chapter 7 explains the systematic review. Chapter 8 provides a detailed description of meta-analysis.

Lambert, Mike. 2012. A beginner’s guide to doing your education research project . Los Angeles: SAGE.

Chapter 6 (pp. 79–100) presents a thumbnail sketch for doing a literature review.

Machi, Lawrence A., and Brenda T. McEvoy. 2016. The literature review: Six steps to success . 3d ed. Thousand Oaks, CA: Corwin.

The introduction of this text differentiates between a simple and an advanced review and concisely defines a literature review.

Ridley, Diana. 2012. The literature review: A step-by-step guide for students . 2d ed. Sage Study Skills. London: SAGE.

In the introductory chapter, Ridley reviews many definitions of the literature review, literature reviews at the master’s and doctoral level, and placement of literature reviews within the thesis or dissertation document. She also defines and differentiates literature reviews produced for degree-affiliated research from the more advanced systematic review projects.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login .

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here .

• About Education »
• Meet the Editorial Board »
• Academic Achievement
• Academic Audit for Universities
• Academic Freedom and Tenure in the United States
• Action Research in Education
• Adjuncts in Higher Education in the United States
• Administrator Preparation
• Adolescence
• Advanced Placement and International Baccalaureate Courses
• Advocacy and Activism in Early Childhood
• African American Racial Identity and Learning
• Alaska Native Education
• Alternative Certification Programs for Educators
• Alternative Schools
• American Indian Education
• Animals in Environmental Education
• Art Education
• Artificial Intelligence and Learning
• Assessing School Leader Effectiveness
• Assessment, Behavioral
• Assessment, Educational
• Assessment in Early Childhood Education
• Assistive Technology
• Augmented Reality in Education
• Beginning-Teacher Induction
• Bilingual Education and Bilingualism
• Black Undergraduate Women: Critical Race and Gender Perspe...
• Black Women in Academia
• Blended Learning
• Case Study in Education Research
• Changing Professional and Academic Identities
• Character Education
• Children’s and Young Adult Literature
• Children's Beliefs about Intelligence
• Children's Rights in Early Childhood Education
• Citizenship Education
• Civic and Social Engagement of Higher Education
• Classroom Learning Environments: Assessing and Investigati...
• Classroom Management
• Coherent Instructional Systems at the School and School Sy...
• College Admissions in the United States
• College Athletics in the United States
• Community Relations
• Comparative Education
• Computer-Assisted Language Learning
• Computer-Based Testing
• Conceptualizing, Measuring, and Evaluating Improvement Net...
• Continuous Improvement and "High Leverage" Educational Pro...
• Counseling in Schools
• Critical Approaches to Gender in Higher Education
• Critical Perspectives on Educational Innovation and Improv...
• Critical Race Theory
• Crossborder and Transnational Higher Education
• Cross-National Research on Continuous Improvement
• Cross-Sector Research on Continuous Learning and Improveme...
• Cultural Diversity in Early Childhood Education
• Culturally Responsive Leadership
• Culturally Responsive Pedagogies
• Culturally Responsive Teacher Education in the United Stat...
• Curriculum Design
• Data Collection in Educational Research
• Data-driven Decision Making in the United States
• Deaf Education
• Desegregation and Integration
• Design Thinking and the Learning Sciences: Theoretical, Pr...
• Development, Moral
• Dialogic Pedagogy
• Digital Age Teacher, The
• Digital Citizenship
• Digital Divides
• Disabilities
• Distance Learning
• Distributed Leadership
• Doctoral Education and Training
• Early Childhood Education and Care (ECEC) in Denmark
• Early Childhood Education and Development in Mexico
• Early Childhood Education in Aotearoa New Zealand
• Early Childhood Education in Australia
• Early Childhood Education in China
• Early Childhood Education in Europe
• Early Childhood Education in Sub-Saharan Africa
• Early Childhood Education in Sweden
• Early Childhood Education Pedagogy
• Early Childhood Education Policy
• Early Childhood Education, The Arts in
• Early Childhood Mathematics
• Early Childhood Science
• Early Childhood Teacher Education
• Early Childhood Teachers in Aotearoa New Zealand
• Early Years Professionalism and Professionalization Polici...
• Economics of Education
• Education For Children with Autism
• Education for Sustainable Development
• Education Leadership, Empirical Perspectives in
• Education of Native Hawaiian Students
• Education Reform and School Change
• Educational Research Approaches: A Comparison
• Educational Statistics for Longitudinal Research
• Educator Partnerships with Parents and Families with a Foc...
• Emotional and Affective Issues in Environmental and Sustai...
• Emotional and Behavioral Disorders
• English as an International Language for Academic Publishi...
• Environmental and Science Education: Overlaps and Issues
• Environmental Education
• Environmental Education in Brazil
• Epistemic Beliefs
• Equity and Improvement: Engaging Communities in Educationa...
• Equity, Ethnicity, Diversity, and Excellence in Education
• Ethical Research with Young Children
• Ethics and Education
• Ethics of Teaching
• Ethnic Studies
• Evidence-Based Communication Assessment and Intervention
• Family and Community Partnerships in Education
• Family Day Care
• Federal Government Programs and Issues
• Feminization of Labor in Academia
• Finance, Education
• Financial Aid
• Formative Assessment
• Future-Focused Education
• Gender and Achievement
• Gender and Alternative Education
• Gender, Power and Politics in the Academy
• Gender-Based Violence on University Campuses
• Gifted Education
• Global Mindedness and Global Citizenship Education
• Global University Rankings
• Governance, Education
• Grounded Theory
• Growth of Effective Mental Health Services in Schools in t...
• Higher Education and Globalization
• Higher Education and the Developing World
• Higher Education Faculty Characteristics and Trends in the...
• Higher Education Finance
• Higher Education Governance
• Higher Education Graduate Outcomes and Destinations
• Higher Education in Africa
• Higher Education in China
• Higher Education in Latin America
• Higher Education in the United States, Historical Evolutio...
• Higher Education, International Issues in
• Higher Education Management
• Higher Education Policy
• Higher Education Student Assessment
• High-stakes Testing
• History of Early Childhood Education in the United States
• History of Education in the United States
• History of Technology Integration in Education
• Homeschooling
• Inclusion in Early Childhood: Difference, Disability, and ...
• Inclusive Education
• Indigenous Education in a Global Context
• Indigenous Learning Environments
• Indigenous Students in Higher Education in the United Stat...
• Infant and Toddler Pedagogy
• Inservice Teacher Education
• Integrating Art across the Curriculum
• Intelligence
• Intensive Interventions for Children and Adolescents with ...
• International Perspectives on Academic Freedom
• Intersectionality and Education
• Knowledge Development in Early Childhood
• Leadership Development, Coaching and Feedback for
• Leadership in Early Childhood Education
• Leadership Training with an Emphasis on the United States
• Learning Analytics in Higher Education
• Learning Difficulties
• Learning, Lifelong
• Learning, Multimedia
• Learning Strategies
• Legal Matters and Education Law
• LGBT Youth in Schools
• Linguistic Diversity
• Linguistically Inclusive Pedagogy
• Literacy Development and Language Acquisition
• Literature Reviews
• Mathematics Identity
• Mathematics Instruction and Interventions for Students wit...
• Mathematics Teacher Education
• Measurement for Improvement in Education
• Measurement in Education in the United States
• Methodological Approaches for Impact Evaluation in Educati...
• Mindfulness, Learning, and Education
• Motherscholars
• Multiliteracies in Early Childhood Education
• Multiple Documents Literacy: Theory, Research, and Applica...
• Multivariate Research Methodology
• Museums, Education, and Curriculum
• Music Education
• Narrative Research in Education
• Native American Studies
• Nonformal and Informal Environmental Education
• Note-Taking
• Numeracy Education
• One-to-One Technology in the K-12 Classroom
• Online Education
• Open Education
• Organizing for Continuous Improvement in Education
• Organizing Schools for the Inclusion of Students with Disa...
• Outdoor Play and Learning
• Outdoor Play and Learning in Early Childhood Education
• Pedagogical Leadership
• Pedagogy of Teacher Education, A
• Performance Objectives and Measurement
• Performance-based Research Assessment in Higher Education
• Performance-based Research Funding
• Phenomenology in Educational Research
• Physical Education
• Podcasts in Education
• Policy Context of United States Educational Innovation and...
• Portable Technology Use in Special Education Programs and ...
• Post-humanism and Environmental Education
• Pre-Service Teacher Education
• Problem Solving
• Productivity and Higher Education
• Professional Development
• Professional Learning Communities
• Program Evaluation
• Programs and Services for Students with Emotional or Behav...
• Psychology Learning and Teaching
• Psychometric Issues in the Assessment of English Language ...
• Qualitative, Quantitative, and Mixed Methods Research Samp...
• Qualitative Research Design
• Quantitative Research Designs in Educational Research
• Queering the English Language Arts (ELA) Writing Classroom
• Race and Affirmative Action in Higher Education
• Reading Education
• Refugee and New Immigrant Learners
• Relational and Developmental Trauma and Schools
• Relational Pedagogies in Early Childhood Education
• Reliability in Educational Assessments
• Religion in Elementary and Secondary Education in the Unit...
• Researcher Development and Skills Training within the Cont...
• Research-Practice Partnerships in Education within the Uni...
• Response to Intervention
• Restorative Practices
• Risky Play in Early Childhood Education
• Scale and Sustainability of Education Innovation and Impro...
• Scaling Up Research-based Educational Practices
• School Accreditation
• School Choice
• School Culture
• School District Budgeting and Financial Management in the ...
• School Improvement through Inclusive Education
• School Reform
• Schools, Private and Independent
• School-Wide Positive Behavior Support
• Science Education
• Secondary to Postsecondary Transition Issues
• Self-Regulated Learning
• Self-Study of Teacher Education Practices
• Service-Learning
• Severe Disabilities
• Single Salary Schedule
• Single-sex Education
• Single-Subject Research Design
• Social Context of Education
• Social Justice
• Social Network Analysis
• Social Pedagogy
• Social Science and Education Research
• Social Studies Education
• Sociology of Education
• Standards-Based Education
• Statistical Assumptions
• Student Access, Equity, and Diversity in Higher Education
• Student Assignment Policy
• Student Engagement in Tertiary Education
• Student Learning, Development, Engagement, and Motivation ...
• Student Participation
• Student Voice in Teacher Development
• Sustainability Education in Early Childhood Education
• Sustainability in Early Childhood Education
• Sustainability in Higher Education
• Teacher Beliefs and Epistemologies
• Teacher Collaboration in School Improvement
• Teacher Evaluation and Teacher Effectiveness
• Teacher Preparation
• Teacher Training and Development
• Teacher Unions and Associations
• Teacher-Student Relationships
• Teaching Critical Thinking
• Technologies, Teaching, and Learning in Higher Education
• Technology Education in Early Childhood
• Technology, Educational
• Technology-based Assessment
• The Bologna Process
• The Regulation of Standards in Higher Education
• Theories of Educational Leadership
• Three Conceptions of Literacy: Media, Narrative, and Gamin...
• Tracking and Detracking
• Traditions of Quality Improvement in Education
• Transformative Learning
• Transitions in Early Childhood Education
• Tribally Controlled Colleges and Universities in the Unite...
• Understanding the Psycho-Social Dimensions of Schools and ...
• University Faculty Roles and Responsibilities in the Unite...
• Using Ethnography in Educational Research
• Value of Higher Education for Students and Other Stakehold...
• Virtual Learning Environments
• Vocational and Technical Education
• Wellness and Well-Being in Education
• Women's and Gender Studies
• Young Children and Spirituality
• Young Children's Learning Dispositions
• Young Children's Working Theories
• Privacy Policy
• Cookie Policy
• Legal Notice
• Accessibility

Powered by:

• [185.66.14.133]
• 185.66.14.133

• Library Search
• View Library Account
• The Claremont Colleges Library
• Research Guides

Educational Studies

• Conducting a Literature Review
• Getting Started With Research
• Finding Scholarly Articles
• Finding Books & Papers
• Finding Education Data & Testing Resources

What is a Literature Review?

Basics of a literature review, types of literature reviews.

• Citing Your Information (Attribution)

A Literature Review is a systematic and comprehensive analysis of books, scholarly articles and other sources relevant to a specific topic providing a base of knowledge on a topic. Literature reviews are designed to identify and critique the existing literature on a topic to justify your research by exposing gaps in current research .  This investigation should provide a description, summary, and critical evaluation of works related to the research problem and should also add to the overall knowledge of the topic as well as demonstrating how your research will fit within a larger field of study.  A literature review should offer critical analysis of the current research on a topic and that analysis should direct your research objective. This should not be confused with a book review or an annotated bibliography both research tools but very different in purpose and scope.  A Literature Review can be a stand alone element or part of a larger end product, know your assignment.  Key to a good Literature Review is to document your process. For more information see:

Planning a Literature Review .

There are many different ways to organize your references in a literature review, but most reviews contain certain basic elements.

• Objective of the literature review - Clearly describe the purpose of the paper and state your objectives in completing the literature review.
•   Overview of the subject, issue or theory under consideration – Give an overview of your research topic and what prompted it.
• Categorization of sources – Grouping your research either historic, chronologically or thematically
• Organization of Subtopics – Subtopics should be grouped and presented in a logical order starting with the most prominent or significant and moving to the least significant
• Discussion – Provide analysis of both the uniqueness of each source and its similarities with other source
• Conclusion   - Summary of your analysis and evaluation of the reviewed works and how it is related to its parent discipline, scientific endeavor or profession

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147.

All content is from The Literature Review created by Dr. Robert Larabee USC

• << Previous: Finding Education Data & Testing Resources
• Next: Citing Your Information (Attribution) >>
• : Aug 5, 2024 12:42 PM
• URL: https://libguides.libraries.claremont.edu/education

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• CBE Life Sci Educ
• v.21(3); Fall 2022

Literature Reviews, Theoretical Frameworks, and Conceptual Frameworks: An Introduction for New Biology Education Researchers

Julie a. luft.

† Department of Mathematics, Social Studies, and Science Education, Mary Frances Early College of Education, University of Georgia, Athens, GA 30602-7124

Sophia Jeong

‡ Department of Teaching & Learning, College of Education & Human Ecology, Ohio State University, Columbus, OH 43210

Robert Idsardi

§ Department of Biology, Eastern Washington University, Cheney, WA 99004

Grant Gardner

∥ Department of Biology, Middle Tennessee State University, Murfreesboro, TN 37132

Associated Data

To frame their work, biology education researchers need to consider the role of literature reviews, theoretical frameworks, and conceptual frameworks as critical elements of the research and writing process. However, these elements can be confusing for scholars new to education research. This Research Methods article is designed to provide an overview of each of these elements and delineate the purpose of each in the educational research process. We describe what biology education researchers should consider as they conduct literature reviews, identify theoretical frameworks, and construct conceptual frameworks. Clarifying these different components of educational research studies can be helpful to new biology education researchers and the biology education research community at large in situating their work in the broader scholarly literature.

INTRODUCTION

Discipline-based education research (DBER) involves the purposeful and situated study of teaching and learning in specific disciplinary areas ( Singer et al. , 2012 ). Studies in DBER are guided by research questions that reflect disciplines’ priorities and worldviews. Researchers can use quantitative data, qualitative data, or both to answer these research questions through a variety of methodological traditions. Across all methodologies, there are different methods associated with planning and conducting educational research studies that include the use of surveys, interviews, observations, artifacts, or instruments. Ensuring the coherence of these elements to the discipline’s perspective also involves situating the work in the broader scholarly literature. The tools for doing this include literature reviews, theoretical frameworks, and conceptual frameworks. However, the purpose and function of each of these elements is often confusing to new education researchers. The goal of this article is to introduce new biology education researchers to these three important elements important in DBER scholarship and the broader educational literature.

The first element we discuss is a review of research (literature reviews), which highlights the need for a specific research question, study problem, or topic of investigation. Literature reviews situate the relevance of the study within a topic and a field. The process may seem familiar to science researchers entering DBER fields, but new researchers may still struggle in conducting the review. Booth et al. (2016b) highlight some of the challenges novice education researchers face when conducting a review of literature. They point out that novice researchers struggle in deciding how to focus the review, determining the scope of articles needed in the review, and knowing how to be critical of the articles in the review. Overcoming these challenges (and others) can help novice researchers construct a sound literature review that can inform the design of the study and help ensure the work makes a contribution to the field.

The second and third highlighted elements are theoretical and conceptual frameworks. These guide biology education research (BER) studies, and may be less familiar to science researchers. These elements are important in shaping the construction of new knowledge. Theoretical frameworks offer a way to explain and interpret the studied phenomenon, while conceptual frameworks clarify assumptions about the studied phenomenon. Despite the importance of these constructs in educational research, biology educational researchers have noted the limited use of theoretical or conceptual frameworks in published work ( DeHaan, 2011 ; Dirks, 2011 ; Lo et al. , 2019 ). In reviewing articles published in CBE—Life Sciences Education ( LSE ) between 2015 and 2019, we found that fewer than 25% of the research articles had a theoretical or conceptual framework (see the Supplemental Information), and at times there was an inconsistent use of theoretical and conceptual frameworks. Clearly, these frameworks are challenging for published biology education researchers, which suggests the importance of providing some initial guidance to new biology education researchers.

Fortunately, educational researchers have increased their explicit use of these frameworks over time, and this is influencing educational research in science, technology, engineering, and mathematics (STEM) fields. For instance, a quick search for theoretical or conceptual frameworks in the abstracts of articles in Educational Research Complete (a common database for educational research) in STEM fields demonstrates a dramatic change over the last 20 years: from only 778 articles published between 2000 and 2010 to 5703 articles published between 2010 and 2020, a more than sevenfold increase. Greater recognition of the importance of these frameworks is contributing to DBER authors being more explicit about such frameworks in their studies.

Collectively, literature reviews, theoretical frameworks, and conceptual frameworks work to guide methodological decisions and the elucidation of important findings. Each offers a different perspective on the problem of study and is an essential element in all forms of educational research. As new researchers seek to learn about these elements, they will find different resources, a variety of perspectives, and many suggestions about the construction and use of these elements. The wide range of available information can overwhelm the new researcher who just wants to learn the distinction between these elements or how to craft them adequately.

Our goal in writing this paper is not to offer specific advice about how to write these sections in scholarly work. Instead, we wanted to introduce these elements to those who are new to BER and who are interested in better distinguishing one from the other. In this paper, we share the purpose of each element in BER scholarship, along with important points on its construction. We also provide references for additional resources that may be beneficial to better understanding each element. Table 1 summarizes the key distinctions among these elements.

Comparison of literature reviews, theoretical frameworks, and conceptual reviews

This article is written for the new biology education researcher who is just learning about these different elements or for scientists looking to become more involved in BER. It is a result of our own work as science education and biology education researchers, whether as graduate students and postdoctoral scholars or newly hired and established faculty members. This is the article we wish had been available as we started to learn about these elements or discussed them with new educational researchers in biology.

LITERATURE REVIEWS

Purpose of a literature review.

A literature review is foundational to any research study in education or science. In education, a well-conceptualized and well-executed review provides a summary of the research that has already been done on a specific topic and identifies questions that remain to be answered, thus illustrating the current research project’s potential contribution to the field and the reasoning behind the methodological approach selected for the study ( Maxwell, 2012 ). BER is an evolving disciplinary area that is redefining areas of conceptual emphasis as well as orientations toward teaching and learning (e.g., Labov et al. , 2010 ; American Association for the Advancement of Science, 2011 ; Nehm, 2019 ). As a result, building comprehensive, critical, purposeful, and concise literature reviews can be a challenge for new biology education researchers.

Building Literature Reviews

There are different ways to approach and construct a literature review. Booth et al. (2016a) provide an overview that includes, for example, scoping reviews, which are focused only on notable studies and use a basic method of analysis, and integrative reviews, which are the result of exhaustive literature searches across different genres. Underlying each of these different review processes are attention to the s earch process, a ppraisa l of articles, s ynthesis of the literature, and a nalysis: SALSA ( Booth et al. , 2016a ). This useful acronym can help the researcher focus on the process while building a specific type of review.

However, new educational researchers often have questions about literature reviews that are foundational to SALSA or other approaches. Common questions concern determining which literature pertains to the topic of study or the role of the literature review in the design of the study. This section addresses such questions broadly while providing general guidance for writing a narrative literature review that evaluates the most pertinent studies.

The literature review process should begin before the research is conducted. As Boote and Beile (2005 , p. 3) suggested, researchers should be “scholars before researchers.” They point out that having a good working knowledge of the proposed topic helps illuminate avenues of study. Some subject areas have a deep body of work to read and reflect upon, providing a strong foundation for developing the research question(s). For instance, the teaching and learning of evolution is an area of long-standing interest in the BER community, generating many studies (e.g., Perry et al. , 2008 ; Barnes and Brownell, 2016 ) and reviews of research (e.g., Sickel and Friedrichsen, 2013 ; Ziadie and Andrews, 2018 ). Emerging areas of BER include the affective domain, issues of transfer, and metacognition ( Singer et al. , 2012 ). Many studies in these areas are transdisciplinary and not always specific to biology education (e.g., Rodrigo-Peiris et al. , 2018 ; Kolpikova et al. , 2019 ). These newer areas may require reading outside BER; fortunately, summaries of some of these topics can be found in the Current Insights section of the LSE website.

In focusing on a specific problem within a broader research strand, a new researcher will likely need to examine research outside BER. Depending upon the area of study, the expanded reading list might involve a mix of BER, DBER, and educational research studies. Determining the scope of the reading is not always straightforward. A simple way to focus one’s reading is to create a “summary phrase” or “research nugget,” which is a very brief descriptive statement about the study. It should focus on the essence of the study, for example, “first-year nonmajor students’ understanding of evolution,” “metacognitive prompts to enhance learning during biochemistry,” or “instructors’ inquiry-based instructional practices after professional development programming.” This type of phrase should help a new researcher identify two or more areas to review that pertain to the study. Focusing on recent research in the last 5 years is a good first step. Additional studies can be identified by reading relevant works referenced in those articles. It is also important to read seminal studies that are more than 5 years old. Reading a range of studies should give the researcher the necessary command of the subject in order to suggest a research question.

Given that the research question(s) arise from the literature review, the review should also substantiate the selected methodological approach. The review and research question(s) guide the researcher in determining how to collect and analyze data. Often the methodological approach used in a study is selected to contribute knowledge that expands upon what has been published previously about the topic (see Institute of Education Sciences and National Science Foundation, 2013 ). An emerging topic of study may need an exploratory approach that allows for a description of the phenomenon and development of a potential theory. This could, but not necessarily, require a methodological approach that uses interviews, observations, surveys, or other instruments. An extensively studied topic may call for the additional understanding of specific factors or variables; this type of study would be well suited to a verification or a causal research design. These could entail a methodological approach that uses valid and reliable instruments, observations, or interviews to determine an effect in the studied event. In either of these examples, the researcher(s) may use a qualitative, quantitative, or mixed methods methodological approach.

Even with a good research question, there is still more reading to be done. The complexity and focus of the research question dictates the depth and breadth of the literature to be examined. Questions that connect multiple topics can require broad literature reviews. For instance, a study that explores the impact of a biology faculty learning community on the inquiry instruction of faculty could have the following review areas: learning communities among biology faculty, inquiry instruction among biology faculty, and inquiry instruction among biology faculty as a result of professional learning. Biology education researchers need to consider whether their literature review requires studies from different disciplines within or outside DBER. For the example given, it would be fruitful to look at research focused on learning communities with faculty in STEM fields or in general education fields that result in instructional change. It is important not to be too narrow or too broad when reading. When the conclusions of articles start to sound similar or no new insights are gained, the researcher likely has a good foundation for a literature review. This level of reading should allow the researcher to demonstrate a mastery in understanding the researched topic, explain the suitability of the proposed research approach, and point to the need for the refined research question(s).

The literature review should include the researcher’s evaluation and critique of the selected studies. A researcher may have a large collection of studies, but not all of the studies will follow standards important in the reporting of empirical work in the social sciences. The American Educational Research Association ( Duran et al. , 2006 ), for example, offers a general discussion about standards for such work: an adequate review of research informing the study, the existence of sound and appropriate data collection and analysis methods, and appropriate conclusions that do not overstep or underexplore the analyzed data. The Institute of Education Sciences and National Science Foundation (2013) also offer Common Guidelines for Education Research and Development that can be used to evaluate collected studies.

Because not all journals adhere to such standards, it is important that a researcher review each study to determine the quality of published research, per the guidelines suggested earlier. In some instances, the research may be fatally flawed. Examples of such flaws include data that do not pertain to the question, a lack of discussion about the data collection, poorly constructed instruments, or an inadequate analysis. These types of errors result in studies that are incomplete, error-laden, or inaccurate and should be excluded from the review. Most studies have limitations, and the author(s) often make them explicit. For instance, there may be an instructor effect, recognized bias in the analysis, or issues with the sample population. Limitations are usually addressed by the research team in some way to ensure a sound and acceptable research process. Occasionally, the limitations associated with the study can be significant and not addressed adequately, which leaves a consequential decision in the hands of the researcher. Providing critiques of studies in the literature review process gives the reader confidence that the researcher has carefully examined relevant work in preparation for the study and, ultimately, the manuscript.

A solid literature review clearly anchors the proposed study in the field and connects the research question(s), the methodological approach, and the discussion. Reviewing extant research leads to research questions that will contribute to what is known in the field. By summarizing what is known, the literature review points to what needs to be known, which in turn guides decisions about methodology. Finally, notable findings of the new study are discussed in reference to those described in the literature review.

Within published BER studies, literature reviews can be placed in different locations in an article. When included in the introductory section of the study, the first few paragraphs of the manuscript set the stage, with the literature review following the opening paragraphs. Cooper et al. (2019) illustrate this approach in their study of course-based undergraduate research experiences (CUREs). An introduction discussing the potential of CURES is followed by an analysis of the existing literature relevant to the design of CUREs that allows for novel student discoveries. Within this review, the authors point out contradictory findings among research on novel student discoveries. This clarifies the need for their study, which is described and highlighted through specific research aims.

A literature reviews can also make up a separate section in a paper. For example, the introduction to Todd et al. (2019) illustrates the need for their research topic by highlighting the potential of learning progressions (LPs) and suggesting that LPs may help mitigate learning loss in genetics. At the end of the introduction, the authors state their specific research questions. The review of literature following this opening section comprises two subsections. One focuses on learning loss in general and examines a variety of studies and meta-analyses from the disciplines of medical education, mathematics, and reading. The second section focuses specifically on LPs in genetics and highlights student learning in the midst of LPs. These separate reviews provide insights into the stated research question.

Suggestions and Advice

A well-conceptualized, comprehensive, and critical literature review reveals the understanding of the topic that the researcher brings to the study. Literature reviews should not be so big that there is no clear area of focus; nor should they be so narrow that no real research question arises. The task for a researcher is to craft an efficient literature review that offers a critical analysis of published work, articulates the need for the study, guides the methodological approach to the topic of study, and provides an adequate foundation for the discussion of the findings.

In our own writing of literature reviews, there are often many drafts. An early draft may seem well suited to the study because the need for and approach to the study are well described. However, as the results of the study are analyzed and findings begin to emerge, the existing literature review may be inadequate and need revision. The need for an expanded discussion about the research area can result in the inclusion of new studies that support the explanation of a potential finding. The literature review may also prove to be too broad. Refocusing on a specific area allows for more contemplation of a finding.

It should be noted that there are different types of literature reviews, and many books and articles have been written about the different ways to embark on these types of reviews. Among these different resources, the following may be helpful in considering how to refine the review process for scholarly journals:

• Booth, A., Sutton, A., & Papaioannou, D. (2016a). Systemic approaches to a successful literature review (2nd ed.). Los Angeles, CA: Sage. This book addresses different types of literature reviews and offers important suggestions pertaining to defining the scope of the literature review and assessing extant studies.
• Booth, W. C., Colomb, G. G., Williams, J. M., Bizup, J., & Fitzgerald, W. T. (2016b). The craft of research (4th ed.). Chicago: University of Chicago Press. This book can help the novice consider how to make the case for an area of study. While this book is not specifically about literature reviews, it offers suggestions about making the case for your study.
• Galvan, J. L., & Galvan, M. C. (2017). Writing literature reviews: A guide for students of the social and behavioral sciences (7th ed.). Routledge. This book offers guidance on writing different types of literature reviews. For the novice researcher, there are useful suggestions for creating coherent literature reviews.

THEORETICAL FRAMEWORKS

Purpose of theoretical frameworks.

As new education researchers may be less familiar with theoretical frameworks than with literature reviews, this discussion begins with an analogy. Envision a biologist, chemist, and physicist examining together the dramatic effect of a fog tsunami over the ocean. A biologist gazing at this phenomenon may be concerned with the effect of fog on various species. A chemist may be interested in the chemical composition of the fog as water vapor condenses around bits of salt. A physicist may be focused on the refraction of light to make fog appear to be “sitting” above the ocean. While observing the same “objective event,” the scientists are operating under different theoretical frameworks that provide a particular perspective or “lens” for the interpretation of the phenomenon. Each of these scientists brings specialized knowledge, experiences, and values to this phenomenon, and these influence the interpretation of the phenomenon. The scientists’ theoretical frameworks influence how they design and carry out their studies and interpret their data.

Within an educational study, a theoretical framework helps to explain a phenomenon through a particular lens and challenges and extends existing knowledge within the limitations of that lens. Theoretical frameworks are explicitly stated by an educational researcher in the paper’s framework, theory, or relevant literature section. The framework shapes the types of questions asked, guides the method by which data are collected and analyzed, and informs the discussion of the results of the study. It also reveals the researcher’s subjectivities, for example, values, social experience, and viewpoint ( Allen, 2017 ). It is essential that a novice researcher learn to explicitly state a theoretical framework, because all research questions are being asked from the researcher’s implicit or explicit assumptions of a phenomenon of interest ( Schwandt, 2000 ).

Selecting Theoretical Frameworks

Theoretical frameworks are one of the most contemplated elements in our work in educational research. In this section, we share three important considerations for new scholars selecting a theoretical framework.

The first step in identifying a theoretical framework involves reflecting on the phenomenon within the study and the assumptions aligned with the phenomenon. The phenomenon involves the studied event. There are many possibilities, for example, student learning, instructional approach, or group organization. A researcher holds assumptions about how the phenomenon will be effected, influenced, changed, or portrayed. It is ultimately the researcher’s assumption(s) about the phenomenon that aligns with a theoretical framework. An example can help illustrate how a researcher’s reflection on the phenomenon and acknowledgment of assumptions can result in the identification of a theoretical framework.

In our example, a biology education researcher may be interested in exploring how students’ learning of difficult biological concepts can be supported by the interactions of group members. The phenomenon of interest is the interactions among the peers, and the researcher assumes that more knowledgeable students are important in supporting the learning of the group. As a result, the researcher may draw on Vygotsky’s (1978) sociocultural theory of learning and development that is focused on the phenomenon of student learning in a social setting. This theory posits the critical nature of interactions among students and between students and teachers in the process of building knowledge. A researcher drawing upon this framework holds the assumption that learning is a dynamic social process involving questions and explanations among students in the classroom and that more knowledgeable peers play an important part in the process of building conceptual knowledge.

It is important to state at this point that there are many different theoretical frameworks. Some frameworks focus on learning and knowing, while other theoretical frameworks focus on equity, empowerment, or discourse. Some frameworks are well articulated, and others are still being refined. For a new researcher, it can be challenging to find a theoretical framework. Two of the best ways to look for theoretical frameworks is through published works that highlight different frameworks.

When a theoretical framework is selected, it should clearly connect to all parts of the study. The framework should augment the study by adding a perspective that provides greater insights into the phenomenon. It should clearly align with the studies described in the literature review. For instance, a framework focused on learning would correspond to research that reported different learning outcomes for similar studies. The methods for data collection and analysis should also correspond to the framework. For instance, a study about instructional interventions could use a theoretical framework concerned with learning and could collect data about the effect of the intervention on what is learned. When the data are analyzed, the theoretical framework should provide added meaning to the findings, and the findings should align with the theoretical framework.

A study by Jensen and Lawson (2011) provides an example of how a theoretical framework connects different parts of the study. They compared undergraduate biology students in heterogeneous and homogeneous groups over the course of a semester. Jensen and Lawson (2011) assumed that learning involved collaboration and more knowledgeable peers, which made Vygotsky’s (1978) theory a good fit for their study. They predicted that students in heterogeneous groups would experience greater improvement in their reasoning abilities and science achievements with much of the learning guided by the more knowledgeable peers.

In the enactment of the study, they collected data about the instruction in traditional and inquiry-oriented classes, while the students worked in homogeneous or heterogeneous groups. To determine the effect of working in groups, the authors also measured students’ reasoning abilities and achievement. Each data-collection and analysis decision connected to understanding the influence of collaborative work.

Their findings highlighted aspects of Vygotsky’s (1978) theory of learning. One finding, for instance, posited that inquiry instruction, as a whole, resulted in reasoning and achievement gains. This links to Vygotsky (1978) , because inquiry instruction involves interactions among group members. A more nuanced finding was that group composition had a conditional effect. Heterogeneous groups performed better with more traditional and didactic instruction, regardless of the reasoning ability of the group members. Homogeneous groups worked better during interaction-rich activities for students with low reasoning ability. The authors attributed the variation to the different types of helping behaviors of students. High-performing students provided the answers, while students with low reasoning ability had to work collectively through the material. In terms of Vygotsky (1978) , this finding provided new insights into the learning context in which productive interactions can occur for students.

Another consideration in the selection and use of a theoretical framework pertains to its orientation to the study. This can result in the theoretical framework prioritizing individuals, institutions, and/or policies ( Anfara and Mertz, 2014 ). Frameworks that connect to individuals, for instance, could contribute to understanding their actions, learning, or knowledge. Institutional frameworks, on the other hand, offer insights into how institutions, organizations, or groups can influence individuals or materials. Policy theories provide ways to understand how national or local policies can dictate an emphasis on outcomes or instructional design. These different types of frameworks highlight different aspects in an educational setting, which influences the design of the study and the collection of data. In addition, these different frameworks offer a way to make sense of the data. Aligning the data collection and analysis with the framework ensures that a study is coherent and can contribute to the field.

New understandings emerge when different theoretical frameworks are used. For instance, Ebert-May et al. (2015) prioritized the individual level within conceptual change theory (see Posner et al. , 1982 ). In this theory, an individual’s knowledge changes when it no longer fits the phenomenon. Ebert-May et al. (2015) designed a professional development program challenging biology postdoctoral scholars’ existing conceptions of teaching. The authors reported that the biology postdoctoral scholars’ teaching practices became more student-centered as they were challenged to explain their instructional decision making. According to the theory, the biology postdoctoral scholars’ dissatisfaction in their descriptions of teaching and learning initiated change in their knowledge and instruction. These results reveal how conceptual change theory can explain the learning of participants and guide the design of professional development programming.

The communities of practice (CoP) theoretical framework ( Lave, 1988 ; Wenger, 1998 ) prioritizes the institutional level , suggesting that learning occurs when individuals learn from and contribute to the communities in which they reside. Grounded in the assumption of community learning, the literature on CoP suggests that, as individuals interact regularly with the other members of their group, they learn about the rules, roles, and goals of the community ( Allee, 2000 ). A study conducted by Gehrke and Kezar (2017) used the CoP framework to understand organizational change by examining the involvement of individual faculty engaged in a cross-institutional CoP focused on changing the instructional practice of faculty at each institution. In the CoP, faculty members were involved in enhancing instructional materials within their department, which aligned with an overarching goal of instituting instruction that embraced active learning. Not surprisingly, Gehrke and Kezar (2017) revealed that faculty who perceived the community culture as important in their work cultivated institutional change. Furthermore, they found that institutional change was sustained when key leaders served as mentors and provided support for faculty, and as faculty themselves developed into leaders. This study reveals the complexity of individual roles in a COP in order to support institutional instructional change.

It is important to explicitly state the theoretical framework used in a study, but elucidating a theoretical framework can be challenging for a new educational researcher. The literature review can help to identify an applicable theoretical framework. Focal areas of the review or central terms often connect to assumptions and assertions associated with the framework that pertain to the phenomenon of interest. Another way to identify a theoretical framework is self-reflection by the researcher on personal beliefs and understandings about the nature of knowledge the researcher brings to the study ( Lysaght, 2011 ). In stating one’s beliefs and understandings related to the study (e.g., students construct their knowledge, instructional materials support learning), an orientation becomes evident that will suggest a particular theoretical framework. Theoretical frameworks are not arbitrary , but purposefully selected.

With experience, a researcher may find expanded roles for theoretical frameworks. Researchers may revise an existing framework that has limited explanatory power, or they may decide there is a need to develop a new theoretical framework. These frameworks can emerge from a current study or the need to explain a phenomenon in a new way. Researchers may also find that multiple theoretical frameworks are necessary to frame and explore a problem, as different frameworks can provide different insights into a problem.

Finally, it is important to recognize that choosing “x” theoretical framework does not necessarily mean a researcher chooses “y” methodology and so on, nor is there a clear-cut, linear process in selecting a theoretical framework for one’s study. In part, the nonlinear process of identifying a theoretical framework is what makes understanding and using theoretical frameworks challenging. For the novice scholar, contemplating and understanding theoretical frameworks is essential. Fortunately, there are articles and books that can help:

• Creswell, J. W. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Los Angeles, CA: Sage. This book provides an overview of theoretical frameworks in general educational research.
• Ding, L. (2019). Theoretical perspectives of quantitative physics education research. Physical Review Physics Education Research , 15 (2), 020101-1–020101-13. This paper illustrates how a DBER field can use theoretical frameworks.
• Nehm, R. (2019). Biology education research: Building integrative frameworks for teaching and learning about living systems. Disciplinary and Interdisciplinary Science Education Research , 1 , ar15. https://doi.org/10.1186/s43031-019-0017-6 . This paper articulates the need for studies in BER to explicitly state theoretical frameworks and provides examples of potential studies.
• Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice . Sage. This book also provides an overview of theoretical frameworks, but for both research and evaluation.

CONCEPTUAL FRAMEWORKS

Purpose of a conceptual framework.

A conceptual framework is a description of the way a researcher understands the factors and/or variables that are involved in the study and their relationships to one another. The purpose of a conceptual framework is to articulate the concepts under study using relevant literature ( Rocco and Plakhotnik, 2009 ) and to clarify the presumed relationships among those concepts ( Rocco and Plakhotnik, 2009 ; Anfara and Mertz, 2014 ). Conceptual frameworks are different from theoretical frameworks in both their breadth and grounding in established findings. Whereas a theoretical framework articulates the lens through which a researcher views the work, the conceptual framework is often more mechanistic and malleable.

Conceptual frameworks are broader, encompassing both established theories (i.e., theoretical frameworks) and the researchers’ own emergent ideas. Emergent ideas, for example, may be rooted in informal and/or unpublished observations from experience. These emergent ideas would not be considered a “theory” if they are not yet tested, supported by systematically collected evidence, and peer reviewed. However, they do still play an important role in the way researchers approach their studies. The conceptual framework allows authors to clearly describe their emergent ideas so that connections among ideas in the study and the significance of the study are apparent to readers.

Constructing Conceptual Frameworks

Including a conceptual framework in a research study is important, but researchers often opt to include either a conceptual or a theoretical framework. Either may be adequate, but both provide greater insight into the research approach. For instance, a research team plans to test a novel component of an existing theory. In their study, they describe the existing theoretical framework that informs their work and then present their own conceptual framework. Within this conceptual framework, specific topics portray emergent ideas that are related to the theory. Describing both frameworks allows readers to better understand the researchers’ assumptions, orientations, and understanding of concepts being investigated. For example, Connolly et al. (2018) included a conceptual framework that described how they applied a theoretical framework of social cognitive career theory (SCCT) to their study on teaching programs for doctoral students. In their conceptual framework, the authors described SCCT, explained how it applied to the investigation, and drew upon results from previous studies to justify the proposed connections between the theory and their emergent ideas.

In some cases, authors may be able to sufficiently describe their conceptualization of the phenomenon under study in an introduction alone, without a separate conceptual framework section. However, incomplete descriptions of how the researchers conceptualize the components of the study may limit the significance of the study by making the research less intelligible to readers. This is especially problematic when studying topics in which researchers use the same terms for different constructs or different terms for similar and overlapping constructs (e.g., inquiry, teacher beliefs, pedagogical content knowledge, or active learning). Authors must describe their conceptualization of a construct if the research is to be understandable and useful.

There are some key areas to consider regarding the inclusion of a conceptual framework in a study. To begin with, it is important to recognize that conceptual frameworks are constructed by the researchers conducting the study ( Rocco and Plakhotnik, 2009 ; Maxwell, 2012 ). This is different from theoretical frameworks that are often taken from established literature. Researchers should bring together ideas from the literature, but they may be influenced by their own experiences as a student and/or instructor, the shared experiences of others, or thought experiments as they construct a description, model, or representation of their understanding of the phenomenon under study. This is an exercise in intellectual organization and clarity that often considers what is learned, known, and experienced. The conceptual framework makes these constructs explicitly visible to readers, who may have different understandings of the phenomenon based on their prior knowledge and experience. There is no single method to go about this intellectual work.

Reeves et al. (2016) is an example of an article that proposed a conceptual framework about graduate teaching assistant professional development evaluation and research. The authors used existing literature to create a novel framework that filled a gap in current research and practice related to the training of graduate teaching assistants. This conceptual framework can guide the systematic collection of data by other researchers because the framework describes the relationships among various factors that influence teaching and learning. The Reeves et al. (2016) conceptual framework may be modified as additional data are collected and analyzed by other researchers. This is not uncommon, as conceptual frameworks can serve as catalysts for concerted research efforts that systematically explore a phenomenon (e.g., Reynolds et al. , 2012 ; Brownell and Kloser, 2015 ).

Sabel et al. (2017) used a conceptual framework in their exploration of how scaffolds, an external factor, interact with internal factors to support student learning. Their conceptual framework integrated principles from two theoretical frameworks, self-regulated learning and metacognition, to illustrate how the research team conceptualized students’ use of scaffolds in their learning ( Figure 1 ). Sabel et al. (2017) created this model using their interpretations of these two frameworks in the context of their teaching.

Conceptual framework from Sabel et al. (2017) .

A conceptual framework should describe the relationship among components of the investigation ( Anfara and Mertz, 2014 ). These relationships should guide the researcher’s methods of approaching the study ( Miles et al. , 2014 ) and inform both the data to be collected and how those data should be analyzed. Explicitly describing the connections among the ideas allows the researcher to justify the importance of the study and the rigor of the research design. Just as importantly, these frameworks help readers understand why certain components of a system were not explored in the study. This is a challenge in education research, which is rooted in complex environments with many variables that are difficult to control.

For example, Sabel et al. (2017) stated: “Scaffolds, such as enhanced answer keys and reflection questions, can help students and instructors bridge the external and internal factors and support learning” (p. 3). They connected the scaffolds in the study to the three dimensions of metacognition and the eventual transformation of existing ideas into new or revised ideas. Their framework provides a rationale for focusing on how students use two different scaffolds, and not on other factors that may influence a student’s success (self-efficacy, use of active learning, exam format, etc.).

In constructing conceptual frameworks, researchers should address needed areas of study and/or contradictions discovered in literature reviews. By attending to these areas, researchers can strengthen their arguments for the importance of a study. For instance, conceptual frameworks can address how the current study will fill gaps in the research, resolve contradictions in existing literature, or suggest a new area of study. While a literature review describes what is known and not known about the phenomenon, the conceptual framework leverages these gaps in describing the current study ( Maxwell, 2012 ). In the example of Sabel et al. (2017) , the authors indicated there was a gap in the literature regarding how scaffolds engage students in metacognition to promote learning in large classes. Their study helps fill that gap by describing how scaffolds can support students in the three dimensions of metacognition: intelligibility, plausibility, and wide applicability. In another example, Lane (2016) integrated research from science identity, the ethic of care, the sense of belonging, and an expertise model of student success to form a conceptual framework that addressed the critiques of other frameworks. In a more recent example, Sbeglia et al. (2021) illustrated how a conceptual framework influences the methodological choices and inferences in studies by educational researchers.

Sometimes researchers draw upon the conceptual frameworks of other researchers. When a researcher’s conceptual framework closely aligns with an existing framework, the discussion may be brief. For example, Ghee et al. (2016) referred to portions of SCCT as their conceptual framework to explain the significance of their work on students’ self-efficacy and career interests. Because the authors’ conceptualization of this phenomenon aligned with a previously described framework, they briefly mentioned the conceptual framework and provided additional citations that provided more detail for the readers.

Within both the BER and the broader DBER communities, conceptual frameworks have been used to describe different constructs. For example, some researchers have used the term “conceptual framework” to describe students’ conceptual understandings of a biological phenomenon. This is distinct from a researcher’s conceptual framework of the educational phenomenon under investigation, which may also need to be explicitly described in the article. Other studies have presented a research logic model or flowchart of the research design as a conceptual framework. These constructions can be quite valuable in helping readers understand the data-collection and analysis process. However, a model depicting the study design does not serve the same role as a conceptual framework. Researchers need to avoid conflating these constructs by differentiating the researchers’ conceptual framework that guides the study from the research design, when applicable.

Explicitly describing conceptual frameworks is essential in depicting the focus of the study. We have found that being explicit in a conceptual framework means using accepted terminology, referencing prior work, and clearly noting connections between terms. This description can also highlight gaps in the literature or suggest potential contributions to the field of study. A well-elucidated conceptual framework can suggest additional studies that may be warranted. This can also spur other researchers to consider how they would approach the examination of a phenomenon and could result in a revised conceptual framework.

It can be challenging to create conceptual frameworks, but they are important. Below are two resources that could be helpful in constructing and presenting conceptual frameworks in educational research:

• Maxwell, J. A. (2012). Qualitative research design: An interactive approach (3rd ed.). Los Angeles, CA: Sage. Chapter 3 in this book describes how to construct conceptual frameworks.
• Ravitch, S. M., & Riggan, M. (2016). Reason & rigor: How conceptual frameworks guide research . Los Angeles, CA: Sage. This book explains how conceptual frameworks guide the research questions, data collection, data analyses, and interpretation of results.

CONCLUDING THOUGHTS

Literature reviews, theoretical frameworks, and conceptual frameworks are all important in DBER and BER. Robust literature reviews reinforce the importance of a study. Theoretical frameworks connect the study to the base of knowledge in educational theory and specify the researcher’s assumptions. Conceptual frameworks allow researchers to explicitly describe their conceptualization of the relationships among the components of the phenomenon under study. Table 1 provides a general overview of these components in order to assist biology education researchers in thinking about these elements.

It is important to emphasize that these different elements are intertwined. When these elements are aligned and complement one another, the study is coherent, and the study findings contribute to knowledge in the field. When literature reviews, theoretical frameworks, and conceptual frameworks are disconnected from one another, the study suffers. The point of the study is lost, suggested findings are unsupported, or important conclusions are invisible to the researcher. In addition, this misalignment may be costly in terms of time and money.

Conducting a literature review, selecting a theoretical framework, and building a conceptual framework are some of the most difficult elements of a research study. It takes time to understand the relevant research, identify a theoretical framework that provides important insights into the study, and formulate a conceptual framework that organizes the finding. In the research process, there is often a constant back and forth among these elements as the study evolves. With an ongoing refinement of the review of literature, clarification of the theoretical framework, and articulation of a conceptual framework, a sound study can emerge that makes a contribution to the field. This is the goal of BER and education research.

Supplementary Material

• Allee, V. (2000). Knowledge networks and communities of learning . OD Practitioner , 32 ( 4 ), 4–13. [ Google Scholar ]
• Allen, M. (2017). The Sage encyclopedia of communication research methods (Vols. 1–4 ). Los Angeles, CA: Sage. 10.4135/9781483381411 [ CrossRef ] [ Google Scholar ]
• American Association for the Advancement of Science. (2011). Vision and change in undergraduate biology education: A call to action . Washington, DC. [ Google Scholar ]
• Anfara, V. A., Mertz, N. T. (2014). Setting the stage . In Anfara, V. A., Mertz, N. T. (eds.), Theoretical frameworks in qualitative research (pp. 1–22). Sage. [ Google Scholar ]
• Barnes, M. E., Brownell, S. E. (2016). Practices and perspectives of college instructors on addressing religious beliefs when teaching evolution . CBE—Life Sciences Education , 15 ( 2 ), ar18. https://doi.org/10.1187/cbe.15-11-0243 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Boote, D. N., Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation . Educational Researcher , 34 ( 6 ), 3–15. 10.3102/0013189x034006003 [ CrossRef ] [ Google Scholar ]
• Booth, A., Sutton, A., Papaioannou, D. (2016a). Systemic approaches to a successful literature review (2nd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Booth, W. C., Colomb, G. G., Williams, J. M., Bizup, J., Fitzgerald, W. T. (2016b). The craft of research (4th ed.). Chicago, IL: University of Chicago Press. [ Google Scholar ]
• Brownell, S. E., Kloser, M. J. (2015). Toward a conceptual framework for measuring the effectiveness of course-based undergraduate research experiences in undergraduate biology . Studies in Higher Education , 40 ( 3 ), 525–544. https://doi.org/10.1080/03075079.2015.1004234 [ Google Scholar ]
• Connolly, M. R., Lee, Y. G., Savoy, J. N. (2018). The effects of doctoral teaching development on early-career STEM scholars’ college teaching self-efficacy . CBE—Life Sciences Education , 17 ( 1 ), ar14. https://doi.org/10.1187/cbe.17-02-0039 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Cooper, K. M., Blattman, J. N., Hendrix, T., Brownell, S. E. (2019). The impact of broadly relevant novel discoveries on student project ownership in a traditional lab course turned CURE . CBE—Life Sciences Education , 18 ( 4 ), ar57. https://doi.org/10.1187/cbe.19-06-0113 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Creswell, J. W. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• DeHaan, R. L. (2011). Education research in the biological sciences: A nine decade review (Paper commissioned by the NAS/NRC Committee on the Status, Contributions, and Future Directions of Discipline Based Education Research) . Washington, DC: National Academies Press. Retrieved May 20, 2022, from www7.nationalacademies.org/bose/DBER_Mee ting2_commissioned_papers_page.html [ Google Scholar ]
• Ding, L. (2019). Theoretical perspectives of quantitative physics education research . Physical Review Physics Education Research , 15 ( 2 ), 020101. [ Google Scholar ]
• Dirks, C. (2011). The current status and future direction of biology education research . Paper presented at: Second Committee Meeting on the Status, Contributions, and Future Directions of Discipline-Based Education Research, 18–19 October (Washington, DC). Retrieved May 20, 2022, from http://sites.nationalacademies.org/DBASSE/BOSE/DBASSE_071087 [ Google Scholar ]
• Duran, R. P., Eisenhart, M. A., Erickson, F. D., Grant, C. A., Green, J. L., Hedges, L. V., Schneider, B. L. (2006). Standards for reporting on empirical social science research in AERA publications: American Educational Research Association . Educational Researcher , 35 ( 6 ), 33–40. [ Google Scholar ]
• Ebert-May, D., Derting, T. L., Henkel, T. P., Middlemis Maher, J., Momsen, J. L., Arnold, B., Passmore, H. A. (2015). Breaking the cycle: Future faculty begin teaching with learner-centered strategies after professional development . CBE—Life Sciences Education , 14 ( 2 ), ar22. https://doi.org/10.1187/cbe.14-12-0222 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Galvan, J. L., Galvan, M. C. (2017). Writing literature reviews: A guide for students of the social and behavioral sciences (7th ed.). New York, NY: Routledge. https://doi.org/10.4324/9781315229386 [ Google Scholar ]
• Gehrke, S., Kezar, A. (2017). The roles of STEM faculty communities of practice in institutional and departmental reform in higher education . American Educational Research Journal , 54 ( 5 ), 803–833. https://doi.org/10.3102/0002831217706736 [ Google Scholar ]
• Ghee, M., Keels, M., Collins, D., Neal-Spence, C., Baker, E. (2016). Fine-tuning summer research programs to promote underrepresented students’ persistence in the STEM pathway . CBE—Life Sciences Education , 15 ( 3 ), ar28. https://doi.org/10.1187/cbe.16-01-0046 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Institute of Education Sciences & National Science Foundation. (2013). Common guidelines for education research and development . Retrieved May 20, 2022, from www.nsf.gov/pubs/2013/nsf13126/nsf13126.pdf
• Jensen, J. L., Lawson, A. (2011). Effects of collaborative group composition and inquiry instruction on reasoning gains and achievement in undergraduate biology . CBE—Life Sciences Education , 10 ( 1 ), 64–73. https://doi.org/10.1187/cbe.19-05-0098 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kolpikova, E. P., Chen, D. C., Doherty, J. H. (2019). Does the format of preclass reading quizzes matter? An evaluation of traditional and gamified, adaptive preclass reading quizzes . CBE—Life Sciences Education , 18 ( 4 ), ar52. https://doi.org/10.1187/cbe.19-05-0098 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Labov, J. B., Reid, A. H., Yamamoto, K. R. (2010). Integrated biology and undergraduate science education: A new biology education for the twenty-first century? CBE—Life Sciences Education , 9 ( 1 ), 10–16. https://doi.org/10.1187/cbe.09-12-0092 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lane, T. B. (2016). Beyond academic and social integration: Understanding the impact of a STEM enrichment program on the retention and degree attainment of underrepresented students . CBE—Life Sciences Education , 15 ( 3 ), ar39. https://doi.org/10.1187/cbe.16-01-0070 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lave, J. (1988). Cognition in practice: Mind, mathematics and culture in everyday life . New York, NY: Cambridge University Press. [ Google Scholar ]
• Lo, S. M., Gardner, G. E., Reid, J., Napoleon-Fanis, V., Carroll, P., Smith, E., Sato, B. K. (2019). Prevailing questions and methodologies in biology education research: A longitudinal analysis of research in CBE — Life Sciences Education and at the Society for the Advancement of Biology Education Research . CBE—Life Sciences Education , 18 ( 1 ), ar9. https://doi.org/10.1187/cbe.18-08-0164 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Lysaght, Z. (2011). Epistemological and paradigmatic ecumenism in “Pasteur’s quadrant:” Tales from doctoral research . In Official Conference Proceedings of the Third Asian Conference on Education in Osaka, Japan . Retrieved May 20, 2022, from http://iafor.org/ace2011_offprint/ACE2011_offprint_0254.pdf
• Maxwell, J. A. (2012). Qualitative research design: An interactive approach (3rd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Miles, M. B., Huberman, A. M., Saldaña, J. (2014). Qualitative data analysis (3rd ed.). Los Angeles, CA: Sage. [ Google Scholar ]
• Nehm, R. (2019). Biology education research: Building integrative frameworks for teaching and learning about living systems . Disciplinary and Interdisciplinary Science Education Research , 1 , ar15. https://doi.org/10.1186/s43031-019-0017-6 [ Google Scholar ]
• Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice . Los Angeles, CA: Sage. [ Google Scholar ]
• Perry, J., Meir, E., Herron, J. C., Maruca, S., Stal, D. (2008). Evaluating two approaches to helping college students understand evolutionary trees through diagramming tasks . CBE—Life Sciences Education , 7 ( 2 ), 193–201. https://doi.org/10.1187/cbe.07-01-0007 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Posner, G. J., Strike, K. A., Hewson, P. W., Gertzog, W. A. (1982). Accommodation of a scientific conception: Toward a theory of conceptual change . Science Education , 66 ( 2 ), 211–227. [ Google Scholar ]
• Ravitch, S. M., Riggan, M. (2016). Reason & rigor: How conceptual frameworks guide research . Los Angeles, CA: Sage. [ Google Scholar ]
• Reeves, T. D., Marbach-Ad, G., Miller, K. R., Ridgway, J., Gardner, G. E., Schussler, E. E., Wischusen, E. W. (2016). A conceptual framework for graduate teaching assistant professional development evaluation and research . CBE—Life Sciences Education , 15 ( 2 ), es2. https://doi.org/10.1187/cbe.15-10-0225 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Reynolds, J. A., Thaiss, C., Katkin, W., Thompson, R. J. Jr. (2012). Writing-to-learn in undergraduate science education: A community-based, conceptually driven approach . CBE—Life Sciences Education , 11 ( 1 ), 17–25. https://doi.org/10.1187/cbe.11-08-0064 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Rocco, T. S., Plakhotnik, M. S. (2009). Literature reviews, conceptual frameworks, and theoretical frameworks: Terms, functions, and distinctions . Human Resource Development Review , 8 ( 1 ), 120–130. https://doi.org/10.1177/1534484309332617 [ Google Scholar ]
• Rodrigo-Peiris, T., Xiang, L., Cassone, V. M. (2018). A low-intensity, hybrid design between a “traditional” and a “course-based” research experience yields positive outcomes for science undergraduate freshmen and shows potential for large-scale application . CBE—Life Sciences Education , 17 ( 4 ), ar53. https://doi.org/10.1187/cbe.17-11-0248 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sabel, J. L., Dauer, J. T., Forbes, C. T. (2017). Introductory biology students’ use of enhanced answer keys and reflection questions to engage in metacognition and enhance understanding . CBE—Life Sciences Education , 16 ( 3 ), ar40. https://doi.org/10.1187/cbe.16-10-0298 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Sbeglia, G. C., Goodridge, J. A., Gordon, L. H., Nehm, R. H. (2021). Are faculty changing? How reform frameworks, sampling intensities, and instrument measures impact inferences about student-centered teaching practices . CBE—Life Sciences Education , 20 ( 3 ), ar39. https://doi.org/10.1187/cbe.20-11-0259 [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Schwandt, T. A. (2000). Three epistemological stances for qualitative inquiry: Interpretivism, hermeneutics, and social constructionism . In Denzin, N. K., Lincoln, Y. S. (Eds.), Handbook of qualitative research (2nd ed., pp. 189–213). Los Angeles, CA: Sage. [ Google Scholar ]
• Sickel, A. J., Friedrichsen, P. (2013). Examining the evolution education literature with a focus on teachers: Major findings, goals for teacher preparation, and directions for future research . Evolution: Education and Outreach , 6 ( 1 ), 23. https://doi.org/10.1186/1936-6434-6-23 [ Google Scholar ]
• Singer, S. R., Nielsen, N. R., Schweingruber, H. A. (2012). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington, DC: National Academies Press. [ Google Scholar ]
• Todd, A., Romine, W. L., Correa-Menendez, J. (2019). Modeling the transition from a phenotypic to genotypic conceptualization of genetics in a university-level introductory biology context . Research in Science Education , 49 ( 2 ), 569–589. https://doi.org/10.1007/s11165-017-9626-2 [ Google Scholar ]
• Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes . Cambridge, MA: Harvard University Press. [ Google Scholar ]
• Wenger, E. (1998). Communities of practice: Learning as a social system . Systems Thinker , 9 ( 5 ), 2–3. [ Google Scholar ]
• Ziadie, M. A., Andrews, T. C. (2018). Moving evolution education forward: A systematic analysis of literature to identify gaps in collective knowledge for teaching . CBE—Life Sciences Education , 17 ( 1 ), ar11. https://doi.org/10.1187/cbe.17-08-0190 [ PMC free article ] [ PubMed ] [ Google Scholar ]

• Skip to Guides Search
• Skip to breadcrumb
• Skip to main content
• Skip to footer
• Skip to chat link
• Report accessibility issues and get help
• Go to Penn Libraries Home
• Go to Franklin catalog

Education: Literature Reviews

• Handbooks, Encyclopedias, Dictionaries
• Books and Searchable E-Book Collections
• Articles in Education Databases
• Articles in Subject Databases
• Articles in Multidisciplinary Databases
• Full-text Dissertations
• Open Access
• Publications, Reports, Working Papers
• Literature Reviews
• Research Methods
• How to Search Better
• Manage your Citations
• Journal Info / Metrics
• Author Citation Metrics

Getting Started on Literature Reviews

• "Reviewing the Literature" Project Planner SAGE Research Methods. Provides checklists and bullet points for the literature review process, with "Search for reources" links to relevant SAGE Research Methods fulltext books and book chapters.
• "Literature reviews" / Lawrence A. Machi & Brenda T. McEvoy Oxford Bibliographies : Education, 2016. An annotated bibliography identifying and describing books and articles on the theory of literature reviews, their variety, and how to write them.

Selected Books on Writing Literature Reviews

Search Databases for Literature Review Articles and Overview Publications

• ERIC (ProQuest) Filter search results for Document Type = 070 : Information Analyses and 130 : Reference Materials - Bibliographies. ERIC Digests, research syntheses produced by ERIC ceased after ERIC's reform that closed its research-monitoring clearinghouses in the early 2000s. The ERIC database continues to make available the 3,000+ ERIC Digest published in 1980-2003. They may be found in ERIC (ProQuest) using the search filter for Document type = 073
• APA PsycInfo PsycINFO has a Methodology limit, with values of Literature review, Systematic Review, or Meta-analysis.
• Web of Science (WOS) Don't be misled by "science" in the title. WOS also covers the humanities and social sciences. On the left, under Refine Results, Select REVIEWS under Document Types. This is a limit for literature reviews or overview articles. THis may not get all lit reviews. Consider also searching the TS field (Title, Abstract, Author Keyword, Keywords Plus®) with meta-analysis, metaanalysis, synthesis, overview.
• Scopus Do a search in Scopus for a keyword. Then refine the results by selected under "Document type" - review.
• PubMed Perform a search, then under Article Type on the right, see Reviews or Systematic Reviews.
• ProQuest Dissertations & Theses Global‎: Full Collection Dissertations can sometimes be useful for review-type surveys of the literature on a specific field. Most often authors begin their study with a review of the literature in order to offer context for their contribution.

Selected Journals with Review Articles

• Review of Educational Research (RER) SAGE, for American Educational Research Association (AERA), 1931- . Publishes critical, integrative reviews of research literature bearing on education, including conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research.
• Review of Research in Education (RRE) SAGE, for AERA, 1973- . Each RRE annual volume is devoted to a single topic, with research syntheses and literature reviews.
• Educational Research Review Elsevier, for European Association for Research on Learning and Instruction (EARLI), 2006- .
• Campbell Systematic Reviews Wiley, for Campbell Collaboration, 2004- . CSR publishes finalized systematic reviews developed through the Campbell Collaboration. Subjects include education and child welfare.
• Educational Psychology Review An international forum for the publication of peer-reviewed integrative review articles, special thematic issues, reflections or comments on previous research or new research directions, interviews, and research-based advice for practitioners - all pertaining to the field of educational psychology.
• Annual Reviews AnnRev publishes literature-review journals in physical, life and social sciences. Includes anthropology, economics, linguistics, public health, psychology, and sociology. Education topics may be found in many of these discipline-specific journals.
• Annals of the American Academy of Political and Social Science SAGE, for AAPSS, 1890- . Penn's house journal for the social sciences, and one of the oldest US scholarly journals. Each issue presents research syntheses on a specific topic, with one issue per year focusing on education or child welfare.
• Emerging Trends in the Social and Behavioral sciences Wiley, 2015-201?. Review articles on new research fronts. Includes a recurring section on "Educational Institutions" .

Systematic Review Databases

• Systematic Review Data Repository (SRDR) U.S. Department of Health and Human Services - open and searchable archive of systematic reviews and their data.
• Campbell Collaboration - Library of Systematic Reviews Systematic reviews in areas such as education, criminal justice, social policy and social care. (The Campbell Collaboration was formally established at a meeting at the University of Pennsylvania on 24-25 February 2000.)
• EPPI-Centre The Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) is part of the Social Science Research Unit at the Institute of Education, University of London. EPPI develops systematic reviews and developing review methods in social science and public policy.
• Cochrane Collaboration Cochrane Reviews are systematic reviews of primary research in human health care and health policy. Since 2011, Cochrane has an official partnership with the WHO.
• What Works Clearinghouse - U.S. Department of Education
• << Previous: Publications, Reports, Working Papers
• Next: Research Methods >>
• Last Updated: Apr 16, 2024 8:23 AM
• URL: https://guides.library.upenn.edu/education

• Research Guides
• CECH Library

Education Complete

• Checklist for Education Literature Review
• Search Word Help
• ASL and Deaf Studies Journals
• Children's and Young Adult Literature Journals
• Content-Subject Area Journals
• Early Childhood, Middle Childhood, and Secondary Education Journals
• Educational Psychology Journals
• Gifted Education Journals
• Higher Education Journals
• Instructional Design and Technology Journals
• Multicultural Education Journals
• Special Education Journals
• TEFL Journals
• Tests and Measurements
• Theses & Dissertations
• Professional Organizations
• Systematic Review Resources
• Other Relevant Library Guides

There are eight general steps in conducting an education literature review.  Please follow the eight numbered boxes, starting below.

Please note that the general framework for this guide is derived from the work of Joyce P. Gall, M.D. Gall, and Walter R. Borg in Applying Educational Research: a Practical Guide (5th ed., 2005). Also, much of the information on framing the research question comes from Emily Grimm's "Selected Reference Sources for Graduate Students in Education and Education Related Areas" (1995).

Step 1: Frame Your Research Question

Basic questions.

• What do I want to know?  For what purpose? Consider subject terms, synonyms, related concepts and approaches.
• What do I know already?
• Who else might have performed similar research and why? Consider individuals, institutions, governmental agencies and other groups.
• What summarizing or descriptive information is already available? Consider the secondary sources found below.

Time Questions

• For which time span(s) do I need information?
• Would recurrent or temporal events in education affect my research?  For example: school terms, budget hearings, conference proceedings, legislative sessions, policy decisions, elections, administrative procedural changes.

Limitation(s) Questions

• Do I have other limitations?  For example:  language, age group, grade level, type of student, type of school, type of district, geography, curricular area, or style of teaching.

Aspect Questions

• What aspects of education interest me?  For example:  financial, administrative, teaching, legislative, gender, parental, theoretical, research, developmental, practical or other.

Subjective Aspect Questions

• What are my values, prejudices, biases, and areas of ignorance in regard to my research question(s)?
• Will I let these prejudices limit my research?
• Will I let these prejudices influence my note taking, choice of vocabulary and indexing terms, selection of data, evaluations of the work of other researchers, inclusion of conflicting theories, reporting of data, or my conclusions?

Step 2: Contact Experts to Get Answers or for Guidance to Relevant Publications

Consider consulting other educators, faculty or government officials who may specialize in your research area.

You may also want to consult the American Educational Research Association's Special Interest Groups (SIGs) for the names of groups and individuals who have expertise in different educational areas.  

Step 3: Read Secondary Sources to Gain a Broad Overview of the Literature Related to Your Research Area

Use secondary sources to further define your research question and to expand your literature search.  Secondary sources include encyclopedias, handbooks, dictionaries, and thesauri. Secondary sources are resources that review research that others have done.  They provide a general overview, will give you ideas for key search terms, and often include useful bibliographies for further reading.

Here are some key secondary sources and books on doing educational research:

• Educational Psychology Review Educational Psychology Review is an international forum for the publication of peer-reviewed integrative review articles, special thematic issues, reflections or comments on previous research or new research directions, interviews, and research-based advice for practitioners.
• NSSE Yearbook (Yearbook of the National Society for the Study of Education) Two issues are published annually; each focuses on a major education topic.
• The Phi Delta Kappan Contains many articles that cite research and analyze practical implications.
• Review of Educational Research Quarterly journal that consists of reviews of educational research literature.

Handbooks and Encyclopedias 

• Encyclopedia of Education by James W. Guthrie Call Number: Online Book Publication Date: 2004
• Encyclopedia of Educational Philosophy and Theory by Michael A. Peters (Editor) Call Number: Online Book Publication Date: 2017
• Encyclopedia of Special Education by Cecil R. Reynolds (Editor); Elaine Fletcher-Janzen (Editor); Kimberly J. Vannest (Editor) Call Number: Online Book Publication Date: 2014
• Encyclopedia of the Social and Cultural Foundations of Education by Eugene F. Provenzo (Editor); John Philip Renaud (Editor) Call Number: Online Book Publication Date: 2008
• Handbook of Research on Educational Communications and Technology by J. Michael Spector (Editor); M. David Merrill (Editor); Jan Elen (Editor); M. J. Bishop (Editor) Call Number: Online Book ISBN: 9781461431855 Publication Date: 2013
• Handbook of Research on Teaching by Drew Gitomer Call Number: Online Book Publication Date: 2016
• Handbook of Research on the Education of Young Children by Olivia N. Saracho (Editor); Bernard Spodek (Editor) Call Number: Online Book Publication Date: 2012
• Philosophy : Education by Brian Warnick Call Number: Online book Publication Date: 2017

Step 4: Select Preliminary Sources that Index Relevant Research Literature

Preliminary sources index primary research resources such as journal articles, conference proceeding papers, technical reports, government documents, dissertations and more. See below for key education databases:

Step 5: Identify Subject Terms, or Descriptors, and Use Them to Search Preliminary Sources

Choosing the most appropriate subject search terms, or descriptors, for searching indexes and catalogs can greatly influence your search results.  A good place to start is ERIC's thesaurus of descriptors:

Step 6: Read and Evaluate Primary Sources Discovered Through Indexes

For assistance in obtaining copies of primary sources, please consult online tutorials from UC Libraries .

As you print out copies of articles, review copies of books or reports, remember to look in the sources for bibliographies, names of individuals or groups who have done research on the topic, and for additional subject terms to help you narrow or broaden your research.

Step 7: Classify the Publications You Have Reviewed into Meaningful Categories

As you review the sources you find, classify them into meaningful categories.  This will help you prioritize reading them and may indicate useful ways to synthesize what you discover.  You may want to create a simple code for the different categories.

Step 8: Prepare Your Literature Review Report

The following books can help you assemble a literature review report. The resource icons indicate whether the book is available in print or as an e-book.

• << Previous: Theses & Dissertations
• Next: Professional Organizations >>
• Last Updated: Jul 29, 2024 1:58 PM
• URL: https://guides.libraries.uc.edu/edcomplete

University of Cincinnati Libraries

PO Box 210033 Cincinnati, Ohio 45221-0033

Phone: 513-556-1424

Contact Us | Staff Directory

University of Cincinnati

Alerts | Clery and HEOA Notice | Notice of Non-Discrimination | eAccessibility Concern | Privacy Statement | Copyright Information

© 2021 University of Cincinnati

• Technical Support
• Find My Rep

You are here

Review of Educational Research

Preview this book.

• Description
• Aims and Scope
• Editorial Board
• Abstracting / Indexing
• Submission Guidelines

The Review of Educational Research ( RER , quarterly, begun in 1931; approximately 640 pp./volume year) publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research. RER encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues. RER does not publish original empirical research unless it is incorporated in a broader integrative review. RER will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented.

The Review of Educational Research publishes critical, integrative reviews of research literature bearing on education. Such reviews should include conceptualizations, interpretations, and syntheses of literature and scholarly work in a field broadly relevant to education and educational research. RER encourages the submission of research relevant to education from any discipline, such as reviews of research in psychology, sociology, history, philosophy, political science, economics, computer science, statistics, anthropology, and biology, provided that the review bears on educational issues. RER does not publish original empirical research, and all analyses should be incorporated in a broader integrative review. RER will occasionally publish solicited, but carefully refereed, analytic reviews of special topics, particularly from disciplines infrequently represented. The following types of manuscripts fall within the journal’s purview:

Integrative reviews pull together the existing work on an educational topic and work to understand trends in that body of scholarship. In such a review, the author describes how the issue is conceptualized within the literature, how research methods and theories have shaped the outcomes of scholarship, and what the strengths and weaknesses of the literature are. Meta-analyses are of particular interest when they are accompanied by an interpretive framework that takes the article beyond the reporting of effect sizes and the bibliographic outcome of a computer search.

Theoretical reviews should explore how theory shapes research. To the extent that research is cited and interpreted, it is in the service of the specification, explication, and illumination of a theory. Theoretical reviews and integrative reviews have many similarities, but the former are primarily about how a theory is employed to frame research and our understandings, and refer to the research as it relates to the theory.

Methodological reviews are descriptions of research design, methods, and procedures that can be employed in literature reviews or research in general. The articles should highlight the strengths and weaknesses of methodological tools and explore how methods constrain or open up opportunities for learning about educational problems. They should be written in a style that is accessible to researchers in education rather than methodologists.

Historical reviews provide analyses that situate literature in historical contexts. Within these reviews, explanations for educational phenomena are framed within the historical forces that shape language and understanding.

Commissioned reviews and thematic issues. The editors may commission and solicit authors to review areas of literature. In all other respects, commissioned reviews are subject to the same review process as submitted reviews. The editors also encourage readers to propose thematic topics for special issues and, as potential guest editors, to submit plans for such issues.

In addition to review articles, RER will occasionally publish notes and responses which are short pieces of no more than 1,200 words on any topic that would be of use to reviewers of research. Typically, they point out shortcomings and differences in interpretation in RER articles and policy.

The standards and criteria for review articles in RER are the following:

1. Quality of the Literature. Standards used to determine quality of literature in education vary greatly. Any review needs to take into account the quality of the literature and its impact on findings. Authors should attempt to review all relevant literature on a topic (e.g., international literature, cross-disciplinary work, etc.).

2. Quality of Analysis. The review should go beyond description to include analysis and critiques of theories, methods, and conclusions represented in the literature. This analysis should also examine the issue of access—which perspectives are included or excluded in a body of work? Finally, the analysis should be reflexive—how does the scholars’ framework constrain what can be known in this review?

3. Significance of the Topic. The review should seek to inform and/or illuminate questions important to the field of education. While these questions may be broad-based, they should have implications for the educational problems and issues affecting our national and global societies.

4. Impact of the Article. The review should be seen as an important contribution and tool for the many different educators dealing with the educational problems and issues confronting society.

5. Advancement of the Field. The review should validate or inform the knowledge of researchers and guide and improve the quality of their research and scholarship.

6. Style. The review must be well written and conform to style of the Publication Manual of the American Psychological Association (6th edition). Authors should avoid the use of unexplained jargon and parochialism.

7. Balance and Fairness. The review should be careful not to misrepresent the positions taken by others, or be disrespectful of contrary positions.

8. Purpose. Any review should be accessible to the broad readership of RER. The purpose of any article should be to connect the particular problem addressed by the researcher(s) to a larger context of education.

We also encourage all authors interested in submitting a manuscript to RER to read our Editorial Vision for more information on our publication aims.

• Academic Search - Premier
• Academic Search Alumni Edition
• Academic Search Elite
• Clarivate Analytics: Current Contents - Physical, Chemical & Earth Sciences
• EBSCO: MasterFILE Elite
• EBSCO: MasterFILE Premier
• EBSCO: Professional Development Collection
• EBSCO: Sales & Marketing Source
• EBSCOhost: Current Abstracts
• ERIC (Education Resources Information Center)
• Educational Research Abstracts Online (T&F)
• Higher Education Abstracts
• MasterFILE Select - EBSCO
• ProQuest Education Journals
• Social SciSearch
• Social Sciences Citation Index (Web of Science)
• Teacher Reference Center
• Wilson Education Index/Abstracts

1. Publication Standards 2. Submission Preparation Checklist 3. How to Get Help With the Quality of English in Your Submission 4. Copyright Information 5. For authors who use figures or other materials for which they do not own copyright 6. Right of Reply 7. Sage Choice and Open Access

The Review of Educational Research (RER) publishes comprehensive reviews of literature related to education and does not publish new empirical work, except in the context of meta-analytic reviews of an area. Please check the journal’s Aims and Scope to see if your manuscript is appropriate to submit to RER.

All manuscripts should be submitted electronically to the editorial team at http://mc.manuscriptcentral.com/rer . For questions or inquiries about manuscripts, email us at [email protected] . Manuscripts may not be submitted via e-mail.

Publication Standards

Researchers who intend to submit studies for publication should consult the Standards for Research Conduct adopted by the AERA Council. We also recommend consulting (a) the Guidelines for Reviewers , which outline the criteria under which manuscripts are reviewed for publication by AERA and (b) recent previous editions of the journal. Individuals submitting systematic reviews or meta-analyses should also consult The PRISMA Statement ( http://www.prisma-statement.org ) as well the article on “Reporting Standards for Research in Psychology” in American Psychologist, 63 , 839 – 851 (doi:10.1037/0003-066X.63.9.839).

Submission Preparation Checklist

When you upload your initial submission, upload (1) a separate title page that is not anonymized. Please format the title page as described by the 7th edition of the APA Manual and (2) the main manuscript, which includes an ANONYMIZED title page, an abstract with keywords at the bottom, and the rest of the document including tables and figures, and finally (c) Author Bios.

Please ensure that your manuscript complies with the “ RER Formatting Requirements and Common Formatting Errors ” (see PDF on the RER website). If your submission does not meet these requirements, it will be returned to you.

Additionally, your submission should meet the following guidelines:

1. The submission has not been previously published and is not under consideration for publication elsewhere; or an explanation has been provided in the Cover Letter. Authors should indicate in the Author Note on the separate title page if sections of the manuscript have been published in other venues.

2. THE MANUSCRIPT CONTAINS NO IDENTIFYING INFORMATION, EVEN ON THE ANONYMIZED TITLE PAGE. Please anonymize any work of limited circulation (e.g., in press papers, manuscripts under submission) that would point to the author, both in the body of the manuscript and the reference list. More information on anonymizing is described subsequently. Please double check that the author’s name has been removed from the document’s Properties, which in Microsoft Word is found in the File menu (select “File,” “Properties,” “Summary,” and remove the author’s name; select “OK” to save).

3. The text conforms to APA style (currently the 7th ed.). Consult the guidelines spelled out under “Manuscript Style, Length, and Format” on this webpage and in the RER Formatting Requirements PDF included on our website.

4. The submission must be in Microsoft Word format (.doc or .docx), which will be converted into a PDF file. Please do not upload PDF files, or they will be returned to you.

5. All URL addresses and DOIs in the manuscript (e.g., http://www.aera.net ) should be activated and ready to click.

6. An abstract of 150 words maximum is included (both separately and on the second page of the main document after the ANONYMIZED title page). Please also include three to five keywords—the terms that researchers will use to find your article in indexes and databases.

Manuscript Style, Length, and Format

The style guide for the Review of Educational Research and all AERA journals is the Publication Manual of the American Psychological Association, 7th ed., 2020. The manual is available for purchase here . Guidelines are also available on the APA website .

Manuscripts should NOT exceed 65 pages (or 15,000 words), including tables, figures, appendices, notes, and references, but excluding anonymized title page, abstract, and any supplementary files. Pages should be numbered consecutively in the top right-hand corner, with a fully capitalized running head in the top-left corner. All manuscripts should begin with the anonymized title page (p.1). Manuscripts should be typed for 8½” x 11” paper, in upper and lower case, with 1-inch margins on all sides. Manuscripts should be typed in 12-point Times New Roman font. Manuscripts that exceed 65 pages may be returned without review.

All text, from the title page to the end of the manuscript should be double-spaced , including the abstract, block quotations, bulleted text, and the reference list. Single-spacing is allowed in tables when it is useful in making the table clearer. Do not leave blank lines after paragraphs or before sub-headings. However, if a heading or subheading is the last line on a page, use a page break to move it to the top of the next page. The Abstract, Introduction (beginning with the title), the References, and all tables and figures begin on new pages.

Please use the five subheadings as appropriate based on the 7th edition of the APA style manual. In addition to being on the title page, the title should also be placed at the beginning of the Introduction (in lieu of the word, “Introduction,” which should not appear) and the title at the beginning of the Introduction should be a Level 1 heading.

Tables and figures are to be placed after the references—all tables precede all figures—and should not be included in the body of the text. Each figure and table should begin on a separate page. Do NOT use the “Place Table 5 here” or “Place Figure 1 here” convention. The tables and figures will be placed nearest to where they are mentioned as appropriate when copyediting is done.

Figures and tables should present data to the reader in a clear and unambiguous manner, and should be referred to in the text. If the illustration/table/figure and text are redundant, eliminate the illustration or reduce the amount of detail provided in text. The use of lines in tables is limited (please consult the APA style manual for formatting guidelines ). Figure captions should be placed at the bottom of the figure. One high-quality electronic version of each figure must be submitted with the manuscript. Tables will be typeset. Note that any figures and tables uploaded separately from the main manuscript will still count toward the total 65-page limit.

Italics can be used for emphasis or contrast in special situations but should be used sparingly. Ideally, sentence structure should be used for these issues. All words to be set in italics (e.g., book titles, journal names) should be typed in italics. There should be no underlined text . Abbreviations and acronyms should be spelled out the first time they are mentioned unless they are found as entries in their abbreviated form in Merriam-Webster’s Collegiate Dictionary , 11th ed., 2003 (e.g., “IQ” can be used without being spelled out). Mathematical symbols and symbols for vectors should be clearly formatted in italics and boldface, respectively.

You can use the footnote or endnote feature of Microsoft Word. However, notes are only for explanations or amplifications of textual material that cannot be incorporated into the regular text; they are not for reference information. Moreover, notes are distracting to readers and expensive to produce and should be used sparingly and avoided whenever possible.

The reference list should contain only references that are cited in the text. Its accuracy and completeness are the responsibility of the authors. Reference each publicly available dataset with its title, author, date, and a persistent Web identifier such as a digital object identifier (doi), a handle, or a uniform resource name (URN). If necessary, this last element may be replaced by a web address. Additionally, any references that were included in the analysis but not cited in-text in the main manuscript can be included in a separate reference list that is uploaded as a Supplementary File for Review (this may assist in meeting the page limit).

Authors should anonymize their manuscripts for review . Anonymizing does not mean removing all self-citations. Authors should only anonymize citations of limited circulation (e.g., forthcoming, in press, unpublished) that point to the author. Publications already in the extant literature (e.g., books, book chapters, journal articles) should be cited normally, but authors should include self-citations judiciously . When anonymizing, please use “Author” or “Authors” as in the examples below and place this alphabetically in the reference list and not where the author’s actual name would typically appear.

For examples of common types of references, consult the APA 7th edition manual, or visit the webpage here: https://apastyle.apa.org/style-grammar-guidelines/references

How to Get Help with the Quality of English in Your Submission

Authors who would like to refine the use of English in their manuscripts might consider using the services of a professional English-language editing company. We highlight some of these companies at  https://languageservices.sagepub.com/en/ .

Please be aware that Sage has no affiliation with these companies and makes no endorsement of them. An author's use of these services in no way guarantees that his or her submission will ultimately be accepted. Any arrangement an author enters into will be exclusively between the author and the particular company, and any costs incurred are the sole responsibility of the author.

Copyright Information Accepted authors will be asked to  assign copyright  to AERA, in return for which AERA grants several rights to authors.

Permission to reproduce your own published material

No written or oral permission is necessary to reproduce a table, a figure, or an excerpt of fewer than 500 words from this journal, or to make photocopies for classroom use. Authors are granted permission, without fee, to photocopy their own material or make printouts from the final pdf of their article. Copies must include a full and accurate bibliographic citation and the following credit line: “Copyright [year] by the American Educational Research Association; reproduced with permission from the publisher.” Written permission must be obtained to reproduce or reprint material in circumstances other than those just described. Please review Sage Publishing’s  Journal Permissions  for further information on policies and fees.

Permission to submit material for which you do not own copyright

Authors who wish to use material, such as figures or tables, for which they do not own the copyright must obtain written permission from the copyright holder (usually the publisher) and submit it along with their manuscript. However, no written or oral permission is necessary to reproduce a table, a figure, or an excerpt of fewer than 500 words from an AERA journal.

Copyright transfer agreements for accepted works with more than one author

This journal uses a transfer of copyright agreement that requires just one author (the corresponding author) to sign on behalf of all authors. Please identify the corresponding author for your work when submitting your manuscript for review. The corresponding author will be responsible for the following:

1. Ensuring that all authors are identified on the copyright agreement, and notifying the editorial office of any changes in the authorship.

2. Securing written permission (by letter or e-mail) from each co-author to sign the copyright agreement on the co-author’s behalf.

3. Warranting and indemnifying the journal owner and publisher on behalf of all co-authors. Although such instances are very rare, you should be aware that in the event that a co-author has included content in his or her portion of the article that infringes the copyright of another or is otherwise in violation of any other warranty listed in the agreement, you will be the sole author indemnifying the publisher and the editor of the journal against such violation.

Please contact the publications office at  AERA  if you have questions or if you prefer to use a copyright agreement for all coauthors to sign.

Right of Reply

The right of reply policy encourages comments on recently published articles in AERA publications. They are, of course, subject to the same editorial review and decision process as articles. If the comment is accepted for publication, the editor shall inform the author of the original article. If the author submits a reply to the comment, the reply is also subject to editorial review and decision. The editor may allot a specific amount of journal space for the comment (ordinarily about 1,500 words) and for the reply (ordinarily about 750 words). The reply may appear in the same issue as the comment or in a later issue (Council, June 1980).

If an article is accepted for publication in an AERA journal that, in the judgment of the editor, has as its main theme or thrust a critique of a specific piece of work or a specific line of work associated with an individual or program of research, then the individual or representative of the research program whose work is critiqued should be notified in advance about the upcoming publication and given the opportunity to reply, ideally in the same issue. The author of the original article should also be notified. Normal guidelines for length and review of the reply and publication of a rejoinder by the original article’s author(s) should be followed. Articles in the format “an open letter to …” may constitute prototypical exemplars of the category defined here, but other formats may well be used, and would be included under the qualifications for response prescribed here (Council, January 2002).

Sage Choice and Open Access

If you or your funder wish your article to be freely available online to nonsubscribers immediately upon publication (gold open access), you can opt for it to be included in Sage Choice, subject to payment of a publication fee. The manuscript submission and peer review procedure is unchanged. On acceptance of your article, you will be asked to let Sage know directly if you are choosing Sage Choice. To check journal eligibility and the publication fee, please visit  Sage Choice . For more information on open access options and compliance at Sage, including self author archiving deposits (green open access) visit  Sage Publishing Policies  on our Journal Author Gateway.

• Read Online
• Sample Issues
• Current Issue
• Email Alert
• Permissions
• Foreign rights
• Reprints and sponsorship
• Advertising

Individual Subscription, Combined (Print & E-access)

Institutional Subscription, E-access

Institutional Subscription & Backfile Lease, E-access Plus Backfile (All Online Content)

Institutional Subscription, Print Only

Institutional Subscription, Combined (Print & E-access)

Institutional Subscription & Backfile Lease, Combined Plus Backfile (Current Volume Print & All Online Content)

Institutional Backfile Purchase, E-access (Content through 1998)

Individual, Single Print Issue

Institutional, Single Print Issue

To order single issues of this journal, please contact SAGE Customer Services at 1-800-818-7243 / 1-805-583-9774 with details of the volume and issue you would like to purchase.

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

On "Literature Reviews of, and for, Educational Research": A Response to the Critique by Joseph Maxwell

2006, Educational Researcher

We thank Joseph Maxwell (this issue of Educational Researcher, pp. 28–31) for accepting our invitation to examine the roles and expectations of dissertation literature reviews. We agree that most are inadequate but disagree why. Maxwell argues that dissertations should emulate research articles and include a conceptual framework that only discusses relevant literature. Candidates need not present a thorough analysis and synthesis of the scholarship and research, Maxwell argues, or justify claims made about the literature. Finally, he asserts that we misunderstand the relationship between literature and research, leading to an accusation of “foundationalism.” However, these specific disagreements ignore fundamental differences about the purposes of the doctoral dissertation and the relationship between those purposes and doctoral program goals. In this rejoinder, we briefly explore these conceptual and normative differences.

Related Papers

Educational Researcher

Penny Beile

A thorough, sophisticated literature review is the foundation and inspiration for substantial, useful research. The complex nature of education research demands such thorough, sophisticated reviews. Although doctoral education is a key means for improving education research, the literature has given short shrift to the dissertation literature review. This article suggests criteria to evaluate the quality of dissertation literature reviews and reports a study that examined dissertation at three universities. Acquiring the skills and knowledge required to be education scholars, able to analyze and synthesize the research in a field of specialization, should be the focal, integrative activity of predissertation doctoral education. Such scholarship is a prerequisite for increased methodological sophistication and for improving the usefulness of education research.

Sadruddin Qutoshi

The main purpose of this review of the article is to analyze critically to what extent the author has been successful in capturing the essence of literature review by addressing the key issues like: how faulty literature review leads to derail of a dissertation; how to write high-quality dissertation literature review; purpose of writing literature review; taxonomy of literature review; steps to conduct literature review; common mistakes and framework for the self-evaluation of a literature review. The researcher used Cooper’s (1988) taxonomy of literature reviews (CTLR) as methodological tool to critically review the claims of the author. The researcher found the article an interesting and useful work of the author in providing guidelines about writing literature reviews. The findings support most of the claims of the author regarding the guidelines provided. The author’ discourse on the topic found to be consistent and engaging except a few points which still demand more clarity to understand, which opens new space for writers to develop a comprehensive framework like using an integrative framework of literature review.

Justus Randolph

InSITE Conference

shardul pandya

Aim/Purpose: Identify the prerequisites, the corequisites, and the iteration processes in organizing and writing the literature review chapter of doctoral dissertations. Background: Writing the literature review chapter of doctoral dissertations presents unique challenges. Students waste a lot of time identifying material to write, and the experience is generally that of frustration and time delay. Methodology: Paper reviews literature to identify levels of information helpful for writing the literature review chapter: prerequisites, corequisites, and iteration process. Contribution: Paper identifies and explains the prerequisites, the corequisites, and iteration steps that go into organizing the reviewed literature and suggests putting them into use when starting the literature review chapter of a doctoral dissertation. Findings: Writing a doctoral dissertation literature review is long and complicated be-cause some delve into the writing without much preparation. By identifying wh...

This study had two purposes: to examine the expertise of doctoral students in their use of the scholarly literature and to investigate the use of citation analysis as a tool for collection development. Analysis of 1,842 coded citations gleaned from 30 education dissertations awarded in 2000 from 3 institutions in the United States revealed that journal articles, at 45%, were cited most frequently, followed by monographs (33.9%) and "other" (18.3%), with magazines and Web sites contributing less than 2% each of the total material types cited. The study examined 858 journal and magazine citations, which were found in 239 unique titles. A relatively small number of journals contained a high percentage of the references found in the dissertations analyzed. Based on a design by D. Kohl and L. Wilson (1986), dissertation citations were also scored for scholarliness, currency, and appropriateness of format, and scores on the three criteria were averaged to arrive at a quality rating. Results of interinstitutional comparisons revealed a significant amount of variation and were considered in conjunction with institutional characteristics and published criteria for quality bibliographies. The data suggest that the assumption of doctoral student expertise in their use of the scholarly literature may be overstated and should be examined in relation to their preparation for professional status. For purposes of developing a library's research collection, a core list of titles, generated on the basis of multiple, rather than single, institutional analysis is indicated. (Contains 8 figures, 11 tables, and 28 references.)

London Review of Education

Stella Mi-Cheong Cheong (정미정 She/Her/Hers)

Sarah Elaine Eaton

Purpose: The purpose of this paper is to provide students of educational research with clear guidance on how to choose high quality sources for research papers and theses. Methods: Using an information-analysis approach, this evidence-based guide draws from social sciences research methodology literature. It addresses a gap in the literature by offering pragmatic guidance on how to engage in an informed decision-making process about what types of literature are appropriate for inclusion in educational research reports. Results: The primary outcome of this work is a clear, plain-language guide for students to help them build their understanding of the kinds of sources to include in a literature review. Implications: As a result of better understanding the types of sources that constitute a credible literature review, students may develop both their competence and confidence selecting appropriate sources for their research writing. Although written for an educational research audience, researchers in other disciplines may find it applicable. Additional Materials: 12 references, 1 figure.

Anne Switzer

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Gary Anderson

School Leadership Review

Joseph Murphy

Communications in Computer and Information Science

Elizabeth Lee

intakhab khan

Education Review

Ivet Parra Gaete

Educational Research and Reviews

Abebayehu Tekleselassie

Alfi Rahman

Impacting Education: Journal on Transforming Professional Practice

Gwynne Rife

Gavin Mount

Carla Thompson

QUEST JOURNALS

KnE Social Sciences

Ivan Hanafi

International Review of Qualitative Research

Sarah Bridges-Rhoads , Hilary E . Hughes

Wiley Blackwell

Martyn Hammersley

Auxiliadora Padilha

Publications

Cherley C Du Plessis

Kristina Hesbol , Valerie A Storey

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

EDUC 760 Literature Review in Education Research

• Course Description

This course provides education practitioners in the doctoral program with research skills and strategies for conducting a literature review. Critical thinking skills, synthesis of information, and application of the literature are emphasized to support the feasibility and relevance of a research study.

For information regarding prerequisites for this course, please refer to the  Academic Course Catalog .

Course Guide

View this course’s outcomes, policies, schedule, and more.*

Requires a student login to access.

*The information contained in our Course Guides is provided as a sample. Specific course curriculum and requirements for each course are provided by individual instructors each semester. Students should not use Course Guides to find and complete assignments, class prerequisites, or order books.

A literature review is a comprehensive investigation of previous research and scholarly articles on a topic. Beginning with a theory that serves to frame, or orient, the study, Chapter Two continues with a comprehensive review of the literature that addresses sufficient directly and indirectly related bodies of literature to show the reader that the researcher has a complete understanding of the topic. By conducting a thorough review, the researcher identifies any gaps in the literature and, therefore, a need for a study. Literature reviews are not merely listings or descriptions of the literature but are critiques of the literature that, when synthesized, create a conversation, or dialogue, among the various scholars that comes alive and inspires your reader to be even more interested in the topic and to be ready to consume the study’s findings. Poor literature reviews reflect poorly on the scholar, and, in the case of Christian scholars, reflect poorly on Christianity. As such, the need for Christian scholars to generate exemplary literature reviews is paramount to representing God as the supreme authority and creator of all knowledge. 

Course Assignment

Textbook readings and presentations.

The Claxton and Dolan (2021) textbook is used for this course and includes any readings necessary for the candidate to ensure competence with basic literature review development. Additionally, the Galvan and Galvan (2017) text will be used for supplemental material. Finally, the latest APA Style Guide and dissertation templates are required for this course.

Course Requirements Checklist

As the first activity in this course, the candidate will read the Syllabus and Student Expectations and complete the related checklist found in the Course Overview.

Discussions (2)

Discussions are collaborative learning experiences. Therefore, the candidate will complete the discussions in two parts. First, the candidate will post a new thread of 400-500 words in response to the provided discussion prompt. Then, the candidate will reply to at least two classmates, 150-200 words each reply.  (CLO: A, E)

Literature Review: Topic Identification Essay Assignment

The candidate will submit a document containing two paragraphs identifying his/her proposed topic for the dissertation. The first paragraph will identify and discuss the topic, including a brief summary of its social and historical contexts, as well as its relationship to education. The second paragraph will give an explanation as why the candidate considers this a worthwhile topic, which may include personal experience and research goals. At least two scholarly, peer-reviewed articles published within the past five years must be cited in the essay with an appropriate reference page attached.  (CLO: A, E)

Literature Review: Overview Assignment

The candidate will submit a document containing a one-paragraph overview for the literature review. The overview will follow the template provided, be written in present tense, and be specific to the candidate’s topic from Module 1 (based upon professor feedback).  (CLO: E)

Literature Review: Theory Identification Essay Assignment

The candidate will identify 1-2 theories related to the selected topic of study and write a 2-4 paragraph essay. The first paragraph for each theory will explain the theory using the publication(s) of the primary theorist and at least one other work. In the second paragraph, the candidate will justify why the theory was chosen and connect it to the topic. A correctly formatted reference page must be included. (CLO: B, E)

Literature Review Outline Assignment

The candidate will develop a complete literature review outline that will include an improved overview and identify the theory (or theories) of the Theoretical Framework section based on previous professor feedback. The outline must include a title page, the four required level 1 headings (Overview, Theoretical Framework, Related Literature, and Summary), a one-paragraph (improved) Overview, and the name(s) of the theory (or theories) in the Theoretical Framework. The outline must also include the level 2 and 3 APA headings (headings only) identifying the themes and subthemes of the Related Literature section, the Summary section, and the completed reference list., The outline should be 2-3 pages in length, excluding the title page and reference list section. (CLO: A, B, C, E)

Literature Review: Theoretical Framework Section Assignment

The candidate will write the theoretical framework section of the literature review and include it in the working literature review document with the title page, revised Overview, revised Related Literature outline headings (with sources only, no written content), revised Summary, and reference list pages. The assignment should be a total of 2-3 pages, excluding title and reference list pages.  (CLO: B, E)

Literature Review: Synthesis Essay Assignment

The candidate will demonstrate an understanding of how to synthesize literature by researching at least 5 sources related to his/her topic, identifying 2 themes evident across those 5 sources, and synthesizing the concepts and source material related to those two themes. Synthesizing and writing concisely are learned skills, and this assignment is an opportunity to practice those skills. This essay must follow APA standards and will include a title page, 2-3 full pages of content, and a reference list. (CLO: D, E)

Literature Review Rough Draft Assignment

The candidate will develop a rough draft of the literature review including the title page, Overview, Theoretical Framework, Related Literature section, Summary, and reference list. All work must be updated and developed based on previous professor feedback. While the Related Literature section may not include content under every level 2 (or level 3) APA heading, all headings must be updated per professor feedback, and the Summary section must be revised. The document must include a minimum of 4 full pages of content (excluding the title and reference list pages). (CLO: C, D, E)

Literature Review Final Assignment

The candidate will develop a final draft of the literature review containing a minimum of 8 full pages of content (excluding the title and reference list pages). The literature review must also use appropriate APA format, incorporate professor feedback. (CLO: E)

Quizzes (4)

Each quiz will cover the Learn section material for the module in which it is assigned. Each quiz will be open-book/open-notes; contain multiple-choice, true/false, or checklist completion questions; and have a 1-hour time limit. (CLO: D, E)

Have questions about this course or a program?

Speak to one of our admissions specialists.

Inner Navigation

• Assignments

Have questions?

Are you ready to change your future?

Apply FREE This Week*

Request Information

*Some restrictions may occur for this promotion to apply. This promotion also excludes active faculty and staff, military, non-degree-seeking, DGIA, Continuing Education, WSB, and certificate students.

Request Information About a Program

Request info about liberty university online, what program are you interested in, choose a program level.

Choose a program level

Bachelor’s

Master’s

Certificate

Select a Field of Study

Select a field of study

Select a Program

Select a program

Next: Contact Info

Legal first name.

Enter legal first name

Legal Last Name

Enter legal last name

Enter an email address

Enter a phone number

Full Address

Enter an address

Apt., P.O. Box, or can’t find your address? Enter it manually instead .

Select a Country

Street Address

Enter Street Address

Enter State

ZIP/Postal Code

Enter Zip Code

Back to automated address search

Start my application now for FREE

Research in Education

• Lit Review - Best Sources
• Books and Dissertations
• Interlibrary Loan
• How-To for Google Scholar
• How-To for ERIC (EBSCO)
• How-To for ERIC (Free)
• How-To for APA PsycInfo
• How-To for OneSearch
• Is it Peer Reviewed?
• Finding the DOI

Contact Your Librarian

Top Sources for Literature Reviews in Education

Eric , education full text , and google scholar - best options for education research.

Use these databases to broaden your literature searches.              

Additional Sources for Locating Education Research

Multidisciplinary databases for education research.

Use these databases to broaden your literature searches.            

Historical education research .

• Next: Books and Dissertations >>
• Last Updated: Aug 22, 2023 3:29 PM
• URL: https://guides.lib.uni.edu/education-research

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Author Biographies
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Impacts of generative artificial intelligence in higher education: research trends and students’ perceptions.

1. Introduction

2. materials and methods.

• “Generative Artificial Intelligence” or “Generative AI” or “Gen AI”, AND;
• “Higher Education” or “University” or “College” or “Post-secondary”, AND;
• “Impact” or “Effect” or “Influence”.
• Q1— Does GenAI have more positive or negative effects on higher education? Options (to choose one): 1. It has more negative effects than positives; 2. It has more positive effects than negative; 3. There is a balance between positive and negative effects; 4. Don’t know.
• Q2— Identify the main positive effect of Gen AI in an academic context . Open-ended question.
• Q3— Identify the main negative effect of Gen AI in an academic context . Open-ended question.

3.1. Impacts of Gen AI in HE: Research Trends

3.1.1. he with gen ai, the key role that pedagogy must play, new ways to enhance the design and implementation of teaching and learning activities.

• Firstly, prompting in teaching should be prioritized as it plays a crucial role in developing students’ abilities. By providing appropriate prompts, educators can effectively guide students toward achieving their learning objectives.
• Secondly, configuring reverse prompting within the capabilities of Gen AI chatbots can greatly assist students in monitoring their learning progress. This feature empowers students to take ownership of their education and fosters a sense of responsibility.
• Furthermore, it is essential to embed digital literacy in all teaching and learning activities that aim to leverage the potential of the new Gen AI assistants. By equipping students with the necessary skills to navigate and critically evaluate digital resources, educators can ensure that they are prepared for the digital age.

The Student’s Role in the Learning Experience

The key teacher’s role in the teaching and learning experience, 3.1.2. assessment in gen ai/chatgpt times, the need for new assessment procedures, 3.1.3. new challenges to academic integrity policies, new meanings and frontiers of misconduct, personal data usurpation and cheating, 3.2. students’ perceptions about the impacts of gen ai in he.

• “It harms the learning process”: ▪ “What is generated by Gen AI has errors”; ▪ “Generates dependence and encourages laziness”; ▪ “Decreases active effort and involvement in the learning/critical thinking process”.

4. Discussion

• Training: providing training for both students and teachers on effectively using and integrating Gen AI technologies into teaching and learning practices.
• Ethical use and risk management: developing policies and guidelines for ethical use and risk management associated with Gen AI technologies.
• Incorporating AI without replacing humans: incorporating AI technologies as supplementary tools to assist teachers and students rather than replacements for human interaction.
• Continuously enhancing holistic competencies: encouraging the use of AI technologies to enhance specific skills, such as digital competence and time management, while ensuring that students continue to develop vital transferable skills.
• Fostering a transparent AI environment: promoting an environment in which students and teachers can openly discuss the benefits and concerns associated with using AI technologies.
• Data privacy and security: ensuring data privacy and security using AI technologies.
• The dynamics of technological support to align with the most suitable Gen AI resources;
• The training policy to ensure that teachers, students, and academic staff are properly trained to utilize the potential of Gen AI and its tools;
• Security and data protection policies;
• Quality and ethical action policies.

5. Conclusions

• Database constraints: the analysis is based on existing publications in SCOPUS and the Web of Science, potentially omitting relevant research from other sources.
• Inclusion criteria: due to the early stage of scientific production on this topic, all publications were included in the analysis, rather than focusing solely on articles from highly indexed journals and/or with a high number of citations as recommended by bibliometric and systematic review best practices.
• Dynamic landscape: the rate of publications on Gen AI has been rapidly increasing and diversifying in 2024, highlighting the need for ongoing analysis to track trends and changes in scientific thinking.

Author Contributions

Institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Akakpo, Martin Gameli. 2023. Skilled for the Future: Information Literacy for AI Use by University Students in Africa and the Role of Librarians. Internet Reference Services Quarterly 28: 19–26. [ Google Scholar ] [ CrossRef ]
• AlAfnan, Mohammad Awad, Samira Dishari, Marina Jovic, and Koba Lomidze. 2023. ChatGPT as an Educational Tool: Opportunities, Challenges, and Recommendations for Communication, Business Writing, and Composition Courses. Journal of Artificial Intelligence and Technology 3: 60–68. [ Google Scholar ] [ CrossRef ]
• Almaraz-López, Cristina, Fernando Almaraz-Menéndez, and Carmen López-Esteban. 2023. Comparative Study of the Attitudes and Perceptions of University Students in Business Administration and Management and in Education toward Artificial Intelligence. Education Sciences 13: 609. [ Google Scholar ] [ CrossRef ]
• Al-Zahrani, Abdulrahman. 2023. The impact of generative AI tools on researchers and research: Implications for academia in higher education. Innovations in Education and Teaching International , 1–15. [ Google Scholar ] [ CrossRef ]
• Athilingam, Ponrathi, and Hong-Gu He. 2023. ChatGPT in nursing education: Opportunities and challenges. Teaching and Learning in Nursing 19: 97–101. [ Google Scholar ] [ CrossRef ]
• Álvarez-Álvarez, Carmen, and Samuel Falcon. 2023. Students’ preferences with university teaching practices: Analysis of testimonials with artificial intelligence. Educational Technology Research and Development 71: 1709–24. [ Google Scholar ] [ CrossRef ]
• Bannister, Peter, Elena Alcalde Peñalver, and Alexandra Santamaría Urbieta. 2023. Transnational higher education cultures and generative AI: A nominal group study for policy development in English medium instruction. Journal for Multicultural Education . ahead-of-print . [ Google Scholar ] [ CrossRef ]
• Bearman, Margaret, and Rola Ajjawi. 2023. Learning to work with the black box: Pedagogy for a world with artificial intelligence. British Journal of Educational Technology 54: 1160–73. [ Google Scholar ] [ CrossRef ]
• Boháček, Matyas. 2023. The Unseen A+ Student: Evaluating the Performance and Detectability of Large Language Models in the Classroom. CEUR Workshop Proceedings 3487: 89–100. Available online: https://openreview.net/pdf?id=9ZKJLYg5EQ (accessed on 7 January 2024).
• Chan, Cecilia Ka Yuk. 2023. A comprehensive AI policy education framework for university teaching and learning. International Journal of Educational Technology in Higher Education 20: 38. [ Google Scholar ] [ CrossRef ]
• Chan, Cecilia Ka Yuk, and Wenjie Hu. 2023. Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education 20: 43. [ Google Scholar ] [ CrossRef ]
• Chan, Cecilia Ka Yuk, and Wenxin Zhou. 2023. An expectancy value theory (EVT) based instrument for measuring student perceptions of generative AI. Smart Learning Environments 10: 64. [ Google Scholar ] [ CrossRef ]
• Chang, Daniel H., Michael Pin-Chuan Lin, Shiva Hajian, and Quincy Q. Wang. 2023. Educational Design Principles of Using AI Chatbot That Supports Self-Regulated Learning in Education: Goal Setting, Feedback, and Personalization. Sustainability 15: 12921. [ Google Scholar ] [ CrossRef ]
• Chiu, Thomas. 2023. The impact of Generative AI (GenAI) on practices, policies and research direction in education: A case of ChatGPT and Midjourney. Interactive Learning Environments , 1–17. [ Google Scholar ] [ CrossRef ]
• Chun, John, and Katherine Elkins. 2023. The Crisis of Artificial Intelligence: A New Digital Humanities Curriculum for Human-Centred AI. International Journal of Humanities and Arts Computing 17: 147–67. [ Google Scholar ] [ CrossRef ]
• Cowling, Michael, Joseph Crawford, Kelly-Ann Allen, and Michael Wehmeyer. 2023. Using leadership to leverage ChatGPT and artificial intelligence for undergraduate and postgraduate research supervision. Australasian Journal of Educational Technology 39: 89–103. [ Google Scholar ] [ CrossRef ]
• Crawford, Joseph, Carmen Vallis, Jianhua Yang, Rachel Fitzgerald, Christine O’Dea, and Michael Cowling. 2023a. Editorial: Artificial Intelligence is Awesome, but Good Teaching Should Always Come First. Journal of University Teaching & Learning Practice 20: 01. [ Google Scholar ] [ CrossRef ]
• Crawford, Joseph, Michael Cowling, and Kelly-Ann Allen. 2023b. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). Journal of University Teaching & Learning Practice 20: 02. [ Google Scholar ] [ CrossRef ]
• Currie, Geoffrey. 2023a. A Conversation with ChatGPT. Journal of Nuclear Medicine Technology 51: 255–60. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Currie, Geoffrey. 2023b. GPT-4 in Nuclear Medicine Education: Does It Outperform GPT-3.5? Journal of Nuclear Medicine Technology 51: 314–17. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Currie, Geoffrey, and Kym Barry. 2023. ChatGPT in Nuclear Medicine Education. Journal of Nuclear Medicine Technology 51: 247–54. [ Google Scholar ] [ CrossRef ]
• Currie, Geoffrey, Clare Singh, Tarni Nelson, Caroline Nabasenja, Yazan Al-Hayek, and Kelly Spuur. 2023. ChatGPT in medical imaging higher education. Radiography 29: 792–99. [ Google Scholar ] [ CrossRef ]
• Dai, Yun, Ang Liu, and Cher P. Lim. 2023. Reconceptualizing Chatgpt and Generative AI as a Student-driven Innovation in Higher Education. Procedia CIRP Volume 119: 84–90. [ Google Scholar ] [ CrossRef ]
• Dogru, Tarik, Nathana Line, Lydia Hanks, Fulya Acikgoz, Je’Anna Abbott, Selim Bakir, Adiyukh Berbekova, Anil Bilgihan, Ali Iskender, Murat Kizildag, and et al. 2023. The implications of generative artificial intelligence in academic research and higher education in tourism and hospitality. Tourism Economics 30: 1083–94. [ Google Scholar ] [ CrossRef ]
• Duong, Cong Doanh, Trong Nghia Vu, and Thi Viet Nga Ngo. 2023. Applying a modified technology acceptance model to explain higher education students’ usage of ChatGPT: A serial multiple mediation model with knowledge sharing as a moderator. The International Journal of Management Education 21: 100883. [ Google Scholar ] [ CrossRef ]
• Eager, Bronwyn, and Ryan Brunton. 2023. Prompting Higher Education Towards AI-Augmented Teaching and Learning Practice. Journal of University Teaching & Learning Practice 20: 5. [ Google Scholar ] [ CrossRef ]
• Elkhodr, Mahmoud, Ergun Gide, Robert Wu, and Omar Darwish. 2023. ICT students’ perceptions towards ChatGPT: An experimental reflective lab analysis. STEM Education 3: 70–88. [ Google Scholar ] [ CrossRef ]
• Farrelly, Tom, and Nick Baker. 2023. Generative Artificial Intelligence: Implications and Considerations for Higher Education Practice. Education Sciences 13: 1109. [ Google Scholar ] [ CrossRef ]
• Farrokhnia, Mohammadreza, Seyyed Banihashem, Seyyed Kazem Banihashem, Omid Noroozi, and Arjen Wals. 2023. A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International 61: 460–74. [ Google Scholar ] [ CrossRef ]
• Gong, Furong. 2023. The Impact of Generative AI like ChatGPT on Digital Literacy Education in University Libraries. Documentation, Information & Knowledge 40: 97–106, 156. [ Google Scholar ] [ CrossRef ]
• Han, Bingyi, Sadia Nawaz, George Buchanan, and Dana McKay. 2023. Ethical and Pedagogical Impacts of AI in Education. In Artificial Intelligence in Education . Edited by Ning Wang, Genaro Rebolledo-Mendez, Noboru Matsuda, Olga Santos and Vania Dimitrova. Lecture Notes in Computer Science. Cham: Springer, pp. 667–73. [ Google Scholar ] [ CrossRef ]
• Hassoulas, Athanasios, Ned Powell, Lindsay Roberts, Katja Umla-Runge, Laurence Gray, and Marcus J. Coffey. 2023. Investigating marker accuracy in differentiating between university scripts written by students and those produced using ChatGPT. Journal of Applied Learning and Teaching 6: 71–77. [ Google Scholar ] [ CrossRef ]
• Hernández-Leo, Davinia. 2023. ChatGPT and Generative AI in Higher Education: User-Centered Perspectives and Implications for Learning Analytics. CEUR Workshop Proceedings , 1–6. Available online: https://ceur-ws.org/Vol-3542/paper2.pdf (accessed on 7 January 2024).
• Hidayat-ur-Rehman, Imdadullah, and Yasser Ibrahim. 2023. Exploring factors influencing educators’ adoption of ChatGPT: A mixed method approach. Interactive Technology and Smart Education . ahead-of-print . [ Google Scholar ] [ CrossRef ]
• Ilieva, Galina, Tania Yankova, Stanislava Klisarova-Belcheva, Angel Dimitrov, Marin Bratkov, and Delian Angelov. 2023. Effects of Generative Chatbots in Higher Education. Information 14: 492. [ Google Scholar ] [ CrossRef ]
• Javaid, Mohd, Abid Haleem, Ravi Pratap Singh, Shahbaz Khan, and Haleem Ibrahim. 2023. Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system. Bench Council Transactions on Benchmarks, Standards and Evaluations 3: 100115. [ Google Scholar ] [ CrossRef ]
• Kaplan-Rakowski, Regina, Kimberly Grotewold, Peggy Hartwick, and Kevin Papin. 2023. Generative AI and Teachers’ Perspectives on Its Implementation in Education. Journal of Interactive Learning Research 34: 313–38. Available online: https://www.learntechlib.org/primary/p/222363/ (accessed on 7 January 2024).
• Karunaratne, Thashmee, and Adenike Adesina. 2023. Is it the new Google: Impact of ChatGPT on Students’ Information Search Habits. Paper presented at the European Conference on e-Learning (ECEL 2023), Pretoria, South Africa, October 26–27; pp. 147–55. [ Google Scholar ] [ CrossRef ]
• Kelly, Andrew, Miriam Sullivan, and Katrina Strampel. 2023. Generative artificial intelligence: University student awareness, experience, and confidence in use across disciplines. Journal of University Teaching & Learning Practice 20: 12. [ Google Scholar ] [ CrossRef ]
• Kohnke, Lucas, Benjamin Luke Moorhouse, and Di Zou. 2023. Exploring generative artificial intelligence preparedness among university language instructors: A case study. Computers and Education: Artificial Intelligence 5: 100156. [ Google Scholar ] [ CrossRef ]
• Laker, Lauren, and Mark Sena. 2023. Accuracy and detection of student use of ChatGPT in business analytics courses. Issues in Information Systems 24: 153–63. [ Google Scholar ] [ CrossRef ]
• Lemke, Claudia, Kathrin Kirchner, Liadan Anandarajah, and Florian Herfurth. 2023. Exploring the Student Perspective: Assessing Technology Readiness and Acceptance for Adopting Large Language Models in Higher Education. Paper presented at the European Conference on e-Learning, (ECEL 2023), Pretoria, South Africa, October 26–27; pp. 156–64. [ Google Scholar ] [ CrossRef ]
• Limna, Pongsakorn, Tanpat Kraiwanit, Kris Jangjarat, and Prapasiri Klayklung. 2023a. The use of ChatGPT in the digital era: Perspectives on chatbot implementation. Journal of Applied Learning and Teaching 6: 64–74. [ Google Scholar ] [ CrossRef ]
• Limna, Pongsakorn, Tanpat Kraiwanit, Kris Jangjarat, and Yarnaphat Shaengchart. 2023b. Applying ChatGPT as a new business strategy: A great power comes with great responsibility [Special issue]. Corporate & Business Strategy Review 4: 218–26. [ Google Scholar ] [ CrossRef ]
• Lopezosa, Carlos, Carles Lluís Codina, Carles Pont-Sorribes, and Mari Vállez. 2023. Use of Generative Artificial Intelligence in the Training of Journalists: Challenges, Uses and Training Proposal. Profesional De La información Information Professional 32: 1–12. [ Google Scholar ] [ CrossRef ]
• Martineau, Kim. 2023. What Is Generative AI? IBM Research Blog . April 20. Available online: https://research.ibm.com/blog/what-is-generative-AI (accessed on 7 January 2024).
• Mondal, Himel, Shaikat Mondal, and Indrashis Podder. 2023. Using ChatGPT for Writing Articles for Patients’ Education for Dermatological Diseases: A Pilot Study. Indian Dermatology Online Journal 14: 482–86. [ Google Scholar ] [ CrossRef ]
• Moorhouse, Benjamin, Marie Alina Wan, and Yuwei Wan. 2023. Generative AI tools and assessment: Guidelines of the world’s top-ranking universities. Computers and Education Open 5: 100151. [ Google Scholar ] [ CrossRef ]
• Overono, Acacia L., and Annie Ditta. 2023. The Rise of Artificial Intelligence: A Clarion Call for Higher Education to Redefine Learning and Reimagine Assessment. College Teaching , 1–4. [ Google Scholar ] [ CrossRef ]
• Page, Matthew J., Joanne E. McKenzie, Patrick M. Bossuyt, Isabelle Boutron, Tammy C. Hoffmann, Cynthia D. Mulrow, Larissa Shamseer, Jennifer M. Tetzlaff, Elie A. Akl, Sue E. Brennan, and et al. 2021. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 372: n71. [ Google Scholar ] [ CrossRef ]
• Pechenkina, Ekaterina. 2023. Artificial intelligence for good? Challenges and possibilities of AI in higher education from a data justice perspective. In Higher Education for Good: Teaching and Learning Futures . Edited by Laura Czerniewicz and Catherine Cronin. Cambridge, UK: Open Book Publishers, pp. 239–66. [ Google Scholar ] [ CrossRef ]
• Perkins, Mike, Jasper Roe, Darius Postma, James McGaughran, and Don Hickerson. 2023. Detection of GPT-4 Generated Text in Higher Education: Combining Academic Judgement and Software to Identify Generative AI Tool Misuse. Journal of Academic Ethics 22: 89–113. [ Google Scholar ] [ CrossRef ]
• Pitso, Teboho. 2023. Post-COVID-19 Higher Learning: Towards Telagogy, A Web-Based Learning Experience. IAFOR Journal of Education 11: 39–59. [ Google Scholar ] [ CrossRef ]
• Plata, Sterling, Maria Ana De Guzman, and Arthea Quesada. 2023. Emerging Research and Policy Themes on Academic Integrity in the Age of Chat GPT and Generative AI. Asian Journal of University Education 19: 743–58. [ Google Scholar ] [ CrossRef ]
• Rudolph, Jürgen, Samson Tan, and Shannon Tan. 2023a. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. Journal of Applied Learning and Teaching 6: 364–89. [ Google Scholar ] [ CrossRef ]
• Rudolph, Jürgen, Samson Tan, and Shannon Tan. 2023b. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching 6: 342–63. [ Google Scholar ] [ CrossRef ]
• Ryall, Adelle, and Stephen Abblitt. 2023. “A Co-Pilot for Learning Design?”: Perspectives from Learning Designers on the Uses, Challenges, and Risks of Generative Artificial Intelligence in Higher Education. In People, Partnerships and Pedagogies. Proceedings ASCILITE 2023 . Edited by Thomas Cochrane, Vickel Narayan, Cheryl Brown, MacCallum Kathryn, Elisa Bone, Christopher Deneen, Robert Vanderburg and Brad Hurren. Christchurch: Te Pae Conference Center, pp. 525–30. [ Google Scholar ] [ CrossRef ]
• Santiago, Cereneo S., Steve I. Embang, Ricky B. Acanto, Kem Warren P. Ambojia, Maico Demi B. Aperocho, Benedicto B. Balilo, Erwin L. Cahapin, Marjohn Thomas N. Conlu, Samson M. Lausa, Ester Y. Laput, and et al. 2023. Utilization of Writing Assistance Tools in Research in Selected Higher Learning Institutions in the Philippines: A Text Mining Analysis. International Journal of Learning, Teaching and Educational Research 22: 259–84. [ Google Scholar ] [ CrossRef ]
• Solopova, Veronika, Eiad Rostom, Fritz Cremer, Adrian Gruszczynski, Sascha Witte, Chengming Zhang, Fernando Ramos López, Lea Plößl, Florian Hofmann, Ralf Romeike, and et al. 2023. PapagAI: Automated Feedback for Reflective Essays. In KI 2023: Advances in Artificial Intelligence. KI 2023 . Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Cham: Springer, vol. 14236, pp. 198–206. [ Google Scholar ] [ CrossRef ]
• Sridhar, Pragnya, Aidan Doyle, Arav Agarwal, Christopher Bogart, Jaromir Savelka, and Majd Sakr. 2023. Harnessing LLMs in Curricular Design: Using GPT-4 to Support Authoring of Learning Objectives. CEUR Workshop Proceedings 3487: 139–50. [ Google Scholar ]
• Sullivan, Miriam, Andrew Kelly, and Paul McLaughlan. 2023. ChatGPT in higher education: Considerations for academic integrity and student learning. Journal of Applied Learning and Teaching 6: 31–40. [ Google Scholar ] [ CrossRef ]
• Tominc, Polona, and Maja Rožman. 2023. Artificial Intelligence and Business Studies: Study Cycle Differences Regarding the Perceptions of the Key Future Competences. Education Sciences 13: 580. [ Google Scholar ] [ CrossRef ]
• van den Berg, Geesje, and Elize du Plessis. 2023. ChatGPT and Generative AI: Possibilities for Its Contribution to Lesson Planning, Critical Thinking and Openness in Teacher Education. Education Sciences 13: 998. [ Google Scholar ] [ CrossRef ]
• Walczak, Krzysztof, and Wojciech Cellary. 2023. Challenges for higher education in the era of widespread access to Generative AI. Economics and Business Review 9: 71–100. [ Google Scholar ] [ CrossRef ]
• Wang, Ting, Brady D. Lund, Agostino Marengo, Alessandro Pagano, Nishith Reddy Mannuru, Zoë A. Teel, and Jenny Pange. 2023. Exploring the Potential Impact of Artificial Intelligence (AI) on International Students in Higher Education: Generative AI, Chatbots, Analytics, and International Student Success. Applied Sciences 13: 6716. [ Google Scholar ] [ CrossRef ]
• Watermeyer, Richard, Lawrie Phipps, Donna Lanclos, and Cathryn Knight. 2023. Generative AI and the Automating of Academia. Postdigital Science and Education 6: 446–66. [ Google Scholar ] [ CrossRef ]
• Wolf, Leigh, Tom Farrelly, Orna Farrell, and Fiona Concannon. 2023. Reflections on a Collective Creative Experiment with GenAI: Exploring the Boundaries of What is Possible. Irish Journal of Technology Enhanced Learning 7: 1–7. [ Google Scholar ] [ CrossRef ]
• Yilmaz, Ramazan, and Fatma Gizem Karaoglan Yilmaz. 2023. The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation. Computers and Education: Artificial Intelligence 4: 100147. [ Google Scholar ] [ CrossRef ]
• Zawiah, Mohammed, Fahmi Y. Al-Ashwal, Lobna Gharaibeh, Rana Abu Farha, Karem H. Alzoubi, Khawla Abu Hammour, Qutaiba A. Qasim, and Fahd Abrah. 2023. ChatGPT and Clinical Training: Perception, Concerns, and Practice of Pharm-D Students. Journal of Multidisciplinary Healthcare 16: 4099–110. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Saúde, S.; Barros, J.P.; Almeida, I. Impacts of Generative Artificial Intelligence in Higher Education: Research Trends and Students’ Perceptions. Soc. Sci. 2024 , 13 , 410. https://doi.org/10.3390/socsci13080410

Saúde S, Barros JP, Almeida I. Impacts of Generative Artificial Intelligence in Higher Education: Research Trends and Students’ Perceptions. Social Sciences . 2024; 13(8):410. https://doi.org/10.3390/socsci13080410

Saúde, Sandra, João Paulo Barros, and Inês Almeida. 2024. "Impacts of Generative Artificial Intelligence in Higher Education: Research Trends and Students’ Perceptions" Social Sciences 13, no. 8: 410. https://doi.org/10.3390/socsci13080410

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Research on comprehensive evaluation & development of aesthetic education based on PCA and CEM model

Roles Writing – original draft

Affiliation School of Art, Anhui University of Finance and Economics, Bengbu, China

Roles Supervision, Validation, Writing – review & editing

* E-mail: [email protected]

Roles Conceptualization, Data curation, Funding acquisition

Affiliation Graduate School, Hankuk University of Foreign Studies, Seoul, Korea

Roles Investigation, Methodology, Project administration

Affiliation School of Statistics and Applied Mathematics, Anhui University of Finance and Economics, Bengbu, China

Roles Project administration, Resources, Software

• Xin-Hong Xu, 
• Yu-Ting Niu, 
• Zhi-Min Li, 
• Yue-Yang Xu, 
• Ke-Wang Cao

• Published: August 9, 2024
• https://doi.org/10.1371/journal.pone.0308446
• Reader Comments

Aesthetic education, conveyed through public art courses, serves as a vital form of humanistic literacy education. It represents an effective approach to fostering innovative and creative thinking among college students. In order to effectively analyze the aesthetic education work of 46 universities, an aesthetic education index evaluation system is constructed, involving indicators including faculty strength, curriculum setting, teaching management, artistic practice, and teaching support. The secondary indicators corresponding to the five indicators are statistically analyzed, and a comprehensive evaluation analysis of the current development status of aesthetic education in 46 universities in Anhui Province is conducted by combining theoretical analysis with empirical analysis. Based on principal component analysis, an integrated evaluation model for the development of aesthetic education in universities in Anhui Province is further constructed. The model designed quantifies the influence weight of each aesthetic education index on the development of aesthetic education in Anhui Province, and forges a theoretical basis for determining the precursors of rapid development of aesthetic education in Anhui Province. Additionally, a novel approach is introduced to gauge the progression of aesthetic education within universities in Anhui Province, considering the dispersion of aesthetic education index data across the province. The comprehensive evaluation model for the development of aesthetic education in Anhui Province exhibits an overall declining trend. Hence, it is suggested to utilize the maximum value of the first derivative of the comprehensive evaluation model as an indicator of the imminent rapid development of aesthetic education in Anhui Province. On this basis, the probability equation of sustainable development of aesthetic education in Anhui Province is defined. Overall, the research results lay a theoretical foundation for the development of aesthetic education in Anhui Province.

Citation: Xu X-H, Niu Y-T, Li Z-M, Xu Y-Y, Cao K-W (2024) Research on comprehensive evaluation & development of aesthetic education based on PCA and CEM model. PLoS ONE 19(8): e0308446. https://doi.org/10.1371/journal.pone.0308446

Editor: Mc Rollyn Daquiado Vallespin, Far Eastern University - Manila, PHILIPPINES

Received: April 11, 2024; Accepted: July 23, 2024; Published: August 9, 2024

Copyright: © 2024 Xu et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data are fully available without restriction.

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

1. Introduction

Aesthetic education, also referred to as aesthetic or art education, holds significant importance within higher education. It typically pertains to undergraduate programs lasting four or five years, excluding vocational and technical colleges, as well as graduate education for the time being. Based on the National General Art Education Curriculum Guidelines 2006 and the Opinions on Solidly Improving the Aesthetic Education Work in Colleges and Universities in the New Era 2020 [ 1 , 2 ], the results of aesthetic education in colleges and universities over the past 15 years indicate that on the one hand, aesthetic education in colleges and universities is still in the stage of theoretical exploration and practical education, while on the other, there is no feasible evaluation system available for the actual effect of aesthetic education. This results in an imbalance between excessive aesthetic education theory and the lack of evaluation system.

According to relevant documents such as China’s National Medium-to-Long-Term Education Reform and Development Plan 2010–2020 [ 3 ], Opinions on Comprehensively Strengthening and Improving Art Education in Schools [ 4 ], and Regulations on Art Education Work in Schools [ 5 ], this article examines 46 undergraduate universities in Anhui Province, adhering to principles of scientificity, comprehensiveness, and hierarchy. Meanwhile, it employs reverse reasoning and considers multiple observation points such as faculty strength, curriculum design, teaching management, artistic practice, and teaching support. Through the utilization of methods like principal component analysis, entropy weight method, and BP neural network method, the article offers a comprehensive evaluation of the current state of aesthetic education in universities in Anhui Province. Additionally, it highlights future challenges that need to be addressed.

2. Literature review

Aesthetic education is an education that cultivates people’s aesthetic perception, appreciation, creativity, and judgment. It holds irreplaceable significance in the higher education system, and presents long-term effectiveness, strong infectiousness, and transcends time, space, and ethnic and national aesthetic universality. Plato advocated for the cultivation of citizens in both body and mind as the ultimate goal of education, while Confucius emphasized the importance of poetry and music education as primary avenues for aesthetic education. Indeed, aesthetic education is an emotional education that concerns people mentality and personality development. When individuals are in crisis and pressure, aesthetic education can furnish them with a balanced, stable, and positive attitude to life [ 6 ]. Filipović and Vojvodić [ 7 ] believed that aesthetic education provides opportunities for students’ creative ability, creative communication ability, and aesthetic ability development, thereby affecting their personality emancipation and all-round development. In this way, art education makes people the most humanized and complete individual [ 8 ]. The implementation of aesthetic education in universities primarily involves utilizing public art courses as the foundation of art education. It centers on art works within the context of art history as the core content of the educational process. Art education refers to the education of music, dance, drama and visual arts disciplines [ 9 ]. Specifically, art encompasses various forms, including visual arts such as painting, drawing, sculpture, filmmaking, architecture, photography, and ceramic art. Additionally, it includes literary arts such as poetry, drama, prose, and novels, as well as performing arts like drama, dance, and music [ 10 ]. According to the National Common Propaganda on Public Art Education in Colleges and Universities in China , public art courses refer to those emphasizing aesthetics, art history and theory, art appreciation, and artistic practice represented by music, dance, painting, calligraphy, drama, film and television.

From the perspective of function and method of aesthetics, reconstructing STEM education by integrating art education is a way to implant art education [ 11 ]. STEAM combines science, technology, engineering, art, and mathematics education [ 12 ]. Meanwhile, art education can enhance students’ innovation ability and promote the cultivation of comprehensive innovative talents [ 13 ]. Scientific work requires creativity and critical thinking, while art and science mutually inspire each other in a subtle and covert way [ 14 ]. Wang and Zeng [ 15 ] believed that imaginative thinking ability is a kind of creative thinking ability, which is not only the core of art but also the key to scientific research. However, it is a challenge to scientifically evaluate the actual function of art in STEAM education from the student-centered perspective [ 16 ]. Teachers should have good adaptive teaching abilities to ensure that the aesthetic effect of public art courses is achieved [ 17 ]. In order to shift from the traditional aesthetic evaluation model centered on teachers, classrooms, and textbooks, a more student-centered approach can be adopted. This involves conducting aesthetic ability evaluations based on indicators such as artistic cultural knowledge and skills, artistic abilities, and artistic achievements. However, despite these efforts, the evaluation may still exhibit some bias towards a student-centered perspective [ 18 ]. In 2018, in order to construct a comprehensive evaluation system for higher education aesthetic education, Shandong Province established an evaluation index system for higher education aesthetic education work from five dimensions of curriculum design, teaching management, teacher team, artistic practice, and condition guarantee [ 19 ]. Only by adopting a more student-centered approach to aesthetic education evaluation can the aesthetic development of universities in the province truly benefit.

3. Comprehensive evaluation analysis

3.1 construction of the comprehensive evaluation index.

Aesthetic education as an educational phenomenon can be described, measured, calculated, and analyzed to help depict the quality of aesthetic education. Observing, describing, or researching the development of aesthetic education in various universities in Anhui Province aids in cultivating a reasoned comprehension of its status, distinctive features, and operational mechanisms. Comprehensive evaluation serves as a quantitative empirical method enabling the comparison of research objects through evaluation outcomes. Evaluating and comparing the progress of aesthetic education in universities across Anhui Province necessitates the establishment of a viable comprehensive evaluation index system, which should adhere to the following four principles:

The first principle is systematicity, where various indicators have logical relationships, reflecting the main characteristics and status of the overall system of aesthetic education from different aspects. The second is the principle of typicality, which emphasizes that indicators should possess certain representative characteristics to comprehensively reflect the current development status of aesthetic education. Selected indicators should have the capacity to guide the evaluation process effectively. The third principle involves independence, where the interaction of indicators within the system manifests as the characteristics of subsystems. The evaluation index system of aesthetic education includes independent subsystems reflecting teacher conditions, curriculum settings, teaching management, practical results, and teaching guarantees. The fourth principle encompasses comparability, operability, and quantifiability. The index system utilizes existing data and standards, ensuring consistency in the calculation methods and measurement of indicators. The indicators should be as concise as possible, with strong micro-level characteristics, easy to collect, and quantifiable.

Aesthetic education quality evaluation entails a comprehensive assessment and acknowledgment of the situation, processes, methods, objectives, and outcomes of education. This contributes significantly to the management, supervision, and enhancement of the quality and standard of education [ 20 ]. The evaluation of aesthetic education involves complexity and multidimensionality. A single indicator is far from sufficient to comprehensively and typically reflect its characteristics. Therefore, constructing a comprehensive evaluation index system requires decomposition, summarization, and refinement from multiple attributes, perspectives, and features [ 21 ]. Drawing on normative documents and prior scholarly research, an evaluation index system for the advancement of aesthetic education in universities within Anhui Province is hereby formulated, which comprises five sub-dimensions, i.e., teacher qualifications, curriculum design, teaching management, artistic practice, and teaching support, as delineated in Table 1 .

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0308446.t001

3.2 Data source and processing

To measure the development of aesthetic education in universities in Anhui Province, while ensuring the authenticity, validity, and accessibility of data, a variety of sources can be utilized. These may include annual reports on art development from each university, public documents from the Anhui Provincial Department of Education, official documents from each university’s website, and publications like the China Education Statistical Yearbook . These sources can be organized and compiled to form original cross-sectional data with a time variable set to 2020 for empirical analysis.

In a multi-indicator evaluation system, each indicator often varies in units and magnitudes due to their different nature. To address this, raw data undergo preprocessing, involving standardization and quantification of qualitative indicators. This ensures that all indicator values are on a uniform scale, facilitating comprehensive evaluation analysis.

This formula standardizes the raw data x nm , k for each indicator across all schools and years, ensuring comparability across different indicators.

3.3 Method description

In comprehensive evaluations, utilizing different methods provides analyses from various perspectives. To overcome the many differences in preference and adaptability inherent in single comprehensive evaluation methods, employing multiple evaluation methods is essential, followed by the integration of results [ 22 ]. This paper takes the 46 universities in Anhui Province as samples, constructs an indicator system, and summarizes and analyzes the development of aesthetic education in each university. Methods such as the entropy method, BP neural network method, and principal component analysis method are employed, followed by a combined evaluation approach.

The Back Propagation (BP) neural network comprehensive evaluation method offers a comprehensive and interactive approach. It effectively bypasses the inaccuracies associated with manually assigning weights and the complexities of solving correlation coefficients. Moreover, it enables comprehensive evaluation of instances with large quantities and numerous indicators [ 24 ]. The BP neural network is an artificial neural network composed of input, hidden, and output layers. The input layer nodes only receive signal inputs, while the output layer nodes perform linear weighting, and the hidden layer nodes primarily handle the most significant mathematical processing of information. During forward propagation, training samples traverse through the input layer, followed by the hidden layer, and finally to the output layer. If the output results fail to meet the expected output, the process will enter the backpropagation stage. Herein, the weights and thresholds of each neuron are continuously adjusted based on the set prediction error until approaching the expected output. The aesthetic education development in Anhui Province’s universities is assessed by inputting indicators from teacher qualifications, curriculum design, teaching management, artistic practice, and teaching support subsystems into Python. Leveraging the potent nonlinear mapping capability of BP neural networks, the data undergo iterative learning and optimization for a comprehensive evaluation.

Principal Component Analysis (PCA) is a multivariate statistical analysis method that reduces the dimensionality of data by linear transformation to select a smaller number of important variables. The basic idea is to recombine the original set of correlated indicators into a smaller set of uncorrelated composite indicators F m to replace the original indicators.

The above equation satisfies that F i and F j are uncorrelated, that is, cov( F i , F j ) = 0, and have nonzero covariances. F i is the linear combination of X 1 , X 2 , X 3 …. X p with the largest variance, that is, it is the combination of all linear combinations that are uncorrelated with each other and have the largest variance. It is constructed as the new variable indicator, namely the first, second,…, mth principal component of the original variable indicators. In practical applications, the specific steps of using Principal Component Analysis (PCA) are as follows: to standardize the original data; to calculate the covariance matrix; to find the eigenvalue λ i of R and the corresponding eigenvector a i of the orthogonalized unit; and to select principal components: through m in F 1 , F 2 , F 3 ….. F m is determined by the cumulative variance contribution rate G m .

Scores of aesthetic education in universities in Anhui Province are calculated, Principal Component Analysis is conducted using the Sklearn toolkit in Python, with the cumulative variance contribution rate set to be greater than 80%, and the scores for each university are finally computed.

3.4 Result interpretation

The entropy method, BP neural network and principal component analysis method are used to calculate the development scores of aesthetic education in 46 universities in Anhui Province. The weights of each index are shown in Table 2 :

https://doi.org/10.1371/journal.pone.0308446.t002

The weights of the indicators calculated by the three methods vary. The high rankings of both the number of teaching and research papers published by teachers and the number of art inheritance bases underscore their significance in supporting the development of aesthetic education. Research papers serve as vital data supporting the advancement of aesthetic education, while art inheritance bases enable schools, associations, and enterprises to leverage their strengths. They facilitate the discovery and cultivation of aesthetic education models and help standardize and guide aesthetic education behavior effectively. Hence, it is not difficult to understand that these two indicators occupy a large weight. In addition, significant weight is placed on the number of teachers participating in discipline competitions rated B or above, the ratio of traditional cultural associations to school associations, investment in aesthetic education funds, and the number of awards received in the previous National College Students Art Performance in Anhui Province.

After calculating the weights, the development scores of aesthetic education in 46 universities in Anhui Province are calculated according to three methods and evaluated jointly. Python software is used to draw the heat map, and the results are presented in visual form. The results are shown in Fig 1 .

https://doi.org/10.1371/journal.pone.0308446.g001

Fig 1 obviously reveals that darker colors correspond to higher scores, indicating a higher ranking in the development of aesthetic education. Observation over the last column of color blocks indicates that following the comprehensive evaluation, Anhui University, Anhui Normal University, Anqing Normal University, Chuzhou University, Anhui University of Finance and Economics, Fuyang Normal University, Anhui Academy of Fine Arts, Huangshan University, Anhui University of Engineering, and Bengbu University rank among the top ten. Reasons behind this include the fact that these universities offer five or more arts-related majors. Among them, Anhui Normal University, Anhui University, and Anqing Normal University offering over ten such majors. With sufficient funding and effective support for artistic venues, these universities prioritize professional faculty, thereby fostering comprehensive development in art courses, artistic heritage sites, and practical art endeavors. Additionally, institutions like Anhui Academy of Fine Arts specialize in cultivating artistic talents, positioning these universities at the forefront of aesthetic education development in the province.

4. Further analysis

Fig 2 illustrates a comparison of weighted indicators between research paper publications and the number of awards for artistic exhibitions across 46 universities in Anhui Province. From 2005 to 2022, in terms of the total number of awards at the national university art exhibitions organized by the Ministry of Education, universities with art schools, fine arts colleges, design colleges, and music colleges ranked prominently. A strong correlation between the quantity of research paper publications and the number of awards for artistic exhibitions is suggested by a calculated correlation coefficient of 0.899. Anhui Normal University, Anhui University, Suzhou University, Hefei Normal University, Anhui University of Finance and Economics, along with several other universities, exhibit a notable trend of both higher quantities of research paper publications related to aesthetic education and a greater number of awards for artistic exhibitions. This pattern implies the establishment of a systematic mechanism for nurturing talents, encompassing aspects such as faculty allocation, curriculum design, practical teaching methods, artistic practice, and participation in artistic exhibitions.

https://doi.org/10.1371/journal.pone.0308446.g002

In terms of overall statistical award results, participation in artistic exhibitions and subject competitions serves as a dual-purpose platform. It not only allows universities to showcase their achievements in aesthetic education but also serves as an observation point for assessing the degree to which universities prioritize efforts in aesthetic education. Through retrospective analysis, with a focus on professional settings, faculty allocation, and curriculum design for aesthetic education, it becomes evident that certain universities excel in constructing flagship or advantageous disciplines. However, this specialization may lead to fewer entries in exhibitions and competitions, consequently resulting in fewer awards. This situation could potentially relegate aesthetic education to a subsidiary or even marginal position within these universities.

An analysis is conducted on research papers regarding aesthetic education with significant weightage. Based on statistics from the China National Knowledge Infrastructure (CNKI) from 1990 to 2020, 624 research papers are examined. Python is employed for text segmentation and word frequency analysis to generate a word cloud, as shown in Fig 3 . The word cloud demonstrates "aesthetic education," "curriculum," and "art" as the three most frequently occurring terms, suggesting that art education, primarily focusing on public art courses, holds a dominant position. Among these, music and dance often occupy the forefront of aesthetic education. However, courses such as art appreciation, film and television appreciation, drama appreciation, calligraphy appreciation, and traditional Chinese opera appreciation are relatively marginalized. This is attributed to the prevalence of music and dance education at the primary and secondary school levels. Statistics also reveal that 26% of universities do not offer courses related to calligraphy, Chinese folk art, paper cutting, or intangible cultural heritage. Notably, Chinese calligraphy education appears to lack prominence, accounting for only 0.048% of research paper contributions. Furthermore, through a parallel comparison of public art courses, the structural imbalance in course offerings highlights the uneven distribution of such courses among the 46 universities.

https://doi.org/10.1371/journal.pone.0308446.g003

Differences in aesthetic education development in teaching management among the 46 universities in Anhui Province are further explored. Considering the small sample size and the teaching management indicators being categorical variables, a systematic cluster analysis method is adopted. The Ward method is used to calculate the distances between clusters using the squared Euclidean distance, and the number of clusters is determined based on the constructed pseudo-F statistic and pseudo-t 2 statistic. According to the statistics in Table 3 , nine clusters are finally obtained, as shown in Fig 4 .

https://doi.org/10.1371/journal.pone.0308446.g004

https://doi.org/10.1371/journal.pone.0308446.t003

The aesthetic education evaluation index system encompasses both basic statistics and variables. Hard indicators in teaching management include the presence of specialized public art course management departments, the number of teaching institutions, and the count of full-time staff in aesthetic education institutions or teaching departments. They have specific data requirements and are highly operational. The aesthetic management departments of the 46 universities vary widely, involving departments such as Art Education Centers, Cultural Arts and Quality Education Departments, Public Art Education Centers, Aesthetic Teaching Departments, Undergraduate Art Research Offices, Undergraduate Art Education Centers, Sports and Art Clubs, Public Art Education Clubs, Aesthetic Research Offices, etc. The diversity of management departments implies the flexibility of aesthetic education work, indicating an exploratory and practical stage in aesthetic education at universities. First, professional faculty and aesthetic education faculty may share resources, but this arrangement is uncertain and does not form a fixed teaching team. At the same time, soft indicators include the teaching syllabus of public art courses, the methods of public art course evaluation and assessment, public art course evaluations, awards, and teaching observation activities. Among these, cultural courses offered by engineering and medical universities occupy a relatively high proportion, but they are somewhat related to aesthetic education. The absence of standardized indicators among provincial universities limits the effectiveness of assessing procedural soft indicators, making it challenging to operationalize. The convergence of hard and soft indicators leads to a diverse range of teaching management approaches.

As shown in Fig 4 , a cluster analysis is conducted on the 46 universities in Anhui Province based on teaching management. The results indicate that Anqing Normal University, Chuzhou University, Anhui University of Finance and Economics, Tongling University, and Anhui Academy of Fine Arts are grouped into one cluster; Anhui University, Anhui Normal University, Anhui Agricultural University, Huangshan University, Bozhou University, Anhui Science and Technology University, and Hefei Normal University form another cluster; Bengbu Medical College, Wannan Medical College, Anhui Wenyi Information Engineering College, Fuyang Normal University School of Information Engineering, Anhui Medical University Clinical Medicine College, and Huaibei Normal University constitute a third cluster. The distribution of different types can be attributed to the setup of teaching management institutions. Specifically, 10 universities have dedicated institutions solely responsible for aesthetic education, supported by full-time staff. In contrast, 36 universities lack such dedicated staff, resulting in the delegation of aesthetic education responsibilities to various management departments. Regarding the establishment of art-related majors, six universities have more than 10 art-related majors, eight have between 7 and 9 art-related majors, 13 have between 4 and 6 art-related majors, 10 have between 1 and 3 art-related majors, and eight do not have any art-related majors. In terms of the student-teacher ratio for aesthetic education, according to the Guidelines for Public Art Courses in National Regular Higher Education Institutions , the number of teachers responsible for teaching public art courses in each school should account for 0.15% to 0.2% of the total number of students, with full-time teachers taking up 50% of the total number of art teachers. Based on the given ratio, certain medical and science/engineering universities have only one-third the necessary number of instructors for public art classes. Furthermore, universities with adequate faculty often fail to differentiate between art educators dedicated to specialized training and those teaching public art courses, thereby not offering a comprehensive array of public art courses. In terms of teaching management, responsibilities and personnel regarding faculty, courses, and practical activities should be further refined.

5. Aesthetic education comprehensive evaluation model

5.1 establishment of the model.

Herein, multiple indicators of aesthetic education development are selected to analyze and evaluate the status of aesthetic education in universities in Anhui Province. However, it remains challenging to quantitatively analyze aesthetic education development. The difficulty arises partly from some indicators being binary outcomes and the potential for the research conclusions to be limited to specific scenarios if only intuitive analysis of aesthetic education indicators is conducted. Therefore, five nonlinear indicators are hereby selected, which include the number of teacher publications on aesthetic education, the frequency of organizing school-level and above art activities, the number of awards in Anhui University Student Art Exhibition, the number of awards in provincial-level arts competitions, and the proportion of funding for cultural and folk heritage activities on campus-as the benchmark indicators for principal component analysis. A comprehensive evaluation model for aesthetic education development, considering data dispersion, is established. If the original variables are denoted as x 1 , x 2 ,⋅⋅⋅⋅, x n , and the new variables after principal component analysis are denoted as F 1 , F 2 ,⋅⋅⋅⋅, F m , where m < n, the new variables are linear combinations of the original variables. The new variable coordinate system is derived by translating and rotating the original one. The resulting space, comprising the new variables, is termed the m-dimensional principal hyperplane. On this plane, the first principal component F 1 corresponds to the direction of the largest data variation (contribution rate e1), while for F, there are e 2 ≥ e 3 ≥ ⋅⋅⋅≥ e m in sequence. As a result, the new variables effectively encapsulate the essence of the original data, while the m-dimensional hyperplane holds the utmost information content. Despite the possible presence of a slight loss of data information, this method effectively captures the primary contradictions, extracting the majority of variance from the original variables. By reducing the number of variables and focusing on the essential information, it facilitates problem analysis and processing, streamlining the overall approach.

Through principal component analysis, some of the original data information with minor contributions to the final results can be eliminated, while the main characteristic information of aesthetic education development indicators is retained. This is conducive to the analysis and processing of aesthetic education development in Anhui Province. The calculation process of the comprehensive evaluation model for aesthetic education is illustrated in Fig 5 .

https://doi.org/10.1371/journal.pone.0308446.g005

5.2 Precursors of rapid development in art education

The weight analysis in Principal Component Analysis (PCA) establishes a theoretical foundation for pinpointing sensitive indicators that mirror the advancement of art education in Anhui Province. Through PCA, a comprehensive evaluation model of art education development in Anhui Province is constructed, which incorporates multiple parameters such as the number of published papers on art education by teachers, the frequency of organizing arts events at the school and higher levels, the number of awards won by university students in the Anhui division of art exhibitions, the number of awards won in arts competitions at the provincial level and above, and the proportion of funding allocated to cultural heritage and folk art projects entering campuses. This model streamlines a comprehensive analysis of art education development in Anhui Province. Its essence lies in summing the product of each indicator and its corresponding weight across different dimensions of art education in higher education institutions within the province. A thorough investigation into the precursors of rapid development in art education aids in understanding the influence of various factors such as societal, policy, and technological factors on art education. It provides a basis for devising more effective strategies for art education development and lays the theoretical groundwork for the sustainable development of art education in Anhui Province.

Fig 6 depicts the evolution curve of the comprehensive evaluation model of art education development in Anhui Province as a function of funding input. As illustrated in the figure, the curve of the comprehensive evaluation model for art education development in Anhui Province exhibits an overall decreasing trend, with values ranging from -0.8 to 0.2. Despite the variations in numerical values and diverse changing trends among different art education indicators, indicating a certain degree of dispersion, this study takes into account the dispersion of data across these indicators. Polynomial fitting is applied to the curve of the comprehensive evaluation model for art education development in Anhui Province, using logarithmic, exponential, and polynomial functions. It is found that the correlation coefficient of the fitting curve reaches its highest value of 0.999 when a third-degree term is used, indicating a near-complete overlap between the fitting curve and the comprehensive evaluation model curve. Subsequently, the equation of the fitting curve is differentiated once and twice. The first derivative’s significance lies in indicating the rate of change of the curve in the comprehensive evaluation model for art education development in Anhui Province. It indirectly reflects the model’s sensitivity to changes in art education funding. Extreme values of the first derivative signify transitions in sensitivity states. The derivative curve of the comprehensive evaluation model for art education development in Anhui Province is shown in Fig 6 , which exhibits an overall increasing followed by decreasing trend. As art education funding increases, the first derivative peaks, with the second derivative reaching 0. Herein, the authors pinpoint this maximum of the first derivative as the precursor to rapid development in art education. At this juncture, funding input for art education in Anhui Province stands at 1,331,000 RMB, presenting a corresponding comprehensive evaluation model value of -0.502.

https://doi.org/10.1371/journal.pone.0308446.g006

5.3 Sustainable development probability of aesthetic education

Art education holds considerable significance in the sustainable development of society, emphasizing the cultivation of students’ creative thinking and innovation skills, both essential components for the enduring development of society. Creativity is a key factor driving social progress and problem-solving. Art education stimulates individuals’ independent thinking, fostering creativity and providing ongoing momentum for societal innovation. It helps to inherit and promote local culture while also facilitating communication and integration between different cultures. Art facilitates a deeper understanding and respect for multiculturalism, fostering a more harmonious and inclusive society. Besides, art education emphasizes cultivating students’ aesthetic tastes and cultural literacy, enriching individuals in the arts and enhancing their overall comprehensive qualities. Additionally, participation in artistic activities cultivates skills in interpersonal communication, cooperation, and collaboration, which are also crucial for societal sustainable development. The cultural and creative industries are integral components of modern society’s economy. Also, art education nurtures creative talents like artists, designers, and cultural professionals, fostering the development of the creative industry and establishing a cultural and economic foundation for societal sustainability. Moreover, it cultivates environmental awareness by drawing attention to environmental issues. At the same time, artworks serve as powerful tools for conveying environmental concepts, arousing concern through visual and aesthetic means, and advocating for sustainable environmental development.

When t = T, the art education funding input in Anhui Province reaches its peak. Hence, the probability value of sustainable development of art education at the peak state is 1. By computing the probability density curve of sustainable development of art education in Anhui Province based on Eq (16) , Fig 7 illustrates the evolutionary trend of the probability density of sustainable development of art education in Anhui Province. As shown in the figure, the probability density curve of sustainable development of art education in Anhui Province exhibits an overall trend of accelerating increase. The probability value of sustainable development corresponding to the precursor point of rapid development of art education in Anhui Province is 0.388. The probability values of sustainable development of art education corresponding to art education funding of 1.85 million yuan and 1.95 million yuan are 0.8 and 0.9, respectively. Examining the probability curve of sustainable art education development in Anhui Province provides access to probability values for sustainable development across different levels of art education funding.

https://doi.org/10.1371/journal.pone.0308446.g007

6. Discussions and recommendations

In Anhui Province, 10 colleges and universities offer one to three art majors, while 8 lack any art programmes. Some medical and scientific institutions have a mere one-third of the public art faculty compared to the program’s needs. Additionally, colleges with adequate faculty do not segregate those focused on professional art education from public art teachers, resulting in an incomplete public art curriculum. The authors believe that a sound teaching management organisation should be established. On the basis of the 10 colleges and universities that already have had an organisation specifically responsible for aesthetic education, it is recommended that a specialised aesthetic education management organisation with full-time staff also be established in the remaining colleges and universities. This ensures a specialised team is dedicated to overseeing and advancing aesthetic education. Secondly, the setting of art majors should be optimised. For the 8 colleges and universities without art majors, more relevant majors should be gradually set up to enrich the disciplines. Meanwhile, institutions with a limited number of art majors should be motivated to expand their offerings and evolve towards a more diverse and holistic approach. This strategy aims to accommodate the varied demands of society for artistic talent and to bolster the overall competitiveness of these colleges and universities. Colleges and universities, particularly those focused on medical and technical fields, should bolster their recruitment efforts for public art programs. This is to ensure that the faculty count meets the necessary standards, thereby enhancing the quality and breadth of aesthetic education. In addition, it is recommended that full-time art teachers and public art course teachers be divided to undertake the teaching tasks of professional education and public art courses, respectively. This ensures that the courses are fully opened. In terms of teacher management, the teacher training and assessment mechanism should be strengthened, and art teachers should be organised to participate in training and exchange activities on a regular basis, thereby improving their teaching ability and professionalism. At the same time, a scientific teacher appraisal system can be established to evaluate the teaching effectiveness and performance, so as to motivate the teachers for consistent self-improvement and optimisation of the teaching quality. In terms of curriculum, the public art curriculum system should be enriched and improved. In alignment with the distinct profiles of colleges and universities and the varied interests of their students, a diverse curriculum of art courses, spanning music, visual arts, dance, theater, and other disciplines, should be established. This curriculum is designed to cater to the broad artistic learning needs of students, enrich their practical experience, and enhance their practical skills and overall quality through a combination of campus-based and extramural internships and art practice activities.

Finally, delineating clear responsibilities and assigning specific roles is essential. Create a personalized responsibility system to ensure that the task of aesthetic education work is decomposed into specific posts and individuals, so that every task has a responsible person in charge and is subject to proper supervision. By clearly defining roles and enforcing accountability, work efficiency is enhanced, ensuring the seamless execution of aesthetic education initiatives. For example, by adding a number of art majors and actively bringing in high-calibre teachers, Zhejiang University has rapidly formed a competitive cluster of art disciplines and increased the university’s influence in the field of art education. Meanwhile, the Chinese University of Hong Kong has significantly enhanced students’ artistic qualities and practical abilities through the provision of diversified art courses and the organisation of rich art practice activities. However, the above proposal also faces challenges on various fronts, including problems in funding, resources, time and management co-ordination. Specifically, the introduction of additional specialisations necessitates considerable financial and resource investment, potentially burdening HEIs with scarce resources. Recruiting and training adequate faculty, notably in medical and polytechnic institutions, is a protracted process. Crafting and launching a varied curriculum demands significant time and resources, potentially overwhelming current educational capacities. Moreover, organizing and overseeing practical sessions involves multi-party resource coordination, presenting logistical challenges. In order to achieve an overall improvement in aesthetic education, successful implementation of these recommendations requires detailed planning and step-by-step progress by the universities according to their own specific conditions, balancing short-term goals with long-term development.

The comprehensive evaluation model established in this paper serves as a novel method for quantitatively analysing the development degree of aesthetic education. Its theoretical basis relies on the discrete characteristics of aesthetic education metrics, utilizing mathematical curve-fitting techniques, along with the application of primary and secondary derivatives for analysis. The model can effectively analyse the relationship between the input of aesthetic education and the degree of development. The primary derivative reflects the sensitivity of the comprehensive evaluation model to changes in the funding of aesthetic education, while the point of extreme value indicates a shift in the state of sensitivity. The extreme value point of the primary derivative is considered the precursor point of rapid development of aesthetic education, similar to the inflection point in the theory of marginal effect, revealing the rapid growth of aesthetic education input funding upon reaching a certain degree. In order to measure the sustainable development characteristics of aesthetic education, the authors have integrated the concept of probability value. They have delineated the probability of sustainable development in aesthetic education, grounded in a comprehensive evaluation model.

In terms of practical application, the model proposed in this paper can assist policy makers and educational administrators in identifying the key points of investment in aesthetic education, optimising the funding allocation strategy, and ensuring the effective use of resources. Analysing the derivative curves enables educational administrators to dynamically evaluate the responsiveness of aesthetic education development to inputs and the effectiveness of those inputs. This provides data support for the continuous improvement of the effectiveness of the aesthetic education teaching and learning. By identifying the precursor points of rapid development, the model can facilitate colleges and government departments to formulate long-term planning and predict the future trend of aesthetic education development. The points at which both the first and second derivatives’ extreme values equal zero signify a pivotal shift in the sensitivity of funding inputs to the advancement of aesthetic education. These points corroborate the hypothesis that the field is poised for rapid development. In terms of model validation, in future studies, the authors will apply the model to the data on aesthetic education in other provinces to validate the generalisability of the model. In addition, the accuracy of the model predictions will be verified by tracking the actual data on the development of aesthetic education in Anhui Province over time. This paper constructs a comprehensive evaluation model of aesthetic education in universities in Anhui Province based on principal component analysis. Besides, it identifies the precursor points for the rapid development of aesthetic education, and defines the probability value of the sustainable development of aesthetic education in Anhui Province. The paper’s model excels by accommodating the discrete nature of data, making it versatile for comprehensive aesthetic education evaluations across diverse datasets. Yet, its efficacy is critically dependent on the quality and completeness of aesthetic education indicator data.

In addition, the model mainly analyses the relationship between the investment in aesthetic education and the development degree, failing to comprehensively consider other factors affecting the development of aesthetic education, such as cultural background, policy environment, and students’ interests. As artificial intelligence advances rapidly, the authors intend to leverage AI technology in their forthcoming research for mining and analysing aesthetic education big data. This will encompass a broader spectrum of data related to aesthetic education, including student feedback, assessments of teaching quality, and the social impact of these educational efforts. More comprehensive analytical support will be provided. Furthermore, the authors aim to employ deep learning models to assess the collective impact of various factors on the progression of aesthetic education. These models will predict how the field may evolve under diverse conditions, thereby enhancing the precision of decision-making support. On this basis, natural language processing technology will also be utilized to analyse literature, policy documents and expert opinions related to aesthetic education. Valuable information can be extracted for model optimisation and strategy formulation, while an intelligent recommendation system can be developed to recommend optimal aesthetic education development strategies and resource allocation plans according to the specific conditions of universities. In this case, the efficiency and effectiveness of aesthetic education work can be comprehensively enhanced. The paper’s research contribution is exemplified through the case study of Anhui Province, where it introduces a comprehensive evaluation model for aesthetic education. Besides, it defines the probability value of sustainable development of aesthetic education in Anhui Province. The paper’s model stands out for its ability to handle the discrete nature of data, making it suitable for conducting comprehensive evaluations of aesthetic education across a range of data scenarios. However, the size of the data volume may affect the analysis scope of the model, yet this limitation does not impact data accuracy or the model’s predictive trends and policy suggestions for aesthetic education. The authors plan to collect a larger volume of data in future studies to enhance the comprehensive evaluation model’s analysis capabilities.

7. Conclusion

Following a thorough evaluation of art education development across 46 universities in Anhui Province utilizing a construction of 25 indicators, it becomes evident that there are variations in the development of art education among these universities. For instance, Anhui University and Anhui Normal University, representing comprehensive and normal universities respectively, rank higher in terms of overall strength and university rankings. However, universities such as the University of Science and Technology of China, Hefei University of Technology, and Anhui Medical University, despite their high overall strength, do not necessarily lead in art education. Despite advancements in higher education, art education continues to be a weak aspect within the system. However, universities such as Chuzhou University and Huangshan University, while ranking average in overall university standings, demonstrate relatively advanced development in art education.

In summary, several observations arise: Firstly, universities housing art colleges boast advantages in faculty expertise. However, there is still a need to differentiate between specialized courses for art students and those for the general public. Secondly, universities without art colleges generally offer public arts courses in literature and culture. Thirdly, while some universities offer distinctive interdisciplinary courses such as Contemporary Science and Technology Arts (USTC), Medical Humanities Film (BBMC), High-tech Ceramic Materials (AHUT), and Architectural Aesthetics (AHJZU), there is still a relative scarcity of courses focusing on interdisciplinary studies and fostering students’ innovative thinking.

According to the undergraduate talent training regulations of the 46 universities, students must complete 2 credits of public arts courses to graduate during their school years. However, most universities have not fully offered public arts courses, leading to varying degrees of imbalance between supply and demand. Consequently, students may not necessarily fulfill their 2-credit art education based on their interests, hobbies, and specialties. Art education may even become utilitarian education driven by credit orientation, neglecting the importance of aesthetic consciousness, personality development, and innovative thinking, making it necessarily important to strengthen faculty construction and curriculum system development for public arts courses.

Among the forty-six universities in the province, the total number of student art groups and art-related clubs ranges from 5 to 30. However, a higher quantity of literary and art clubs does not necessarily indicate more artistic practice activities, as some clubs may be dormant. Indeed, art education extends beyond formal coursework in public arts courses; it encompasses practical experiences like visiting art galleries, museums, and attending art exhibitions. Additionally, various professional education programs also incorporate elements of art education. Therefore, the construction of a comprehensive evaluation index system for faculty strength, curriculum design, teaching management, artistic practice, and teaching support matters considerably for improving the quality of higher education. The next step in enhancing art education involves focusing on faculty quality, uniqueness, practicality, and innovation in the reform of public arts course teaching.

Herein, a comprehensive evaluation model for the development of art education in Anhui Province is constructed based on principal component analysis. This model quantifies the impact weights of various art education indicators on the development of art education in Anhui Province, and also provides a theoretical basis for identifying the precursors of rapid development of art education in Anhui Province. Furthermore, a new method for determining the development of art education in Anhui Province is also proposed, which takes into account the discreteness of art education indicator data in Anhui Province. The comprehensive evaluation model curve for art education development in Anhui Province exhibits an overall declining trend. Building upon this observation, the authors suggest utilizing the maximum value of the first derivative of the comprehensive evaluation model as the precursor point for rapid art education development in Anhui Province. This very approach leads to the definition of a probability equation for the sustainable development of art education in the province, forging a theoretical foundation for its future development.

• 1. MOE of PRC. The Guidance Program of Public Art Curriculum of National Higher Education Institutions, Retrieved 24 May, 2022, from http://www.moe.gov.cn//srcsite/A17/moe_794/moe_624/200603/t20060308_80347.html .
• 2. General Office of the CPC Central Committee, General Office of the State Council of PRC. Opinions on Effectively Strengthening Aesthetic Education in Higher Education in the New Era, Retrieved 20 March, 2022, from http://www.gov.cn/gongbao//content/2020/content_5554511.htm .
• 3. MOE of PRC. Outline of National Medium and Long-term Education Reform and Development Plan (2010), Retrieved 17 June, from http://www.moe.gov.cn/srcsite/A17/moe_794/moe_624/201904/t20190411_377523.html .
• 4. MOE of PRC. Opinions on Comprehensively Strengthening and Improving Aesthetic Education in Schools (2015), Retrieved 8 August, 2022, from http://www.moe.gov.cn/s78/A01/s4561/jgfwzx_zcwj/202010/t20201019_495584.html .
• 5. MOE of PRC. School Art Education Work Regulations (2002), Retrieved 25 October, 2022, from http://www.moe.gov.cn/srcsite/A02/s5911/moe_621/200207/t20020725_81854.html .
• View Article
• Google Scholar
• 8. Richard PJ, Thelma CA (2012). The Art of Being Human, tenth edition, New York: Pearson Education, inc.
• 11. Howes A, Kaneva D, Swanson S, Williams J (2013). Re-envisioning STEM Education: Curriculum, Assessment and Integrated, Interdisciplinary Studies. The University of Manchester, pp.1–55.
• 15. Wang M, Zeng FR (2017). The Modern Construction of a Large Aesthetic Education System in Colleges and Universities. Chinese Higher Education, no.7, (General Serial 583), pp.7-10.
• 18. Shi CX (2020). Research on the Evaluation System and Promotion Strategies of College Students’ Artistic Quality under the Background of Aesthetic Education. China University Teaching, no.6 (General Serial 358), pp.60-63.
• 19. Shandong Provincial Department of Education. Evaluation Index System of Aesthetic Education Work in Higher Education Institutions in Shandong Province. https://edu.shandong.gov.cn . Accessed 24 March, 2022.
• 21. Yin YX (2020). Research on the Current Situation and Countermeasures of Aesthetic Education in Science and Engineering Colleges and Universities. Nanjing University of Technology. https://doi.org/10.27238/d.cnki.gnjhu.2020.000064
• 24. Wang XH, Li TJ, Ding LL (2010). Evaluation Theory and Techniques for Complex Large Systems. Jinan: Shandong University Press, pp. 80–89.
• 25. Yuan YC (2017) Evolution Law of Precursor Information of Sudden Water Influx in Tunnels and Fusion Warning Method and Engineering Application [D]. Jinan: Shandong University.
• 28. Sheng Z, Xie SQ, Pan CY (2008). Probability Theory and Mathematical Statistics[M]. Beijing: Higher Education Press.

• Open access
• Published: 01 August 2024

Literature review of complementary and alternative therapies: using text mining and analysis of trends in nursing research

• Jihye Nam   ORCID: orcid.org/0000-0002-5534-2660 1 ,
• Hyejin Lee   ORCID: orcid.org/0000-0002-8501-0560 1 ,
• Seunghyeon Lee   ORCID: orcid.org/0009-0005-6411-364X 1 &
• Hyojung Park   ORCID: orcid.org/0000-0002-7804-0593 1  

BMC Nursing volume  23 , Article number:  526 ( 2024 ) Cite this article

235 Accesses

Metrics details

This study aimed to review the literature on complementary and alternative therapies, utilizing text mining and trend analysis in nursing research. As CAM becomes increasingly prevalent in healthcare settings, a comprehensive understanding of the current research landscape is essential to guide evidence-based practice, inform clinical decision-making, and ultimately enhance patient outcomes.

This study aimed to identify CAM-related literature published from 2018 to 2023. Using the search terms 'complementary therap*', 'complementary medicine', 'alternative therap*', and 'alternative medicine', we performed a comprehensive search in eight databases, including EMBASE, Cochrane Central, PubMed Central, Korea Education and Research Information Service (RISS), Web of Science, KMbase, KISS, and CINAHL. From the text network and topic modeling analysis of 66,490 documents, 15 topics were identified. These topics were classified into two nursing-related topics through an academic classification process involving three doctors with doctoral degrees, three nurses, and three pharmacists. Based on the classified topics, research trends were comparatively analyzed by re-searching the database for 12 nursing and 22 non-nursing literature.

This study found that in nursing literature, yoga is used to improve mental symptoms such as stress and anxiety. In non-nursing literature, most of the experimental studies on complementary and alternative therapies were conducted in a randomized manner, confirming that a variety of physiological and objective indicators were used. Additionally, it was discovered that there were differences in the diversity of research subjects and research design methods for the same intervention method. Therefore, future research should focus on broadening the scope of subjects and measurement tools in nursing studies. Additionally, such studies should be conducted with randomization and generalizability in the experimental design in mind.

This study employed text network analysis and text mining to identify domestic and international CAM research trends. Our novel approach combined big data-derived keywords with a systematic classification method, proposing a new methodological strategy for trend analysis. Future nursing research should focus on broadening the scope of subjects, diversifying measurement tools, and emphasizing randomization and generalizability in experimental designs.

Peer Review reports

The World Health Organization (WHO) defines complementary and alternative medicine (CAM) as healthcare practices outside a country's traditional or conventional medicine [ 1 ]. According to the National Center for Complementary and Integrative Health (NCCIH), CAM encompasses nutritional approaches (e.g., herbs), psychological methods (e.g., mindfulness), physical therapies (e.g., massage), integrated mind–body practices (e.g., yoga or auricular acupressure), and modalities that combine psychology and nutrition [ 2 ]. This definition suggests CAM may facilitate holistic nursing by addressing both psychological and physical aspects [ 3 ]. Consequently, substantial CAM research is conducted in nursing internationally [ 4 , 5 ], spanning areas like pain, depression, anxiety, chronic disease symptoms, sleep disturbances, and vomiting [ 4 , 5 , 6 ]. Classification systems exist, with the Korean Nursing Association (2023) delineating 12 CAM subcategories [ 6 ] and NCCIH outlining 76 therapies across major categories like nutrition, body, and psychotherapies [ 4 , 5 , 6 , 7 ]. The multitude of CAM types has prompted trend identification research, including reviews on Chinese medicine for allergic rhinitis, aromatherapy, auricular acupressure, and CAM for COVID-19 [ 6 , 8 ]. However, many previous studies have significant limitations in comprehensively identifying overall research trends in CAM. First, they tend to focus narrowly on specific diseases or treatments, lacking a broader perspective on the field as a whole [ 6 , 8 ]. Second, the use of search queries containing keywords from a specific discipline or arbitrarily selected by researchers introduces bias and hinders the identification of overarching trends [ 9 , 10 ]. These limitations highlight the need for a more systematic and data-driven approach to analyzing CAM research trends [ 11 , 12 , 13 ]. A previous study [ 14 , 15 ] suggested the use of text mining technique as an approach for literature review [ 16 ]. To date, the analysis on research trend in nursing has been conducted more than five years after publication or has only been conducted with partial analyses through literature reviews and text mining [ 17 , 18 , 19 ].

The overarching goal was to extract keywords identifying domestic and international CAM research trends using text network analysis and analyze these trends within the nursing field. Specific objectives were: 1) Identify frequency, degree centrality, closeness centrality, and betweenness centrality for keywords appearing in domestic and international CAM studies; 2) Identify key themes within these studies; 3) Discern nursing keywords among sub-topic groups; 4) Analyze and compare nursing and other disciplinary literature based on findings; and 5) Analyze the trend of CAM in nursing based on extracted nursing keywords.

Study design and methodological framework

This study employs a novel methodological framework that combines text mining techniques with expert validation to identify and analyze CAM research trends in a comprehensive and data-driven manner. The framework consists of the following key steps.

Data collection: A comprehensive search of multiple databases is conducted to collect a broad range of CAM-related literature across various disciplines.

Text preprocessing involves several techniques to prepare the data for analysis. These include natural language processing, stopword removal, and synonym standardization.

Keyword extraction and network analysis: Text mining techniques, including term frequency-inverse document frequency (TF-IDF) and centrality analysis, are applied to extract key topics and analyze their relationships within the literature.

Topic modeling: Latent Dirichlet Allocation (LDA) is used to identify latent topics within the literature and visualize their proportions and relationships.

Expert validation: An expert panel of physicians, nurses, and pharmacists is consulted to validate the relevance and credibility of the identified topics and classify them into respective academic fields.

Focused literature analysis: Based on the expert-validated nursing-related topics, a focused re-search and analysis of the literature are conducted to identify trends specific to nursing research on CAM.

This multi-step framework allows for a more comprehensive and less biased exploration of CAM research trends by leveraging text mining techniques to process large volumes of literature, identify key topics, and uncover patterns that may not be apparent through traditional review methods [ 14 , 15 , 16 ]. The integration of expert validation ensures the relevance and credibility of the findings, while the focused analysis of nursing literature provides insights specific to the nursing discipline within the broader context of CAM research. The process of selecting studies for our analysis is illustrated in Fig.  1 , which provides a clear visual representation of the key steps involved, from the initial database search to the final classification of studies into nursing and other disciplines. This multi-step approach, combined with the visual aid, enhances the clarity and transparency of our methodology, allowing readers to better understand and contextualize the subsequent data analysis steps.

Flow diagram for literature selection process

Literature selection

This study focused on complementary and alternative medicine studies conducted in the fields of medicine, public health, and nursing in Korea and abroad. After specifying the research title and abstract as the search scope to extract the literature and build a database, the literature related to nursing was classified based on the topics derived through text network analysis and then, the literature that met the selection criteria was secondarily extracted and analyzed through the abstract screening. The three researchers checked the consistency of the study selection process and if there was any discrepancy, the final decision was made through consensus among the researchers.

The selection criteria for the literature were: (1) domestic and foreign studies published within the last five years (January 2018 to September 2023) that conducted studies on complementary and alternative medicine; and

The exclusion criteria for the literature were grey literatures, dissertations, and studies for which original texts are not available.

Data collection strategies

In this study, the database was selected by referring to the COSI (Core, Standard, Ideal) [ 20 ] model presented by the National Library of Medicine for literature search. EMBASE, Cochrane Central, and PubMed Central were selected as the core databases.

On the other hand, the standard databases selected were the Cumulative Index to Nursing and Allied Health Literature (CINAHL); and Korean database services such as the Research Information Sharing Service (RISS), KMbase, and Korean studies Information Service System (KISS). These Korean databases were included to ensure a comprehensive coverage of potentially relevant studies published in South Korea, as they index a wide range of domestic and international journals, conference proceedings, and dissertations across various disciplines, including those related to CAM. However, it is important to note that the inclusion of these Korean databases does not limit the scope of our study to Korean literature only, as the majority of our analysis focuses on studies published in English and indexed in the core and standard international databases.

In addition, the Web of Science was selected to include a wider range of literature for the ideal database, and the period of literature search focused on the last five years, from 1 January 2018 to 15 September 2023. to capture the most recent trends in CAM research following the last comprehensive analysis of CAM research trends conducted in 2018 by Sung et al. [ 19 ]. This time frame was chosen to provide an updated and comprehensive analysis of CAM research trends, building upon the findings of previous studies and identifying new patterns and areas of focus that have emerged in recent years, given the rapid evolution of CAM research and the increasing integration of CAM into mainstream healthcare.

The data collection procedure was limited for both domestic and foreign studies. In case of foreign studies, ‘English’ was limited as the search language, ‘abstract and title’ were identified as the field, ‘article’ was set as the document form, and the keywords were ‘complementary therap*,’ ‘complementary medicine,’ ‘alternative therap*,’ and ‘alternative medicine.’ For the Korean studies, ‘Korean’ was limited as the search language, ‘abstract and title’ were identified as the field, ‘article’ was set as the document form, and the search keywords used were ‘보완대체,’ ‘대체요법,’ and ‘대체의학.’ In searching for the secondary literature, studies in the field of nursing were presented to a group of nine experts including physicians, nurses, and pharmacists with a master's degree or higher, and then the relevant areas were classified to extract the keywords. These keywords were then used in the text mining search. Topic words, the majority of which were classified as nursing, were re-searched in the collected database. The literature selection and classification process were carried out independently by three researchers and promoted through discussions between the researchers.

Data analysis process

Data extraction.

A comprehensive literature search was conducted across eight databases: CINAHL, Cochrane, EMBASE, KISS, Kmbase, PubMed, RISS, and Web of Science. This extensive search yielded a total of 77,062 studies. To ensure the integrity and non-redundancy of our dataset, we employed a rigorous two-step deduplication process. First, we utilized the 'Find Duplicates' function in EndNote software for initial automatic deduplication. This function systematically identifies and groups potential duplicate records based on shared metadata such as title, authors, year, and DOI. Through this automated process, 12,107 duplicate records were identified and removed.

Following the automated process, we conducted a manual review to identify and remove any remaining duplicates that the software might have missed. This careful manual screening allowed us to catch subtle duplicates that automated systems might overlook, such as those with slight variations in titles or author names. Through this manual review, an additional 465 duplicate records were identified and removed. In total, our rigorous two-step deduplication process resulted in the removal of 12,572 duplicate records. Of these, 12,107 were removed through automated deduplication and 465 through manual review. After deduplication, 64,490 unique studies were retained for further analysis. These studies were systematically organized by title and subjected to a thorough text preprocessing phase. During this phase, unstructured words were sorted and cleaned using the social networking program Netminer 4.3.3 and text editor Notepad + + (version 8.5.8).

Also, stopwords such as pronouns, adverbs, and numbers were deleted through natural language processing, while exception list, defined words, and thesaurus were registered. The exception list and thesaurus were selected by the three researchers, and if they failed to reach a unanimous agreement, the keywords were refined through consultation and the abstracts and preambles were reviewed again to examine the context in which the words were used. In case of the exception list, literature search keywords or stopwords such as pronouns, adverbs, numbers, and special symbols were considered, while ‘complementary,’ ‘medicine,’ ‘alternative,’ ‘therapeutic,’ ‘therapy,’ ‘therap,’ ‘therapies,’ ‘the,’ ‘a,’ ‘and,’ ‘of,’ ‘for,’ ‘in,’ ‘to,’ and ‘among’ were excluded. Special symbols like ‘’,:'"()&-?# <  >  + "",‘ were excluded as well. As for defined words, ‘cells → cell,’ ‘effects → effect,’ ‘staphylococcus aureus → staphylococcus,’ ‘aureus → staphylococcus,’ ‘characteristics → characterization,’ ‘efficacy → effect,’ ‘rat → mice,’ ‘radio → radiation,’ ‘systems → system,’ ‘agents → agent,’ ‘activity → activation,’ ‘carcinoma → cancer,’ ‘cases → case,’ ‘mouse → mice,’ ‘practices → practice,’ ‘radio sensitization → radiation,’ ‘years → year,’ ‘α → alpha,’ and ‘β → beta’ were selected, and data sorting for synonyms was performed. As a result of the analysis, a database consisting of 464,625 words was constructed.

Data analysis

In this study, text mining and topic modeling analysis were employed using textom and RStudio (4.3) to identify keywords related to CAM. Word analysis, TF-IDF, and degree centrality analysis were performed through text mining, with results presented via visualization. TF-IDF determines if a keyword holds actual significance within a document, as words with high TF and TF-IDF values appear frequently and are more likely keywords or important terms [ 21 , 22 ]. Following previous studies [ 22 , 23 ], the minimum word length was set to two, with the top 20 words extracted per topic. Text network analysis created word networks expressing co-occurrence frequency as links [ 24 ]. To gauge word occurrence frequency, words were converted to word-word one-mode, and degree centrality analysis identified highly influential network words. The results of these analyses, including frequency, TF-IDF, degree centrality, closeness centrality, and betweenness centrality of core keywords, can be found in Table 1 .

This study utilized Latent Dirichlet Allocation (LDA) for topic modeling, a statistical method that estimates the probability distribution of topics within documents based on the Document Term Matrix (DTM). Following established practices in the literature, we set the Markov Chain Monte Carlo (MCMC) parameters to alpha = 1.44, beta = 0.001, and iterations = 1,000 [ 25 ]. To determine the optimal number of topics, we iteratively tested configurations ranging from 1 to 20 topics. Through a combination of silhouette clustering analysis and researcher consensus, we identified that a 15-topic model best represented the research trends in our corpus.

LDA visualization indicated that larger topic sizes represented greater proportions within the analyzed studies [ 25 ]. We confirmed that the ideal number of topics, where topics do not overlap and have distinct boundaries, is 15, as shown in Fig.  2 . To validate the relevance and credibility of the topic modeling results, we consulted an expert panel consisting of physicians, nurses, and pharmacists with master's or doctoral degrees. The panel members were asked to classify the 15 derived topics into their respective academic fields. Based on the survey results, two topics (Topics 4 and 7) were identified as nursing-related, with the majority of the expert panel categorizing them as such.

LDA topic modeling visualization

Using the words from these two nursing-related topics, a keyword search was conducted within the database to identify the final set of literature containing these terms. The selected literature was then classified as either nursing-related or non-nursing-related based on the following criteria: (1) the study was published by a nursing school or department, (2) the authors were nurses or nursing researchers, (3) the authors were hospital-affiliated nurses, or (4) the study was published in a nursing journal. The classification process was carried out independently by three authors, and the final categorization was determined through a verification process among them.

Literature review

After the three researchers re-searched the database built based on the sub-words of the extracted topics, a total of 35 articles were selected, including 13 nursing-related literatures and 22 other discipline-related literatures. The sub-words used for the re-search were derived from Topic 4 and Topic 7 in Table 2  and were classified using the PICO (Population, Intervention, Comparison, Outcome) framework. The population-related sub-words included 'patient,' 'students,' and 'nursing.' The intervention-related sub-words were 'yoga,' 'treatment,' 'radiation,' 'acupuncture,' 'education,' and 'cam.' The comparison-related sub-word was 'placebo,' and the outcome-related sub-words included 'anxiety,' 'depression,' 'symptoms,' 'knowledge,' 'attitudes,' and 'perceptions.' These PICO-classified sub-words were used to conduct the database re-search.

In order to examine the research trends in nursing and other related fields, general characteristics (author, country of publication, year of publication) and research characteristics (research design model, statistical method, research subject, intervention method, outcome variable, measurement instruments) were identified, presented, and compared. Meanwhile, the three researchers independently prepared a characteristic table to ensure the accuracy of the extracted contents and if there was any discrepancy, one data was selected through the discussion process until a consensus was reached and a characteristic table was constructed.

To assess the quality of the selected studies, we employed the Mixed Methods Appraisal Tool (MMAT), a concise tool designed to evaluate various study designs, including qualitative, randomized controlled trials, non-randomized studies, quantitative descriptive studies, and mixed methods studies [ 26 ]. This comprehensive tool allowed us to systematically evaluate the methodological rigor of our diverse selection of studies. Each study was evaluated against five MMAT criteria specific to its design, focusing on aspects such as research question appropriateness, data collection methods, and result interpretation. Our assessment revealed varying levels of methodological quality. Among nursing studies (A1-A12), 25% were high quality (5/5 criteria met), 58.3% moderate quality (4/5 criteria), and 16.7% low quality (3/5 criteria). Importantly, all included studies met at least 3 out of the 5 MMAT criteria, indicating an overall moderate to high quality across the selected literature. This suggests that the studies included in our analysis provide a reliable foundation for drawing conclusions. Studies that did not meet all criteria were carefully reviewed, and their potential limitations were considered when interpreting their findings. The MMAT provided a useful overview of study quality and was deemed suitable for assessing methodological rigor while maintaining the feasibility of our analysis. This approach ensured a balanced and nuanced interpretation of the evidence in the field of complementary and alternative medicine. The detailed results of this quality assessment can be found in Tables 3  and 4 .

Data collection and ethical considerations

Since the data used in this study did not contain information that can identify individuals, the study was conducted after obtaining an IRB approval (IRB No: ewha-202311–0008-01) from the Institutional Review Board of Ewha Womans University.

Analysis of word frequency and centrality

The frequency and percentage of the top 20 words related to complementary and alternative medicine are shown in Table 1 . The frequency and percentage of the top 20 words related to complementary and alternative medicine are shown in Table 1 . The table presents the top 20 keywords ranked by frequency, TF-IDF, degree centrality, closeness centrality, and betweenness centrality. The frequency column indicates the number of times each keyword appears in the analyzed documents, while the TF-IDF column represents the importance of each keyword within the entire document set. Degree centrality, closeness centrality, and betweenness centrality are network analysis measures that indicate the importance and influence of each keyword within the text network. The words with the highest frequency included ‘cell’ (7,653 times), ‘patient’ (6,910 times), ‘treatment’ (6,851 times), ‘cancer’ (6,722 times), ‘study’ (6,295 times), and ‘effect’ (6,203 times). The words with the highest values of TF-IDF, in order, were ‘cell,’ ‘effect,’ ‘cancer,’ ‘patient,’ ‘treatment,’ and ‘study.’ As a result of centrality analysis, the top six common words, in order, were ‘effect,’ ‘treatment,’ ‘study,’ ‘analysis,’ ‘disease,’ and ‘approach.’ Except for common words, the words with the highest values in the centrality analysis, in order, were ‘model,’ ‘patient,’ ‘activation,’ and ‘use.’ The words with the highest values for closeness centrality were ‘factor,’ ‘model,’ ‘patient,’ and ‘activation,’ while the words with the highest values for betweenness centrality were ‘factor,’ ‘model,’ ‘type,’ and ‘activation.’

Results of the topic modeling

The LDA visualization provides insights into the relative importance and distinctiveness of identified topics. In this visualization, the size of each topic circle is proportional to its prevalence within the analyzed corpus, with larger circles indicating topics that are more frequently discussed across the literature. Interestingly, we observed that some topics, despite being represented by smaller circles, were positioned at considerable distances from other topics. This spatial separation suggests that these topics, while perhaps less prevalent, possess high discriminant validity and represent distinct thematic areas within the field of complementary and alternative medicine research. This interpretation is consistent with established principles in topic modeling, where spatial relationships in visualizations can indicate semantic distinctiveness. An expert panel of 9 individuals (3 doctors, 3 nurses, and 3 pharmacists), each holding a master's or doctoral degree, conducted a survey to classify the topics based on the keywords. The topic that received the most votes from the panel was designated as the representative field for that topic. Based on the resulting values of the topic modeling, 20 sub-words for each topic were presented and provided in Table 2 , Topics 1–3, 5–6, and 9–15 were classified as Medicine, Topics 4 and 7 as Nursing, and Topics 8 and 10 as Pharmacology.

The process of selecting studies for our analysis is illustrated in Fig.  2 . To determine the optimal number of topics for our analysis, we conducted Latent Dirichlet Allocation (LDA) visualization. As Greene et al. [ 25 ] suggest, larger topic sizes in LDA visualization indicate a greater proportion of that topic within the analyzed studies. We tested topic numbers ranging from 1 to 20, seeking a configuration where topics were visually distinct and non-overlapping. This approach aligns with Liu et al. [ 24 ], who note that topics with high discriminant validity appear as small but clearly separated clusters. After careful visual analysis, we determined that 15 topics provided the most coherent and distinct groupings, as shown in Fig.  2 . This visualization demonstrates the independence and non-overlapping nature of our identified topics, supporting the robustness of our topic modeling approach. Based on the resulting values of the topic modeling, 20 sub-words for each topic were presented and provided in Table 2 . The expert panel's classification suggested that Topics 4 and 7 had relevance to nursing research. However, upon closer examination of the keywords included in these topics, it became apparent that they also encompassed literature from other medical disciplines. While the expert panel's classification indicated these topics were nursing-related, the presence of medical terminology suggested a broader interdisciplinary scope. This highlighted the limitations in identifying nursing-specific research using the current topic modeling approach. To address this issue and clarify the nursing-specific research within these topics, a further refinement of the literature search was conducted using the PICO framework. The keywords from Topics 4 and 7 were used to formulate a focused research question and search strategy. This targeted approach yielded a final selection of 12 nursing-specific articles and 22 articles from other disciplines. By employing the PICO framework and leveraging the keywords from the identified nursing-related topics, it was possible to isolate the nursing research within the broader interdisciplinary landscape.

The words included in topic 4 were the following: ‘trial,’ ‘effect,’ ‘yoga,’ ‘treatment,’ ‘radiation,’ ‘phage,’ ‘protocol,’ ‘anxiety,’ ‘dose,’ ‘zinc,’ ‘symptoms,’ ‘depression,’ ‘placebo,’ ‘acupuncture,’ ‘feasibility,’ ‘training,’ ‘insights,’ ‘toxicity,’ ‘mri,’ and ‘emergency.’ The words included in the topic 7 were: ‘role,’ ‘survey,’ ‘practice,’ ‘evidence,’ ‘failure,’ ‘utilization,’ ‘heart,’ ‘students,’ ‘cam,’ education,’ ‘healthcare,’ ‘valve,’ ‘knowledge,’ ‘communication,’ ‘narrative,’ ‘practitioners,’ ‘attitudes,’ ‘nursing,’ ‘perceptions,’ and ‘pseudomonas.’

The characteristics of the 12 studies included in the literature review analysis are shown in Table 3 .

Of the 12 final literature selections in nursing, there were four randomized controlled trials [A2] [A4] [A7] [A8], three non-randomized comparative trials [A3] [A5] [A6], four descriptive survey studies [A1] [A9] [A10] [A11], and one qualitative study [A12]. Regarding the country of the study’s publication, there were five studies from the United States, three from the United Kingdom, two from Germany and Turkey, and one from Australia. As for the statistical techniques that appeared with high frequency, 10 studies, which were [A1] [A2] [A3] [A4] [A5] [A7] [A8] [A9] [A10] [A11,] used independent t-test, and it was used in most studies. On the other hand, χ2 test was used in seven studies [A3] [A4] [A7] [A8] [A9] [A10] [A11] and one-way analysis of variance was used in four studies [A1] [A9] [A10] [A11]. Regarding the studies that were conducted targeting patients, there was one study conducted on cancer patients [A5], one study on women with post-traumatic stress disorder caused by a car accident [A8], one study on hypertension patients [A7], and one study on breast cancer patients undergoing chemoradiotherapy [A4]. There were seven studies conducted on medical staffs [A1] [A3] [A6] [A9] [A10] [A11] [A12] and one study conducted on nursing students [A2]. Among the interventional therapies used in clinical trials, the most common one was yoga, which was identified in three studies. Specifically, there was one study that used yoga therapy for chemotherapy patients [A5], laughter yoga for nursing students [A2], and yoga therapy for women with post-traumatic disorder [A8]. There were also studies conducted on virtual cancer education program [A6], education on complementary and alternative medicine [A3], auricular acupressure for hypertensive patients [A7], and music therapy for those with breast cancer [A4]. In the studies conducted among medical professionals and nursing students, knowledge [A1] [A3] [A6] [A9] [A10] [A11], attitudes [A1] [A3] [A10] [A11], and usage surveys [A1] [A11] were identified as measurement variables, whereas depression [A8], pain [A7], quality of life [A7], and anxiety [A8] [A4] were identified as the measurement variables in the studies conducted on patients.

Other disciplines

The detailed characteristics of these studies, including the study design, sample, intervention, statistical methods, and outcome measures, are presented in Table 4 .

Of the 22 final literature selections in other disciplines, there were 20 randomized controlled trials [B1] [B2] [B3] [B5] [B6] [B7] [B8] [B9] [B10] [B11] [B12] [B13] [B14] [B15] [B16] [B17] [B18] [B20] [B21] [B22], one pre- and post-hoc comparative study [B4], and one scoping review [B19]. The detailed characteristics of these studies, including the study design, sample, intervention, statistical methods, and outcome measures, are presented in Table 4 . Regarding the country of the study's publication, there were seven studies from the United States of America and the United Kingdom, three studies from China, two studies from the Netherlands, and one study each from Germany, India, and Hong Kong. As for the statistical techniques that appeared with high frequency, there were 10 studies that used independent t-test [B2] [B3] [B5] [B6] [B8] [B11] [B13] [B15] [B18] and one-way ANOVA [B3] [B6] [B7] [B9] [B11] [B14] [B18] [B21] [B20] [B22], while seven studies used repeated measures ANOVA [B2] [B4] [B10] [B11] [B15] [B20] [B22]. All studies for the literature review were conducted on patients. The most common intervention used was auricular acupressure, which was applied on patients with Parkinson’s disease [B11], poststroke depression [B6] [B14], insomnia and depression [B20] [B21], carpal tunnel syndrome [B7], soldiers with PTSD [B19], migraine [B15], pelvic organ prolapse [B8], and gallbladder stones [B22]. The second most common intervention used was yoga therapy, and the subjects were those with active arthritis [B18], generalized anxiety disorder [B17], hemodialysis [B4], and hypertension [B2]. Other subjects and interventions shown in the studies were the following: irritable bladder syndrome patients treated with cinnamon patch [B13]; depression patients treated with bouldering psychotherapy [B12]; dementia patients treated with aromatherapy [B10]; insomnia patients treated with Tai-chi and meridian pressure [B9]; Crohn’s disease patients treated with moxibustion [B3]; HIV patients treated with green tea [B5]; and peripheral arterial disease patients treated with laser acupuncture [B1]. On the other hand, the following were identified as the measurement variables for yoga intervention: level of depression, arthritis stage, anxiety level, quality of life, treatment response rate, sleep, and autonomic function [B2] [B4] [B16] [B17] [B18]. Measurement variables for auricular acupressure included level of depression, sleep quality, level of pain, physical and psychological symptoms, severity of depressive symptoms pelvic organ prolapse, and gastrointestinal symptoms [B3] [B6] [B7] [B8] [B11] [B14] [B15] [B19] [B20] [B21] [B22].

In the study conducted using cinnamon patches, the overactive bladder symptom scores and residual urine volume after urination were identified [B13]. In the study which used green tea, the level of depression was assessed while measuring the severity of depressive symptoms through bouldering [B12]. In the study that used aromatherapy, the behavior, psychology, daily living ability, and cognitive function of the patients with dementia were also assessed [B10].

The present study employed text mining techniques to analyze the literature on CAM published over the past five years and identify trends in nursing research. The text network analysis revealed keywords with high TF-IDF and degree centrality, such as 'cell', 'patient', 'treatment', 'cancer', 'study', and 'effect', suggesting a strong focus on cellular mechanisms, patient-centered approaches, and treatment effects, particularly in the context of cancer [ 22 , 23 ]. The high centrality of these keywords indicates their importance and influence within the broader network of CAM research [ 24 , 25 ]. The topic modeling approach identified 15 major topics, providing a comprehensive overview of the key areas of focus in recent CAM research. This data-driven method offers a more nuanced understanding of research trends compared to previous studies that relied on arbitrary searches or focused on narrow populations or interventions [ 27 , 28 , 29 , 30 , 31 ]. By employing this systematic approach, the present study captures the breadth and diversity of CAM research, overcoming the limitations of previous nursing studies.

An expert panel of 9 individuals (3 doctors, 3 nurses, and 3 pharmacists), each holding a master's or doctoral degree, conducted a survey to classify topics based on keywords. According to the expert classification results shown in Table 2 , Topics 1–3, 5–6, and 9–15 were classified as Medicine, Topics 4 and 7 as Nursing, and Topics 8 and 10 as Pharmacology. While Topics 4 and 7 were found to be nursing-related, closer examination revealed the presence of literature from other medical disciplines within these topics. To address this issue and clarify the nursing-specific research, a further refinement of the literature search was conducted using the PICO framework. The keywords from Topics 4 and 7 were used to formulate a focused research question and search strategy, yielding a final selection of 34 articles, with 12 nursing-specific articles and 22 articles from other disciplines. Analyzing trends in nursing and interdisciplinary studies within the context of the existing literature provides a more comprehensive understanding of CAM research trends. From a nursing perspective, the identification of topics related to patient care, such as symptom management, quality of life, and patient education, highlights the potential for CAM interventions to improve patient outcomes and experiences. The prominence of keywords such as 'patient', 'treatment', and 'effect' highlights the need for evidence-based practice and the need for rigorous studies to evaluate the efficacy and safety of CAM interventions in nursing care. Furthermore, the expert panel's validation of Topics 4 and 7 as relevant to nursing research emphasizes the relevance of these areas within the nursing discipline. Topic 4, which includes keywords such as 'trial', 'effect', 'yoga', 'anxiety', and 'depression', suggests a focus on the psychological benefits of CAM interventions, particularly in the context of clinical trials. This aligns with the growing recognition of the importance of holistic, patient-centered care in nursing practice [ 3 , 4 ]. Topic 7, which includes keywords such as 'practice', 'evidence', 'education', 'knowledge', and 'attitudes', highlights the importance of evidence-based practice and the need for nurse education and training in CAM. As CAM interventions become increasingly popular among patients, it is crucial for nurses to have the knowledge and skills needed to provide safe and effective care [ 5 , 6 ]. The insights gained from this study highlight the potential of text mining and topic modeling techniques for investigating research trends in various fields [ 11 , 12 , 13 ]. By leveraging these methods, researchers can systematically analyze large volumes of literature, identify key areas of focus, and uncover patterns and trends that may not be apparent through traditional review methods [ 14 , 15 ]. This approach can lead to a more comprehensive understanding of the current state of research and inform future directions for investigation.

In conclusion, the present study demonstrates the value of text mining and topic modeling techniques in analyzing research trends, particularly in the field of CAM [ 9 , 10 ]. The systematic approach employed in this study allowed for a more comprehensive and data-driven exploration of the research landscape, overcoming the limitations of previous studies and providing valuable insights into the trends in nursing research on CAM. The findings of this study have significant implications for nursing practice, highlighting the need for evidence-based approaches, patient-centered care, and the integration of CAM interventions into nursing education and training. Future studies should consider adopting similar methodological approaches to investigate research trends in other fields, as this can lead to a more complete understanding of the current state of research and inform future directions for investigation.

The trends analysis of nursing and interdisciplinary studies on CAM revealed notable differences in research design, subject characteristics, intervention types, and assessment methods. Nursing studies exhibited a more balanced distribution of research designs, including randomized controlled trials [A2, A4, A7, A8], non-randomized comparative trials [A3, A5, A6], descriptive survey studies [A1, A9-A11], and a qualitative study [A12]. In contrast, other disciplines predominantly utilized experimental designs, with 95.2% of the studies being randomized controlled trials [B1-B3, B5-B18, B20-B22]. This disparity suggests that nursing research on CAM should expand its focus on experimental studies to enhance the evidence base and align with the methodological approaches of other disciplines.

The subject characteristics of nursing studies differed significantly from those of other disciplines, with nursing research primarily focusing on healthcare professionals and students [A1, A3, A6, A9-A12], while other disciplines exclusively studied patient populations [B1-B22]. This highlights the need for nursing research to diversify its study subjects and investigate the effects of CAM interventions on patients and healthcare providers [ 28 , 29 ], as well as broader community and general health populations [ 3 , 6 ]. By expanding its scope, nursing research can provide valuable insights into the effectiveness and applicability of CAM interventions in promoting health and well-being across diverse settings and populations [ 4 , 5 , 7 , 8 ]. Nurses, as frontline healthcare providers, are uniquely positioned to bridge the gap between healthcare settings and the community, engaging with patients and community members to assess their health needs and provide evidence-based recommendations for CAM interventions [ 1 , 2 ]. This expanded focus, coupled with interdisciplinary collaboration and knowledge exchange [ 9 , 10 ], can lead to the development of innovative, culturally sensitive, and evidence-based CAM interventions that address the complex health needs of individuals and communities alike.

A closer examination of the intervention types in nursing studies reveals that although they focused on a relatively limited range of CAM modalities, such as yoga [A2, A5, A8] and auricular acupressure [A7], these interventions demonstrated promising potential for managing various symptoms and conditions. For instance, yoga was found to be effective in reducing psychological symptoms and cortisol levels in college students [A2], alleviating chemotherapy-related symptoms in cancer patients [A5], and improving post-traumatic stress disorder among traffic accident survivors [A8]. Similarly, auricular acupressure was shown to help decrease angina symptoms in hypertensive patients [A7]. These research findings suggest that even though the scope of CAM interventions in nursing research may be limited, they can provide significant benefits to diverse patient populations [ 2 , 4 , 22 ]. In contrast, the wide array of CAM interventions investigated in other disciplines, such as aromatherapy for dementia [B10], green tea for depression in HIV patients [B5], laser acupuncture for peripheral arterial disease [B1], cinnamon patch for irritable bladder syndrome [B13], bouldering psychotherapy for depression [B12], Tai-chi and meridian pressure for insomnia [B9], and moxibustion for Crohn's disease [B3], demonstrates the potential for nursing research to explore and apply new therapies. The safety, efficacy, and potential of these diverse CAM modalities, as evidenced in other disciplines [ 23 , 24 ], should encourage nursing researchers to investigate their applicability in patient care. By conducting rigorous studies on the safety and efficacy of various CAM interventions, nursing research can provide valuable evidence to support the integration of complementary therapies into nursing practice [ 2 , 4 , 22 ]. Moreover, this trends analysis emphasizes the importance of studying CAM interventions for chronic disease management. With the increasing prevalence of chronic conditions [ 1 , 9 , 10 ], nursing research can play a pivotal role in evaluating the effectiveness of CAM for managing these diseases. Studies on yoga for hypertension [B2] and arthritis [B18], auricular acupressure for insomnia and depression [B20, B21], and moxibustion for Crohn's disease [B3] demonstrate the potential of CAM in improving patient outcomes and quality of life. As nurses have more direct and prolonged contact with patients compared to other healthcare professionals, they are well-positioned to assess the effectiveness of CAM interventions in both clinical and community settings [ 3 , 5 ]. By conducting well-designed studies on the safety and efficacy of various CAM modalities, nursing research can provide the necessary evidence to support the integration of complementary therapies into chronic disease management plans, ultimately enhancing patient care and outcomes across diverse settings. Leveraging their unique role in patient care and conducting rigorous studies on the safety and efficacy of various CAM interventions, particularly for chronic disease management, can enable nursing research to make significant contributions to the integration of complementary therapies into nursing practice. This approach has the potential to not only improve patient outcomes and experiences but also strengthen the evidence base for CAM in healthcare, fostering interdisciplinary collaboration in CAM research and advancing the field of nursing.

The analysis of assessment methods revealed that nursing studies heavily relied on self-developed measurement instruments (58.3%) [A3, A5, A6, A9-A12], while other disciplines predominantly used previously validated tools [B1-B22]. Furthermore, nursing studies rarely incorporated physiological indicators (8.3%) [A2], in contrast to the more frequent use of such measures in other disciplines (36.3%) [B1-B22]. These findings underscore the importance of utilizing validated assessment tools and physiological indicators in nursing research to enhance the reliability and validity of study results [ 31 ]. By incorporating these objective measures, nursing research can more clearly identify significant factors and strengthen the level of evidence, ultimately improving the credibility and applicability of the results.

The trends analysis of statistical techniques revealed a higher prevalence of independent t-tests in nursing research (83.3%) [A1-A5, A7-A11], while other disciplines showed a more balanced use of various techniques, including one-way ANOVA (45.5%) [B3, B6, B7, B9, B11, B14, B18, B20-B22] and repeated measures ANOVA (31.8%) [B2, B4, B10, B11, B15, B20, B22]. This difference can be attributed to the nature of the dependent variables assessed in each field, with nursing studies primarily focusing on single assessments of knowledge, attitudes, education, beliefs, and symptoms [A1, A3-A11], whereas other disciplines frequently employed repeated measures of pain, depression, response rate, serum levels, and neurological outcomes [B2-B4, B6-B8, B10, B11, B14-B22]. These findings underscore the importance of aligning the choice of statistical techniques with the nature of the outcome measures to ensure the validity and reliability of the research findings.

In conclusion, the trends analysis of nursing and interdisciplinary studies on CAM highlights the need for nursing research to expand its focus on experimental designs, diversify study subjects, explore various CAM interventions, utilize validated assessment tools and physiological indicators, and employ robust statistical techniques. By addressing these methodological considerations, nursing research can strengthen the evidence base for CAM interventions, facilitate their integration into nursing practice, and contribute to interdisciplinary dialogue in the field of CAM research [ 11 , 12 , 13 ]. As CAM use becomes increasingly prevalent among patients, particularly those with chronic conditions [ 1 , 9 , 10 ], nursing research has a crucial role to play in investigating the safety and efficacy of various CAM modalities [ 2 , 4 , 22 ]. This approach not only has the potential to improve patient outcomes and experiences but also enables nursing research to make valuable contributions to interdisciplinary collaboration in the field of CAM [ 3 , 5 ]. By embracing the diversity of CAM interventions and fostering interdisciplinary interactions, nursing research can broaden its scope, enhance the efficiency of patient-focused care, and move closer to providing truly holistic care that addresses the multifaceted needs of patients. Also, the integration of CAM into nursing practice, supported by robust research evidence, has the power to transform healthcare delivery and improve the lives of patients, particularly those with chronic conditions who stand to benefit greatly from a more comprehensive and individualized approach to care.

The trends analysis of nursing and interdisciplinary studies on CAM highlights the potential for nursing research to draw inspiration from the diverse CAM interventions studied in other disciplines and adapt them for nursing practice. For example, the use of aromatherapy for dementia [B10], green tea for depression in HIV patients [B5], and cinnamon patch for irritable bladder syndrome [B13] could be explored in nursing research to assess their feasibility and effectiveness in nursing care settings. By learning from the experiences of other disciplines and adapting promising CAM interventions for nursing practice, researchers can expand the scope of nursing research on CAM and contribute to the development of innovative, evidence-based complementary therapies for various patient populations. Given the current trends in nursing research on CAM, it is essential for future studies to consider the research directions and methodologies employed in other disciplines to guide the advancement of nursing science in this field. In summary, this trends analysis emphasizes the need for nursing research to embrace a more diverse and rigorous approach to CAM research, drawing inspiration from the methodologies and interventions studied in other disciplines. By expanding the focus on experimental designs, diversifying study subjects, exploring novel CAM interventions, utilizing validated assessment tools and physiological indicators, nursing research can strengthen the evidence base for CAM interventions, facilitate their integration into nursing practice.

Limitations

This study aimed to identify research trends in CAM through text network analysis and to analyze nursing research trends based on the findings. The use of text mining and big data analysis allowed for a more comprehensive and less biased approach to data collection and processing compared to arbitrary search strategies. However, there were still limitations in defining each field intuitively due to the diverse and wide-ranging areas of CAM used in different disciplines. Future studies should focus on analyzing overall topics across various fields as well as keyword extraction through text mining to gain a more holistic understanding of CAM research trends. Another limitation of this study is that the search languages were restricted to Korean and English. This may have excluded relevant studies published in other languages and might limit the generalizability of the findings. As CAM is rooted in diverse cultures and traditions worldwide, it is important to include studies conducted in various languages for a comprehensive understanding. Future research should incorporate more languages to provide a global perspective on CAM research trends.

Despite these limitations, this study offers a novel methodological strategy for trend analysis by combining keywords extracted using big data rather than relying on researchers' arbitrary settings. The keyword-based classification and literature analysis provide a new approach to identifying research trends and directions. The trends analysis between nursing literature and other disciplines revealed differences in subject selection, study design, statistical techniques, and measurement of dependent variables, highlighting the need for nursing research to broaden the range of subjects and measurement tools while considering randomization and generalization in experimental designs. Furthermore, this study emphasizes the importance of using design techniques that facilitate the sharing of research results beyond the nursing community.

Conclusions

This study significantly advances CAM research in nursing by providing a comprehensive, data-driven overview of research trends. We have identified key areas for improvement, such as the need for more randomized controlled trials and broader subject diversity, and have proposed innovative methodological strategies. Our findings underscore the importance of interdisciplinary collaboration and the adoption of diverse, rigorous research approaches. By addressing these gaps, nursing research in CAM can be strengthened, ultimately enhancing the integration of evidence-based CAM practices in nursing care and improving patient outcomes.

Availability of data and materials

The data and materials of this study are available from the corresponding author upon reasonable request.

Abbreviations

Complementary and Alternative Medicine

World Health Organization

National Center for Complementary and Integrative Health

Research Information Sharing Service

Korean studies Information Service System

Cumulative Index to Nursing and Allied Health Literature

Term Frequency-Inverse Document Frequency

Latent Dirichlet Allocation

Markov Chain Monte Carlo

Document Term Matrix

Mixed Methods Appraisal Tool

Institutional Review Board

World Health Organization. Traditional complementary and integrative medicine. Geneva: WHO; 2023. Available from:  https://www.who.int/health-topics/traditional-complementary-and-integrative-medicine#tab=tab_1 . Cited 7 Oct 2023.

Google Scholar  

National Center for Complementary and Integrative Health. Complementary, alternative, or integrative health: what's in a name? Bethesda (MD): NCCIH; 2023. Available from: https://www.nccih.nih.gov/health/complementary-alternative-or-integrative-health-whats-in-a-name . Cited 7 Oct 2023.

Shin GR. Development of Korean nursing intervention: alternative (complementary) therapy. J Korean Acad Nurs. 1999;29(6):1403–18.

Article   Google Scholar  

Kang MA, Kim BH, Kim HJ. Complementary and alternative therapy use and health promotion activities among adult women in urban areas. J Korean Wellness Soc. 2017;12(1):677–87. https://doi.org/10.21097/ksw.2017.02.12.677 .

Balouchi A, Mahmoudirad G, Hastings-Tolsma M, Shorofi SA, Shahdadi H, Abdollahimohammad A. Knowledge, attitude and use of complementary and alternative medicine among nurses: A systematic review. Complement Ther Clin Pract. 2018;31:146–57. https://doi.org/10.1016/j.ctcp.2018.02.008 .

Article   PubMed   Google Scholar  

Korean Nurses Association of Complementary and Alternative Therapies. Introduction to the divisions of the Korean Nurses Association of complementary and alternative therapies. Seoul: KNACAT; 2023. Available from:  http://catnurse.co.kr/html/frontend/ .

Oh K, Kim KS, Kwon SH, Park JW. Research trend of complementary and alternative medicine. J Korean Acad Nurs. 2006;36(5):721–31. https://doi.org/10.4040/jkan.2006.36.5.721 .

Pratt M, Wieland S, Ahmadzai N, Butler C, Wolfe D, Pussagoda K, et al. A scoping review of network meta-analyses assessing the efficacy and safety of complementary and alternative medicine interventions. Syst Rev. 2020;9:97. https://doi.org/10.1186/s13643-020-01328-3 .

Article   PubMed   PubMed Central   Google Scholar  

Baek EM, Lee HJ. Analysis of research trends and articles related to work-related overwork using text mining. Occup Health Res. 2023;5(2):79–90. https://doi.org/10.35861/KJOH.2023.5.2.79 .

Kim EJ, Kim JY. Exploring the online news trends of the metaverse in South Korea: A data-mining-driven semantic network analysis. Sustainability. 2023;15(23): 16279. https://doi.org/10.3390/su152316279 .

Karami A, Spinel MY, White CN, Ford K, Swan S. A systematic literature review of sexual harassment studies with text mining. Sustainability. 2021;13(12): 6589. https://doi.org/10.3390/su13126589 .

Lee, HJ, Youn KH. The Analysis of Research Trends in Social Service Quality Using Text Mining and Topic Modeling. J Int Things Converg. 2022;8(3):29–40. https://doi.org/10.20465/KIOTS.2022.8.3.029 .

Kim HY, Seo DH. A Topic Modeling Approach to the Analysis of Domestic Performance. Official J Korean Soc Dance Sci. 2019;36(3):67–78. https://doi.org/10.21539/Ksds.2019.36.3.99 .

O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4(1):1–22.  http://www.systematicreviewsjournal.com/content/4/1/5 .

Kim HJ, Jang HJ, Shin MS. Analysis of research trends on stuttering in adolescence using text mining and literature review. Lang Ther Res. 2020;29(4):1–12. https://doi.org/10.15724/jslhd.2020.29.4.001 .

Gurcan F, Cagiltay NE. Research trends on distance learning: A text mining-based literature review from 2008 to 2018. Interact Learn Environ. 2023;31(2):1007–28. https://doi.org/10.1080/10494820.2020.1815795 .

Yang NY, Shin KL. Analysis of research trends in nursing papers on complementary and alternative therapies in Korea. J Adult Nurs. 2003;15(2):226–35. https://doi.org/10.7739/jkafn.2021.28.2.174 .

Han KS, Lee PS, Choi WH, Ahn KH. The analysis of trends in complementary and alternative therapy (CAT) in nursing research in Korea. J Korean Acad Fundam Nurs. 2003;10(3):392–8.

Sung HR, Kim YS, Kim HY, Ha NY, Chae SM. Research trend about complementary and alternative therapy in Korea using text network analysis. Korean J Rehabil Nurs. 2018;21(2):61–70. https://doi.org/10.7587/kjrehn.2018.61 .

Bidwell S, Jensen MF. Using a search protocol to identify sources of information: The COSI model. In: Topfer LA, Auston I, editors. Etext on Health Technology Assessment Information Resources [Internet]. Bethesda (MD): National Library of Medicine; 2006. Available from: https://www.nlm.nih.gov/archive/20060905/nichsr/ehta/chapter3.html#COSI . Cited 2023 Mar 20.

Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res. 2003;3:993–1022.

Lee SJ, Kim HJ. Keyword extraction from news corpus using modified TF-IDF. J Soc e-Bus Stud. 2009;14(4):59–73.

Lee JH. Semantic network analysis of 2019 Gangwon-do wild fire news reporting: focusing on media agenda analysis. J Korea Contents Assoc. 2019;19(11):153–67. https://doi.org/10.5392/JKCA.2019.19.11.153 .

Liu S, Zhou MX, Pan S, Song Y, Qian W, Cai W, et al. Tiara: Interactive, topic-based visual text summarization and analysis. ACM Trans Intell Syst Technol. 2012;3(2):1–28. https://doi.org/10.1145/2089094.2089101 .

Greene D, O'Callaghan D, Cunningham P. How many topics? stability analysis for topic models. In: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2014; 2014 Sep 15–19; Nancy, France: Springer Berlin Heidelberg; 2014. p. 498–513. https://doi.org/10.1007/978-3-662-44848-9_32

Leung L. Validity, reliability, and generalizability in qualitative research. J Family Med Prim Care. 2015;4(3):324–7. https://doi.org/10.4103/2249-4863.161306 .

Park S, Lee JL. Research trend analysis of Korean new graduate nurses using topic modeling. J Korean Acad Soc Nurs Educ. 2021;27(3):240–50. https://doi.org/10.5977/jkasne.2021.27.3.240 .

Kim HJ, Lee HN, Oh HS, Yang YJ, Shin SH. Trend Analysis of Articles Published in the Journal of East-West Nursing Research. J East-West Nurs Res. 2014;20(2):167–76. https://doi.org/10.14370/jewnr.2014.20.2.167 .

v SH. Analysis of research trends in complementary and alternative therapy studies published in the Journal of Adult Nursing (2009-2018). In: Adult Nursing Academic Conference. 2019. p. 179–80.

Shin KR, Ha JY. The analysis of research trends about reflexology in Korea. Korean J Adult Nurs. 2006;18(4):603–11.

Lee HY, Kim SY. The trends of nursing research on aromatherapy in Korea. J East-West Nurs Res. 2010;16(2):85–95. https://doi.org/10.14370/JEWNR.2010.16.2.085 .

Download references

Acknowledgements

The authors thank all participants who accepted to be part of this work.

This study was supported by the National Research Foundation of Korea (NRF-2022R1F1A1071533)

Author information

Authors and affiliations.

College of Nursing, Ewha Womans University, 52, Ewhayeodae-Gil, Seodaemun-Gu, Seoul, 03760, South Korea

Jihye Nam, Hyejin Lee, Seunghyeon Lee & Hyojung Park

You can also search for this author in PubMed   Google Scholar

Contributions

Study design: JN, HP, and HL; Data collection and Data analysis : JN and HL ;Study supervision: HP; Manuscript writing: JN, HL, and SL Critical revisions for important intellectual content: HP. All authors reviewed and approved the final manuscript.

Corresponding author

Correspondence to Hyojung Park .

Ethics declarations

Ethics approval and consent to participate.

This study was granted an exemption from requiring ethics approval, as it utilized secondary data. The exemption was approved by the Institutional Review Board of Ewha Womans University (IRB No: ewha-202304–0016-01). Informed consent was not required due to the nature of the study, which involved the analysis of existing data that did not include any personally identifiable information. All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nam, J., Lee, H., Lee, S. et al. Literature review of complementary and alternative therapies: using text mining and analysis of trends in nursing research. BMC Nurs 23 , 526 (2024). https://doi.org/10.1186/s12912-024-02172-9

Download citation

Received : 01 April 2024

Accepted : 11 July 2024

Published : 01 August 2024

DOI : https://doi.org/10.1186/s12912-024-02172-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Alternative
• Complementary
• Text network

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]