in research external validity

• Privacy Policy

Home » External Validity – Threats, Examples and Types

External Validity – Threats, Examples and Types

Table of Contents

External Validity

Definition:

External validity refers to the extent to which the results of a study can be generalized or applied to a larger population, settings, or conditions beyond the specific context of the study. It is a measure of how well the findings of a study can be considered representative of the real world.

How To Increase External Validity

To increase external validity in research, researchers can employ several strategies to enhance the generalizability of their findings. Here are some common approaches:

Representative Sampling

Ensure that the sample used in the study is representative of the target population of interest. Random sampling techniques, such as simple random sampling or stratified sampling, can help reduce sampling bias and increase the likelihood of obtaining a representative sample.

Diverse Participant Characteristics

Include participants with diverse demographic characteristics, such as age, gender, socioeconomic status, and cultural backgrounds. This helps to ensure that the findings are applicable to a wider range of individuals.

Multiple Settings

Conduct the study in multiple settings or contexts to assess the robustness of the findings across different environments. This could involve replicating the study in different geographical locations, institutions, or organizations.

Large Sample Size

Increasing the sample size can improve the statistical power of the study and enhance the reliability of the findings. Larger samples are generally more representative of the population, making it easier to generalize the results.

Longitudinal Studies

Consider conducting longitudinal studies that span a longer duration. By observing changes and trends over time, researchers can provide a more comprehensive understanding of the phenomenon under investigation and increase the applicability of their findings.

Real-world Conditions

Strive to create conditions in the study that closely resemble real-world situations. This can be achieved by conducting field experiments, using naturalistic observation, or implementing interventions in real-life settings.

External Validation of Measures

Use established and validated measurement instruments to assess variables of interest. By employing recognized measures, researchers increase the likelihood that their findings can be compared and replicated in other studies.

Meta-Analysis

Conducting a meta-analysis, which involves systematically analyzing and combining the results of multiple studies on the same topic, can provide a more comprehensive view and increase the external validity by pooling findings from various sources.

Replication

Encourage replication of the study by other researchers. When multiple studies yield similar results, it strengthens the external validity of the findings.

Transparent Reporting

Clearly document the study design, methodology, and limitations in research publications. Transparent reporting allows readers to evaluate the study’s external validity and consider the potential generalizability of the findings.

Threats to External Validity

There are several threats to external validity that researchers should be aware of when interpreting the generalizability of their findings. These threats include:

Selection Bias

Participants in a study may not be representative of the target population due to the way they were selected or recruited. This can limit the generalizability of the findings to the broader population.

Sampling Bias

Even with random sampling techniques, there is a possibility of sampling bias. This occurs when certain segments of the population are underrepresented or overrepresented in the sample, leading to a skewed representation of the population.

Reactive or Interaction Effects of Testing

The act of participating in a study or being exposed to a specific experimental condition can influence participants’ behaviors or responses. This can lead to artificial results that may not occur in natural settings.

Experimental Setting

The controlled environment of a laboratory or research setting may differ significantly from real-world situations, potentially influencing participant behavior and limiting the generalizability of the findings.

Demand Characteristics

Participants may alter their behavior based on their perception of the study’s purpose or the researcher’s expectations. This can introduce biases and limit the external validity of the findings.

Novelty Effects

Participants may respond differently to novel or unusual conditions in a study, which may not accurately reflect their behavior in everyday life.

Hawthorne Effect

Participants may change their behavior simply because they are aware they are being observed. This effect can distort the findings and limit generalizability.

Experimenter Bias

The actions or behaviors of the researchers conducting the study can inadvertently influence participant responses or outcomes, impacting the generalizability of the findings.

Time-related Threats

The passage of time can affect the external validity of findings. Social, cultural, or technological changes that occur between the study and the application of the findings may limit their relevance.

Specificity of the Intervention or Treatment

If the study involves a specific intervention or treatment, the findings may be limited to that particular intervention and may not generalize to other similar interventions or treatments.

Publication Bias

The tendency of researchers or journals to publish studies with significant or positive findings can introduce a bias in the literature and limit the generalizability of research findings.

Types of External Validity

Types of External Validity are as follows:

Population Validity

Population validity refers to the extent to which the findings of a study can be generalized to the larger target population from which the study sample was drawn. If the sample is representative of the population in terms of relevant characteristics, such as age, gender, socioeconomic status, and ethnicity, the study’s findings are more likely to have high population validity.

Ecological Validity

Ecological validity refers to the extent to which the findings of a study can be generalized to real-world settings or conditions. It assesses whether the experimental conditions and procedures accurately represent the complexity and dynamics of the natural environment. High ecological validity suggests that the findings are applicable to everyday situations.

Temporal Validity

Temporal validity, also known as historical validity or generalizability over time, refers to the extent to which the findings of a study can be generalized across different time periods. It assesses whether the relationships or effects observed during the study remain consistent or change over time.

Cross-Cultural Validity

Cross-cultural validity refers to the extent to which the findings of a study can be generalized to different cultural contexts or populations. It examines whether the relationships or effects observed in one culture hold true in other cultures. Conducting research in multiple cultural settings can help establish cross-cultural validity.

Setting Validity

Setting validity refers to the extent to which the findings of a study can be generalized to different settings or environments. It assesses whether the relationships or effects observed in one specific setting can be replicated in other similar settings.

Task Validity

Task validity refers to the extent to which the findings of a study can be generalized to different tasks or activities. It examines whether the relationships or effects observed during a specific task are applicable to other tasks that share similar characteristics.

Measurement Validity

Measurement validity refers to the extent to which the chosen measurements or instruments accurately capture the constructs or variables of interest. It examines whether the relationships or effects observed are robust across different measurement tools or techniques.

Examples of External Validity

Here are some real-time examples of external validity:

Medical Research: A pharmaceutical company conducts a clinical trial to test the efficacy of a new drug on a specific population group (e.g., adults with diabetes). To ensure external validity, the company includes participants from diverse backgrounds, ages, and geographical locations to ensure that the results can be generalized to a broader population.

Educational Research: A study examines the effectiveness of a teaching method in improving student performance in mathematics. Researchers choose a sample of schools from different regions, representing various socioeconomic backgrounds, to ensure the findings can be applied to a wider range of schools and students.

Opinion Polls: A polling agency conducts a survey to understand public opinion on a particular political issue. To ensure external validity, the agency ensures a representative sample of respondents, considering factors such as age, gender, ethnicity, education level, and geographic location. This approach allows the findings to be generalized to the broader population.

Social Science Research: A study investigates the impact of a social intervention program on reducing crime rates in a specific neighborhood. To enhance external validity, researchers select neighborhoods that represent diverse socio-economic conditions and urban and rural settings. This approach increases the likelihood that the findings can be applied to similar neighborhoods in other locations.

Psychological Research: A psychology study examines the effects of a therapy technique on reducing anxiety levels in individuals. To improve external validity, the researchers recruit a diverse sample of participants, including individuals of different ages, genders, and cultural backgrounds. This ensures that the findings can be applicable to a broader range of individuals experiencing anxiety.

Applications of External Validity

External validity has several practical applications across various fields. Here are some specific applications of external validity:

Policy Development:

External validity helps policymakers make informed decisions by considering research findings from different contexts and populations. By examining the external validity of studies, policymakers can determine the applicability and generalizability of research results to their target population and policy goals.

Program Evaluation:

External validity is crucial in evaluating the effectiveness of programs or interventions. By assessing the external validity of evaluation studies, policymakers and program administrators can determine if the findings are applicable to their target population and whether similar interventions can be implemented in different settings.

Market Research:

External validity is essential in market research to understand consumer behavior and preferences. By conducting studies with representative samples, companies can extrapolate the findings to the broader consumer population, allowing them to make informed marketing and product development decisions.

Health Interventions:

External validity plays a significant role in healthcare research. It helps researchers and healthcare practitioners understand the generalizability of treatment outcomes to diverse patient populations. By considering external validity, healthcare providers can determine if a specific treatment or intervention will be effective for their patients.

Education and Training:

External validity is important in educational research to ensure that instructional methods, educational interventions, and training programs are effective across diverse student populations and different educational settings. It helps educators and trainers make evidence-based decisions about instructional strategies that are likely to have positive outcomes in different contexts.

Public Opinion Research:

External validity is crucial in public opinion research, such as political polling or survey research. By ensuring a representative sample and considering external validity, researchers can generalize their findings to the larger population, providing insights into public sentiment and informing decision-making processes.

Advantages of External Validity

Here are some advantages of external validity:

• Generalizability: External validity allows researchers to generalize their findings to broader populations, settings, or conditions. It enables them to make inferences about how the results of a study might hold true in real-world situations beyond the controlled environment of the study.
• Real-world applicability: When a study has high external validity, the findings are more likely to be applicable and relevant to real-world scenarios. This is particularly important in fields such as medicine, psychology, and social sciences, where the goal is often to understand and improve human behavior and well-being.
• Increased confidence in findings: Studies with high external validity provide stronger evidence and increase confidence in the findings. When the results can be generalized to diverse populations or different contexts, it suggests that the observed effects are more robust and reliable.
• Enhanced ecological validity: External validity enhances ecological validity, which refers to the degree to which a study reflects real-life situations. When a study has good external validity, it increases the likelihood that the findings accurately represent the complexities and nuances of the real world.
• Policy implications: Research findings with high external validity are more likely to have practical implications for policy-making. Policymakers are interested in studies that can inform decisions and interventions on a larger scale. Studies with strong external validity provide a basis for making informed decisions and implementing effective policies.
• Replication and meta-analysis: External validity facilitates replication studies and meta-analyses, which involve combining the results of multiple studies. When studies have high external validity, it becomes easier to replicate the findings in different contexts or conduct meta-analyses to examine the overall effects across a range of studies.
• Improved understanding of causal relationships: External validity allows researchers to test the generalizability of causal relationships. By replicating studies in different settings or populations, researchers can examine whether the causal relationships observed in one context hold true in other contexts, providing a more comprehensive understanding of the phenomenon under investigation.

Limitations of External Validity

While external validity offers several advantages, it also has limitations that researchers need to consider. Here are some limitations of external validity:

• Specificity of conditions: The specific conditions and settings of a study may limit the generalizability of the findings. Factors such as the time period, location, and sample characteristics can influence the results. For example, cultural, socioeconomic, or geographical differences between the study sample and the target population may affect the generalizability of the findings.
• Selection bias: In many studies, participants are recruited through convenience sampling or other non-random methods, which can introduce selection bias. This means that the sample may not be representative of the larger population, reducing the external validity of the findings. Selection bias can limit the generalizability of the results to other populations or contexts.
• Artificiality of experimental settings: Studies conducted in controlled laboratory or experimental settings may lack ecological validity. The artificial conditions and controlled variables may not accurately reflect real-world complexities. Participants’ behavior in a laboratory setting may differ from their behavior in naturalistic settings, leading to limited generalizability.
• Novelty and awareness effects: Participants in research studies may behave differently simply because they are aware they are being studied. This awareness can lead to the novelty effect or demand characteristics, where participants alter their behavior in response to the study context or the researchers’ expectations. As a result, the observed effects may not accurately represent real-world behavior.
• Time-dependent effects: The relevance and applicability of research findings can change over time due to societal, technological, or cultural shifts. What may be true and valid today may not hold true in the future. Therefore, the external validity of a study’s findings may diminish as time progresses.
• Lack of contextual variation: Studies often focus on a narrow range of contexts or populations, limiting the understanding of how findings may vary across different contexts. The external validity of a study may be compromised if it fails to account for contextual variations that can influence the generalizability of the results.
• Replication challenges: While replication is important for assessing the external validity of a study, it can be challenging to replicate studies in different contexts or with diverse populations. Replication studies may encounter practical constraints, such as resource limitations, time constraints, or ethical considerations, which can limit the ability to establish external validity.

Also see Validity

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Alternate Forms Reliability – Methods, Examples...

Face Validity – Methods, Types, Examples

Content Validity – Measurement and Examples

Internal Validity – Threats, Examples and Guide

Test-Retest Reliability – Methods, Formula and...

Parallel Forms Reliability – Methods, Example...

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2024 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

Internal Validity vs. External Validity in Research

What they tell us about the meaningfulness and trustworthiness of research

Arlin Cuncic, MA, is the author of The Anxiety Workbook and founder of the website About Social Anxiety. She has a Master's degree in clinical psychology.

Rachel Goldman, PhD FTOS, is a licensed psychologist, clinical assistant professor, speaker, wellness expert specializing in eating behaviors, stress management, and health behavior change.

Verywell / Bailey Mariner

• Internal Validity
• External Validity

How do you determine whether a psychology study is trustworthy and meaningful? Two characteristics that can help you assess research findings are internal and external validity.

• Internal validity measures how well a study is conducted (its structure) and how accurately its results reflect the studied group.
• External validity relates to how applicable the findings are in the real world.

These two concepts help researchers gauge if the results of a research study are trustworthy and meaningful.

Conclusions are warranted

Controls extraneous variables

Eliminates alternative explanations

Focus on accuracy and strong research methods

Findings can be generalized

Outcomes apply to practical situations

Results apply to the world at large

Results can be translated into another context

What Is Internal Validity in Research?

Internal validity is the extent to which a research study establishes a trustworthy cause-and-effect relationship. This type of validity depends largely on the study's procedures and how rigorously it is performed.

Internal validity is important because once established, it makes it possible to eliminate alternative explanations for a finding. If you implement a smoking cessation program, for instance, internal validity ensures that any improvement in the subjects is due to the treatment administered and not something else.

Internal validity is not a "yes or no" concept. Instead, we consider how confident we can be with study findings based on whether the research avoids traps that may make those findings questionable. The less chance there is for "confounding," the higher the internal validity and the more confident we can be.

Confounding refers to uncontrollable variables that come into play and can confuse the outcome of a study, making us unsure of whether we can trust that we have identified the cause-and-effect relationship.

In short, you can only be confident that a study is internally valid if you can rule out alternative explanations for the findings. Three criteria are required to assume cause and effect in a research study:

• The cause preceded the effect in terms of time.
• The cause and effect vary together.
• There are no other likely explanations for the relationship observed.

Factors That Improve Internal Validity

To ensure the internal validity of a study, you want to consider aspects of the research design that will increase the likelihood that you can reject alternative hypotheses. Many factors can improve internal validity in research, including:

• Blinding : Participants—and sometimes researchers—are unaware of what intervention they are receiving (such as using a placebo on some subjects in a medication study) to avoid having this knowledge bias their perceptions and behaviors, thus impacting the study's outcome
• Experimental manipulation : Manipulating an independent variable in a study (for instance, giving smokers a cessation program) instead of just observing an association without conducting any intervention (examining the relationship between exercise and smoking behavior)
• Random selection : Choosing participants at random or in a manner in which they are representative of the population that you wish to study
• Randomization or random assignment : Randomly assigning participants to treatment and control groups, ensuring that there is no systematic bias between the research groups
• Strict study protocol : Following specific procedures during the study so as not to introduce any unintended effects; for example, doing things differently with one group of study participants than you do with another group

Internal Validity Threats

Just as there are many ways to ensure internal validity, a list of potential threats should be considered when planning a study.

• Attrition : Participants dropping out or leaving a study, which means that the results are based on a biased sample of only the people who did not choose to leave (and possibly who all have something in common, such as higher motivation)
• Confounding : A situation in which changes in an outcome variable can be thought to have resulted from some type of outside variable not measured or manipulated in the study
• Diffusion : This refers to the results of one group transferring to another through the groups interacting and talking with or observing one another; this can also lead to another issue called resentful demoralization, in which a control group tries less hard because they feel resentful over the group that they are in
• Experimenter bias : An experimenter behaving in a different way with different groups in a study, which can impact the results (and is eliminated through blinding)
• Historical events : May influence the outcome of studies that occur over a period of time, such as a change in the political leader or a natural disaster that occurs, influencing how study participants feel and act
• Instrumentation : This involves "priming" participants in a study in certain ways with the measures used, causing them to react in a way that is different than they would have otherwise reacted
• Maturation : The impact of time as a variable in a study; for example, if a study takes place over a period of time in which it is possible that participants naturally change in some way (i.e., they grew older or became tired), it may be impossible to rule out whether effects seen in the study were simply due to the impact of time
• Statistical regression : The natural effect of participants at extreme ends of a measure falling in a certain direction due to the passage of time rather than being a direct effect of an intervention
• Testing : Repeatedly testing participants using the same measures influences outcomes; for example, if you give someone the same test three times, it is likely that they will do better as they learn the test or become used to the testing process, causing them to answer differently

What Is External Validity in Research?

External validity refers to how well the outcome of a research study can be expected to apply to other settings. This is important because, if external validity is established, it means that the findings can be generalizable to similar individuals or populations.

External validity affirmatively answers the question: Do the findings apply to similar people, settings, situations, and time periods?

Population validity and ecological validity are two types of external validity. Population validity refers to whether you can generalize the research outcomes to other populations or groups. Ecological validity refers to whether a study's findings can be generalized to additional situations or settings.

Another term called transferability refers to whether results transfer to situations with similar characteristics. Transferability relates to external validity and refers to a qualitative research design.

Factors That Improve External Validity

If you want to improve the external validity of your study, there are many ways to achieve this goal. Factors that can enhance external validity include:

• Field experiments : Conducting a study outside the laboratory, in a natural setting
• Inclusion and exclusion criteria : Setting criteria as to who can be involved in the research, ensuring that the population being studied is clearly defined
• Psychological realism : Making sure participants experience the events of the study as being real by telling them a "cover story," or a different story about the aim of the study so they don't behave differently than they would in real life based on knowing what to expect or knowing the study's goal
• Replication : Conducting the study again with different samples or in different settings to see if you get the same results; when many studies have been conducted on the same topic, a meta-analysis can also be used to determine if the effect of an independent variable can be replicated, therefore making it more reliable
• Reprocessing or calibration : Using statistical methods to adjust for external validity issues, such as reweighting groups if a study had uneven groups for a particular characteristic (such as age)

External Validity Threats

External validity is threatened when a study does not take into account the interaction of variables in the real world. Threats to external validity include:

• Pre- and post-test effects : When the pre- or post-test is in some way related to the effect seen in the study, such that the cause-and-effect relationship disappears without these added tests
• Sample features : When some feature of the sample used was responsible for the effect (or partially responsible), leading to limited generalizability of the findings
• Selection bias : Also considered a threat to internal validity, selection bias describes differences between groups in a study that may relate to the independent variable—like motivation or willingness to take part in the study, or specific demographics of individuals being more likely to take part in an online survey
• Situational factors : Factors such as the time of day of the study, its location, noise, researcher characteristics, and the number of measures used may affect the generalizability of findings

While rigorous research methods can ensure internal validity, external validity may be limited by these methods.

Internal Validity vs. External Validity

Internal validity and external validity are two research concepts that share a few similarities while also having several differences.

Similarities

One of the similarities between internal validity and external validity is that both factors should be considered when designing a study. This is because both have implications in terms of whether the results of a study have meaning.

Both internal validity and external validity are not "either/or" concepts. Therefore, you always need to decide to what degree a study performs in terms of each type of validity.

Each of these concepts is also typically reported in research articles published in scholarly journals . This is so that other researchers can evaluate the study and make decisions about whether the results are useful and valid.

Differences

The essential difference between internal validity and external validity is that internal validity refers to the structure of a study (and its variables) while external validity refers to the universality of the results. But there are further differences between the two as well.

For instance, internal validity focuses on showing a difference that is due to the independent variable alone. Conversely, external validity results can be translated to the world at large.

Internal validity and external validity aren't mutually exclusive. You can have a study with good internal validity but be overall irrelevant to the real world. You could also conduct a field study that is highly relevant to the real world but doesn't have trustworthy results in terms of knowing what variables caused the outcomes.

Examples of Validity

Perhaps the best way to understand internal validity and external validity is with examples.

Internal Validity Example

An example of a study with good internal validity would be if a researcher hypothesizes that using a particular mindfulness app will reduce negative mood. To test this hypothesis, the researcher randomly assigns a sample of participants to one of two groups: those who will use the app over a defined period and those who engage in a control task.

The researcher ensures that there is no systematic bias in how participants are assigned to the groups. They do this by blinding the research assistants so they don't know which groups the subjects are in during the experiment.

A strict study protocol is also used to outline the procedures of the study. Potential confounding variables are measured along with mood , such as the participants' socioeconomic status, gender, age, and other factors. If participants drop out of the study, their characteristics are examined to make sure there is no systematic bias in terms of who stays in.

External Validity Example

An example of a study with good external validity would be if, in the above example, the participants used the mindfulness app at home rather than in the laboratory. This shows that results appear in a real-world setting.

To further ensure external validity, the researcher clearly defines the population of interest and chooses a representative sample . They might also replicate the study's results using different technological devices.

Setting up an experiment so that it has both sound internal validity and external validity involves being mindful from the start about factors that can influence each aspect of your research.

It's best to spend extra time designing a structurally sound study that has far-reaching implications rather than to quickly rush through the design phase only to discover problems later on. Only when both internal validity and external validity are high can strong conclusions be made about your results.

Andrade C. Internal, external, and ecological validity in research design, conduct, and evaluation .  Indian J Psychol Med . 2018;40(5):498-499. doi:10.4103/IJPSYM.IJPSYM_334_18

San Jose State University. Internal and external validity .

Kemper CJ. Internal validity . In: Zeigler-Hill V, Shackelford TK, eds. Encyclopedia of Personality and Individual Differences . Springer International Publishing; 2017:1-3. doi:10.1007/978-3-319-28099-8_1316-1

Patino CM, Ferreira JC. Internal and external validity: can you apply research study results to your patients?   J Bras Pneumol . 2018;44(3):183. doi:10.1590/S1806-37562018000000164

Matthay EC, Glymour MM. A graphical catalog of threats to validity: Linking social science with epidemiology .  Epidemiology . 2020;31(3):376-384. doi:10.1097/EDE.0000000000001161

Amico KR. Percent total attrition: a poor metric for study rigor in hosted intervention designs .  Am J Public Health . 2009;99(9):1567-1575. doi:10.2105/AJPH.2008.134767

Kemper CJ. External validity . In: Zeigler-Hill V, Shackelford TK, eds. Encyclopedia of Personality and Individual Differences . Springer International Publishing; 2017:1-4. doi:10.1007/978-3-319-28099-8_1303-1

Desjardins E, Kurtz J, Kranke N, Lindeza A, Richter SH. Beyond standardization: improving external validity and reproducibility in experimental evolution . BioScience. 2021;71(5):543-552. doi:10.1093/biosci/biab008

Drude NI, Martinez Gamboa L, Danziger M, Dirnagl U, Toelch U. Improving preclinical studies through replications .  Elife . 2021;10:e62101. doi:10.7554/eLife.62101

Michael RS. Threats to internal & external validity: Y520 strategies for educational inquiry .

Pahus L, Burgel PR, Roche N, Paillasseur JL, Chanez P. Randomized controlled trials of pharmacological treatments to prevent COPD exacerbations: applicability to real-life patients . BMC Pulm Med . 2019;19(1):127. doi:10.1186/s12890-019-0882-y

By Arlin Cuncic, MA Arlin Cuncic, MA, is the author of The Anxiety Workbook and founder of the website About Social Anxiety. She has a Master's degree in clinical psychology.

External Validity

• Living reference work entry
• First Online: 10 February 2017
• Cite this living reference work entry

• Christoph J. Kemper 3  

492 Accesses

External validity refers to the degree to which conclusions from experimental scientific studies can be generalized from the specific set of conditions under which the study is conducted to other populations, settings, treatments, measurements, times, and experimenters.

Introduction

The ultimate goal of experimental scientific studies is to advance our understanding of real-life processes and phenomena. In research on individual differences it is rarely feasible to design experiments that involve thousands of participants and conditions that closely resemble the real world. Researchers usually seek to study an assumed cause-effect relationship without the interference of myriads of extraneous variables in real-life settings. To this purpose, they set up an experimental situation which allows to focus on the assumed cause-effect relationship and to control potentially confounding effects of extraneous variables. As a result, an artificial situation that differs from the real...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American Educational Research Journal, 5 (4), 437–474.

Article   Google Scholar  

Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research . Chicago: Rand-McNally.

Google Scholar  

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). Most people are not WEIRD. Nature, 466 (7302), 29–29.

Article   PubMed   Google Scholar  

Lynch, J. G. (1982). On the external validity of experiments in consumer research. Journal of Consumer Research, 9 (3), 225–239.

Download references

Author information

Authors and affiliations.

University of Luxembourg, Esch-sur-Alzette, Belval, Luxembourg

Christoph J. Kemper

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Christoph J. Kemper .

Editor information

Editors and affiliations.

Oakland University, Rochester, USA

Virgil Zeigler-Hill

Todd K. Shackelford

Section Editor information

Humboldt University, Germany, Berlin, Germany

Matthias Ziegler

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this entry

Cite this entry.

Kemper, C.J. (2017). External Validity. In: Zeigler-Hill, V., Shackelford, T. (eds) Encyclopedia of Personality and Individual Differences. Springer, Cham. https://doi.org/10.1007/978-3-319-28099-8_1303-1

Download citation

DOI : https://doi.org/10.1007/978-3-319-28099-8_1303-1

Received : 03 February 2017

Accepted : 03 February 2017

Published : 10 February 2017

Publisher Name : Springer, Cham

Print ISBN : 978-3-319-28099-8

Online ISBN : 978-3-319-28099-8

eBook Packages : Springer Reference Behavioral Science and Psychology Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Internal vs. External Validity In Psychology

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

Internal validity centers on demonstrating clear casual relationships within the bounds of a specific study and external validity relates to demonstrating the applicability of findings beyond that original study situation or population.

Researchers have to weigh these considerations in designing methodologically rigorous and generalizable studies.

Internal Validity 

Internal validity refers to the degree of confidence that the causal relationship being tested exists and is trustworthy.

It tests how likely it is that your treatment caused the differences in results that you observe. Internal validity is largely determined by the study’s experimental design and methods . 

Studies that have a high degree of internal validity provide strong evidence of causality, so it makes it possible to eliminate alternative explanations for a finding.

Studies with low internal validity provide weak evidence of causality. The less chance there is for confounding or extraneous variables , the higher the internal validity and the more confident we can be in our findings. 

In order to assume cause and effect in a research study, the cause must precede the effect in terms of time, the cause and effect must vary together, and there must be no other explanations for the relationship observed. If these three criteria are observed, you can be confident that a study is internally valid. 

An example of a study with high internal validity would be if you wanted to run an experiment to see if using a particular weight-loss pill will help people lose weight.

To test this hypothesis, you would randomly assign a sample of participants to one of two groups: those who will take the weight-loss pill and those who will take a placebo pill.

You can ensure that there is no bias in how participants are assigned to the groups by blinding the research assistants , so they don’t know which participants are in which groups during the experiment. The participants are also blinded, so they do not know whether they are receiving the intervention or not.

If participants drop out of the study, their characteristics are examined to ensure there is no systematic bias regarding who left.

It is important to have a well-thought-out research procedure to mitigate the threats to internal validity.

External Validity

External validity refers to the extent to which the results of a research study can be applied or generalized to another context.

This is important because if external validity is established, the studies’ findings can be generalized to a larger population as opposed to only the relatively few subjects who participated in the study. Unlike internal validity, external validity doesn’t assess causality or rule out confounders.

There are two types of external validity: ecological validity and population validity.

• Ecological validity refers to whether a study’s findings can be generalized to other situations or settings. A high ecological validity means that there is a high degree of similarity between the experimental setting and another setting, and thus we can be confident that the results will generalize to that other setting.
• Population validity refers to how well the experimental sample represents other populations or groups. Using random sampling techniques , such as stratified sampling or cluster sampling, significantly helps increase population validity. 

An example of a study with high external validity would be if you hypothesize that practicing mindfulness two times per week will improve the mental health of those diagnosed with depression.

You recruit people who have been diagnosed with depression for at least a year and are between 18–29 years old. Choosing this representative sample with a clearly defined population of interest helps ensure external validity. 

You give participants a pre-test and a post-test measuring how often they experienced symptoms of depression in the past week.

During the study, all participants were given individual mindfulness training and asked to practice mindfulness daily for 15 minutes as part of their morning routine. 

You can also replicate the study’s results using different methods of mindfulness or different samples of participants. 

Trade-off Between Internal and External Validity

There tends to be a negative correlation between internal and external validity in experimental research. This means that experiments that have high internal validity will likely have low external validity and vice versa. 

This happens because experimental conditions that produce higher degrees of internal validity (e.g., artificial labs) tend to be highly unlikely to match real-world conditions. So, the external validity is weaker because a lab environment is much different than the real world. 

On the other hand, to produce higher degrees of external validity, you want experimental conditions that match a real-world setting (e.g., observational studies ).

However, this comes at the expense of internal validity because these types of studies increase the likelihood of confounding variables and alternative explanations for differences in outcomes. 

A solution to this trade-off is replication! You want to conduct the research in multiple environments and settings – first in a controlled, artificial environment to establish the existence of a causal relationship and then in a “real-world” setting to analyze if the results are generalizable. 

Threats to Internal Validity

Attrition refers to the loss of study participants over time. Participants might drop out or leave the study which means that the results are based solely on a biased sample of only the people who did not choose to leave.

Differential rates of attrition between treatment and control groups can skew results by affecting the relationship between your independent and dependent variables and thus affect the internal validity of a study. 

Confounders

A confounding variable is an unmeasured third variable that influences, or “confounds,” the relationship between an independent and a dependent variable by suggesting the presence of a spurious correlation.

Confounders are threats to internal validity because you can’t tell whether the predicted independent variable causes the outcome or if the confounding variable causes it.

Participant Selection Bias

This is a bias that may result from the selection or assignment of study groups in such a way that proper randomization is not achieved.

If participants are not randomly assigned to groups, the sample obtained might not be representative of the population intended to be studied. For example, some members of a population might be less likely to be included than others due to motivation, willingness to take part in the study, or demographics. 

Experimenter Bias

Experimenter bias occurs when an experimenter behaves in a different way with different groups in a study, impacting the results and threatening internal validity. This can be eliminated through blinding. 

Social Interaction (Diffusion)

Diffusion refers to when the treatment in research spreads within or between treatment and control groups. This can happen when there is interaction or observation among the groups.

Diffusion poses a threat to internal validity because it can lead to resentful demoralization. This is when the control group is less motivated because they feel resentful over the group that they are in. 

Historical Events

Historical events might influence the outcome of studies that occur over longer periods of time. For example, changes in political leadership, natural disasters, or other unanticipated events might change the conditions of the study and influence the outcomes.

Instrumentation

Instrumentation refers to any change in the dependent variable in a study that arises from changes in the measuring instrument used. This happens when different measures are used in the pre-test and post-test phases. 

Maturation refers to the impact of time on a study. If the outcomes of the study vary as a natural result of time, it might not be possible to determine whether the effects seen in the study were due to the study treatment or simply due to the impact of time. 

Statistical Regression

Regression to the mean refers to the fact that if one sample of a random variable is extreme, the next sampling of the same random variable is likely going to be closer to its mean.

This is a threat to internal validity as participants at extreme ends of treatment can naturally fall in a certain direction due to the passage of time rather than being a direct effect of an intervention. 

Repeated Testing

Testing your research participants repeatedly with the same measures will influence your research findings because participants will become more accustomed to the testing. Due to familiarity, or awareness of the study’s purpose, many participants might achieve better results over time.

Threats to External Validity 

Sample features.

If some feature(s) of the sample used were responsible for the effect, this could lead to limited generalizability of the findings.

This is a bias that may result from the selection or assignment of study groups in such a way that proper randomization is not achieved. If participants are not randomly assigned to groups, the sample obtained might not be representative of the population intended to be studied.

For example, some members of a population might be less likely to be included than others due to motivation, willingness to take part in the study, or demographics. 

Situational Factors

Factors such as the setting, time of day, location, researchers’ characteristics, noise, or the number of measures might affect the generalizability of the findings.

Aptitude-Treatment Interaction → Aptitude-Treatment Interaction to the concept that some treatments are more or less effective for particular individuals depending upon their specific abilities or characteristics. 

Hawthorne Effect

The Hawthorne Effect refers to the tendency for participants to change their behaviors simply because they know they are being studied.

Experimenter Effect

Experimenter bias occurs when an experimenter behaves in a different way with different groups in a study, impacting the results and threatening the external validity.

John Henry Effect

The John Henry Effect refers to the tendency for participants in a control group to actively work harder because they know they are in an experiment and want to overcome the “disadvantage” of being in the control group.

Factors that Improve Internal Validity

Blinding refers to a practice where the participants (and sometimes the researchers) are unaware of what intervention they are receiving.

This reduces the influence of extraneous factors and minimizes bias, as any differences in outcome can thus be linked to the intervention and not to the participant’s knowledge of whether they were receiving a new treatment or not. 

Random Sampling

Using random sampling to obtain a sample that represents the population that you wish to study will improve internal validity. 

Random Assignment

Using random assignment to assign participants to control and treatment groups ensures that there is no systematic bias among the research groups. 

Strict Study Protocol

Highly controlled experiments tend to improve internal validity. Experiments that occur in lab settings tend to have higher validity as this reduces variability from sources other than the treatment. 

Experimental Manipulation

Manipulating an independent variable in a study as opposed to just observing an association without conducting an intervention improves internal validity. 

Factors that Improve External Validity

Replication.

Conducting a study more than once with a different sample or in a different setting to see if the results will replicate can help improve external validity.

If multiple studies have been conducted on the same topic, a meta-analysis can be used to determine if the effect of an independent variable can be replicated, thus making it more reliable.

Replication is the strongest method to counter threats to external validity by enhancing generalizability to other settings, populations, and conditions.

Field Experiments

Conducting a study outside the laboratory, in a natural, real-world setting will improve external validity (however, this will threaten the internal validity) 

Probability Sampling

Using probability sampling will counter selection bias by making sure everyone in a population has an equal chance of being selected for a study sample.

Recalibration

Recalibration is the use of statistical methods to maintain accuracy, standardization, and repeatability in measurements to assure reliable results.

Reweighting groups, if a study had uneven groups for a particular characteristic (such as age), is an example of calibration. 

Inclusion and Exclusion Criteria

Setting criteria as to who can be involved in the research and who cannot be involved will ensure that the population being studied is clearly defined and that the sample is representative of the population.

Psychological Realism

Psychological realism refers to the process of making sure participants perceive the experimental manipulations as real events so as to not reveal the purpose of the study and so participants don’t behave differently than they would in real life based on knowing the study’s goal.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 23, Issue 1
• External validity, generalisability, applicability and directness: a brief primer
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-5502-5975 Mohammad H Murad 1 ,
• Abdulrahman Katabi 1 ,
• Raed Benkhadra 1 ,
• Victor M Montori 2
• 1 Evidence-Based Practice Center , Mayo Clinic , Rochester , Minnesota , USA
• 2 Knowledge and Evaluation Research Unit , Mayo Clinic , Rochester , Minnesota , USA
• Correspondence to Dr Mohammad H Murad, Evidence-Based Practice Center, Mayo Clinic, Rochester, MN 55905, USA; murad.mohammad{at}mayo.edu

https://doi.org/10.1136/ebmed-2017-110800

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• quality in health care
• public health

Introduction

The central question in the use of research evidence in practice is how confident can we be that what is true for study participants (internal validity) is also true for other people (external validity). For survey researchers, the study of a sample produces claims on the population from which the sample was drawn. In clinical practice, it is rarely important to narrowly apply clinical trial results to the population from which trial participants were drawn. Rather, the most common situation is to apply those results to other populations that are deemed sufficiently similar to the trial population in relevant characteristics. These are issues of external validity that are often described using terms such as generalisability, applicability, transferability, representativeness, directness and others. These terms reveal different underlying concepts. In this brief primer, we describe two unique underlying concepts that are part of the construct of external validity. We aim to explain these two concepts beyond semantics and describe the impact of these concepts on our confidence in the evidence (also called quality of the evidence or certainty in evidence). 1

Generalisability versus applicability

When the concern is how confidently I can use inferences drawn from study participants in the care of patients drawn from any populations, the problem is one of applicability. This concept can be evaluated by determining how similar the two populations are in terms that affect prognosis and outcomes. Pertinent factors include clinical and psycho-socioeconomic characteristics, and healthcare factors including clinician and healthcare system characteristics. 2 3 The concepts of generalisability and applicability are depicted in figure 1 .

• Download figure
• Open in new tab
• Download powerpoint

The concepts of generalisability and applicability.

Although random sampling can lead to more generalisable results, it is rarely used in clinical trials. Trial participants are rarely drawn from known populations, and are almost never randomly selected. Trialists invite convenient samples and patients volunteer to participate (even when allocation to interventions is made at random). Therefore, evaluating generalisability is easier for surveys than for clinical trials. Conversely, clinicians, guideline developers and policymakers do not struggle with generalisability, but with applicability. Narrow eligibility criteria that apply to a small sliver of the population narrow the applicability of the results, but results remain highly applicable to anyone who closely fits these inclusion criteria. The most extreme example of this is the n-of-1 trial in which the trial results are highly applicable to the study participant. To demand generalisability from clinical trials would require that clinicians not use research evidence from clinical trials in the care of their patients. This would be foolish and wasteful. When clinicians judge trial results to be applicable to their patients, they are deciding that the treatment effect (benefits and harms) in practice is expected to be similar to the treatment effect observed in the trial. To the extent that this is not the case, that is, to the extent that there is indirectness between the trial situation and the clinical situation, they must reduce their confidence in the applicability of the estimates of effect from this evidence to their patients.

This reduction in confidence in the estimates of effect due to indirectness is akin to the effect of other factors that negatively affect (lower) the quality of evidence, such as bias and imprecision. 4 The GRADE approach to describe the certainty in evidence uses the term indirectness (not directness) because differences between the study situation and the clinical situation are almost always expected, even though they are seldom important. Figure 2 depicts how to evaluate generalisability, applicability and indirectness. Table 1 offers two examples to illustrate this evaluation.

Evaluating generalisability, applicability and indirectness.

• View inline

Examples illustrating generalisability, applicability and indirectness

Because external validity is a construct that entails several concepts, it is better to use specific terms that relate to the purpose. The central concern for generalisability is representativeness, and these terms are best used in survey research and other observational studies designed to produce population-based estimates. The central concern for applicability is similarity or directness, and these terms are best used in evidence-based care, since clinicians rarely treat study populations and guideline developers must recommend care for populations well outside the narrowly selected group enrolled in clinical studies. A study with good applicability enrols a population that is similar to the patients for whom decisions are being made, and uses interventions feasible to implement in their setting and measures outcomes relevant to the patients.

One can try to predict generalisability and applicability based on study design. However, exceptions are likely common. Animal studies are expected to have poor generalisability and applicability. Case series and case reports have poor generalisability but may have occasionally good applicability (ie, may fit the characteristics of specific patients). Longitudinal studies of convenience samples may have limited generalisability (compared with those with random or consecutive sampling), but may have good applicability if they included patients commonly excluded from randomised trials (eg, elderly or those with comorbidities).

Study results can be at high risk of bias due to confounding, but be highly applicable to a particular patient population. Conversely, a well-done study at low risk of bias that enrols highly selected patients based on factors that would enhance treatment response and reduce its harm, would produce trustworthy results for that specific population, but could be applied to usual patient populations with trepidation. A study can be generalisable but have poor applicability. A good example will be a study with rigorous sampling (ie, well done random sampling) from a population that has strict inclusion criteria (making the results difficult to apply in clinical practice).

Guidelines developers and evidence-based practitioners should consider the applicability of a body of evidence to determine if results are sufficiently indirect to warrant reducing confidence that their results will translate into similar effects in the targeted clinical population. Reasons to reduce one’s confidence in the effects reported in the research evidence also include so-called threats to the internal validity of study results. These threats can increase the risk of bias and reduce the precision of the estimates. An optimal scenario 5 6 for decision makers is when they have pragmatic randomised trials offering low risk of bias evidence that are corroborated by complementary observational studies with good applicability. Such studies can provide important contextual and implementation information.

• Hultcrantz M ,
• Akl EA , et al
• Dekkers OM ,
• von Elm E ,
• Algra A , et al
• Fernandez-Hermida JR ,
• Calafat A ,
• Becoña E , et al
• Guyatt GH ,
• Kunz R , et al
• Rothman KJ ,
• Gallacher JE ,
• Monte LD , et al
• Bourhis V ,
• Bangou J , et al
• Lachin JM , et al

Contributors MHM and VMM conceived the idea and drafted the manuscript. All authors critically revised the manuscript and approved it.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research Research Tools and Apps

External Validity: Types, Research Methods & Examples

External validity is one of the main goals of researchers who want to find reliable cause-and-effect relationships in qualitative research.

When research has this validity, the results can be used with other people in different situations or places. Because without this validity, analysis can’t be generalized, and researchers can’t apply the results of studies to the real world. So, psychology research needs to be conducted outside a lab setting.

Still, sometimes they prefer to research how variables cause each other instead of being able to generalize the results.

In this article, we’ll talk about what external validity means, its types, and its research design methods.

LEARN ABOUT: Theoretical Research

What is external validity?

External validity describes how effectively the findings of an experiment may be generalized to different people, places, or times. Most scientific investigations do not intend to obtain outcomes that only apply to the few persons who participated in the study.

Instead, researchers want to be able to take the results of an experiment and use them with a larger group of people. It is a big part of what inferential statistics try to do.

For example, if you’re looking at a new drug or educational program, you don’t want to know that it works for only a few people. You want to use those results outside the experiment and beyond those participating. It is called “generalizability,” the essential part of this validity.

Types of external validity

Generally, there are three main types of this validity. We’ll discuss each one below and give examples to help you understand.

Population validity

Population validity is a kind of external validity that looks at how well the study’s results applied to a larger group of people. In this case, “population” refers to the group of people about whom a researcher is trying to conclude. On the other hand, a sample is a particular group of people who participate in the research.

If the results from the sample can apply to a larger group of people, then the study is valid for a large population.

Example: low population validity

You want to test the theory about how exercise and sleep are linked. You think that adults will sleep better when they do physical activities regularly. Your target group is adults in the United States, but your sample comprises about 300 college students. 

Even though they are all adults, it might be hard to ensure the population validity in this case because the sampling model of students only represents some adults in the US.

So, your study has a limited amount of population validity, and you can only apply the results to some of the population.

Ecological validity

Ecological validity is another type of external validity that shows how well the research results can be used in different situations. In simple terms, ecological validity is about whether or not your results can be used in the real world.

So, if a study has a lot of ecological validity, the results can be used in the real world. On the other hand, low validity means that the results can’t be used outside the experiment.

Example: low ecological validity

The Milgram Experiment is a classic example of low ecological validity.

Stanley Milgram studied authority in the 1960s. He randomly chose participants and directed them to employ higher and higher voltage shocks to penalize wrong-answering actors. The study showed great obedience to authorities despite fake shock and victim behaviors.

The results of this study are revolutionary for the field of social psychology. However, it is often criticized because it has little ecological validity. Milgram’s set-up was not like real-life situations.

In the experiment, he set up a situation where the participants couldn’t avoid obeying the rules. But the reality of the issue can be very different.

Temporal validity

When figuring out external validity, time is just as important as the number of people involved and confusing factors.

The concept of temporal validity refers to how findings evolve. Particularly, this form of validity refers to how well the research results can be extended to another period.

High temporal validity means that research results can be used correctly in different times and places and that factors will be important in the future.

Imagine that you’re a psychologist, and you’re studying how people act the same.

You found out that social pressure from the majority group has a big effect on the choices of the minority. Because of this, people act similarly. Even though famous psychologist Solomon Asch did this research in the 1950s, the results can still be used in the real world today. 

This study, therefore, has temporal validity even after nearly a century.

Research methods of external validity

There are a lot of methods you can do to improve the external validity of your research. Some things that can improve are given below:

Field experiments

Field experiments are like conducting research outside rather than in a controlled environment like a laboratory.

Criteria for inclusion and exclusion

Establishing criteria for who can participate in the research and ensuring that the group being examined is properly identified

Realism in psychology

If you want the participants to believe that the events that take place throughout the study are true, you should provide them with a cover story regarding the purpose of the research. So that they don’t behave any differently than they would in real life based on the fact.

Replication

Doing the study again with different samples or in a different place to see if you get the same results. When many studies have been done on the same topic, a meta-analysis can be used to see if the effect of an independent variable can be repeated to make it more reliable.

Reprocessing

It is like using statistical methods to fix problems with external validity, like reweighting groups if they were different in a certain way, such as age.

LEARN ABOUT: 12 Best Tools for Researchers

As stated in the article, the ability to replicate the results of an experiment is a key component of its external validity. Using the sampling methods the external validity can be improved in the research.

Researchers compare the results to other relevant data to determine the external validity. They can also do the research with more people from the target population. It’s hard to figure out external validity in research, but it’s important to use the results in the future.

The QuestionPro research suite is an enterprise-level research tool that can help you with your research process and surveys.

We at QuestionPro provide tools for data collection, such as our survey software, and a library of insights for any lengthy study. If you’re interested in seeing a demo or learning more, visit the Insight Hub.

LEARN MORE         FREE TRIAL

MORE LIKE THIS

You Can’t Please Everyone — Tuesday CX Thoughts

Oct 22, 2024

Edit survey: A new way of survey building and collaboration

Oct 10, 2024

Pulse Surveys vs Annual Employee Surveys: Which to Use

Oct 4, 2024

Employee Perception Role in Organizational Change

Oct 3, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence