difference between experimental work and observation

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Observational Study vs Experiment with Examples

By Jim Frost 1 Comment

Comparing Observational Studies vs Experiments

Observational studies and experiments are two standard research methods for understanding the world. Both research designs collect data and use statistical analysis to understand relationships between variables. Beyond that commonality, they are vastly different and have dissimilar sets of pros and cons.

Experiments are controlled investigations where researchers actively manipulate one or more variables to observe the effect on another variable, all within a carefully controlled environment. Researchers must be able to control the treatment condition each subject experiences. Experiments typically use randomization to equalize the experimental groups at the start of the study to control potential confounders.

In this post, we’ll compare an observational study vs experiment, highlighting their definitions, strengths, and when to use them effectively. I work through an example showing how a study can use either approach to answer the same research question.

Learn more about Experimental Design: Definition and Types and Confounding Variable Bias .

Strengths of Observational Studies

Real-World Insights : Observational studies reflect real-world scenarios, providing valuable insights into how things naturally occur. Well-designed observational studies have high external validity , specifically ecological validity .

Does Not Require Randomization : Observational studies shine when researchers can’t manipulate treatment conditions or ethical constraints prevent randomization. For example, studying the long-term effects of smoking requires an observational approach because we can’t ethically assign people to smoke or abstain from smoking.

Cost-Effective : Observational studies are generally less expensive and time-consuming than experiments.

Longitudinal Research : They are well-suited for long-term studies or those tracking trends over time.

Strengths of Experiments

Causality : Experiments are the gold standard for establishing causality. By controlling variables and randomly assigning treatment conditions to participants, researchers can confidently attribute changes to the manipulated factor . Well-designed experiments have high internal validity . Learn more about Correlation vs. Causation: Understanding the Differences .

Controlled Environment : Experiments offer a controlled environment, reducing the influence of confounding variables and enhancing the reliability of results.

Replicability : Well-designed experiments are often easier to replicate, increasing researchers’ ability to compare and confirm results.

Randomization : Random assignment in experiments minimizes bias, ensuring all groups are comparable. Learn more about Random Assignment in Experiments .

When to Choose Observational Studies vs Experiments

Observational studies vs experiments are two vital tools in the statistician ’s arsenal, each offering unique advantages.

Experiments excel in establishing causality, controlling variables, and minimizing the impact of confounders. However, they are more expensive and randomly assigning subjects to the treatment groups is impossible in some settings. Learn more about Randomized Controlled Trials .

Meanwhile, observational studies provide real-world insights, are less expensive, and do not require randomization but are more susceptible to the effects of confounders. Identifying causal relationships is problematic in these studies. Learn more about Observational Studies: Definition & Examples  and Correlational Studies .

Observational studies can be prospective or retrospective studies . On the other hand, randomized experiments must be prospective studies .

The choice between an observational study vs experiment hinges on your research objectives, the context in which you’re working, available time and resources, and your ability to assign subjects to the experimental groups and control other variables.

If you’re looking for a middle ground choice between observational studies vs experiments, consider using a quasi-experimental design. These methods don’t require you to randomly assign participants to the experimental groups and still allow you to draw better causal conclusions about an intervention than an observational study. Learn more about Quasi-Experimental Design Overview & Examples .

Understanding their strengths and differences will help you make the right choice for your statistical endeavors.

Observational Study vs Experiment Example

Suppose you want to assess the health benefits of consuming a daily multivitamin. Let’s explore how an observational study vs experiment would evaluate this research question and their pros and cons.

An observational study will recruit subjects and have them record their vitamin consumption, various health outcomes, and, ideally, record confounding variables. The participants choose whether or not to take vitamins during the study based on their existing habits. Some medical measurements might occur in a lab setting, but researchers are not administering treatments (vitamins). Then, using statistical models, researchers can evaluate the relationship between vitamin consumption and health outcomes while controlling for potential confounders they measured.

An experiment will recruit subjects and then randomly assign them to the treatment group that takes daily vitamins or the control group taking a placebo . Randomization controls all confounders whether the researchers know of them or not. Finally, the researchers compare the treatment to the control group. Learn more about Control Groups in Experiments .

Most vitamin studies are observational because the randomization process would be challenging to implement, and it raises ethical concerns in this context. The random assignment process would override the participants’ preferences for taking vitamins by randomly forcing subjects to consume vitamins or placebos for decades . That’s how long it takes for the differences in health outcomes to manifest. Consequently, enforcing the rigid protocol for so long would be difficult and unethical.

For an observational study, a critical downside is that the pre-existing differences between those who do and do not take vitamins daily comprise a pretty long list of health-related habits and medical measures. Any of them can potentially explain the difference in outcomes instead of the vitamin consumption!

As you can see, using an observational study vs experiment involves many tradeoffs! Let’s close with a table that summarizes the differences.

Differences between an Observational Study and Experiment

Share this:

Reader Interactions

October 22, 2023 at 11:17 pm

Well stated: ” Both research designs collect data and use statistical analysis to understand relationships between variables” I was not familiar with the terms research designs. 😀

PS, I am already receiving all your wonderful mailing. I binge-read them every few weeks. I am planning on getting your other two books when I can. Thanks, and Cheers!

Comments and Questions Cancel reply

Observational vs. Experimental Study: A Comprehensive Guide

Explore the fundamental disparities between experimental and observational studies in this comprehensive guide by Santos Research Center, Corp. Uncover concepts such as control group, random sample, cohort studies, response variable, and explanatory variable that shape the foundation of these methodologies. Discover the significance of randomized controlled trials and case control studies, examining causal relationships and the role of dependent variables and independent variables in research designs.

This enlightening exploration also delves into the meticulous scientific study process, involving survey members, systematic reviews, and statistical analyses. Investigate the careful balance of control group and treatment group dynamics, highlighting how researchers meticulously assign variables and analyze statistical patterns to discern meaningful insights. From dissecting issues like lung cancer to understanding sleep patterns, this guide emphasizes the precision of controlled experiments and controlled trials, where variables are isolated and scrutinized, paving the way for a deeper comprehension of the world through empirical research.

Introduction to Observational and Experimental Studies

These two studies are the cornerstones of scientific inquiry, each offering a distinct approach to unraveling the mysteries of the natural world.

Observational studies allow us to observe, document, and gather data without direct intervention. They provide a means to explore real-world scenarios and trends, making them valuable when manipulating variables is not feasible or ethical. From surveys to meticulous observations, these studies shed light on existing conditions and relationships.

Experimental studies , in contrast, put researchers in the driver's seat. They involve the deliberate manipulation of variables to understand their impact on specific outcomes. By controlling the conditions, experimental studies establish causal relationships, answering questions of causality with precision. This approach is pivotal for hypothesis testing and informed decision-making.

At Santos Research Center, Corp., we recognize the importance of both observational and experimental studies. We employ these methodologies in our diverse research projects to ensure the highest quality of scientific investigation and to answer a wide range of research questions.

Observational Studies: A Closer Look

In our exploration of research methodologies, let's zoom in on observational research studies—an essential facet of scientific inquiry that we at Santos Research Center, Corp., expertly employ in our diverse research projects.

What is an Observational Study?

Observational research studies involve the passive observation of subjects without any intervention or manipulation by researchers. These studies are designed to scrutinize the relationships between variables and test subjects, uncover patterns, and draw conclusions grounded in real-world data.

Researchers refrain from interfering with the natural course of events in controlled experiment. Instead, they meticulously gather data by keenly observing and documenting information about the test subjects and their surroundings. This approach permits the examination of variables that cannot be ethically or feasibly manipulated, making it particularly valuable in certain research scenarios.

Types of Observational Studies

Now, let's delve into the various forms that observational studies can take, each with its distinct characteristics and applications.

Cohort Studies:  A cohort study is a type of observational study that entails tracking one group of individuals over an extended period. Its primary goal is to identify potential causes or risk factors for specific outcomes or treatment group. Cohort studies provide valuable insights into the development of conditions or diseases and the factors that influence them.

Case-Control Studies:  Case-control studies, on the other hand, involve the comparison of individuals with a particular condition or outcome to those without it (the control group). These studies aim to discern potential causal factors or associations that may have contributed to the development of the condition under investigation.

Cross-Sectional Studies:  Cross-sectional studies take a snapshot of a diverse group of individuals at a single point in time. By collecting data from this snapshot, researchers gain insights into the prevalence of a specific condition or the relationships between variables at that precise moment. Cross-sectional studies are often used to assess the health status of the different groups within a population or explore the interplay between various factors.

Advantages and Limitations of Observational Studies

Observational studies, as we've explored, are a vital pillar of scientific research, offering unique insights into real-world phenomena. In this section, we will dissect the advantages and limitations that characterize these studies, shedding light on the intricacies that researchers grapple with when employing this methodology.

Advantages: One of the paramount advantages of observational studies lies in their utilization of real-world data. Unlike controlled experiments that operate in artificial settings, observational studies embrace the complexities of the natural world. This approach enables researchers to capture genuine behaviors, patterns, and occurrences as they unfold. As a result, the data collected reflects the intricacies of real-life scenarios, making it highly relevant and applicable to diverse settings and populations.

Moreover, in a randomized controlled trial, researchers looked to randomly assign participants to a group. Observational studies excel in their capacity to examine long-term trends. By observing one group of subjects over extended periods, research scientists gain the ability to track developments, trends, and shifts in behavior or outcomes. This longitudinal perspective is invaluable when studying phenomena that evolve gradually, such as chronic diseases, societal changes, or environmental shifts. It allows for the detection of subtle nuances that may be missed in shorter-term investigations.

Limitations: However, like any research methodology, observational studies are not without their limitations. One significant challenge of statistical study lies in the potential for biases. Since researchers do not intervene in the subjects' experiences, various biases can creep into the data collection process. These biases may arise from participant self-reporting, observer bias, or selection bias in random sample, among others. Careful design and rigorous data analysis are crucial for mitigating these biases.

Another limitation is the presence of confounding variables. In observational studies, it can be challenging to isolate the effect of a specific variable from the myriad of other factors at play. These confounding variables can obscure the true relationship between the variables of interest, making it difficult to establish causation definitively. Research scientists must employ statistical techniques to control for or adjust these confounding variables.

Additionally, observational studies face constraints in their ability to establish causation. While they can identify associations and correlations between variables, they cannot prove causality or causal relationship. Establishing causation typically requires controlled experiments where researchers can manipulate independent variables systematically. In observational studies, researchers can only infer potential causation based on the observed associations.

Experimental Studies: Delving Deeper

In the intricate landscape of scientific research, we now turn our gaze toward experimental studies—a dynamic and powerful method that Santos Research Center, Corp. skillfully employs in our pursuit of knowledge.

What is an Experimental Study?

While some studies observe and gather data passively, experimental studies take a more proactive approach. Here, researchers actively introduce an intervention or treatment to an experiment group study its effects on one or more variables. This methodology empowers researchers to manipulate independent variables deliberately and examine their direct impact on dependent variables.

Experimental research are distinguished by their exceptional ability to establish cause-and-effect relationships. This invaluable characteristic allows researchers to unlock the mysteries of how one variable influences another, offering profound insights into the scientific questions at hand. Within the controlled environment of an experimental study, researchers can systematically test hypotheses, shedding light on complex phenomena.

Key Features of Experimental Studies

Central to statistical analysis, the rigor and reliability of experimental studies are several key features that ensure the validity of their findings.

Randomized Controlled Trials:  Randomization is a critical element in experimental studies, as it ensures that subjects are assigned to groups in a random assignment. This randomly assigned allocation minimizes the risk of unintentional biases and confounding variables, strengthening the credibility of the study's outcomes.

Control Groups:  Control groups play a pivotal role in experimental studies by serving as a baseline for comparison. They enable researchers to assess the true impact of the intervention being studied. By comparing the outcomes of the intervention group to those of survey members of the control group, researchers can discern whether the intervention caused the observed changes.

Blinding:  Both single-blind and double-blind techniques are employed in experimental studies to prevent biases from influencing the study or controlled trial's outcomes. Single-blind studies keep either the subjects or the researchers unaware of certain aspects of the study, while double-blind studies extend this blindness to both parties, enhancing the objectivity of the study.

These key features work in concert to uphold the integrity and trustworthiness of the results generated through experimental studies.

Advantages and Limitations of Experimental Studies

As with any research methodology, this one comes with its unique set of advantages and limitations.

Advantages:  These studies offer the distinct advantage of establishing causal relationships between two or more variables together. The controlled environment allows researchers to exert authority over variables, ensuring that changes in the dependent variable can be attributed to the independent variable. This meticulous control results in high-quality, reliable data that can significantly contribute to scientific knowledge.

Limitations:  However, experimental ones are not without their challenges. They may raise ethical concerns, particularly when the interventions involve potential risks to subjects. Additionally, their controlled nature can limit their real-world applicability, as the conditions in experiments may not accurately mirror those in the natural world. Moreover, executing an experimental study in randomized controlled, often demands substantial resources, with other variables including time, funding, and personnel.

Observational vs Experimental: A Side-by-Side Comparison

Having previously examined observational and experimental studies individually, we now embark on a side-by-side comparison to illuminate the key distinctions and commonalities between these foundational research approaches.

Key Differences and Notable Similarities

Methodologies

• Observational Studies : Characterized by passive observation, where researchers collect data without direct intervention, allowing the natural course of events to unfold.
• Experimental Studies : Involve active intervention, where researchers deliberately manipulate variables to discern their impact on specific outcomes, ensuring control over the experimental conditions.
• Observational Studies : Designed to identify patterns, correlations, and associations within existing data, shedding light on relationships within real-world settings.
• Experimental Studies : Geared toward establishing causality by determining the cause-and-effect relationships between variables, often in controlled laboratory environments.
• Observational Studies : Yield real-world data, reflecting the complexities and nuances of natural phenomena.
• Experimental Studies : Generate controlled data, allowing for precise analysis and the establishment of clear causal connections.

Observational studies excel at exploring associations and uncovering patterns within the intricacies of real-world settings, while experimental studies shine as the gold standard for discerning cause-and-effect relationships through meticulous control and manipulation in controlled environments. Understanding these differences and similarities empowers researchers to choose the most appropriate method for their specific research objectives.

When to Use Which: Practical Applications

The decision to employ either observational or experimental studies hinges on the research objectives at hand and the available resources. Observational studies prove invaluable when variable manipulation is impractical or ethically challenging, making them ideal for delving into long-term trends and uncovering intricate associations between certain variables (response variable or explanatory variable). On the other hand, experimental studies emerge as indispensable tools when the aim is to definitively establish causation and methodically control variables.

At Santos Research Center, Corp., our approach to both scientific study and methodology is characterized by meticulous consideration of the specific research goals. We recognize that the quality of outcomes hinges on selecting the most appropriate method of research study. Our unwavering commitment to employing both observational and experimental research studies further underscores our dedication to advancing scientific knowledge across diverse domains.

Conclusion: The Synergy of Experimental and Observational Studies in Research

In conclusion, both observational and experimental studies are integral to scientific research, offering complementary approaches with unique strengths and limitations. At Santos Research Center, Corp., we leverage these methodologies to contribute meaningfully to the scientific community.

Explore our projects and initiatives at Santos Research Center, Corp. by visiting our website or contacting us at (813) 249-9100, where our unwavering commitment to rigorous research practices and advancing scientific knowledge awaits.

Recent Posts

At Santos Research Center, a medical research facility dedicated to advancing TBI treatments, we emphasize the importance of tailored rehabilitation...

Learn about COVID-19 rebound after Paxlovid, its symptoms, causes, and management strategies. Join our study at Santos Research Center. Apply now!

Learn everything about Respiratory Syncytial Virus (RSV), from symptoms and diagnosis to treatment and prevention. Stay informed and protect your health with...

Discover key insights on Alzheimer's disease, including symptoms, stages, and care tips. Learn how to manage the condition and find out how you can...

Discover expert insights on migraines, from symptoms and causes to management strategies, and learn about our specialized support at Santos Research Center.

Explore our in-depth guide on UTIs, covering everything from symptoms and causes to effective treatments, and learn how to manage and prevent urinary tract infections.

Your definitive guide to COVID symptoms. Dive deep into the signs of COVID-19, understand the new variants, and get answers to your most pressing questions.

Santos Research Center, Corp. is a research facility conducting paid clinical trials, in partnership with major pharmaceutical companies & CROs. We work with patients from across the Tampa Bay area.

Contact Details

Navigation menu.

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Experimental vs Observational Studies: Differences & Examples

Understanding the differences between experimental vs observational studies is crucial for interpreting findings and drawing valid conclusions. Both methodologies are used extensively in various fields, including medicine, social sciences, and environmental studies. 

Researchers often use observational and experimental studies to gather comprehensive data and draw robust conclusions about their investigating phenomena. 

This blog post will explore what makes these two types of studies unique, their fundamental differences, and examples to illustrate their applications.

What is an Experimental Study?

An experimental study is a research design in which the investigator actively manipulates one or more variables to observe their effect on another variable. This type of study often takes place in a controlled environment, which allows researchers to establish cause-and-effect relationships.

Key Characteristics of Experimental Studies:

• Manipulation: Researchers manipulate the independent variable(s).
• Control: Other variables are kept constant to isolate the effect of the independent variable.
• Randomization: Subjects are randomly assigned to different groups to minimize bias.
• Replication: The study can be replicated to verify results.

Types of Experimental Study

• Laboratory Experiments: Conducted in a controlled environment where variables can be precisely controlled.
• Field Research : These are conducted in a natural setting but still involve manipulation and control of variables.
• Clinical Trials: Used in medical research and the healthcare industry to test the efficacy of new treatments or drugs.

Example of an Experimental Study:

Imagine a study to test the effectiveness of a new drug for reducing blood pressure. Researchers would:

• Randomly assign participants to two groups: receiving the drug and receiving a placebo.
• Ensure that participants do not know their group (double-blind procedure).
• Measure blood pressure before and after the intervention.
• Compare the changes in blood pressure between the two groups to determine the drug’s effectiveness.

What is an Observational Study?

An observational study is a research design in which the investigator observes subjects and measures variables without intervening or manipulating the study environment. This type of study is often used when manipulating impractical or unethical variables.

Key Characteristics of Observational Studies:

• No Manipulation: Researchers do not manipulate the independent variable.
• Natural Setting: Observations are made in a natural environment.
• Causation Limitations: It is difficult to establish cause-and-effect relationships due to the need for more control over variables.
• Descriptive: Often used to describe characteristics or outcomes.

Types of Observational Studies: 

• Cohort Studies : Follow a control group of people over time to observe the development of outcomes.
• Case-Control Studies: Compare individuals with a specific outcome (cases) to those without (controls) to identify factors that might contribute to the outcome.
• Cross-Sectional Studies : Collect data from a population at a single point to analyze the prevalence of an outcome or characteristic.

Example of an Observational Study:

Consider a study examining the relationship between smoking and lung cancer. Researchers would:

• Identify a cohort of smokers and non-smokers.
• Follow both groups over time to record incidences of lung cancer.
• Analyze the data to observe any differences in cancer rates between smokers and non-smokers.

Difference Between Experimental vs Observational Studies

Choosing Between Experimental and Observational Studies

The researchers relied on statistical analysis to interpret the results of randomized controlled trials, building upon the foundations established by prior research.

Use Experimental Studies When:

• Causality is Important: If determining a cause-and-effect relationship is crucial, experimental studies are the way to go.
• Variables Can Be Controlled: When you can manipulate and control the variables in a lab or controlled setting, experimental studies are suitable.
• Randomization is Possible: When random assignment of subjects is feasible and ethical, experimental designs are appropriate.

Use Observational Studies When:

• Ethical Concerns Exist: If manipulating variables is unethical, such as exposing individuals to harmful substances, observational studies are necessary.
• Practical Constraints Apply: When experimental studies are impractical due to cost or logistics, observational studies can be a viable alternative.
• Natural Settings Are Required: If studying phenomena in their natural environment is essential, observational studies are the right choice.

Strengths and Limitations

Experimental studies.

• Establish Causality: Experimental studies can establish causal relationships between variables by controlling and using randomization.
• Control Over Confounding Variables: The controlled environment allows researchers to minimize the influence of external variables that might skew results.
• Repeatability: Experiments can often be repeated to verify results and ensure consistency.

Limitations:

• Ethical Concerns: Manipulating variables may be unethical in certain situations, such as exposing individuals to harmful conditions.
• Artificial Environment: The controlled setting may not reflect real-world conditions, potentially affecting the generalizability of results.
• Cost and Complexity: Experimental studies can be costly and logistically complex, especially with large sample sizes.

Observational Studies

• Real-World Insights: Observational studies provide valuable insights into how variables interact in natural settings.
• Ethical and Practical: These studies avoid ethical concerns associated with manipulation and can be more practical regarding cost and time.
• Diverse Applications: Observational studies can be used in various fields and situations where experiments are not feasible.
• Lack of Causality: It’s easier to establish causation with manipulation, and results are limited to identifying correlations.
• Potential for Confounding: Uncontrolled external variables may influence the results, leading to biased conclusions.
• Observer Bias: Researchers may unintentionally influence outcomes through their expectations or interpretations of data.

Examples in Various Fields

• Experimental Study: Clinical trials testing the effectiveness of a new drug against a placebo to determine its impact on patient recovery.
• Observational Study: Studying the dietary habits of different populations to identify potential links between nutrition and disease prevalence.
• Experimental Study: Conducting a lab experiment to test the effect of sleep deprivation on cognitive performance by controlling sleep hours and measuring test scores.
• Observational Study: Observing social interactions in a public setting to explore natural communication patterns without intervention.

Environmental Science

• Experimental Study: Testing the impact of a specific pollutant on plant growth in a controlled greenhouse setting.
• Observational Study: Monitoring wildlife populations in a natural habitat to assess the effects of climate change on species distribution.

How QuestionPro Research Can Help in Experimental vs Observational Studies

Choosing between experimental and observational studies is a critical decision that can significantly impact the outcomes and interpretations of a study. QuestionPro Research offers powerful tools and features that can enhance both types of studies, giving researchers the flexibility and capability to gather, analyze, and interpret data effectively.

Enhancing Experimental Studies with QuestionPro

Experimental studies require a high degree of control over variables, randomization, and, often, repeated trials to establish causal relationships. QuestionPro excels in facilitating these requirements through several key features:

• Survey Design and Distribution: With QuestionPro, researchers can design intricate surveys tailored to their experimental needs. The platform supports random assignment of participants to different groups, ensuring unbiased distribution and enhancing the study’s validity.
• Data Collection and Management: Real-time data collection and management tools allow researchers to monitor responses as they come in. This is crucial for experimental studies where data collection timing and sequence can impact the results.
• Advanced Analytics: QuestionPro offers robust analytical tools that can handle complex data sets, enabling researchers to conduct in-depth statistical analyses to determine the effects of the experimental interventions.

Supporting Observational Studies with QuestionPro

Observational studies involve gathering data without manipulating variables, focusing on natural settings and real-world scenarios. QuestionPro’s capabilities are well-suited for these studies as well:

• Customizable Surveys: Researchers can create detailed surveys to capture a wide range of observational data. QuestionPro’s customizable templates and question types allow for flexibility in capturing nuanced information.
• Mobile Data Collection: For field research, QuestionPro’s mobile app enables data collection on the go, making it easier to conduct studies in diverse settings without internet connectivity.
• Longitudinal Data Tracking: Observational studies often require data collection over extended periods. QuestionPro’s platform supports longitudinal studies, allowing researchers to track changes and trends.

Experimental and observational studies are essential tools in the researcher’s toolkit. Each serves a unique purpose and offers distinct advantages and limitations. By understanding their differences, researchers can choose the most appropriate study design for their specific objectives, ensuring their findings are valid and applicable to real-world situations.

Whether establishing causality through experimental studies or exploring correlations with observational research designs, the insights gained from these methodologies continue to shape our understanding of the world around us. 

Whether conducting experimental or observational studies, QuestionPro Research provides a comprehensive suite of tools that enhance research efficiency, accuracy, and depth. By leveraging its advanced features, researchers can ensure that their studies are well-designed, their data is robustly analyzed, and their conclusions are reliable and impactful.

MORE LIKE THIS

QuestionPro: Leading the Charge in Customer Journey Management and Voice of the Customer Platforms

Sep 17, 2024

What is Driver Analysis? Importance and Best Practices

Was The Experience Memorable? (Part II) — Tuesday CX Thoughts

Data Discovery: What it is, Importance, Process + Use Cases

Sep 16, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence

Experiment vs Observational Study: Similarities & Differences

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Learn about our Editorial Process

An experiment involves the deliberate manipulation of variables to observe their effect, while an observational study involves collecting data without interfering with the subjects or variables under study.

This article will explore both, but let’s start with some quick explanations:

• Experimental Study : An experiment is a research design wherein an investigator manipulates one or more variables to establish a cause-effect relationship (Tan, 2022). For example, a pharmaceutical company may conduct an experiment to find out if a new medicine for diabetes is effective by administering it to a selected group (experimental group), while not administering it to another group (control group).
• Observational Study : An observational study is a type of research wherein the researcher observes characteristics and measures variables of interest in a subset of a population, but does not manipulate or intervene (Atkinson et al., 2021). An example may be a sociologist who conducts a cross-sectional survey of the population to determine health disparities across different income groups. 

Experiment vs Observational Study

1. experiment.

An experiment is a research method characterized by a high degree of experimental control exerted by the researcher. In the context of academia, it allows for the testing of causal hypotheses (Privitera, 2022).

When conducting an experiment, the researcher first formulates a hypothesis , which is a predictive statement about the potential relationship between at least two variables.

For instance, a psychologist may want to test the hypothesis that participation in physical exercise ( independent variable ) improves the cognitive abilities (dependent variable) of the elderly.

In an experiment, the researcher manipulates the independent variable(s) and then observes the effects on the dependent variable(s). This method of research involves two or more comparison groups—an experimental group that is subjected to the variable being tested and a control group that is not (Sampselle, 2012).

For instance, in the physical exercise study noted above, the psychologist would administer a physical exercise regime to an experimental group of elderly people, while a control group would continue with their usual lifestyle activities .

One of the unique features of an experiment is random assignment . Participants are randomly allocated to either the experimental or control groups to ensure that every participant has an equal chance of being in either group. This reduces the risk of confounding variables and increases the likelihood that the results are attributable to the independent variable rather than another factor (Eich, 2014).

For instance, in the physical exercise example, the psychologist would randomly assign participants to the experimental or control group to reduce the potential impact of external variables such as diet or sleep patterns.

1. Impacts of Films on Happiness: A psychologist might create an experimental study where she shows participants either a happy, sad, or neutral film (independent variable) then measures their mood afterward (dependent variable). Participants would be randomly assigned to one of the three film conditions.

2. Impacts of Exercise on Weight Loss: In a fitness study, a trainer could investigate the impact of a high-intensity interval training (HIIT) program on weight loss. Half of the participants in the study are randomly selected to follow the HIIT program (experimental group), while the others follow a standard exercise routine (control group).

3. Impacts of Veganism on Cholesterol Levels: A nutritional experimenter could study the effects of a particular diet, such as veganism, on cholesterol levels. The chosen population gets assigned either to adopt a vegan diet (experimental group) or stick to their usual diet (control group) for a specific period, after which cholesterol levels are measured.

Read More: Examples of Random Assignment

Strengths and Weaknesses

Read More: Experimental Research Examples

2. Observational Study

Observational research is a non-experimental research method in which the researcher merely observes the subjects and notes behaviors or responses that occur (Ary et al., 2018).

This approach is unintrusive in that there is no manipulation or control exerted by the researcher. For instance, a researcher could study the relationships between traffic congestion and road rage by just observing and recording behaviors at a set of busy traffic lights, without applying any control or altering any variables.

In observational studies, the researcher distinguishes variables and measures their values as they naturally occur. The goal is to capture naturally occurring behaviors , conditions, or events (Ary et al., 2018).

For example, a sociologist might sit in a cafe to observe and record interactions between staff and customers in order to examine social and occupational roles .

There is a significant advantage of observational research in that it provides a high level of ecological validity – the extent to which the data collected reflects real-world situations – as the behaviors and responses are observed in a natural setting without experimenter interference (Holleman et al., 2020)

However, the inability to control various factors that might influence the observations may expose these studies to potential confounding bias , a consideration researchers must take into account (Schober & Vetter, 2020).

1. Behavior of Animals in the Wild: Zoologists often use observational studies to understand the behaviors and interactions of animals in their natural habitats. For instance, a researcher could document the social structure and mating behaviors of a wolf pack over a period of time.

2. Impact of Office Layout on Productivity: A researcher in organizational psychology might observe how different office layouts affect staff productivity and collaboration. This involves the observation and recording of staff interactions and work output without altering the office setting.

3. Foot Traffic and Retail Sales: A market researcher might conduct an observational study on how foot traffic (the number of people passing by a store) impacts retail sales. This could involve observing and documenting the number of walk-ins, time spent in-store, and purchase behaviors.

Read More: Observational Research Examples

Experimental and Observational Study Similarities and Differences

Experimental and observational research both have their place – one is right for one situation, another for the next.

Experimental research is best employed when the aim of the study is to establish cause-and-effect relationships between variables – that is, when there is a need to determine the impact of specific changes on the outcome (Walker & Myrick, 2016).

One of the standout features of experimental research is the control it gives to the researcher, who dictates how variables should be changed and assigns participants to different conditions (Privitera, 2022). This makes it an excellent choice for medical or pharmaceutical studies, behavioral interventions, and any research where hypotheses concerning influence and change need to be tested.

For example, a company might use experimental research to understand the effects of staff training on job satisfaction and productivity.

Observational research , on the other hand, serves best when it’s vital to capture the phenomena in their natural state, without intervention, or when ethical or practical considerations prevent the researcher from manipulating the variables of interest (Creswell & Poth, 2018).

It is the method of choice when the interest of the research lies in describing what is, rather than altering a situation to see what could be (Atkinson et al., 2021).

This approach might be utilized in studies that aim to describe patterns of social interaction, daily routines, user experiences, and so on. A real-world example of observational research could be a study examining the interactions and learning behaviors of students in a classroom setting.

I’ve demonstrated their similarities and differences a little more in the table below:

Experimental and observational research each have their place, depending upon the study. Importantly, when selecting your approach, you need to reflect upon your research goals and objectives, and select from the vast range of research methodologies , which you can read up on in my next article, the 15 types of research designs .

Ary, D., Jacobs, L. C., Irvine, C. K. S., & Walker, D. (2018). Introduction to research in education . London: Cengage Learning.

Atkinson, P., Delamont, S., Cernat, A., Sakshaug, J. W., & Williams, R. A. (2021). SAGE research methods foundations . New York: SAGE Publications Ltd.

Creswell, J.W., and Poth, C.N. (2018). Qualitative Inquiry and Research Design: Choosing among Five Approaches . New York: Sage Publications.

Eich, E. (2014). Business Research Methods: A Radically Open Approach . Frontiers Media SA.

Holleman, G. A., Hooge, I. T., Kemner, C., & Hessels, R. S. (2020). The ‘real-world approach’and its problems: A critique of the term ecological validity. Frontiers in Psychology , 11 , 721. doi: https://doi.org/10.3389/fpsyg.2020.00721  

Privitera, G. J. (2022). Research methods for the behavioral sciences . Sage Publications.

Sampselle, C. M. (2012). The Science and Art of Nursing Research . South University Online Press.

Schober, P., & Vetter, T. R. (2020). Confounding in observational research. Anesthesia & Analgesia , 130 (3), 635.

Tan, W. C. K. (2022). Research methods: A practical guide for students and researchers . World Scientific.

Walker, D., and Myrick, F. (2016). Grounded Theory: An Exploration of Process and Procedure . New York: Qualitative Health Research.

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 10 Reasons you’re Perpetually Single
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 20 Montessori Toddler Bedrooms (Design Inspiration)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 21 Montessori Homeschool Setups
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 101 Hidden Talents Examples

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

Introduction to Data Science I & II

Observational versus experimental studies, observational versus experimental studies #.

In most research questions or investigations, we are interested in finding an association that is causal (the first scenario in the previous section ). For example, “Is the COVID-19 vaccine effective?” is a causal question. The researcher is looking for an association between receiving the COVID-19 vaccine and contracting (symptomatic) COVID-19, but more specifically wants to show that the vaccine causes a reduction in COVID-19 infections (Baden et al., 2020) 1 .

Experimental Studies #

There are 3 necessary conditions for showing that a variable X (for example, vaccine) causes an outcome Y (such as not catching COVID-19):

Temporal Precedence : We must show that X (the cause) happened before Y (the effect).

Non-spuriousness : We must show that the effect Y was not seen by chance.

No alternate cause : We must show that no other variable accounts for the relationship between X and Y .

If any of the three is not present, the association cannot be causal. If the proposed cause did not happen before the effect, it cannot have caused the effect. In addition, if the effect was seen by chance and cannot be replicated, the association is spurious and therefore not causal. Lastly, if there is another phenomenon that accounts for the association seen, then it cannot be a causal association. These conditions are therefore, necessary to show causality.

The best way to show all three necessary conditions is by conducting an experiment . Experiments involve controllable factors which are measured and determined by the experimenter, uncontrollable factors which are measured but not determined by the experimentor, and experimental variability or noise which is unmeasured and uncontrolled. Controllable factors that the experimenter manipulates in his or her experiment are known as independent variables . In our vaccination example, the independent variable is receipt of vaccine. Uncontrollable factors that are hypothesized to depend on the independent variable are known as dependent variables. The dependent variable in the vaccination example is contraction of COVID-19. The experimentor cannot control whether participants catch the disease, but can measure it, and it is hypothesized that catching the disease is dependent on vaccination status.

Control Groups #

When conducting an experiment, it is important to have a comparison or control group . The control group is used to better understand the effect of the independent variable. For example, if all patients are given the vaccine, it would be impossible to measure whether the vaccine is effective as we would not know the outcome if patients had not received the vaccine. In order to measure the effect of the vaccine, the researcher must compare patients who did not receive the vaccine to patients that did receive the vaccine. This comparison group of patients who did not receive the vaccine is the control group for the experiment. The control group allows the researcher to view an effect or association. When scientists say that the COVID-19 vaccine is 94% effective, this does not mean that only 6% of people who got the vaccine in their study caught COVID-19 (the number is actually much lower!). That would not take into account the rate of catching COVID-19 for those without a vaccine. Rather, 94% effective refers to having 94% lower incidence of infection compared to the control group.

Let’s illustrate this using data from the efficacy trial by Baden and colleagues in 2020. In their primary analysis, 14,073 participants were in the placebo group and 14,134 in the vaccine group. Of these participants, a total of 196 were diagnosed with COVID-19 during the 78 day follow-up period: 11 in the vaccine group and 186 in the placebo group. This means, 0.08% of those in the vaccine group and 1.32% of those in the placebo group were diagnosed with COVID-19. Dividing 0.08 by 1.32, we see that the proportion of cases in the vaccine group was only 6% of the proportion of cases in the placebo group. Therefore, the vaccine is 94% effective.

Chicago has a population of almost 3,000,000. Extrapolating using the numbers from above, without the vaccine, 39,600 people would be expected to catch COVID-19 in the period between 14 and 92 days after their second vaccine. If everyone were vaccinated, the expected number would drop to 2,400. This is a large reduction! However, it is important that the researcher shows this effect is non-spurious and therefore important and significant. One way to do this is through replication : applying a treatment independently across two or more experimental subjects. In our example, researchers conducted many similar experiments for multiple groups of patients to show that the effect can be seen reliably.

Randomization #

A researcher must also be able to show there is no alternate cause for the association in order to prove causality. This can be done through randomization : random assignment of treatment to experimental subjects. Consider a group of patients where all male patients are given the treatment and all female patients are in the control group. If an association is found, it would be unclear whether this association is due to the treatment or the fact that the groups were of differing sex. By randomizing experimental subjects to groups, researchers ensure there is no systematic difference between groups other than the treatment and therefore no alternate cause for the relationship between treatment and outcome.

Another way of ensuring there is no alternate cause is by blocking : grouping similar experimental units together and assigning different treatments within such groups. Blocking is a way of dealing with sources of variability that are not of primary interest to the experimenter. For example, a researcher may block on sex by grouping males together and females together and assigning treatments and controls within the different groups. Best practices are to block the largest and most salient sources of variability and randomize what is difficult or impossible to block. In our example blocking would account for variability introduced by sex whereas randomization would account for factors of variability such as age or medical history which are more difficult to block.

Observational Studies #

Randomized experiments are considered the “Gold Standard” for showing a causal relationship. However, it is not always ethical or feasible to conduct a randomized experiment. Consider the following research question: Does living in Northern Chicago increase life expectancy? It would be infeasible to conduct an experiment which randomly allocates people to live in different parts of the city. Therefore, we must turn to observational data to test this question. Where experiments involve one or more variables controlled by the experimentor (dose of a drug for example), in observational studies there is no effort or intention to manipulate or control the object of study. Rather, researchers collect data without interfering with the subjects. For example, researchers may conduct a survey gathering both health and neighborhood data, or they may have access to administrative data from a local hospital. In these cases, the researchers are merely observing variables and outcomes.

There are two types of observational studies: retrospective studies and prospective studies. In a retrospective study , data is collected after events have taken place. This may be through surveys, historical data, or administrative records. An example of a retrospective study would be using administrative data from a hospital to study incidence of disease. In contrast, a prospective study identifies subjects beforehand and collects data as events unfold. For example, one might use a prospective study to evaluate how personality traits develop in children, by following a predetermined set of children through elementary school and giving them personality assessments each year.

Baden LR, El Sahly HM, Essink B, Kotloff K, Frey S, Novak R, Diemert D, Spector SA, Rouphael N, Creech CB, McGettigan J. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. New England journal of medicine. 2020 Dec 30.

• How It Works
• PhD thesis writing
• Master thesis writing
• Bachelor thesis writing
• Dissertation writing service
• Dissertation abstract writing
• Thesis proposal writing
• Thesis editing service
• Thesis proofreading service
• Thesis formatting service
• Coursework writing service
• Research paper writing service
• Architecture thesis writing
• Computer science thesis writing
• Engineering thesis writing
• History thesis writing
• MBA thesis writing
• Nursing dissertation writing
• Psychology dissertation writing
• Sociology thesis writing
• Statistics dissertation writing
• Buy dissertation online
• Write my dissertation
• Cheap thesis
• Cheap dissertation
• Custom dissertation
• Dissertation help
• Pay for thesis
• Pay for dissertation
• Senior thesis
• Write my thesis

Experiment vs Observational Study: A Deeper Look

When we read about research studies and reports, many are times that we fail to pay attention to the design of the study. For you to know the quality of the research findings, it is paramount to start by understanding some basics of research/study design.

The primary goal of doing a study is to evaluate the relationship between several variables. For example, does eating fast food result in teenagers being overweight? Or does going to college increase the chances of getting a job? Most studies fall into two main categories, observational and experimental studies, but what is the difference? Other widely accepted research types are cohort studies, randomized controls, and case-control studies, but these three are part of either experimental or observational study. Keep reading to understand the difference between observational study and experiment.

What Is An Observational Study?

To understand observational study vs experiment, let us start by looking at each of them.

So, what is an observational study ? This is a form of research where the measurement is done on the selected sample without running a control experiment. Therefore, the researcher observes the impact of a specific risk factor, such as treatment or intervention, without focusing on who is not exposed. It is simply a matter of observing what is happening.

When an observational report is released, it indicates that there might be a relationship between several variables, but this cannot be relied on. It is simply too weak or biased. We will demonstrate this with an example.

A study asking people how they liked a new film that was released a few months ago is a good example of an observational study. The researcher in the study does not have any control over the participants. Therefore, even if the study point to some relationship between the main variables, it is considered too weak. For example, the study did not factor in the possibility of viewers watching other films.

The main difference between an observational study and an experiment is that the latter is randomized . Again, unlike the observational study statistics, which are considered biased and weak, evidence from experimental research is stronger.

Advantages of Observational Studies

If you are thinking of carrying a research and have been wondering whether to go for randomized experiment vs observational study, here are some key advantages of the latter.

• Because the observational study does not require the use of control, it is inexpensive to undertake. Suppose you take the example of a study looking at the impact of introducing a new learning method into a school. In that case, all you need is to ask any interested students to participate in a survey with questions, such as “yes” and “no.”
• Doing observational research can also be pretty simple because you do not have to keep looking into multiple variables, and trying to control some groups.
• Sometimes the observational method is the only way to study some things, such as exposure to specific threats. For example, it might not be ethical to expose people to harmful variables, such as radiation. However, it is possible to study the exposed population living in affected areas using observational studies.

While the advantages of observational research might appear attractive, you need to weigh them against the cons. To run conclusive observational research, you might require a lot of time. Sometimes, this might run for years or decades.

The results from observational studies are also open to a lot of criticism because of confounding biases. For example, a cohort study might conclude that most people who love to meditate regularly suffer less from heart issues. However, this alone might not be the only cause of low cases of heart problems. The people who medicate might also be following healthy diets and doing a lot of exercises to stay healthy.

Types of Observational Studies

Observational studies further branches into several categories, including cohort study, cross-sectional, and case-control. Here is a breakdown of these different types of studies:

• Cohort Study

For study purposes, a “cohort” is a team or group of people who are somehow linked. Example, people born within a specific period might be referred to as a “birth cohort.”

The concept of cohort study edges close to that of experimental research. Here, the researcher records whether every participant in the cohort is affected by the selected variables. In a medical setting, the researcher might want to know whether the cohort population in the study got exposed to a certain variable and if they developed the medical condition of interest. This is the most preferred method of study when urgent response, especially to a public health concern, such as a disease outbreak is reported.

It is important to appreciate that this is different from experimental research because the investigator simply observes but does not determine the exposure status of the participants.

• Case Control Study

In this type of study, the researcher enrolls people with a health issue and another without the problem. Then, the two groups are compared based on exposure. The control group is used to generate an estimate of the expected exposure in the population.

• Cross-Sectional Research

This is the third type of observational type of study, and it involves taking a sample from a population that is exposed to health risk and measuring them to establish the extent of the outcome. This study is very common in health settings when researchers want to know the prevalence of a health condition at any specific moment. For example, in a cross-sectional study, some of the selected persons might have lived with high blood pressure for years, while others might have started seeing the signs recently.

Experimental Studies

Now that you know the observational study definition, we will now compare it with experiment research. So, what is experimental research?

In experimental design, the researcher randomly assigns a selected part of the population some treatment to make a cause and effect conclusion. The random selection of samples is largely what makes the experiment different from the observational study design.

The researcher controls the environment, such as exposure levels, and then checks the response produced by the population. In science, the evidence generated by experimental studies is stronger and less contested compared to that produced by observational studies.

Sometimes, you might find experimental study design being referred to as a scientific study. Always remember that when using experimental studies, you need two groups, the main experiment group (part of the population exposed to a variable) and the control (another group that does not get exposed/ treatment by the researcher).

Benefits of Using Experimental Study Design

Here are the main advantages to expect for using experimental study vs observational experiment.

• Most experimental studies are shorter and smaller compared to observational studies.
• The study, especially the selected sample and control group, is monitored closely to ensure the results are accurate.
• Experimental study is the most preferred method of study when targeting uncontested results.

When using experimental studies, it is important to appreciate that it can be pretty expensive because you are essentially following two groups, the experiment sample and control. The cost also arises from the factor that you might need to control the exposure levels and closely follow the progress before drawing a conclusion.

Observational Study vs Experiment: Examples

Now that we have looked at how each design, experimental and observational, work, we will now turn to examples and identify their application.

To improve the quality of life, many people are trying to quit smoking by following different strategies, but it is true that quitting is not easy. So the methods that are used by smokers include:

• Using drugs to reduce addiction to nicotine.
• Using therapy to train smokers how to stop smoking.
• Combining therapy and drugs.
• Cold turkey (neither of the above).

The variable in the study is (I, II, III, IV), and the outcome or response is success or failure to quit the problem of smoking. If you select to use an observational method, the values of the variables (I, ii, iii, iv) would happen naturally, meaning that you would not control them. In an experimental study, values would be assigned by the researcher, implying that you would tell the participants the methods to use. Here is a demonstration:

• Observational Study: Here, you would imagine a population of people trying to quit smoking. Then, use a survey, such as online or telephone interviews, to reach the smokers trying to stop the habit. After a year later, you reach the same persons again, to enquire whether they were successful. Note that you do not run any control over the population.
• Experimental study: In this case, a representative sample of smokers trying to stop the habit is selected through a survey. Say you reach about 1000. Now, the number is divided into four groups of 250 persons, and each group is allocated one of the four methods above (i, ii, iii, or iv).

The results from the experimental study might be as shown below:

From the results of the experimental study, we can say that combining therapy and drugs method helped most smokers to quit the habit successfully. Therefore, a policy can be developed to adopt the most successful method for helping smokers quit the problem.

It is important to note that both studies commence with a random sample. The difference between an observational study and an experiment is that the sample is divided in the latter while it is not in the former. In the case of the experimental study, the researcher is controlling the main variables and then checking the relationship.

A researcher picked a random sample of learners in a class and asked them about their study habits at home. The data showed that students who used at least 30 minutes to study after school scored better grades than those who never studied at all.

This type of study can be classified as observational because the researcher simply asked the respondents about their study habits after school. Because there was no group given a particular treatment, the study cannot qualify as experimental.

In another study, the researcher randomly picked two groups of students in school to determine the effectiveness of a new study method. Group one was asked to follow the new method for a period of three months, while the other was asked to simply study the way they were used. Then, the researcher checked the scores between the two groups to determine if the new method is better.

So, is this an experimental or observational study? This type of study can be categorized as experimental because the researcher randomly picked two groups of respondents. Then, one group was given some treatment, and the other one was not.

In one of the studies, the researcher took a random sample of people and looked at their eating habits. Then, every member was classified as either healthy or at risk of developing obesity. The researcher also drew recommendations to help people at risk of developing overweight issues to avoid the problem.

This type of study is observational because the researcher took a random sample but did no accord any group a special treatment. The study simply observed the people’s eating habits and classified them.

In one of the studies done in Japan, the researcher wanted to know the levels of radioactive materials in people’s tissues after the bombing of Hiroshima and Nagasaki in 1945. Therefore, he took a random sample of 1000 people in the region and asked them to get checked to determine the levels of radiation in their tissues.

After the study, the researcher concluded that the level of radiation in people’s tissues is still very high and might be associated with different types of diseases being reported in the region. Can you determine what type of study design this is?

The research is an example observational study because it did not have any control. The researcher only observed the levels but did not have any type of control group. Again, there was no special treatment to one of the study populations.

Get Professional Help Whenever You Need It

If you are a researcher, it is very important to be able to define observational study and experiment research before commencing your work. This can help you to determine the different parameters and how to go about the study. As we have demonstrated, observational studies mainly involve gathering the findings from the field without trying to control the variables. Although this study’s results can be contested, it is the most recommended method when using other studies such as experimental design, is unfeasible or unethical.

Experimental studies giving the researcher greater control over the study population by controlling the variables. Although more expensive, it takes a relatively shorter time, and results are less biased.

Now, go ahead and design your study. Always remember that you can seek help from either your lecturer or an expert when designing the study. Once you understand the concept of observational study vs experiment well, research can become so enjoyable and fun.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Comment * Error message

Name * Error message

Email * Error message

Save my name, email, and website in this browser for the next time I comment.

As Putin continues killing civilians, bombing kindergartens, and threatening WWIII, Ukraine fights for the world's peaceful future.

Ukraine Live Updates

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is an Observational Study? | Guide & Examples

What Is an Observational Study? | Guide & Examples

Published on March 31, 2022 by Tegan George . Revised on June 22, 2023.

An observational study is used to answer a research question based purely on what the researcher observes. There is no interference or manipulation of the research subjects, and no control and treatment groups .

These studies are often qualitative in nature and can be used for both exploratory and explanatory research purposes. While quantitative observational studies exist, they are less common.

Observational studies are generally used in hard science, medical, and social science fields. This is often due to ethical or practical concerns that prevent the researcher from conducting a traditional experiment . However, the lack of control and treatment groups means that forming inferences is difficult, and there is a risk of confounding variables and observer bias impacting your analysis.

Table of contents

Types of observation, types of observational studies, observational study example, advantages and disadvantages of observational studies, observational study vs. experiment, other interesting articles, frequently asked questions.

There are many types of observation, and it can be challenging to tell the difference between them. Here are some of the most common types to help you choose the best one for your observational study.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

There are three main types of observational studies: cohort studies, case–control studies, and cross-sectional studies .

Cohort studies

Cohort studies are more longitudinal in nature, as they follow a group of participants over a period of time. Members of the cohort are selected because of a shared characteristic, such as smoking, and they are often observed over a period of years.

Case–control studies

Case–control studies bring together two groups, a case study group and a control group . The case study group has a particular attribute while the control group does not. The two groups are then compared, to see if the case group exhibits a particular characteristic more than the control group.

For example, if you compared smokers (the case study group) with non-smokers (the control group), you could observe whether the smokers had more instances of lung disease than the non-smokers.

Cross-sectional studies

Cross-sectional studies analyze a population of study at a specific point in time.

This often involves narrowing previously collected data to one point in time to test the prevalence of a theory—for example, analyzing how many people were diagnosed with lung disease in March of a given year. It can also be a one-time observation, such as spending one day in the lung disease wing of a hospital.

Observational studies are usually quite straightforward to design and conduct. Sometimes all you need is a notebook and pen! As you design your study, you can follow these steps.

Step 1: Identify your research topic and objectives

The first step is to determine what you’re interested in observing and why. Observational studies are a great fit if you are unable to do an experiment for practical or ethical reasons , or if your research topic hinges on natural behaviors.

Step 2: Choose your observation type and technique

In terms of technique, there are a few things to consider:

• Are you determining what you want to observe beforehand, or going in open-minded?
• Is there another research method that would make sense in tandem with an observational study?
• If yes, make sure you conduct a covert observation.
• If not, think about whether observing from afar or actively participating in your observation is a better fit.
• How can you preempt confounding variables that could impact your analysis?
• You could observe the children playing at the playground in a naturalistic observation.
• You could spend a month at a day care in your town conducting participant observation, immersing yourself in the day-to-day life of the children.
• You could conduct covert observation behind a wall or glass, where the children can’t see you.

Overall, it is crucial to stay organized. Devise a shorthand for your notes, or perhaps design templates that you can fill in. Since these observations occur in real time, you won’t get a second chance with the same data.

Step 3: Set up your observational study

Before conducting your observations, there are a few things to attend to:

• Plan ahead: If you’re interested in day cares, you’ll need to call a few in your area to plan a visit. They may not all allow observation, or consent from parents may be needed, so give yourself enough time to set everything up.
• Determine your note-taking method: Observational studies often rely on note-taking because other methods, like video or audio recording, run the risk of changing participant behavior.
• Get informed consent from your participants (or their parents) if you want to record:  Ultimately, even though it may make your analysis easier, the challenges posed by recording participants often make pen-and-paper a better choice.

Step 4: Conduct your observation

After you’ve chosen a type of observation, decided on your technique, and chosen a time and place, it’s time to conduct your observation.

Here, you can split them into case and control groups. The children with siblings have a characteristic you are interested in (siblings), while the children in the control group do not.

When conducting observational studies, be very careful of confounding or “lurking” variables. In the example above, you observed children as they were dropped off, gauging whether or not they were upset. However, there are a variety of other factors that could be at play here (e.g., illness).

Step 5: Analyze your data

After you finish your observation, immediately record your initial thoughts and impressions, as well as follow-up questions or any issues you perceived during the observation. If you audio- or video-recorded your observations, you can transcribe them.

Your analysis can take an inductive  or deductive approach :

• If you conducted your observations in a more open-ended way, an inductive approach allows your data to determine your themes.
• If you had specific hypotheses prior to conducting your observations, a deductive approach analyzes whether your data confirm those themes or ideas you had previously.

Next, you can conduct your thematic or content analysis . Due to the open-ended nature of observational studies, the best fit is likely thematic analysis .

Step 6: Discuss avenues for future research

Observational studies are generally exploratory in nature, and they often aren’t strong enough to yield standalone conclusions due to their very high susceptibility to observer bias and confounding variables. For this reason, observational studies can only show association, not causation .

If you are excited about the preliminary conclusions you’ve drawn and wish to proceed with your topic, you may need to change to a different research method , such as an experiment.

• Observational studies can provide information about difficult-to-analyze topics in a low-cost, efficient manner.
• They allow you to study subjects that cannot be randomized safely, efficiently, or ethically .
• They are often quite straightforward to conduct, since you just observe participant behavior as it happens or utilize preexisting data.
• They’re often invaluable in informing later, larger-scale clinical trials or experimental designs.

Disadvantages

• Observational studies struggle to stand on their own as a reliable research method. There is a high risk of observer bias and undetected confounding variables or omitted variables .
• They lack conclusive results, typically are not externally valid or generalizable, and can usually only form a basis for further research.
• They cannot make statements about the safety or efficacy of the intervention or treatment they study, only observe reactions to it. Therefore, they offer less satisfying results than other methods.

Prevent plagiarism. Run a free check.

The key difference between observational studies and experiments is that a properly conducted observational study will never attempt to influence responses, while experimental designs by definition have some sort of treatment condition applied to a portion of participants.

However, there may be times when it’s impossible, dangerous, or impractical to influence the behavior of your participants. This can be the case in medical studies, where it is unethical or cruel to withhold potentially life-saving intervention, or in longitudinal analyses where you don’t have the ability to follow your group over the course of their lifetime.

An observational study may be the right fit for your research if random assignment of participants to control and treatment groups is impossible or highly difficult. However, the issues observational studies raise in terms of validity , confounding variables, and conclusiveness can mean that an experiment is more reliable.

If you’re able to randomize your participants safely and your research question is definitely causal in nature, consider using an experiment.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

An observational study is a great choice for you if your research question is based purely on observations. If there are ethical, logistical, or practical concerns that prevent you from conducting a traditional experiment , an observational study may be a good choice. In an observational study, there is no interference or manipulation of the research subjects, as well as no control or treatment groups .

The key difference between observational studies and experimental designs is that a well-done observational study does not influence the responses of participants, while experiments do have some sort of treatment condition applied to at least some participants by random assignment .

A quasi-experiment is a type of research design that attempts to establish a cause-and-effect relationship. The main difference with a true experiment is that the groups are not randomly assigned.

Exploratory research aims to explore the main aspects of an under-researched problem, while explanatory research aims to explain the causes and consequences of a well-defined problem.

Experimental design means planning a set of procedures to investigate a relationship between variables . To design a controlled experiment, you need:

• A testable hypothesis
• At least one independent variable that can be precisely manipulated
• At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

• How you will manipulate the variable(s)
• How you will control for any potential confounding variables
• How many subjects or samples will be included in the study
• How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is an Observational Study? | Guide & Examples. Scribbr. Retrieved September 18, 2024, from https://www.scribbr.com/methodology/observational-study/

Is this article helpful?

Tegan George

Other students also liked, what is a research design | types, guide & examples, guide to experimental design | overview, steps, & examples, naturalistic observation | definition, guide & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

• Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
• Duis aute irure dolor in reprehenderit in voluptate
• Excepteur sint occaecat cupidatat non proident

Keyboard Shortcuts

3.4 - experimental and observational studies.

Now that Jaylen can weigh the different sampling strategies, he might want to consider the type of study he is conduction. As a note, for students interested in research designs, please consult STAT 503 for a much more in-depth discussion. However, for this example, we will simply distinguish between experimental and observational studies.

Now that we know how to collect data, the next step is to determine the type of study. The type of study will determine what type of relationship we can conclude.

There are predominantly two different types of studies: 

Let's say that there is an option to take quizzes throughout this class. In an  observational study , we may find that better students tend to take the quizzes and do better on exams. Consequently, we might conclude that there may be a relationship between quizzes and exam scores.

In an experimental study , we would randomly assign quizzes to specific students to look for improvements. In other words, we would look to see whether taking quizzes causes higher exam scores.

Causation Section  

It is very important to distinguish between observational and experimental studies since one has to be very skeptical about drawing cause and effect conclusions using observational studies. The use of random assignment of treatments (i.e. what distinguishes an experimental study from an observational study) allows one to employ cause and effect conclusions.

Ethics is an important aspect of experimental design to keep in mind. For example, the original relationship between smoking and lung cancer was based on an observational study and not an assignment of smoking behavior.

• Organizations
• Planning & Activities
• Product & Services
• Structure & Systems
• Career & Education
• Entertainment
• Fashion & Beauty
• Political Institutions
• SmartPhones
• Protocols & Formats
• Communication
• Web Applications
• Household Equipments
• Career and Certifications
• Diet & Fitness
• Mathematics & Statistics
• Processed Foods
• Vegetables & Fruits

Difference Between Observational Study and Experiments

• Categorized under Science | Difference Between Observational Study and Experiments

Observational Study vs Experiments

Observational study and experiments are the two major types of study involved in research. The main difference between these two types of study is in the way the observation is done.

In experiments, the researcher will undertake some experiment and not just make observations. In observational study, the researcher simply makes an observation and arrives at a conclusion.

In an experiment, the researcher manipulates every aspect for deriving a conclusion. In observational study, no experiment is conducted. In this type of study, the researcher relies more on data collected. In observational study, the researcher just observes what has happened in the past and what is happening now and draws conclusions based on these data. But in experiments, the researcher observes things through various studies. In other words, it can be said that there is human intervention in experiments whereas there is no human intervention in observational study. Here are examples for observational study and experiments that could clearly define the differences between the two. Hawthorne studies are a good example for experiments. The studies were conducted at the Hawthorne plant of the Western Electric Company. The study was to see the impact of illumination and productivity. First, the productivity was measured, and then the illumination was modified. After this the productivity was again measured which helped the researchers to arrive at a conclusion. The study to determine the relation between smoking and lung cancer is a typical example for observational study. For this the researchers collected data of both smokers and non-smokers. After this, the researchers would make observations with the help of the data and the statistics collected from each group.

1.The main difference between observational study and experiments is in the way the observation is done. 2.In an experiment, the researcher will undertake some experiment and not just make observations. In observational study, the researcher simply makes an observation and arrives at a conclusion. 3.In observational study, no experiment is conducted. In this type of study the researcher relies more on data collected. 4.In an experiment, the researcher observes things through various studies. 5.There is human intervention in experiments whereas there is no human intervention in observational study. 6.Hawthorne studies are a good example for experiments. 7.The study to determine the relation between smoking and lung cancer is a typical example for observational study.

• Recent Posts
• Difference Between CNBC and Fox Business - October 3, 2011
• Difference Between Distilled Water and Boiled Water - September 30, 2011
• Difference Between McDonalds and Burger King - September 30, 2011

Sharing is caring!

Search DifferenceBetween.net :

• Difference Between Study and Experiment
• Difference Between Observation and Conclusion
• Difference Between In Vivo and In Vitro
• Difference Between Observation and Inference
• Difference Between Anonymity and Confidentiality

Cite APA 7 S, P. (2017, June 22). Difference Between Observational Study and Experiments. Difference Between Similar Terms and Objects. http://www.differencebetween.net/science/difference-between-observational-study-and-experiments/. MLA 8 S, Prabhat. "Difference Between Observational Study and Experiments." Difference Between Similar Terms and Objects, 22 June, 2017, http://www.differencebetween.net/science/difference-between-observational-study-and-experiments/.

Leave a Response

Name ( required )

Email ( required )

Please note: comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.

Notify me of followup comments via e-mail

Written by : Prabhat S. and updated on 2017, June 22 Articles on DifferenceBetween.net are general information, and are not intended to substitute for professional advice. The information is "AS IS", "WITH ALL FAULTS". User assumes all risk of use, damage, or injury. You agree that we have no liability for any damages.

Advertisments

More in 'science'.

• Difference Between Rumination and Regurgitation
• Difference Between Pyelectasis and Hydronephrosis 
• Difference Between Cellulitis and Erysipelas
• Difference Between Suicide and Euthanasia
• Difference Between Vitamin D and Vitamin D3

Top Difference Betweens

Get new comparisons in your inbox:, most emailed comparisons, editor's picks.

• Difference Between MAC and IP Address
• Difference Between Platinum and White Gold
• Difference Between Civil and Criminal Law
• Difference Between GRE and GMAT
• Difference Between Immigrants and Refugees
• Difference Between DNS and DHCP
• Difference Between Computer Engineering and Computer Science
• Difference Between Men and Women
• Difference Between Book value and Market value
• Difference Between Red and White wine
• Difference Between Depreciation and Amortization
• Difference Between Bank and Credit Union
• Difference Between White Eggs and Brown Eggs

Section 1.2: Observational Studies versus Designed Experiments

• 1.1 Introduction to the Practice of Statistics
• 1.2 Observational Studies versus Designed Experiments
• 1.3 Random Sampling
• 1.4 Bias in Sampling
• 1.5 The Design of Experiments

By the end of this lesson, you will be able to...

• distinguish between an observational study and a designed experiment
• identify possible lurking variables
• explain the various types of observational studies

For a quick overview of this section, watch this short video summary:

To begin, we're going to discuss some of the ways to collect data. In general, there are a few standards:

• existing sources
• survey sampling
• designed experiments

Most of us associate the word census with the U.S. Census, but it actually has a broader definition. Here's typical definition:

A census is a list of all individuals in a population along with certain characteristics of each individual.

The nice part about a census is that it gives us all the information we want. Of course, it's usually impossible to get - imagine trying to interview every single ECC student . That'd be over 10,000 interviews!

So if we can't get a census, what do we do? A great source of data is other studies that have already been completed. If you're trying to answer a particular question, look to see if someone else has already collected data about that population. The moral of the story is this: Don't collect data that have already been collected!

Observational Studies versus Designed Experiments

Now to one of the main objectives for this section. Two other very common sources of data are observational studies and designed experiments . We're going to take some time here to describe them and distinguish between them - you'll be expected to be able to do the same in homework and on your first exam.

The easiest examples of observational studies are surveys. No attempt is made to influence anything - just ask questions and record the responses. By definition,

An observational study measures the characteristics of a population by studying individuals in a sample, but does not attempt to manipulate or influence the variables of interest.

For a good example, try visiting the Pew Research Center . Just click on any article and you'll see an example of an observational study. They just sample a particular group and ask them questions.

In contrast, designed experiments explicitly do attempt to influence results. They try to determine what affect a particular treatment has on an outcome.

A designed experiment applies a treatment to individuals (referred to as experimental units or subjects ) and attempts to isolate the effects of the treatment on a response variable .

For a nice example of a designed experiment, check out this article from National Public Radio about the effect of exercise on fitness.

So let's look at a couple examples.

Visit this link from Science Daily , from July 8th, 2008. It talks about the relationship between Post-Traumatic Stress Disorder (PTSD) and heart disease. After reading the article carefully, try to decide whether it was an observational study or a designed experiment

What was it?

This was a tricky one. It was actually an observational study . The key is that the researchers didn't force the veterans to have PTSD, they simply observed the rate of heart disease for those soldiers who have PTSD and the rate for those who do not.

Visit this link from the Gallup Organization , from June 17th, 2008. It looks at what Americans' top concerns were at that point. Read carefully and think of the how the data were collected. Do you think this was an observational study or a designed experiment? Why?

Think carefully about which you think it was, and just as important - why? When you're ready, click the link below.

If you were thinking that this was an observational study , you were right!The key here is that the individuals sampled were just asked what was important to them. The study didn't try to impose certain conditions on people for a set amount of time and see if those conditions affected their responses.

This last example is regarding the "low-carb" Atkins diet, and how it compares with other diets. Read through this summary of a report in the New England Journal of Medicine and see if you can figure out whether it's an observational study or a designed experiment.

As expected, this was a designed experiment , but do you know why? The key here is they forced individuals to maintain a certain diet, and then compared the participants' health at the end.

Probably the biggest difference between observational studies and designed experiments is the issue of association versus causation . Since observational studies don't control any variables, the results can only be associations . Because variables are controlled in a designed experiment, we can have conclusions of causation .

Look back over the three examples linked above and see if all three reported their results correctly. You'll often find articles in newspapers or online claiming one variable caused a certain response in another, when really all they had was an association from doing an observational study.

The discussion of the differences between observational studies and designed experiments may bring up an interesting question - why are we worried so much about the difference?

We already mentioned the key at the end of the previous page, but it bears repeating here:

Observational studies only allow us to claim association ,not causation .

The primary reason behind this is something called a lurking variable (sometimes also termed a confounding factor , among other similar terms).

A lurking variable is a variable that affects both of the variables of interest, but is either not known or is not acknowledged.

Consider the following example, from The Washington Post:

Coffee may have health benefits and may not pose health risks for many people

By Carolyn Butler Tuesday, December 22, 2009

Of all the relationships in my life, by far the most on-again, off-again has been with coffee: From that initial, tentative dalliance in college to a serious commitment during my first real reporting job to breaking up altogether when I got pregnant, only to fail miserably at quitting my daily latte the second time I was expecting. More recently the relationship has turned into full-blown obsession and, ironically, I often fall asleep at night dreaming of the delicious, satisfying cup of joe that awaits, come morning.

[...] Rest assured: Not only has current research shown that moderate coffee consumption isn't likely to hurt you, it may actually have significant health benefits. "Coffee is generally associated with a less health-conscious lifestyle -- people who don't sleep much, drink coffee, smoke, drink alcohol," explains Rob van Dam, an assistant professor in the departments of nutrition and epidemiology at the Harvard School of Public Health. He points out that early studies failed to account for such issues and thus found a link between drinking coffee and such conditions as heart disease and cancer, a link that has contributed to java's lingering bad rep. "But as more studies have been conducted-- larger and better studies that controlled for healthy lifestyle issues --the totality of efforts suggests that coffee is a good beverage choice."

Source: Washington Post

What is this article telling us? If you look at the parts in bold, you can see that Professor van Dam is describing a lurking variable: lifestyle. In past studies, this variable wasn't accounted for. Researchers in the past saw the relationship between coffee and heart disease, and came to the conclusion that the coffee was causing the heart disease.

But since those were only observational studies, the researchers could only claim an association . In that example, the lifestyle choices of individuals was affecting both their coffee use and other risks leading to heart disease. So "lifestyle" would be an example of a lurking variable in that example.

For more on lurking variables, check out this link from The Math Forum and this one from The Psychology Wiki . Both give further examples and illustrations.

With all the problems of lurking variables, there are many good reasons to do an observational study. For one, a designed experiment may be impractical or even unethical (imagine a designed experiment regarding the risks of smoking). Observational studies also tend to cost much less than designed experiments, and it's often possible to obtain a much larger data set than you would with a designed experiment. Still, it's always important to remember the difference in what we can claim as a result of observational studies versus designed experiments.

Types of Observational Studies

There are three major types of observational studies, and they're listed in your text: cross-sectional studies, case-control studies, and cohort studies.

Cross-sectional Studies

This first type of observational study involves collecting data about individuals at a certain point in time. A researcher concerned about the effect of working with asbestos might compare the cancer rate of those who work with asbestos versus those who do not.

Cross-sectional studies are cheap and easy to do, but they don't give very strong results. In our quick example, we can't be sure that those working with asbestos who don't report cancer won't eventually develop it. This type of study only gives a bit of the picture, so it is rarely used by itself. Researchers tend to use a cross-sectional study to first determine if their might be a link, and then later do another study (like one of the following) to further investigate.

Case-control Studies

Case-control studies are frequently used in the medical community to compare individuals with a particular characteristic (this group is the case )with individuals who do not have that characteristic (this group is the control ). Researchers attempt to select homogeneous groups, so that on average, all other characteristics of the individuals will be similar, with only the characteristic in question differing.

One of the most famous examples of this type of study is the early research on the link between smoking and lung cancer in the United Kingdom by Richard Doll and A. Bradford Hill. In the 1950's, almost 80% of adults in the UK were smokers, and the connection between smoking and lung cancer had not yet been established. Doll and Hill interviewed about 700 lung cancer patients to try to determine a possible cause.

This type of study is retrospective ,because it asks the individuals to look back and describe their habits(regarding smoking, in this case). There are clear weaknesses in a study like this, because it expects individuals to not only have an accurate memory, but also to respond honestly. (Think about a study concerning drug use and cognitive impairment.) Not only that, we discussed previously that such a study may prove association , but it cannot prove causation .

Cohort Studies

A cohort describes a group of individuals, and so a cohort study is one in which a group of individuals is selected to participate in a study. The group is then observed over a period of time to determine if particular characteristics affect a response variable.

Based on their earlier research, Doll and Hill began one of the largest cohort studies in 1951. The study was again regarding the link between smoking and lung cancer. The study began with 34,439 male British doctors, and followed them for over 50 years. Doll and Hill first reported findings in 1954 in the British Medical Journal , and then continued to report their findings periodically afterward. Their last report was in 2004,again published in the British Medical Journal . This last report reflected on 50 years of observational data from the cohort.

This last type of study is called prospective , because it begins with the group and then collects data over time. Cohort studies are definitely the most powerful of the observational studies,particularly with the quantity and quality of data in a study like the previous one.

Let's look at some examples.

A recent article in the BBC News Health section described a study concerning dementia and "mid-life ills". According to the article, researches followed more than 11,000 people over a period of 12-14 years. They found that smoking, diabetes, and high blood pressure were all factors in the onset of dementia.

What type of observational study was this? Cross-sectional, case-control,or cohort?

Because the researchers tracked the 11,000 participants, this is a cohort study .

In 1993, the National Institute of Environmental Health Sciences funded a study in Iowa regarding the possible relationship between radon levels and the incidence of cancer. The study gathered information from 413 participants who had developed lung cancer and compared those results with 614 participants who did not have lung cancer.

What type of study was this?

This study was retrospective - gathering information about the group of interest (those with cancer) and comparing them with a control group(those without cancer). This is an example of a case-control study .

Thought his may seem similar to a cross-sectional study, it differs in that the individuals are "matched" (with cancer vs. without cancer)and the individuals are expected to look back in time and describe their time spent in the home to determine their radon exposure.

In 2004, researchers published an article in the New England Journal of Medicine regarding the relationship between the mental health of soldiers exposed to combat stress. The study collected information from soldiers in four combat infantry units either before their deployment to Iraq or three to four months after their return from combat duty.

Since this was simply a survey given over a short period of time to try to examine the effect of combat duty, this was a cross-sectional study. Unlike the previous example, it did not ask the participants to delve into their history, nor did it explicitly "match" soldiers with a particular characteristic.

<< previous section | next section >>

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• v.1(3); 2004 Jul

Observational Versus Experimental Studies: What’s the Evidence for a Hierarchy?

John concato.

Department of Internal Medicine, Yale University School of Medicine, New Haven, Connecticut 06510, and the Clinical Epidemiology Research Center, West Haven Veterans Affairs Medical Center, West Haven, Connecticut 06516

Summary: The tenets of evidence-based medicine include an emphasis on hierarchies of research design (i.e., study architecture). Often, a single randomized, controlled trial is considered to provide “truth,” whereas results from any observational study are viewed with suspicion. This paper describes information that contradicts and discourages such a rigid approach to evaluating the quality of research design. Unless a more balanced strategy evolves, new claims of methodological authority may be just as problematic as the traditional claims of medical authority that have been criticized by proponents of evidence-based medicine.

INTRODUCTION

Evidence-based medicine classifies studies into grades of evidence based on research architecture. 1 , 2 This hierarchical approach to study design has been promoted widely in individual reports, meta-analyses, consensus statements, and educational materials for clinicians. For example, a prominent publication 3 reserved the highest grade for “at least one properly randomized, controlled trial,” and the lowest grade for descriptive studies (e.g., case series) and expert opinion. Observational studies, including cohort and case-control, fall into intermediate levels (Table ​ (Table1). 1 ). Although the quality of studies is sometimes evaluated within each grade, each category is considered methodologically superior to level(s) below it.

“Grades of Evidence” Rating the Purported Quality of Study Design 3

The ascendancy of randomized, controlled trials (experimental studies) to become the “gold standard” strategy for assessing the effectiveness of therapeutic agents 4 – 6 was based in part on a landmark paper 7 comparing published articles that used randomized and historical control trial designs. The corresponding results found that the agent being tested was considered effective in 44 of 56 (79%) historical controlled trials, but only 10 of 50 (20%) randomized, controlled trials. The authors concluded “biases in patient selection may irretrievably weight the outcome of historical controlled trials in favor of new therapies.” 7

Although the cited article 7 compared randomized, controlled trials to historical controlled trials only, contemporary criticisms of observational studies also include cohort studies with concurrent (nonhistorical) selection of control subjects as well as case-control designs. 8 A possibility exists, however, that data based on “weaker” forms of observational studies can be used mistakenly to criticize all observational research. The premise of this paper is that evidence-based medicine has contributed to the development of a rigid hierarchy of research design that underestimates the limitations of randomized, controlled trials, and overstates the limitations of observational studies.

WHY USE A HIERARCHY OF RESEARCH DESIGN?

A hierarchy of types of research design would be desirable for providing a “checklist” to evaluate clinical studies, but the complexity of medical research suggests that such approaches are overly simplistic. Although randomization protects against certain types of bias that can threaten the validity of a study (i.e., obtaining the correct answer to the question posed, among the study participants involved), a corresponding randomized, controlled trials protocol may restrict the sample of patients selected, the intervention delivered, or the outcome(s) measured, impairing the so-called generalizability of a study (i.e., the extent to which it applies to patients in the “real world”). For example, a randomized, controlled trial may exclude older patients, it may administer therapy in a manner that is difficult to replicate in actual practice, or it may use short-term or surrogate endpoints. In addition, numerous problems can occur when randomized, controlled trials are conducted improperly. Conversely, if properly-conducted observational studies can overcome threats to validity (using strategies discussed later in this paper), and if such studies incorporate more relevant clinical features, then corresponding results would likely be very generalizable to practicing clinicians. Yet, the conventional wisdom suggests that observational studies consistently provide biased results compared with randomized, controlled trials, regardless of the type of observational study or how well it was conducted. The remainder of this paper will focus on these issues.

EVIDENCE AGAINST A RIGID HIERARCHY

A recent study recognized that systematic reviews and meta-analyses offered an opportunity to test the implicit assumptions of grades (or levels) of evidence and similar hierarchies of research design. 9 We identified particular exposure-outcome associations that were studied with both randomized, controlled trials as well as cohort or case-control studies. The major distinctions of our approach (compared with prior research), however, were that we evaluated observational studies that used concurrent (not historical) control subjects, and we focused on summary results rather than individual study findings. The variation in point estimates of exposure-outcome associations provided data to confirm or refute the assumptions regarding observational studies, as well as the strengths and limitations of a “design hierarchy.”

Our methods involved identifying meta-analyses published in five major journals ( Annals of Internal Medicine , British Medical Journal , Journal of the American Medical Association , Lancet , and New England Journal of Medicine ) from 1991 to 1995, using searches of MEDLINE, with the terms “meta-analysis, ” “meta-analyses,” “pooling,” “combining,” “overview,” and “aggregation.” Additional references were found in Current Contents , supplemented by manual searches of the relevant journals. The meta-analyses identified via this process were then classified by consensus as including clinical trials only, observational studies only, or both. Clinical trials were defined as studies that used randomized interventions; observational studies included cohort or case-control designs. Meta-analyses were excluded if they were based on cohort studies with historical control subjects, or clinical trials with nonrandom assignment of interventions, or if they did not report results in the format of a point estimate (e.g., relative risk, odds ratio) and confidence intervals. The remaining meta-analyses were then reviewed, and the original studies cited in the bibliographies were retrieved.

The search strategy yielded 102 citations for meta-analyses, mainly involving (as expected) randomized, controlled trials only. Data for five clinical topics 10 – 15 met our eligibility criteria and provided sufficient data for analysis, involving 99 original articles and 1,871,681 total study subjects. The summary (pooled) point estimates are presented in Table ​ Table2, 2 , and the ranges of the point estimates are displayed in Figure 1 . For example, the relationship between treatment of hypertension and the first occurrence of stroke (i.e., primary prevention) was examined in meta-analyses of 14 randomized, controlled trials 15 and seven cohort studies. 10 The pooled results from randomized, controlled trials ( N = 36,894) found a point estimate of 0.58 (95% confidence interval 0.50–0.67); the pooled results from observational studies ( N = 405,511) found an adjusted point estimate of 0.62 (95% confidence interval 0.60–0.65). Results for other associations (Table ​ (Table2) 2 ) were also similar, based on data from randomized, controlled trials and observational studies. In another example, the effectiveness of bacillus Calmette-Guerin (BCG) vaccine against tuberculosis was examined in a meta-analysis 11 that included 13 randomized trials ( N = 359,922 subjects) with a pooled relative risk of 0.49 (95% confidence interval 0.34–0.70), and 10 case-control studies ( N = 6511 subjects) with a pooled odds ratio of 0.50 (95% confidence interval 0.39–0.65).

Range of relative risks or odds ratios, based on the following types of research design: bacillus Calmette-Guerin vaccine and tuberculosis (13 randomized, controlled trials and 10 case-control studies), screening mammography and breast cancer mortality (eight randomized, controlled trials and four case-control studies), treatment of hyperlipidemia and traumatic death among men (four randomized, controlled trials and 14 cohort studies), treatment of hypertension and stroke among men (11 randomized, controlled trials and seven cohort studies), treatment of hypertension and coronary heart disease among men (13 randomized, controlled trials and nine cohort studies). Filled circles, randomized, controlled trials; open circles, observational studies. (Reproduced with permission.)

Total Number of Subjects and Summary Estimates for the Impact of Five Interventions (“Clinical Topics”) Based on Type of Research Design

CHD = coronary heart disease; CI = confidence interval; RCT = randomized, controlled trial.

The results of our investigation contradict the idea of a “fixed” hierarchy of study design in clinical research. Importantly, another publication 16 addressing the same general question found “little evidence that estimates of treatment effects in observational studies reported after 1984 are either consistently larger than or qualitatively different from those obtained in randomized, controlled trials.” In addition, an evaluation 17 of the literature on screening mammography found similar results to ours on that particular topic. Thus, contrary to prevailing beliefs, average results from well-designed observational (cohort and case-control) studies did not systematically overestimate the magnitude of exposure-outcome associations reported in randomized, controlled trials. Rather, the summary results from randomized, controlled trials and observational studies were remarkably similar for each clinical question addressed.

Another finding, also contrary to current perceptions, was that observational studies individually demonstrated less variability (heterogeneity) in point estimates compared to the variability in point estimates observed in randomized, controlled trials on the same topic ( FIG. 1 ). Indeed, only among randomized, controlled trials did individual studies report results that were opposite to the direction of the pooled point estimate, representing a “paradoxical” finding (e.g., treatment of hypertension was associated with higher rates of coronary heart disease in several clinical trials).

One possible explanation for the finding that observational studies were less prone to heterogeneity in results (compared with randomized, controlled trials) is that each observational study is more likely to include a broad representation of the at-risk population. In addition, less opportunity exists for differences in the management of subjects “across” observational studies. For example, although general agreement exists that physicians do not use therapeutic agents in a uniform way, an observational study would generally include patients with a wider spectrum of severity (regarding the disease of interest), more comorbid ailments, and treatments that were tailored for each individual patient. In contrast, randomized, controlled trials may have distinct groups of patients based on specific inclusion and exclusion criteria, and the experimental protocol for therapy may not be representative of clinical practice. Therefore, randomized, controlled trials often have limited generalizability.

ADDITIONAL EVIDENCE AGAINST A RIGID HIERARCHY

At the time of our previous study, 9 investigations had already shown that observational cohort studies often produce results similar to those of randomized, controlled trials, when using similar criteria to assemble study participants and suitable methodological precautions. For example, an analysis of 18 randomized and nonrandomized studies in health services research found that treatment effects may differ based on research design but that “one method does not give a consistently greater effect than the other.” 18 In that assessment, results were found to be most similar when exclusion criteria across studies were comparable, and when prognostic factors were accounted for in observational studies. In addition, a specific strategy used to strengthen observational studies (called a “restrictive cohort” design 19 ) adapts principles of randomized, controlled trials to 1) identify a zero-time for determining patient eligibility and baseline prognostic risk, 2) use inclusion and exclusion criteria similar to clinical trials, 3) adjust for differences in baseline susceptibility for the outcome, and 4) use similar statistical strategies (e.g., intention-to-treat) as in randomized, controlled trials. When these procedures were used in a cohort study 19 evaluating the benefit of beta blockers after recovery from myocardial infarction, the restricted cohort produced results consistent with corresponding findings from the Beta-Blocker Heart Attack Trial. 20

A second line of evidence supporting our contention that research design should not be considered a rigid hierarchy is also available in the literature of other scientific disciplines that carry out subject-based intervention trials. Examples include a comprehensive review of psychological, educational, and behavioral treatment research 21 ; the findings from this review did not support a contention that observational studies overestimate effects relative to randomized, controlled trials.

Further evidence against a rigid hierarchy is based on results from the trials themselves. For example, a review of more than 200 randomized, controlled trials found numerous individual trials that were supportive, equivocal, or nonsupportive for each of 36 clinical topics. 22 Several publications have discussed various aspects of randomized, controlled trials in neurology. 23 – 28 Recent publications indicate that randomized, controlled trials continue to generate conflicting results, e.g., addressing the question of whether therapy with monoclonal antibodies improve outcomes among patients with septic shock. 29 , 30 In addition, results of “large, simple” randomized, controlled trials contribute to the evidence of contradictory results from randomized, controlled trials; one report found that results of meta-analyses based on randomized, controlled trials were often discordant with findings from large, simple trials on the same clinical topic. 31 Regardless of the reasons that individual randomized, controlled trials produce heterogeneous results, the available evidence indicates that a single randomized trial (or only one observational study) cannot be expected to provide a gold standard result for all clinical situations.

EXAMPLES FROM THE LITERATURE AND IMPLICATIONS FOR CLINICAL CARE

Vitamin e and coronary heart disease.

The Heart Outcomes Prevention Evaluation (HOPE) study, 32 a randomized, controlled trial, was cited as helping to “restrain earlier observational claims that vitamin E lowers the risk of cardiovascular disease.” 33 A review of this topic illustrates the methodological issues involved. Several observational studies 34 – 36 found a “positive” association; in contrast, the HOPE study suggested that vitamin E has no effect on cardiovascular outcomes. Yet, a thorough examination of randomized, controlled trials on this topic provides a more complete assessment. Although two randomized, controlled trials 37 , 38 also found no effect on mortality, two other randomized, controlled trials 39 , 40 found decreased mortality associated with vitamin E. Thus, data from clinical trials are themselves contradictory, and selecting one randomized, controlled trial as a gold standard to criticize observational studies is overly simplistic.

This clinical topic was used to support the statement that “…society expects us to evaluate new healthcare interventions by the most scientifically sound and rigorous methods available. Although observational studies often are cheaper, quicker, and less difficult to carry out, we should not lose sight of one simple fact: ignorance calls for careful experimentation. This means high-quality randomized, controlled trials, not observations that reflect personal choices and beliefs.” 33 An alternative, more rigorous, and less dogmatic approach would be to compare published studies based on components of their research design, whether randomized or observational (Table ​ (Table3), 3 ), and not make a priori judgments regarding a single randomized, controlled trial constituting a gold standard.

Foci for Comparison of Observational and Experimental Study Designs: Example of Vitamin E and Coronary Disease

Hormone replacement therapy and coronary heart disease

Another example of this controversy involves hormone replacement therapy disease for postmenopausal women. In summary, observational studies (such as the Nurses Health Study 41 ) suggested a protective benefit of hormones; whereas randomized, controlled trials (including the Women’s Health Initiative 42 and the Heart and Estrogen/Progestin Replacement Study 43 ) pointed to no benefit, or even harm. Rather than assume the randomized, controlled trials inherently reveal “truth,” potential explorations for the discordant findings could be explored. First, it should be noted that results of randomized, controlled trials and observational studies are remarkably consistent for most outcomes in studies of hormone replacement therapy, including stroke, breast cancer, colorectal cancer, hip fracture, and pulmonary embolism. The outcome of coronary artery disease has received most attention, and has been described as an anomaly. 44

An assessment of this topic described plausible methodological and biological explanations for the differences in findings. 44 For example, available data indicate that women with higher socioeconomic status are more likely to be hormone replacement therapy users and less likely to have coronary artery disease, suggesting that the observational studies were vulnerable to “healthy user bias” (or “confounding”) in this context. (Confounding, as a general term, occurs when a third variable, socioeconomic status in this situation, is related to both the exposure [hormone therapy] and outcome [coronary artery disease] variables for the association of interest. The exposure variable [hormone therapy] would then be described as a “marker” for the confounding variable, rather than actually causing the outcome.) In addition, the randomized, controlled trials themselves have been criticized for having bias. 45

Another issue involves incomplete capture of early clinical events. 44 Observational studies typically enroll participants who have been taking hormone replacement therapy for some time, whereas randomized clinical trials initiate therapy in nonusers. Accordingly, clinical events that occur soon after initiating the medication would be captured by randomized, controlled trials, but typical observational studies assess what is likely to happen when patients remain on therapy for an extended period of time (patients initiating therapy recently would account for a very small proportion of the overall population). Other explanations for discordant results involve differences in protocols among observational studies and randomized, controlled trials. For example, daily combinations of estrogen and progestin were administered in Women’s Health Initiative 42 and Heart and Estrogen/Progestin Replacement Study, 43 compared with estrogen alone or combined regimens for 10–14 days per month in observational studies such as the Nurses Health Study. 41

These differences are not “fatal flaws” of observational studies, unless a rigid opinion is adopted that designates randomized, controlled trials as infallible. Most of the issues raised involve either methodological differences without a definite “winner” (e.g., examining early vs late clinical events), or true biological differences (e.g., in patients or protocols). Regarding the issue of confounding (e.g., healthy user bias, as described previously), methods are available 19 to measure and adjust for such variables.

A MORE BALANCED VIEW OF OBSERVATIONAL AND EXPERIMENTAL EVIDENCE

Given that randomized, controlled trials have not and often cannot be done for many clinical interventions, much of the clinical care provided in neurology (and all other specialties in medicine) would necessarily be considered unsubstantiated, if observational studies are discounted from consideration. The available evidence suggests, however, that observational studies can be conducted with sufficient rigor to replicate the results of randomized, controlled trials. The key issue is designing appropriate observational studies, usually with suitable (observational) cohort or case-control architecture; a methodological task for investigators to complete and reviewers to evaluate.

Despite the consistency of our results 9 (involving five clinical topics and 99 separate studies), as well as confirmatory evidence available in the literature, 16 – 18 we believe that the role of observational studies may vary in different situations. For example, different exposures (e.g., surgical operations and other invasive therapies) may be more prone to selection bias in observational investigations than the drugs and noninvasive tests examined in our report, 9 and “softer” outcomes (e.g., functional status) may be assessed more readily in randomized, controlled trials. In addition, we emphasized the potential risk associated with poorly done observational studies; for example, to promote ineffective “alternative” therapies. 46

Finally, a point of emphasis involves the general belief that randomization is necessary to balance known and (especially) unknown potential factors that can cause biased estimates of treatment effects through confounding. Given that unknown factors, by definition, would not be recognized by clinicians, a bias in assigning treatment would not occur according to those factors. Although such factors could be associated with outcome, they would not be associated with exposure, and therefore would not be confounding variables and would not affect the validity of results.

Randomized, controlled trials will (and should) remain a prominent tool in clinical research, but the results of a single randomized, controlled trial, or only one observational study, should be interpreted cautiously. If a randomized, controlled trial is later determined to be “wrong” in its conclusions, evidence from both other trials and well designed cohort or case-control studies can and should be used to establish the “right” answers.

The issues raised in this paper are not intended to diminish the important role that randomized, controlled trials play in clinical medicine (e.g., for evaluating interventions or for satisfying regulatory criteria). Yet, the popular belief that randomized, controlled trials inherently produce gold standard results, and that all observational studies are inferior, does a disservice to patient care, clinical investigation, and education of health care professionals. We should recognize the potential problem we face, that “the justification for why studies are included or excluded from the evidence base can rest on competing claims of methodologic authority that look little different from the traditional claims of medical authority that proponents of evidence-based medicine have criticized…interpretive decisions by old pre-evidence-based medicine experts may be replaced by interpretive decisions from a new group of experts with evidence-based medicine credentials…” 47 A more balanced and scientifically justified approach is to evaluate the strengths and limitations of well done experimental and observational studies, recognizing the attributes of each type of design.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• What Is an Observational Study? | Guide & Examples

What Is an Observational Study? | Guide & Examples

Published on 5 April 2022 by Tegan George . Revised on 20 March 2023.

An observational study is used to answer a research question based purely on what the researcher observes. There is no interference or manipulation of the research subjects, and no control and treatment groups .

These studies are often qualitative in nature and can be used for both exploratory and explanatory research purposes. While quantitative observational studies exist, they are less common.

Observational studies are generally used in hard science, medical, and social science fields. This is often due to ethical or practical concerns that prevent the researcher from conducting a traditional experiment . However, the lack of control and treatment groups means that forming inferences is difficult, and there is a risk of confounding variables impacting your analysis.

Table of contents

Types of observation, types of observational studies, observational study example, advantages and disadvantages of observational studies, observational study vs experiment, frequently asked questions.

There are many types of observation, and it can be challenging to tell the difference between them. Here are some of the most common types to help you choose the best one for your observational study.

Prevent plagiarism, run a free check.

There are three main types of observational studies: cohort studies, case–control studies, and cross-sectional studies.

Cohort studies

Cohort studies are more longitudinal in nature, as they follow a group of participants over a period of time. Members of the cohort are selected because of a shared characteristic, such as smoking, and they are often observed over a period of years.

Case–control studies

Case–control studies bring together two groups, a case study group and a control group . The case study group has a particular attribute while the control group does not. The two groups are then compared, to see if the case group exhibits a particular characteristic more than the control group.

For example, if you compared smokers (the case study group) with non-smokers (the control group), you could observe whether the smokers had more instances of lung disease than the non-smokers.

Cross-sectional studies

Cross-sectional studies analyse a population of study at a specific point in time.

This often involves narrowing previously collected data to one point in time to test the prevalence of a theory—for example, analysing how many people were diagnosed with lung disease in March of a given year. It can also be a one-time observation, such as spending one day in the lung disease wing of a hospital.

Observational studies are usually quite straightforward to design and conduct. Sometimes all you need is a notebook and pen! As you design your study, you can follow these steps.

Step 1: Identify your research topic and objectives

The first step is to determine what you’re interested in observing and why. Observational studies are a great fit if you are unable to do an experiment for ethical or practical reasons, or if your research topic hinges on natural behaviors.

Step 2: Choose your observation type and technique

In terms of technique, there are a few things to consider:

• Are you determining what you want to observe beforehand, or going in open-minded?
• Is there another research method that would make sense in tandem with an observational study?
• If yes, make sure you conduct a covert observation.
• If not, think about whether observing from afar or actively participating in your observation is a better fit.
• How can you preempt confounding variables that could impact your analysis?
• You could observe the children playing at the playground in a naturalistic observation.
• You could spend a month at a day care in your town conducting participant observation, immersing yourself in the day-to-day life of the children.
• You could conduct covert observation behind a wall or glass, where the children can’t see you.

Overall, it is crucial to stay organised. Devise a shorthand for your notes, or perhaps design templates that you can fill in. Since these observations occur in real time, you won’t get a second chance with the same data.

Step 3: Set up your observational study

Before conducting your observations, there are a few things to attend to:

• Plan ahead: If you’re interested in day cares, you’ll need to call a few in your area to plan a visit. They may not all allow observation, or consent from parents may be needed, so give yourself enough time to set everything up.
• Determine your note-taking method: Observational studies often rely on note-taking because other methods, like video or audio recording, run the risk of changing participant behavior.
• Get informed consent from your participants (or their parents) if you want to record:  Ultimately, even though it may make your analysis easier, the challenges posed by recording participants often make pen-and-paper a better choice.

Step 4: Conduct your observation

After you’ve chosen a type of observation, decided on your technique, and chosen a time and place, it’s time to conduct your observation.

Here, you can split them into case and control groups. The children with siblings have a characteristic you are interested in (siblings), while the children in the control group do not.

When conducting observational studies, be very careful of confounding or ‘lurking’ variables. In the example above, you observed children as they were dropped off, gauging whether or not they were upset. However, there are a variety of other factors that could be at play here (e.g., illness).

Step 5: Analyse your data

After you finish your observation, immediately record your initial thoughts and impressions, as well as follow-up questions or any issues you perceived during the observation. If you audio- or video-recorded your observations, you can transcribe them.

Your analysis can take an inductive or deductive approach :

• If you conducted your observations in a more open-ended way, an inductive approach allows your data to determine your themes.
• If you had specific hypotheses prior to conducting your observations, a deductive approach analyses whether your data confirm those themes or ideas you had previously.

Next, you can conduct your thematic or content analysis . Due to the open-ended nature of observational studies, the best fit is likely thematic analysis.

Step 6: Discuss avenues for future research

Observational studies are generally exploratory in nature, and they often aren’t strong enough to yield standalone conclusions due to their very high susceptibility to observer bias and confounding variables. For this reason, observational studies can only show association, not causation .

If you are excited about the preliminary conclusions you’ve drawn and wish to proceed with your topic, you may need to change to a different research method , such as an experiment.

• Observational studies can provide information about difficult-to-analyse topics in a low-cost, efficient manner.
• They allow you to study subjects that cannot be randomised safely, efficiently, or ethically .
• They are often quite straightforward to conduct, since you just observe participant behavior as it happens or utilise preexisting data.
• They’re often invaluable in informing later, larger-scale clinical trials or experiments.

Disadvantages

• Observational studies struggle to stand on their own as a reliable research method. There is a high risk of observer bias and undetected confounding variables.
• They lack conclusive results, typically are not externally valid or generalisable, and can usually only form a basis for further research.
• They cannot make statements about the safety or efficacy of the intervention or treatment they study, only observe reactions to it. Therefore, they offer less satisfying results than other methods.

The key difference between observational studies and experiments is that a properly conducted observational study will never attempt to influence responses, while experimental designs by definition have some sort of treatment condition applied to a portion of participants.

However, there may be times when it’s impossible, dangerous, or impractical to influence the behavior of your participants. This can be the case in medical studies, where it is unethical or cruel to withhold potentially life-saving intervention, or in longitudinal analyses where you don’t have the ability to follow your group over the course of their lifetime.

An observational study may be the right fit for your research if random assignment of participants to control and treatment groups is impossible or highly difficult. However, the issues observational studies raise in terms of validity , confounding variables, and conclusiveness can mean that an experiment is more reliable.

If you’re able to randomise your participants safely and your research question is definitely causal in nature, consider using an experiment.

An observational study could be a good fit for your research if your research question is based on things you observe. If you have ethical, logistical, or practical concerns that make an experimental design challenging, consider an observational study. Remember that in an observational study, it is critical that there be no interference or manipulation of the research subjects. Since it’s not an experiment, there are no control or treatment groups either.

The key difference between observational studies and experiments is that, done correctly, an observational study will never influence the responses or behaviours of participants. Experimental designs will have a treatment condition applied to at least a portion of participants.

Exploratory research explores the main aspects of a new or barely researched question.

Explanatory research explains the causes and effects of an already widely researched question.

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts, and meanings, use qualitative methods .
• If you want to analyse a large amount of readily available data, use secondary data. If you want data specific to your purposes with control over how they are generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

George, T. (2023, March 20). What Is an Observational Study? | Guide & Examples. Scribbr. Retrieved 18 September 2024, from https://www.scribbr.co.uk/research-methods/observational-study/

Is this article helpful?

Difference Between Survey and Experiment

While surveys collected data, provided by the informants, experiments test various premises by trial and error method. This article attempts to shed light on the difference between survey and experiment, have a look.

Content: Survey Vs Experiment

Comparison chart.

Definition of Survey

By the term survey, we mean a method of securing information relating to the variable under study from all or a specified number of respondents of the universe. It may be a sample survey or a census survey. This method relies on the questioning of the informants on a specific subject. Survey follows structured form of data collection, in which a formal questionnaire is prepared, and the questions are asked in a predefined order.

Informants are asked questions concerning their behaviour, attitude, motivation, demographic, lifestyle characteristics, etc. through observation, direct communication with them over telephone/mail or personal interview. Questions are asked verbally to the respondents, i.e. in writing or by way of computer. The answer of the respondents is obtained in the same form.

Definition of Experiment

The term experiment means a systematic and logical scientific procedure in which one or more independent variables under test are manipulated, and any change on one or more dependent variable is measured while controlling for the effect of the extraneous variable. Here extraneous variable is an independent variable which is not associated with the objective of study but may affect the response of test units.

In an experiment, the investigator attempts to observe the outcome of the experiment conducted by him intentionally, to test the hypothesis or to discover something or to demonstrate a known fact. An experiment aims at drawing conclusions concerning the factor on the study group and making inferences from sample to larger population of interest.

Key Differences Between Survey and Experiment

The differences between survey and experiment can be drawn clearly on the following grounds:

• A technique of gathering information regarding a variable under study, from the respondents of the population, is called survey. A scientific procedure wherein the factor under study is isolated to test hypothesis is called an experiment.
• Surveys are performed when the research is of descriptive nature, whereas in the case of experiments are conducted in experimental research.
• The survey samples are large as the response rate is low, especially when the survey is conducted through mailed questionnaire. On the other hand, samples required in the case of experiments is relatively small.
• Surveys are considered suitable for social and behavioural science. As against this, experiments are an important characteristic of physical and natural sciences.
• Field research refers to the research conducted outside the laboratory or workplace. Surveys are the best example of field research. On the contrary, Experiment is an example of laboratory research. A laboratory research is nothing but research carried on inside the room equipped with scientific tools and equipment.
• In surveys, the data collection methods employed can either be observation, interview, questionnaire, or case study. As opposed to experiment, the data is obtained through several readings of the experiment.

While survey studies the possible relationship between data and unknown variable, experiments determine the relationship. Further, Correlation analysis is vital in surveys, as in social and business surveys, the interest of the researcher rests in understanding and controlling relationships between variables. Unlike experiments, where casual analysis is significant.

You Might Also Like:

sanjay kumar yadav says

November 17, 2016 at 1:08 am

Ishika says

September 9, 2017 at 9:30 pm

The article was quite helpful… Thank you.

May 21, 2018 at 3:26 pm

Can you develop your Application for Android

Surbhi S says

May 21, 2018 at 4:21 pm

Yeah, we will develop android app soon.

October 31, 2018 at 12:32 am

If I was doing an experiment with Poverty and Education level, which do you think would be more appropriate for me?

Thanks, Chris

Ndaware M.M says

January 7, 2021 at 2:29 am

So interested,

Victoria Addington says

May 18, 2023 at 5:31 pm

Thank you for explaining the topic

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Observation Versus Experiment: An Adequate Framework for Analysing Scientific Experimentation?

• Open access
• Published: 07 May 2016
• Volume 48 , pages 71–95, ( 2017 )

Cite this article

You have full access to this open access article

• Saira Malik 1  

16k Accesses

11 Citations

4 Altmetric

Explore all metrics

Observation and experiment as categories for analysing scientific practice have a long pedigree in writings on science. There has, however, been little attempt to delineate observation and experiment with respect to analysing scientific practice; in particular, scientific experimentation, in a systematic manner. Someone who has presented a systematic account of observation and experiment as categories for analysing scientific experimentation is Ian Hacking. In this paper, I present a detailed analysis of Hacking’s observation versus experiment account. Using a range of cases from various fields of scientific enquiry, I argue that the observation versus experiment account is not an adequate framework for delineating scientific experimentation in a systematic manner.

Similar content being viewed by others

Why Thought Experiments do have a Life of Their Own: Defending the Autonomy of Thought Experimentation Method

Science and Experiment

Generative and Demonstrative Experiments

Avoid common mistakes on your manuscript.

1 Introduction

“They [the Greeks] observed but did not experiment”. Footnote 1

This quote from Desmond Lee, the famous translator of Aristotle’s scientific works, identifies the two categories that form the bedrock of modern scientific practice. Footnote 2

This quote also identifies well one of the principal markers used to delineate modernity. It is now a well-worn axiom that what distinguishes Western modernity—and implicit in this, Western hegemony—is the phenomenon of the Scientific Revolution in the West. What marks out the scientific practices of the Scientific Revolution and thereafter in the West from other scientific enterprises in the past—in the popular imagination—is supposed to be ‘experiment’. It is posited that this is the hallmark of the Scientific Revolution—what went before is ‘observation’. Footnote 3 What Desmond Lee appears to be doing here is setting up a binary of ‘observation versus experiment’ rather than ‘observation and experiment’. Many decades later, the distinguished philosopher of science Ian Hacking echoes Lee by positing, “Observation and experiment are not one thing [,] nor even opposite poles of a smooth continuum”. Footnote 4 The casting of experiment in opposition to observation as Hacking and Lee do, rather than in addition to it, is a very modern turn.

Experiment—as experimentum (and its cognates) in Latin and empeiria or peira in Greek—has a continuity of usage as a category for scientific learning finding its genesis in the works of Hippocrates, Aristotle and Pliny (Pomata 2011 , 45–46). Footnote 5 Observation, as a scientific category, does not enjoy the same continuity as the essays in Histories of Scientific Observation , edited by Lorraine Daston and Elizabeth Lunbeck, show. Footnote 6

In fact, what may be thought of as scientific observational practices were subsumed under a myriad of terms in Latin: experientia , experimentum , contemplatio , consideratio —and the least used— observatio . Where Greek is concerned, there is no equivalent for observation ( observatio )— teresis —in the scientific canon of Hippocrates and Aristotle (Pomata 2011 , 45). It is only in the seventeenth century that observation and experiment as scientific categories- respectively as observatio and experimentum —become established as well as conjoined (Daston 2011 , 81–113). Despite this distinguishment, the terms remained conjoined: “Observation, by the curiosity it inspires and the gaps that it leaves, leads to experiment; experiment returns to observation by the same curiosity that seeks to fill and close the gaps still more; thus one can regard experiment and observation as in some fashion the consequence and complement of one another” (Daston 2011 , 86). The difference implied between the two then, on the eve of the nineteenth century, was that experiment implied intervention and manipulation whereas observation did not (Daston 2011 , 86). In many cases even this implied distinction was subsumed for others, including notables such as Robert Boyle and Robert Hooke, who appear to make no distinction between observation and experiment as long as both were dedicated to the cause of knowledge acquisition of the natural world (Anstey 2014 , 105).

It is in the nineteenth century that one sees observation being cast in opposition to experiment rather than in addition to it (Daston and Lunbeck 2011 , 3). During this period one can see the reconfiguration of vision insofar as it becomes detached from a referent and thus abstracted, leading to the inevitable subjectivity of the observer (Crary 1992 ). Footnote 7 This leads, as Jonathan Crary explains, to the ‘social remaking of the observer’ (Crary 2001 , 4). The observer goes from a passive receiver of the external world to an active producer of it (Crary 2001 , 95–97). This nineteenth century reconfiguration of vision and the observer has been made clear not just in Jonathan Crary’s work on the camera obscura and works of art, but also in Christoph Hoffmann’s work on scientific practices, the senses and instrumentation (astronomical, in particular) during the same period (Hoffmann 2006 ) where the author shows that any qualitative distinction between the observer and instrumentation fades away. Footnote 8

In light of the importance of observation and experiment as categories in scientific practice, particularly scientific experimentation, it is surprising that relatively little attention has been paid to them as a binary within the modern academy of philosophy of science Footnote 9 —in spite of philosophers of experiment such as Hans Radder and David Gooding calling for such. Footnote 10 An exception is Ian Hacking—in Representing and Intervening (Hacking 1983 ). Footnote 11 In this essay, I scrutinise Hacking’s account of observation and experiment in order to assess its efficacy as an adequate account for delineating scientific experimentation. I show that there are significant weaknesses in Hacking’s account when used to analyse a range of cases from different fields of scientific enquiry.

2 Hacking: Observation Versus Experiment

In Representing and Intervening (Hacking 1983 ), Ian Hacking makes a category distinction between experiment and observation. He states, ‘Observation and experiment are not one thing nor even opposite poles of a smooth continuum’ (Hacking 1983 , 173). According to Hacking, ‘Much of the discussion about observation, observation statements and observability is due to our positivist heritage’ (Hacking 1983 , 168). He thinks the need to make these distinctions at all, and to take them seriously, is a task very much confined to professional philosophers. According to Hacking, these distinctions do not worry scientists. He gives the example of Francis Bacon to show what he means (Hacking 1983 , 168–169).

Francis Bacon does not mention the term once in his discussion of the inductive sciences despite the term being in circulation during his time. Observation at this time was restricted in its use—used mainly for observations made of the heavenly bodies via telescopes. That is, the use of the term observation in the natural sciences was associated with the use of instrumentation. Instead of observation, Bacon uses the term ‘prerogative instances’. In his Novum Organum of 1620, he lists 27 different ‘prerogative instances’: these include a range of activities which today one may refer to as scientific practices: experiments, tests to distinguish between hypotheses, notable observations, some are made with devices that ‘aid the immediate actions of the senses’. The latter includes microscopes as well as telescopes, rods, astrolabes and similar devices. He calls devices that aid the senses ‘evoking devices’, devices that ‘reduce the non-sensible to the sensible; that is, make manifest, things not directly perceptible, by means of others which are’ (Hacking 1983 , 168–169).

Bacon recognises the difference between what is directly perceptible and that which is hidden from the senses and needs to be ‘evoked’. He recognises it and does not give it much significance—for Bacon, the difference is not important. For Bacon, there is no difference between directly seeing the sun overhead at noon and seeing a planet via a telescope at night.

Hacking states it is only later, in the nineteenth century, that the difference between things that are directly perceptible and those that are hidden from the senses and have to be ‘evoked’ becomes important. It becomes important because the very notion of ‘seeing’ undergoes a transformation. In the nineteenth century, ‘to see’ is to see the surface—and only the surface—and all knowledge must be derived via this way. This marks the beginnings of positivism and phenomenology. Positivism needs to distinguish between inference and seeing with the unaided eye (Hacking 1983 , 169). Thus, unlike Bacon, there is a difference between seeing the sun overhead at noon and seeing a planet at night via a telescope. For the positivist, the planet seen via a telescope can only be inferred—it is not an observation. According to Hacking, this marks the start of the distinction made between observation and theory in the philosophy of science and is articulated by someone like Bas van Fraassen ( 1980 ). This view has come to be contested in two ways: one that emphasises the scope of observation and the other of theory. Grover Maxwell is a good exemplar of the former view (Maxwell  1962 ) while Paul Feyerabend is an example of the latter (Hacking 1983 , 172–173).

Hacking deals with Maxwell thus (Hacking 1983 , 170). Maxwell makes a historically contingent argument. He suggests that what may be unobservable at some particular time may subsequently become observable—or in Bacon’s language be ‘evoked’—with the development of adequate instrumentation and/or the expansion of the capacity of existing instrumentation. For example, in the case of visual perception, there is a continuum that starts with seeing through a vacuum, through the atmosphere, through a simple microscope and, at present, finishes with seeing through the current batch of advanced microscopes. In this way, what in previous generations would have been unobservable—and according to positivists only inferred, and thus theoretical—becomes observable with the development of appropriate instrumentation. For example, prior to Louis Pasteur, the notion of microbial entities responsible for disease was considered theoretical. However, with the advent of microscopy these entities became observable. Other examples include genes on chromosomes, cell bodies in cells and the fine structure of metals. In all these cases the entities were regarded as theoretical until the development of adequate instruments rendered them observable. For Maxwell there is no significant difference between knowledge gained directly through the senses and that gained indirectly with the aid of instrumentation—Bacon’s ‘evoking’ devices.

The second type of critique of the positivist stance is based on the notion that the distinction between observation and theory is redundant. That is, all observations—whether made directly via the senses or not—are theoretical—that is, there are no pure observations. All observations are ‘theory laden’ to coin Norwood Hanson’s term from his Patterns of Discovery (Hanson 1958 , 19). Hanson states, ‘seeing is a “theory laden” undertaking’.

Paul Feyerabend agrees with Hanson but goes even further (Hacking 1983 , 172–173). For Feyerabend, there is no difference between observation and theory. In fact, he has rejected the term ‘theory laden’ on the grounds that there can be no observation without theory. He states, ‘Nobody will deny that such distinctions [between observation and theory] can be made, but nobody will put great weight on them, for they do not play any decisive role in the business of science’ (Hacking 1983 , 173). He comments on the everyday practices of science, ‘observational reports, experimental results, “factual statements”, either contain theoretical assumptions or assert them by the manner in which they are used (Hacking 1983 , 173).

Hacking chooses to align himself with Grover Maxwell rather than Feyerabend; and is particularly scathing of Feyerabend’s [lack of] understanding of scientific practice exemplified by the statement, ‘observational reports, experimental results, “factual statements”, either contain theoretical assumptions or assert them by the manner in which they are used’. Hacking explains why using two historical examples: the work of Albert Michelson and Edward Morley along with that of William Herschel.

The work of Michelson and Morley is well known to historians of the physical sciences (Hacking 1983 , 174). It is famous because, on reflection, it refuted the existence of ‘electromagnetic aether’ and led to the establishment of the special theory of relativity. Hacking focuses on the scientific practices of Michelson and Morley and what these mean with respect to Feyerabend’s comment, ‘observational reports, experimental results, “factual statements”, either contain theoretical assumptions or assert them by the manner in which they are used’. The published ‘report’ of the experiment of 1887 was 12 pages long. The ‘observations’ made were done so for a total of a couple of hours over 4 days in July. The ‘results’ of the experiment remain controversial: Michelson believed that this work showed that the earth’s motion was independent relative to the [presumed] aether. Hacking goes on to identity the components which (in his view) contributed towards the impact of this work—in its own time and up to the 1920s. These components include, inter alia , the making and re-making of apparatus, getting the apparatus to actually work and, most importantly—knowing when the apparatus was working. Interestingly, the most important result of this work, according to Hacking, had less to do with aether and more to do with the transformation of measurement. Hacking concludes, “In short, ‘Feyerabend’s factual statements, observation reports, and experimental results’ are not even the same kind of thing. To lump them together is to make it impossible to notice anything about what goes on in experimental science” (Hacking 1983 , 174).

Hacking then proceeds to show that Feyerabend’s notion that all observations carry theory is false. Hacking uses the historical case study of William Herschel (d. 1822) as a rebuttal to Feyerabend’s, ‘all observations carry theory’.

William Herschel was an astronomer, who, in the year 1800, is attributed to have discovered radiant heat whilst conducting his astronomical work with his telescope (Hacking 1983 , 176). On using different coloured filters in his telescope, Herschel realised that different colour filters gave off different amounts of heat. Herschel, in the reporting of his work in the Philosophical Transactions of the Royal Society for the year 1800 states, “When I used some of them I felt a sensation of heat, though I had but little light, while others gave me much light with scarce any sensation of heat”. It was this incidental observation to his principal work on the sun which led Herschel in a new experimental direction and the discovery of radiant heat: that the sun emits both visible and invisible rays and that human sight is sensitive to only the visible rays. This incidental observation led him to conduct a whole series of experiments investigating the transmission, reflection and refraction of these rays (Hacking 1983 , 177). Hacking concludes, “Feyarabend says that observations reports, etc., always contain or assert theoretical assumptions. This assertion is hardly worth debating because it is obviously false” (Hacking 1983 , 174).

Thus, Hacking’s notion of observation appears to be very much aligned with Grover Maxwell, and with Francis Bacon’s ‘evoking devices’. Hacking’s anti-positivist stance on observation becomes even clearer when considering his position on observation of sub-atomic particles via indirect methods, and finally, with his view of observation of entities via a microscope.

On observation of sub-atomic particles using indirect methods, Hacking is in agreement with Dudley Shapere (Shapere 1982 ). Shapere uses the discourse of ‘observing the interior of the sun or another star’ as his starting point in his argument to show what is meant by ‘to see’ in modern science—and in the process shows how far we have journeyed along the path of Bacon’s ‘evoking devices’. Shapere analyses the solar neutrino experiment in which physicists claim that the core of the sun (or any star) can be directly observed via the detection of neutrinos. Footnote 12 Shapere shows the various layers of detection—’seeing’—involved in the ‘direct observation’ of neutrinos emitted from the core of the sun. Shapere argues that despite what appears to be a complicated series of events entailed in the detection of neutrinos, it is justifiable to term this process as ‘direct observation’ (Shapere 1982 , 492).

Hacking suggests that it is the fact that the theories underlying the detection mechanism are not entwined with the subject matter under investigation is what gives credence to the claim in the solar neutrino experiment that the “stellar core of the sun can be directly observed” (Hacking 1983 , 185). Footnote 13 For Hacking therefore what would count as an observation would include the detection of electrons in a bubble-chamber, as the theory used in the manufacture and operation of the bubble-chamber does not directly use theory about electrons. Footnote 14 For Hacking, this also holds for the use of microscopes—that is, that the theories, assumptions and norms on which microscopes are built and used (from simple light microscopes to electron-scanning and X-ray diffraction ones) are independent of the subject matter being studied (Hacking 1983 , 186–209) and for him therefore what is seen using these instruments counts as an observation.

Hacking is very much wedded to an anti-positivist stance on what constitutes observation—very much in the tradition of Francis Bacon and his “evoking devices making manifest, things which are not directly perceptible, by means of others which are”. Hacking, thus, aligns himself with working scientists for whom ‘to see’ includes detection methods ranging from the simple microscope to its X-ray diffraction and electron scanning versions—things Bacon could not have imagined when he composed his Novum Organum . What Hacking thus means by observation is detection.

Where experiment is concerned—again—Hacking appears to be in support of Bacon (Hacking 1983 , 246–250) as the following citations from Bacon show, “The secrets of nature reveal themselves more readily under the vexation of art than when they go their own way” (Hacking 1983 , 246), “shake out the folds of nature” and to “twist the lion’s tail” (Hacking 1983 , 246). Hacking says this alludes to “Bacon’s good sense” (Hacking 1983 , 250). However, for Hacking an experiment is not just a case of intervention, or ‘to twist the lion’s tail’, as it is for Bacon as we see below.

What is Hacking saying an experiment is? He stipulates very clearly—It is the “creation of phenomena” (Hacking 1983 , 220). This is made more emphatic in “Experiment is the creation of phenomena” (Hacking 1983 , 229). What does Hacking mean? First—phenomena: Hacking says that he agrees with scientists as to what is meant by phenomena,

A phenomenon is noteworthy . A phenomenon is discernable . A phenomenon is commonly an event or process of certain type that occurs regularly under defined circumstances. When we know the regularity exhibited in a phenomenon we express it in a law-like generalization. The very fact of such a regularity is sometimes called the phenomenon. (Hacking 1983 , 221)

This description fits very closely with the etymology of the (Greek) term phenomenon: a thing, an event or process that can be seen. However, phenomenon, Hacking points out, has quite a different sense in philosophy.

Phenomenon has a long history in its philosophical usage (Hacking 1983 , 220–221) quite different to its etymological roots. Phenomenon, for philosophers—both ancient and modern—has come to indicate something related to the senses. For many ancients, phenomena were in opposition to reality insofar as phenomena—perceived via the senses—were subject to change (Hacking 1983 , 221). The fact of phenomena being the subject of change led to the juxtaposition of phenomena to noumena: phenomena were only appearances of things whereas noumena were things as they actually were. Kant took up this distinction and proposed that only phenomena could be known—the noumena could not. With the advent of positivism, phenomena came to indicate sense-data—things that are “private, personal sensations” (Hacking 1983 , 221)—rendered as ‘phenomenalism’, and according to one of its principal proponents, J. S. Mill, “things are only the permanent possibilities of sensation, and that the external world is constructed out of actual and possible sense-data” (Hacking 1983 , 221).

Hacking breaks from the way philosophers have come to use the term phenomenon, and aligns himself with the scientists. He says,

My use of the word ‘phenomenon’ is like that of the physicists. It must be kept as separate as possible from the philosophers’ phenomenalism, phenomenology and private, fleeting sense-data. A phenomenon, for me, is something public, regular, possibly law-like, but perhaps exceptional. I pattern my use of the word [phenomenon] after physics and astronomy. (Hacking 1983 , 222)

Hacking illustrates what he means by using the ‘Hall effect’ from the field of modern physics as an exemplar.

Edwin (E. J.) Hall’s work on the relationship between a magnetic field and electric potential is referred to as the Hall effect (Hacking 1983 , 224–225). In the late 1870s, Hall, under the supervision of Henry Rowland at John Hopkins University, had been expanding on some of James Clerk Maxwell’s ideas from his Treatise on Electricity and Magnetism . In the Treatise Maxwell had proposed that, where a conductor carrying an electric current was under the influence of a magnetic field, the magnetic field acts on the conductor rather than the current. Hall proposed that if this were the case then there should be two possible outcomes: either the resistance of the conductor would be affected by the magnetic field or that an electric potential across the field would be produced. Hall discarded the first possibility as he failed to observe any effect by the magnetic field on the resistance of the conductor. However, the second possibility bore fruition: he was successful at measuring an electric potential across the magnetic field. He obtained an electric potential when he placed a gold leaf at right angles to the magnetic field and electric current. Hall says,

It seemed hardly safe, even then, to believe that a new phenomenon has been discovered, but now after nearly a fortnight has elapsed, and the experiment has been many times and under various circumstances successfully repeated … it is perhaps not too early to declare that the magnet does have an effect on the electric current or at least an effect on the circuit never before expressly observed or proved. (Hacking 1983 , 225)

Hacking tells us that by the 1880s it was common for physicists to call a phenomenon an effect: as in the Compton effect, Footnote 15 the Zeeman effect Footnote 16 and the photoelectric effect Footnote 17 (Hacking 1983 , 224). He states,

Phenomena and effects are in the same line of business: noteworthy discernable regularities. The words ‘phenomena’ and ‘effects’ can often serve as synonyms, yet they point in different directions. Phenomena remind us, in that semiconscious repository of language, of events that can be recorded by the gifted observer who does not intervene in the world but who watches the stars. Effects remind us of the great experiments after whom, in general, we name the effects: the men and women, the Compton and Curie, who intervened in the course of nature, to create a regularity which, at least at first, can be seen as regular (or anomalous) only against the further background of theory. (Hacking 1983 , 225)

Here Hacking starts by telling us phenomena and effects have similar aims—they yield “noteworthy discernable regularities”—albeit he draws a difference between them in so far as the kinds of activities they are: phenomena as ‘events’ noted by those who do not ‘intervene in the world’ while effects are things which “remind us of the great experiments” done by those “who intervened in the course of nature, to create a regularity…”.

If we now turn to consider what Hacking means by ‘creation’ in his stipulation of experiment—‘creation of phenomena’, we find that Hacking is conferring a very constricted meaning to creation.

Hacking (again) uses Hall’s work to illustrate what he means by creation. He says, “Hall’s effect does not exist outside of certain kinds of apparatus” (Hacking 1983 , 226). This is made more emphatic in, “Hall’s effect did not exist until, with great ingenuity, he [Hall] had discovered how to isolate, purify, create it in the laboratory” (Hacking 1983 , 226). To give even more emphasis, Hacking cites another example: the ‘Josephson effect’ referring to the work of Brian Josephson in the 1960s. Again, the example Hacking chooses is from modern physics and, in this case, concerns the subject of electrical conduction by super-conductors. Footnote 18 He says, “The Josephson effect did not exist in nature until people created the apparatus” (Hacking 1983 , 229). For Hacking, it is these effects—bounded by the apparatus in which they can be demonstrated in laboratory conditions that appear to fulfil his criteria for what qualifies as experiment.

Hacking explains what he means by his statement, “the Hall effect does not exist outside of certain kinds of apparatus” (Hacking 1983 , 226). He asks rhetorically, “Does not a current passing through a conductor, at right angles [ sic ] to a magnetic field, produce a potential, anywhere in nature?”, answering ambivalently, “Yes and no”. According to Hacking, if there were such an event in nature, which occurred in isolation of any other processes, then it could be said that the Hall effect occurs in nature. However, it is only in laboratory conditions that the Hall effect can be produced independent of any other processes. It is with this explanation that it becomes clear that, for Hacking, in order for a phenomenon to be created—it needs to be produced in isolation, or what he calls “in a pure state” (Hacking 1983 , 226).

Hacking’s commentary on the work of Edwin Hall tells us what Hacking means when he stipulates “experiment is the creation of phenomena”. What we see is that Hacking’s stipulation of experiment as ‘creation of phenomena’ becomes highly constricted because of his insistence on the fact that the phenomena under consideration needs to be produced in a ‘pure state’ or in isolation. His repeated emphasis on ‘creation’ gives emphasis on the importance of this aspect in his stipulation of experiment. The emphasis on ‘pure state’ is underlined from the highly selective way he chooses his case studies in support of his position. He cites many examples but chooses either not to deal with them in any sustained way or dismisses them—on occasion, flippantly (Hacking 1983 , 227–228). Amongst the many examples Hacking cites, he chooses to focus only on cases from modern physics, such as the Hall and Josephson effects. This appears to be a deliberate strategy on his part as the following illustrates.

Hacking introduces the medical work of Claude Bernard (published as Introduction to the Study of Experimental Medicine in 1865) as a potential case study to show the distinction between experiment and observation (Hacking 1983 , 173). Hacking states,

Consider Dr Beauchamp [ sic ] who, in the Anglo-American war of 1812 [ sic ], had the fortune to observe, over an extended period of time, the workings of the digestive tract of a man with a dreadful stomach wound. Was that an experiment or just a sequence of fateful observations in almost unique circumstances? (Hacking 1983 , 173)

In this example, Hacking not only makes a couple of errors in transposing historical details from Bernard, Footnote 19 but more importantly, considerably truncates the details of William Beaumont’s study on the digestive physiology of the human stomach. Footnote 20 Hacking finishes by choosing not to engage with this case from medical physiology, saying, “I do not want to pursue such points” (Hacking 1983 , 174).

In contrast, I think it worth pursuing this case from medical physiology, as well as some others from different fields of scientific enquiry, in order to assess how Hacking’s observation versus experiment account maps onto cases from a range of scientific experimentation.

First, returning to Beaumont’s story. William Beaumont himself believed that the work he was doing was an experimental investigation of human digestion (Beaumont 1833 , 5–6). However, more important is how this case fits with Hacking’s account as he chooses not to address this question himself. First—a very brief overview of Beaumont’s work on digestion.

Beaumont, in his capacity as a doctor, treated a patient suffering from a gunshot wound, which had caused damage to his left lung and stomach (Beaumont 1833 , 10). The patient recovered but with a very unusual outcome: the stomach lining did not heal in a uniform way. Instead it formed a fistula with an exterior valve. Beaumont used this valve as the access point in conducting a series of investigations on digestion (Beaumont 1833 , 11–23). He used a pipetting technique to both put substances into the stomach, as well as to draw them out. In this way he examined the digestion of various substances in the stomach.

If one were to use the Hacking’s criteria for experiment, the ‘creation of phenomena’—Beaumont’s work within the stomach falls short of qualifying as experiment. That is, although the digestion process qualifies as a phenomenon (a discernable change) as well as an effect (requires activity and intervention on the part of the investigator); it does not fulfil Hacking’s criteria for creation—that is, the effect is not occurring in isolation of other processes. Footnote 21 However, Beaumont goes on to perform a series of investigations looking at the digestive action of the ‘gastric juice’ in isolation (Beaumont 1833 , 73–101). Footnote 22 This series of investigations would appear to fulfil Hacking’s criteria for experiment as Beaumont sets up apparatus (however rudimentary) which gives rise to an effect—in isolation of others.

Thus, if we use Hacking’s criteria for experiment in respect to William Beaumont’s work, the outcome would be that only part of Beaumont’s work qualifies as experiment—the in vitro part. That is, the part done within the apparatus that Beaumont sets up to investigate digestion outside the stomach. The in vivo part of Beaumont’s work, that is, the work done on the stomach directly, fails to qualify as experiment as the effects occurring are not in isolation of other processes. This ignores the crucial point in William Beaumont’s investigations: the in vitro part (experiment) is contingent on the work that William Beaumont has previously done on the stomach. Beaumont would never have set up the apparatus part of his work if he had not already done the work on the stomach.

According to Hacking, thus, if Beaumont’s in vivo work is not experiment then is it a series of observations? If so, Hacking has told us that, “[o]bservation and experiment are not one thing nor even opposite poles of a smooth continuum” (Hacking 1983 , 173). This statement would imply that, in Beaumont’s case, the work done in vivo and that done in vitro are not part of ‘a smooth continuum’. This is not reasonable given that the work done in vitro is continuous with the work done in the stomach. The limitations of Hacking’s framework are not only confined to this case from physiology.

In evolutionary biology, the name of Henry Kettlewell is well known. Kettlewell’s work on moths in the 1950s was important in understanding the process of natural selection. Footnote 23 Kettlewell used three kinds of moths: typical ( Biston betularia ), intermediate ( Biston insularia ) and dark ( Biston carbonaria ). In Britain, the typical moth had been prevalent in most areas prior to industrialization. However, the proportion of the typical variety in relation to the other two types changed during the twentieth century. Kettlewell showed that this change was due to the change in colour of the landscape. Kettlewell first showed that the different kinds of moths were more or less conspicuous depending on the colour of the background on which they were settled. He did this by using volunteers to rank the degree of conspicuousness of each type of moth on different colour backgrounds. In the next stage, he put all three types of moths in a cage with different colour bark on which they could settle. He then introduced birds (predator to moths) into the cage. He found that the rate at which the moths were eaten depended on the colour of the bark on which they were settled. As three different kinds of moths were used along with different colour barks, the data analysis was very complex in this part of the study. The third part of his study was done in native conditions. Kettlewell released all three kinds of moths in both polluted (dark background) and unpolluted (lighter background) areas and tracked how many survived. This last part of the investigation depended on previously marked moths that had survived being recaptured in traps. Kettlewell showed that the dark species of moths survived better in a polluted (dark) environment than the lighter colour varieties whereas the lighter typical species survived better in the less polluted (light) environment compared with the darker varieties. He showed this was due to the colour of the landscape.

What do we see when we map Hacking’s observation versus experiment account onto this case? The observation part of the account can be done in a straightforward manner. Hacking has told us that the observation part is a source of detection—in this case the numerical values related to what kind of moth species is conspicuous on which colour bark, the numerical values related to different species surviving predation in the cage, the kind of bait used for re-capture in native conditions. However, what, according to Hacking, is the experiment part? The phenomenon under study here—natural selection—is not being produced in isolation of other processes and therefore, in Hacking’s account, cannot—or should not—be included in his category of experiment.

We see this anomalous consequence of Hacking’s account in fields of scientific enquiry other than the two (physiology, evolutionary biology) mentioned already.

In the field of study of animal behavior and psychology, the work of Harry Harlow is well known amongst those working on attachment theory. Footnote 24 Harlow did his experimental work on rhesus monkeys (macaques) during the 1950s and 60s. Footnote 25 His work on isolated infant monkeys had shown that the infants formed a close attachment to the soft materials in their cages (diapers, bedding) whereas those infants who had their mothers in the cage did not form this attachment. Harlow conducted a series of experiments to measure degrees of attachment of an infant monkey to the quality of a carer.

Eight new-born monkeys were separated from their mothers immediately after birth. Each was placed in a cage with two ‘surrogate mothers’—one surrogate was made of wire with a box face while the other surrogate was made of soft cloth with a quasi-monkey face. Milk was dispensed from each surrogate. Harlow measured the time that each infant monkey spent with each surrogate over a period of some months. He found that the infant monkeys spent more time with the cloth-covered surrogate than with the wire one. He then withdrew milk dispensation from the cloth surrogate. He found that the total time that the infants spent with the cloth surrogate was still much greater than that time spent with the wire surrogate—the infants would only go to the wire surrogate to feed when hungry—as soon as their hunger abated, they returned to the cloth covered surrogate. Harlow concluded from these particular experiments that infant monkeys had requirements (social, cognitive, emotional) beyond those of (just) nutrition (milk) in their early years.

In the field of geology, the work of Nevil Maskelyne and colleagues gave an initial indication of the density of the earth (Danson 2009 ). Footnote 26 Their work was based on the notion that a pendulum, placed near a mountain in a uniform gravitational field, would shift from the true vertical. This shift could be measured against a reference such as a fixed star and—given Newton’s proposal that force is proportional to the mass of an object—the density of the earth could be calculated. Isaac Newton himself, in the Principia had indicated that this should be possible but had discarded the idea as he believed the instrumentation of the day would not be able to detect the small changes in the shift of the pendulum. Footnote 27

Just over a century later in 1772, Maskelyne, the Royal Astronomer to George III, believed that the instrumentation at the Royal Observatory in Greenwich was up to the task that Newton had set. The French astronomer Pierre Bouguer had carried out Newton’s proposal of using a mountain in South America some decades earlier—but had not met with much success on account of numerous technical obstacles (Danson 2009 , 40–42; 97–98).

Maskelyne met with greater success at a mountain in central Scotland, Schiehallion (chosen for its symmetry). The investigation was divided into two stages. The first entailed measurement of the deflection of the pendulum with respect to positions of fixed stars for which two observatories were built—one on the north side and one on the south. The measurements taken were in the astronomical measure of arc minutes. The other stage of the investigation involved the survey of the mountain in order to measure its volume. These measurements were expressed in terms of height (feet/inches). The work took until 1778 to complete and the final density of the earth was computed to within 20 % of that calculated by Henry Cavendish some twenty odd years later using a torsion balance to measure the attraction between two lead spheres.

Staying within geology and in Scotland, James Hutton’s extensive investigations on soil erosion helped significantly shape understanding of landscape formation (Dean 1992 ). Footnote 28 Over a span of decades, Hutton made extensive surveys and measurements of various areas of Britain as well as France, Belgium and Holland. Much of this work was subsequently the starting point for Charles Lyell and Charles Darwin in their work on geology (Rudwick 2005a ). Footnote 29 The first outline of Hutton’s work was circulated as Abstract of a Dissertation Concerning the System of Earth, its Duration and Stability in 1786. His work consisted of analysis of rock strata and analysis (chemical, thermogenic) of different kinds of rock formations (granite and gneiss, sediment[ary] and volcanic [igneous] as well as the identification and recording of the frequency of the occurrence of fossils in these different rock strata. Footnote 30 The records of his results consisted of temperature readings at which different kinds of rocks changed appearance, the recording of what these changes entailed, the recording of which (if any) rock kinds reacted with different kinds of chemicals, extensive classification and frequency tabulations of fossil finds, numerous drawings of fossil finds and rock strata.

I now want to turn to physics—the principal focus of study for Hacking. In the early part of the twentieth century, Robert Millikan conducted a series of investigations to establish that the charge of the electron was quantized (had a discrete fundamental value) and occurred in situ as multiples of this value rather than a continuum as had been previously proposed by Thomas Edison, amongst others (Holton 1978 ). Footnote 31

The received narrative of Millikan’s investigations is presented as an ingenious use of the cloud chamber developed by Charles Wilson (Franklin 1986 , 216). In Wilson’s original, within a sealed container, ions act as loci around which water droplets can form. Wilson used a sealed container filled with air and water vapour at the point of condensing—a supersaturated environment—which he produced using a vacuum pump for first compressing and then expanding air inside the sealed container (‘chamber’). Any charged particle in the container containing this supersaturated mixture causes ionization as it moves. This ionization acts as loci around which vapour (‘cloud’) forms as a consequence of condensation. The movement (or fall) of this ‘cloud’ in this ‘chamber’ under gravity can be detected via a viewer (short focal distance telescope) and the visible ionization path measured (by calibrating the eyepiece of the telescope). If an electric field is applied (vertically) across the chamber (in the form of two charged plates—positive at the top and negative at the bottom with a DC voltage applied to each plate via a battery)—then the change in the rate at which the cloud moves/falls under gravity can be detected. Measurement of the velocities of the fall of the cloud under just gravity and then with a known voltage should determine the charge on the electron.

J. J. Thompson had attempted to measure the charge on the electron in this way but had tried to measure the charge of the whole cloud and had met with little success—owing in the main to practical obstacles (Goodstein 2001 , 54).

Millikan, in attempting the same as Thompson, found that applying a much greater electric field across the charged plates resulted not in the cloud being suspended, as had been predicted, but most of the cloud dispersing, leaving only a few drops suspended between the plates. Millikan deduced that working with individual droplets would overcome many of the logistical and numerical obstacles that Thompson had faced in working with a whole cloud (ibid.).

Millikan (and his graduate students) set about repeating Thompson’s work with single droplets of water but found no success as the single water drops tended to evaporate quickly, making reliable measurements impossible. They thus set about adapting Wilson’s cloud chamber, as well as Thompson’s method, over a period of some years. The appearance of simplicity in Millikan’s final investigative set-up belies the various stages it took for the investigation to mature.

The first issue they had to overcome was that of evaporation. They did this by replacing water drops with substances whose evaporation rate would have a negligible effect on their measurements. The first substance they used was oil with a low vapour pressure that would easily form a spray (they produced the oil drops as a spray with a perfume atomizer using watch oil bought at minimal cost at a local market). Although Millikan’s published work dealt with the results from work done with the oil drops, Millikan and his group had done the same investigations with glycerine and mercury. Evaporation issues were only the first of many obstacles they had to overcome to arrive at a working system, including inter alia : temperature within the chamber affecting viscosity of the air, allowing for the evaporation of the oil (as well as glycerine and mercury)—however minimal, the motion of the air inside the chamber, the fluctuation of the charge applied by the battery source (Franklin 1981 ). Footnote 32

Their final set up (which led to Millikan’s published work on the quantization of the charge on the electron in 1910 and 1913) ran as follows. Within a sealed container Millikan et al. placed two charged plates 16 mm apart which were connected to a DC supply (battery). Above the top plate was an aperture through which the atomizer could spray droplets into the container. The top charged plate had a small aperture through which oil (glycerine, mercury) droplets could drop under gravity. In the space between the two plates were three apertures: one for the short focal telescope to view the drops, one for a light source in order to be able to see the drops and the other for an X-ray source to induce ionization of the air. The actual measurements were made in units of time—in seconds (range 11–19 s)—taken for an oil drop to move across a known distance of 10.21 mm (Millikan 1913 ). Footnote 33 The voltage (when used) was set at 5 kV. Differences in the time measured for an oil drop to move across the given distance (10.21 mm) under (just) gravity and then under the given current (5 kV) allowed Millikan to calculate the charge and, with repeated measurements under varying conditions, deduce that the charge was quantized. Footnote 34

As with Beaumont and Kettlewell, how does Hacking’s observation versus experiment account map onto the cases outlined above from various fields of scientific enquiry?

As with Beaumont and Kettlewell, the observation part of Hacking’s account—a means of detection—can be identified easily in the mentioned cases:

In Harlow’s work with infant monkeys, his measurements of time spent with each surrogate;

In Maskelyne’s investigations on the density of the earth, measurements of arc minutes to measure the shift in the pendulum from the true vertical with respect to fixed star positions and those of height in terms of feet and inches in order to measure volume;

In Hutton’s case, the results of chemical and heat changes of different kinds of rock formations, drawings of fossils and rock strata as well as frequency tabulation in the form of integer numbers related to recording of numbers of fossils found in various rock formations;

In Millikan’s case, the measurements of the time taken for an oil drop to traverse the distance of 10.21 mm.

Again as with the cases of Beaumont and Kettlewell, it is much more difficult to see how Hacking’s category of experiment fits with these cases. In none of them is it clear to see where the ‘creation of phenomena’ with its emphasis on ‘pure state’, as we saw with the Hall or Josephson Effects, lies:—emotional attachment for Harlow, density for Maskelyne, landscape erosion for Hutton—even in physics—Millikan’s quantization of charge.

Hacking’s observation versus experiment account—as a means of delineating scientific experimentation as part of practice —thus appears not very helpful when faced with cases from a range of fields of scientific enquiry as those described. Even within modern physics—as Millikan’s work shows—Hacking’s account has limited use. Footnote 35

4 Experiment and Observation as Processes

David Gooding notes that it is in facing real accounts of scientific experimentation that what he calls “the familiar distinction between observation and experiment” collapses; calling the distinction an “artifact of the disembodied, reconstructed character of retrospective accounts” (Gooding 1992 , 68). We should then perhaps not be surprised that Hacking’s observation versus experiment framework does not survive intact when put to the test in a range of cases of scientific experimentation Footnote 36 such as those described.

Hacking’s account, in its attempt to reify and stipulate the notion of experiment, fails to capture the range and complexity of actions (mental and physical) entailed in what is indicated by experiment in scientific practice. If we return to the examples of scientific experimentation described above, in all cases—some undertaken over decades—the investigations consisted of the accumulation of parts: in Beaumont’s case, his in vivo as well as his in vitro work; in Kettlewell’s case, his struggles in finding the appropriate control landscapes; in Harlow’s case, trials with different kinds of ‘soft material’ used as a surrogate with which the infant monkeys could identify; Maskelyne’s case consisted of two distinct parts—the astronomical measurements made in the two observatories and the land survey of Schiehallion which followed the astronomical part and took nearly 2 years to complete (due to weather conditions); Hutton’s work on investigating rock strata and formations, and their relationship with the age of the earth, took decades and consisted of two distinct parts—analysis of the rock strata and work with fossils; Millikan’s experiment—which appears straightforward—went though a number of stages as it was optimized for substances (different kinds of oils, glycerine and mercury) and conditions (such as temperature, air viscosity) and calibration (different scales used to measure distance).

Looking at these examples of scientific work, should we refer to them as experiments or a series of experiments? Gooding proposes a potentially helpful way of thinking about this question. Gooding asks us not to talk about experiment but experimentation and think of it as a process Footnote 37 ( 1992 , 65–67). Hasok Chang too, using a different lexicon, asks us to think of experiment as a series of activities which themselves are composed of processes ( 2011 , 208–210). Footnote 38 The idea of process fits well with the range of examples described above. However, does viewing experimentation as a process help us in delineating experiment from observation as categories distinct from each other as Hacking does in his account? We have seen that observation, for Hacking, is a means of detection. However, this too, more often than not, tends to be a process. If we take just one case from amongst those that Hacking categorizes as observation this becomes apparent. One example (cited earlier) Hacking uses as an example of an observation is of the detection of solar neutrinos ( 1983 , 182). The detection of solar neutrinos runs thus. Footnote 39

Solar neutrinos are produced as a by-product of nuclear fusion in the core of the sun (Pinch 1985 , 5). As they are highly unreactive, they can pass through the outer layers of the sun and pass through the earth’s atmosphere (predominantly) in the state in which they were produced in the sun’s interior. The fact that they are highly unreactive of course makes them very difficult to detect. In the 1960s, Raymond Davis Jr. developed the methodology for detection of solar neutrinos of a particular kind (pp or proton–proton). A 100–400 k gallon container of dry cleaning fluid (perchloroethylene) was buried over a mile underground (in a disused mineshaft). The chlorine in perchloroethylene contains traces of a radioactive chlorine isotope (37) with which the solar neutrinos are able to react. The reaction between the chlorine isotope and solar neutrinos gives rise to the production of a radioactive argon isotope (37). This argon isotope is allowed to accumulate over typically a month (not longer as the half-life of the argon isotope (37) is 35 days). Other isotopes of argon (36 or 38) are added which aids the argon isotope (37) to bind to the helium gas, which is flushed through the container to remove the argon isotope (37). The helium containing the argon isotope (37) is then passed through pre-cooled charcoal, which collects the argon isotope (37). It is the decay of the argon isotope (37), which can be detected via a pre-calibrated Geiger counter.

It is apparent that this process of detection of solar neutrinos is exactly that—a process—with a multitude of different manipulations, practices and interpretations. In fact, very similar to the practices and processes of experimentation described in the cases above, and act for Gooding’s claim that in the face of real cases of scientific practice, to (try to) distinguish between observation and experiment is futile. In Hacking’s account, experiment is defined in a verbal phrase—‘creation of phenomena’—based on activity and ‘endless different tasks’ (Hacking 1983 , 230). However, observation too in this account entails the same as a means of detection and in most fields of scientific enquiry requires ‘endless different tasks’—one could replace the case of the solar neutrinos described above by numerous others including inter alia : other sub-atomic particle decay experiments in physics, chain reactions in chemistry (organic and inorganic), cascade and chain reactions in biochemistry. Both observation and experiment in practice involve undertaking various activities, manipulations, interventions and interpretations.

Hasok Chang has proposed that the pursuit of a systematic analysis of activities entailed in scientific practice is a worthy goal (Chang 2011 ). He proposes a “philosophical grammar of scientific practice” (ibid, 206) where he tentatively draws a taxonomy of what he says are only some of the “epistemic activities” entailed in scientific practice including, inter alia , Describing, Explaining, Hypothesizing, Testing, Observing, Measuring, Classifying, Representing, Modelling, Simulating, Synthesizing, Analyzing, Causing, Abstracting, Idealizing. David Gooding too has made an attempt to describe scientific practice (Gooding 1990 , 1992 ) albeit in diagrammatic form—in what he calls “experimental maps” (Gooding 1992 , 67), rather than discursively as Chang does. However, both are interested in considering the nature (in Chang’s case) and ordering (in Gooding’s case) of the multitude of epistemic activities entailed in scientific practice; rather than in differentiating between them as Hacking appears to be doing in his experiment versus observation account, as a means of categorization in an ‘either/or’ way. It is therefore not surprising that Hacking’s account, based as it is on casting observation and experiment as polarities, rather than seeing them as parts of a continuum within the process of experimentation, is not able to adequately account for cases of scientific experimentation other than from the very narrow area of physics on which he chooses to focus, such as high-energy lasers and such like.

The categorical distinction Hacking makes between observation and experiment would seem to rely in the main on his very particular definition of experiment—‘creation of phenomena’ and the many issues arising out of his stipulation of ‘creation’ as demonstrated earlier. Footnote 40 If one disregards Hacking’s stipulation of ‘creation’ in his definition for experiment, then it is difficult to see how a category distinction can be maintained between observation and experiment. As we saw earlier, both encompass generation of data so this could not act as an adequate marker.

If one were to broaden Hacking’s notion of experiment to consider another candidate as a marker for making a distinction between observation and experiment, then one of the more obvious ones is intervention. Lorraine Daston, in her account of practices of observation in the period 1600–1800, gives a glimpse of the various views circulating around the projected distinction between observation and experiment during this period (Daston 2011 , 85–87). Amongst these views, many gave importance to intervention (or its synonyms) as an important marker for distinguishing between observation and experiment. However, even then (that is, before the use of increasingly complex instrumentation became ubiquitous in scientific experimentation and practice in the modern age) some could see ambiguities arising. Gottfried Wilhelm Liebniz notes, “there are certain experiments that would be better called observations, in which one considers rather than produces the work” (ibid. 86).

This attempt to cast ‘intervention’ as a potential marker to distinguish between observation and experiment as categories, of course, pre-dates the crucial nineteenth century shift towards the dissipation of any qualitative difference between ‘seeing’ with help such as with instrumentation with its associated range of interventions, and that without—as the works of Clary, Hoffmann and Schikore amongst others have shown. Footnote 41 Looking back to the case of William Beaumont and his work on human digestion we see that: both his in vivo and in vitro work needs intervention (of some kind) to be satisfactorily completed making it impossible to distinguish (in any consistent and coherent way) between what should be observation and what experiment. The case of observation of solar neutrinos also makes very clear, with its numerous and complex manipulations, that intervention is not a reasonable candidate for acting as a category distinguisher between observation and experiment. The category distinction Hacking makes between observation and experiment thus rests very much on his narrow definition of experiment—‘creation of phenomena’—with the anomalous consequences that arise when this definition is used across various instances of scientific experimentation as the cases earlier demonstrate.

Other accounts than those of Chang and Gooding have also been advanced to analyse scientific experimentation; although interestingly—but perhaps unsurprisingly in light of our discussion thus far, very few use Hacking’s nomenclature of observation/experiment. Like Gooding and Chang, most believe that scientific experimentation should be viewed as a continuous process rather than one entailing discrete parts—and the terminology used underlines this sense of continuousness. Friedrich Steinle and Richard Burian have coined the term ‘exploratory experimentation’ which gives the same sense of the continuousness of the experimental process as do Chang and Gooding in their work: Steinle working on the early history of electromagnetism (Steinle 1997 , 2002 ), and Burian in his work on molecular biology (Burian 1997 , 2007 ). Footnote 42

Steinle, in analysing the experimental work of Oersted, Ampere and Faraday draws a distinction between two kinds of experiments: those designed with the specific aim of tracing particular effects which were expected because of the field of knowledge within electromagnetism at the time, and those experiments set up where the investigators had, what Steinle calls, “no theory—or—even more fundamentally—no conceptual framework” (Steinle 1997 , S65). Richard Burian, too, has used the term ‘exploratory experimentation’. Burian first used the term in his analysis of the work of Jean Brachet’s experiments on the localization and functioning of nucleic acids (Burian 1997 ). Burian examines the research of Brachet’s on the distribution of nucleic acids across cell life cycles. Burian shows that Brachet was not guided by theoretical considerations about how the nucleic acids may be distributed across the lifetime of cells in various organisms. This was very much in contrast to Brachet’s peer, Francis Crick, working on the same subject matter, who was much more theoretically inclined which greatly influenced the kinds of experiments he chose to undertake (ibid., 40–41). Burian therefore also uses the term in the same sense as Steinle insofar as to distinguish a particular kind of experimentation from theory. Since this inception, ‘exploratory experimentation’ has gradually gained more definitive structure: for example, it is clear that it is not the case that exploratory experimentation is free from theory—rather, the question is how theory influences the experimental process thus leading to a distinction between ‘theory-directed’ and ‘theory-informed’ (Waters 2007 , 277); leading to calls for the creation of its own sub-structure that can account for historical cases more adequately than it does in its present form (O’Malley 2007 ).

Another term, ‘experimental system’, has been used within writing on epistemology of experimentation, which conveys the same sense of ‘continuousness’ as does exploratory experimentation. The term ‘experimental system’ was first used by Hans-Jörg Rheinberger to refer to the experimental research on protein synthesis ( 1997 ). Rheinberger describes experimental systems as “systems of manipulation designed to give unknown answers to questions that the experimenters themselves are not yet clearly to ask” (ibid., 28). As with the term ‘exploratory experimentation’, Rheinberger uses ‘experimental system’ in order to distinguish it from a theory-dominated approach, arguing that experimental work in biology always “begins with the choice of a system rather than with the choice of theoretical framework” (ibid., 25). Other similar terms to experimental systems have been used to indicate scientific experimentation as a process with the sense of continuousness embedded at their centre such as ‘manipulable systems’ (Turnball and Stokes 1990 ) and ‘production systems’ (Kohler 1991 ).

All these terms (exploratory experimentation, experimental system, manipulable system, production system) and their respective accounts emerge with the aim of distinguishing them from theory-dominated accounts such as hypothesis testing. None of these accounts seek to do what Hacking does with his stipulation of experiment: distinguish between different kinds of activities and interventions within the process of scientific experimentation. Hasok Chang’s aim in delineating the various kinds of activities and interventions involved in scientific practice uses a descriptive rather than a stipulative approach (Chang 2011 ). James Woodward uses the terms observation and experiment as distinct categories but with the aim of defending what he calls a “manipulationist account of causation” rather than in an attempt to delineate scientific experimentation as process (Woodward 2003 , 88). Footnote 43

However, elsewhere, James Woodward, together with James Bogen, has sought to put forward an account which seeks to specifically delineate the process of scientific experimentation. It completely abandons the vocabulary of observation and experiment and uses data and phenomena instead. Footnote 44 Bogen and Woodward initially put forward their data phenomena account in 1988 (Bogen and Woodward 1988 ). Footnote 45

Bogen and Woodward tell us that data should be thought of as that which provides evidence for the existence of phenomena (Bogen and Woodward 1988 , 305). Data can (usually) be detected. However, data (usually) cannot be predicted. Phenomena, on the other hand, can only be detected through the use of data (Bogen and Woodward 1988 , 306). Examples of data include bubble chamber photographs, patterns of discharge in electronic particle detectors and records of reaction times and error rates in psychological experiments. These instances of data provide evidence for the following phenomena respectively: weak neutral currents, decay of the proton and chunking effects in human short-term memory.

Bogen and Woodward analyse a number of examples to illustrate the distinction between data and phenomena (Bogen and Woodward 1988 , 308–322). Examples they use to show what they mean include the melting point of lead (from chemistry) and weak neutral currents (from physics).

Bogen and Woodward analyse the following statement about the melting point of lead to show what they mean by their data phenomenon distinction: ‘lead melts at 327.5 ± 0.1 degrees centigrade’. However, this is not what actually happens. It is not possible to determine the melting point of lead by taking a single thermometer reading. Footnote 46 It is necessary to take a series of measurements. Even if systematic errors are reduced, there will be variations in the thermometer readings such as to give a scatter of results that all differ from each other, even if potential sources of error are minimized. The figure 327.5 represents the mean of the scatter of thermometer readings while the figure 0.1 represents the standard deviation.

Within Bogen and Woodward’s account, the thermometer readings fall within the category data while the calculated melting point, 327.5 degree centigrade fall within the category phenomena. It is the latter, phenomena, which becomes the object of systematic scientific explanation. Thus, in the case of the melting point of lead, the figure 327.5 degree centigrade becomes the object of explanation in terms of the molecular structure of metals. This would be expressed in terms such as metallic bonding mechanisms and type of co-ordination.

The data too can become the object of scientific explanation. However, the terms in which explanations regarding data would be made would be different from those made for phenomena. Explanations related to data would include discussion of the accuracy of the thermometer, the purity of the lead sample used, the point at which the thermometer is taken (when the sample of lead starts to melt, at mid-way, when the sample has all melted), the reliability of the heating mechanism and such like. These terms and considerations are very different to those related to discussions in terms of molecular structure. In Bogen and Woodward’s account, thus, data are distinguished from phenomena by the fact that the terms in which phenomena are explained are distinct from the terms in which data are explained.

Another difference between data and phenomena in Bogen and Woodward’s account relates to phenomena possessing regular characteristics, which are detectable from very different kinds of data (or evidence). Bogen and Woodward use the following example to show what they mean.

The evidence for the existence of the phenomenon of weak neutral currents came from two different kinds of investigations. One was at CERN in Switzerland and the other at the NAL (National Accelerator Laboratory) in the US. The data from CERN comprised of bubble chamber photographs (where the detection method depended on the formation of bubbles) while that from NAL consisted of patterns of discharge in particle detectors (where the detection method registered the passage of charged tracks by electronic means). These two very different kinds of data—from very different kinds of apparatus—provided the evidence for the same phenomenon: the weak neutral current.

The terms of explanation for the phenomenon, the weak neutral current, comprise the interaction of the Z particle with the weak force—this is common in both cases: from the data from CERN as well as the very different data from the NAL. However, the terms of explanation of the two different data sets have very little in common: the data set from CERN comprises of terms consisting of, inter alia, the nature of the neutron beam, the shielding chamber, the size of the chamber as well as the type of liquid used in the chamber. The data from the NAL, however, required explanation in terms of, inter alia, the strength of the magnetic field, the characteristics of the calorimeter used to stop, absorb and measure a particle’s energy, and the nature of the tracking device.

For Bogen and Woodward, phenomena are “in the world, as belonging to the natural order itself and not just to the way we talk about or conceptualize that order” ( 1988 , 321). This may include, “particular objects, objects with features, events, processes, and states” (Bogen and Woodward 1988 , 321). To Bogen and Woodward, the key feature of phenomena is that they be the objects of general scientific explanation, rather than the particular explanations, which are the characteristic feature of data, and from which they are distinct ( 1988 , 322). Data are highly localized and idiosyncratic and demand explanations that are framed in very different terms to that of phenomena for which they act as evidence (Bogen and Woodward 1988 , 319).

Mapping the data phenomena account onto the cases cited earlier would thus yield the following outcomes. For Beaumont, the data relates to the results of digestion from both the in vivo and in vitro parts of his investigation and the explanatory terms in which they are framed relate to degrees of acidity, temperature readings and measurements of time, while the terms in which the explanatory terms for the phenomena are framed include peristaltic movement, the anatomy and composition of gastric cell types and the physical topography of the stomach with respect to the rest of the gastrointestinal tract. For Kettlewell, knowledge about data would relate to what kind of moth is conspicuous on which colour bark, the numbers of different kinds of moths surviving exposure to predation in the cage, what kind of bait is used to trap surviving moths in native conditions. The phenomenon is accounted for by discussion in terms of the changing colour of the landscape owing to pollution and degrees of conspicuousness to predators. For Harlow, the data would be framed in terms such as time spent with each kind of surrogate while discussion of phenomena would be framed in terms of emotional bonding, cognitive support and imitation along with the need for physical contact with an animate like material. For Maskelyne, the data would be framed in terms of arc minutes for the astronomical measurements and feet/inches for the physical survey of the mountain while the phenomena concerned (density of the earth) was expressed in terms of a numerical value (4500 kg/m 3 ). For Hutton, the data was framed in terms of chemical, temperature and field measurements and anatomical differentiation in fossil records while the phenomena was framed in terms of soil erosion and the influence of the physical elements (wind, water) on this erosion as indicative of changing climate and its correlation with fossil deposits. For Millikan, the data would be framed in terms of time taken for an oil drop to travel a distance of 10.21 mm, the viscosity of the oil, the temperature of the cloud chamber while the phenomena was framed in terms of the discrete nature of the charge carried by an electron, its interactions with other parts of the atom, the nature of these interactions and the value of the charge itself (1.5924 × 10 −19  C).

Although both Hacking and Bogen and Woodward aim, in each of their accounts of delineating scientific experimentation, to use a criteria based approach, the criteria they use are very different.

It is worth noting some points of conceptual overlap, as well as departure, between the two accounts—notwithstanding the different lexicon of each. There is considerable congruence between Hacking’s category of observation (or results(s) of observation) and Bogen and Woodward’s category of data—the outcomes when mapping each account onto the mentioned cases makes this clear: Beaumont’s time measurements for digestion, Kettlewell’s survival rates, Harlow’s record of time spent with each kind of surrogate, Maskelyne’s astronomical and survey measurements in arc minutes and units of height, Hutton’s complex and varied array of temperature, chemical, maps and diagrammatic records and Millikan’s measurements of time taken for an oil drop to traverse 10.21 mm. It is when we turn to consider the second part of each account—Hacking’s experiment (creation of phenomena) and Bogen and Woodward’s phenomena—that the conceptual overlap starts to dissipate. At first glance, both use phenomena in a similar way. For Hacking, it is a discernable regularity, “[a] phenomenon is noteworthy . A phenomenon is discernable . A phenomenon is commonly an event or process of certain type that occurs regularly under defined circumstances” (Hacking 1983 , 221, emphasis in original). For Bogen and Woodward, a phenomenon has “stable, repeatable characteristics which will be detectable by means of a variety of different procedures, which may yield quite different kinds of data” (Bogen and Woodward 1988 , 317). However, the common use of vocabulary—both in terms of naming and description—should not prevent us from noting the different ways each is conceptualized in each different account—as Bogen and Woodward themselves note (ibid, 306), suggesting that although there are certain similarities between their notion of phenomena and that of Hacking’s, they find Hacking’s notion limited insofar as Hacking’s notion “is not correct as a general characterization of phenomena” and continue on to say that the “features which Hacking ascribes to phenomena are more characteristic of data”. Others too have noted the ambiguity in Hacking’s description of the relationship between experiment and phenomena. Footnote 47

Bogen and Woodward here identify the principal limitation in Hacking’s observation versus experiment account as a means for (systematically) delineating scientific experimentation as practice: in the observation versus experiment account (the results of) observation and experiment both are ways of generating data. It is therefore perhaps not surprising that earlier we saw quite anomalous and ambiguous outcomes on mapping Hacking’s account on to cases of scientific experimentation from a range of fields of enquiry.

Bogen and Woodward use an explanation,—or what Rheinberger has called ‘epistemic object’—based, Footnote 48 criteria. Hacking, however, uses a narrowly construed criteria, centred on (kinds of) action/activity/intervention in his stipulation of experiment as ‘creation of phenomena’. This stipulative approach, as we have seen, has limited value when used in practice across a whole range of fields of enquiry.

5 Concluding Remarks

We have seen from our discussion that the observation versus experiment account has significant weaknesses as a means of delineating scientific experimentation within scientific practice—across a range of cases from various fields of scientific enquiry. This would suggest that the experiment versus observation framework—where observation and experiment are cast as polarities, rather than as complements of each other—as Hooke and Boyle did—is not a sound basis on which to make value judgments.

See Desmond Lee's Introduction to his translation of Aristotle's Meteorologica .

See ‘Introduction' by Lorraine Daston and Elizabeth Lunbeck in Histories of Scientific Observation ; in particular page 3.

This, of course, belies the considerable academic scholarship by historians of science that exists on the nature and characteristics of scientific practice, in particular, scientific experimentation, in pre-modern cultures that has shown the significant limitations of this position; Greek, Latin, Arabic and Chinese to name just a few. See Lloyd ( 2004 , 2006 ) on Greek and Chinese science and references therein. For Arabic science, see Sabra ( 1996 ). For Latin, see Lindberg ( 2007 ). For an example from the exact sciences, see the case of geometrical optics: for Greek, see Smith ( 1996 ), and for Arabic, see Sabra ( 2003 ). For the case of medicine, see Pormann and Savage-Smith ( 2007 ).

See Hacking ( 1983 , 173).

In particular, see footnote 12 in Pomata ( 2011 ).

See Park ( 2011 , 15–44), Pomata ( 2011 , 45–80) and Daston ( 2011 , 81–113).

See pp. 148–149 in particular.

See also Schickore ( 2007 ) for the case of the microscope. Also see Daston and Galison ( 2007 ).

Those who have been interested in the detailed historical accounts of particular experiments include Galison ( 1982 , 1983 ), Pickering ( 1981 ), Gooding ( 1982 ), Worral ( 1982 ), Wheaton ( 1983 ), Stuewer ( 1975 ) and Franklin ( 1986 ). Others have been concerned with the role of experiment in knowledge acquisition such as Gooding ( 2000 ), Kuhn ( 1976 ), Dear ( 1995 ) and Tiles ( 1993 ). Some have been interested in the philosophy of scientific experimentation (Radder 2003a , b ) which takes into account the nexus that experimentation provides for the meeting of theory, technology and modelling amongst others. Others have been concerned with the relationship between theory, observing and experimentation such as Latour and Woolgar ( 1986 ), Collins ( 1985 ), Galison ( 1987 ), Bogen and Woodward ( 1988 ) and Rheinberger ( 1997 ). Philsophers of science interested in observation include Shapere ( 1982 ) and Fodor ( 1983 ).

See Radder ( 2003b , 15) and Gooding ( 1992 , 68).

Hacking's primary aim in Representing and Intervening (Hacking 1983 ), however, lies in the juxtaposition of experiment to theory rather than an analysis of experiment relative to observation per se. Although Hacking takes up the subject of experiment again in some of his later work, there he is more concerned with other matters. He deals with the anti-realist position (see Hacking 1989 , for a response, see Shapere 1993 ) or with trying to defend the stability of laboratory practice (see Hacking 1992 ).

Also see Pinch ( 1985 ).

Brigitte Falkenburg has proposed that this position has limited value as theories of entities such as neutrinos, their detectors and the way information is transmitted from the source are all inextricably linked (see Falkenburg 2000 ).

For a detailed explanation see Galison ( 1985 ).

The Compton effect refers to the scattering of X-rays by electrons in work done by Arthur Compton in the 1920s.

The Zeeman effect refers to the splitting of the energy levels of an atom when it is placed in a magnetic field. Pieter Zeeman and Hendrik Lorentz did this work in the 1890s.

The photoelectric effect refers to the detection of a current when light is shone on some metals and is taken as an indication of the emission of electrons.

Many substances act as superconductors at temperatures near to absolute zero. Brian Josephson (in 1962) predicted that a weak current (subsequently named a ‘super-current’) would flow between two superconductors that were separated by a thin sheet of electrical insulation. Philip Anderson and John Rowell confirmed Josephson’s prediction a year later in 1963.

Dr. Beauchamp should read Dr. Beaumont (see Bernard 1957 , 8). The work was conducted during the 1820s, not a decade earlier as stated (see Bernard 1957 , 8).

See Beaumont ( 1833 ).

Such as neurological processes which control the mechanical and nerve impulse activities of the stomach.

In particular, see p. 82.

For a synopsis of Henry Kettlewell's study on moths, see Franklin ( 2012 ). Kettlewell's work was published in Heredity ( 1955 , 1956 , 1958 ). David Rudge has worked extensively on the history of Kettlewell's work, see Rudge ( 2005a , b , 2006 , 2009 , 2010 ). He has also dealt extensively with the issue of statistical error in Kettlewell's numerical analysis (Rudge 2001 , 2005a , b ) and the issue of validity of control experiments ( 1999 ) in which he deals in particular with Joel Hagen's critique of Kettlewell's use of controls (Hagen 1999 ); for an overview of the issue of the use of controls on Kettlewell's experiments, see Brandon ( 1999 ). The validity of the controls Kettlewell used relate to the geographical areas in which he performed the experiments (Birmingham, UK and Dorset, UK).

‘Attachment theory' relates to the notion that non-material provision from a (primary) carer is significant in the cognitive formation and development of higher mammals.

See Prior and Glaser ( 2006 ), Ainsworth ( 1991 ), Blum ( 2002 ); see a review of the latter at: http://primate.uchicago.edu/2004PC.pdf (accessed 8 Mar 2015).

See Chapters 11–15. See also Smallwood ( 2009 ).

See the edition of Andrew Motte's translation of Newton's Principia: The Mathematical Principles of Natural Philosophy , pp. 527–528.

Also see Repcheck ( 2004 ). For reception of Hutton's work amongst his contemporaries, see Dean ( 1973 ). For a synopsis of Hutton's biography see his entry in the Dictionary of Scientific Biography .

See also Rudwick ( 1985 , 2004 , 2005b ).

A visitor to Hutton's home in Edinburgh remarked, “his study is so full of fossils and chemical apparatus of various kinds that there is barely room to sit down”.

See also Franklin ( 1981 ), Barnes et al. ( 1996 ), and Goodstein ( 2001 ). Also see Niaz ( 2005 ) for an appraisal of the studies of Holton, Franklin, Barnes et al. and Goodstein.

Also see Franklin ( 1986 , 215–224).

See in particular pp. 124–125.

Millikan's conclusions were contested amongst specialists in the field for more than a decade after publication of this work; see Holton ( 1978 ) in particular; for a defence of Millikan, see Goodstein ( 2001 ).

In defence of Hacking, his principal aim in Representing and Intervening in making his observation experiment distinction is in service of other philosophical ends such as entity realism. Further, within its own time, Hacking's drawing of a polarity between observation and experiment served the purpose of challenging the hitherto identification of experiment with observation (as a perceptual rather than a detection form). One may therefore reasonably posit that the criteria Hacking puts forward as his description of experiment should not be applied rigidly. However, I believe he appears quite committed to his stipulation of experiment as ‘creation of phenomena' in a formalistic way—he emphasizes the ‘creation' part of ‘creation of phenomena' in his discussion at length explicitly and reiterates this commitment by the examples with which he chooses to engage at length—certain kinds of cases from physics (Hall effect, Josephson effect) while consciously stepping away from others such as those such as the work of William Beaumont which, as we see above, are not easily receptive to the observation versus experiment account. In addition, Hacking underlines his commitment to ‘creation' in his description of experiment as ‘creation of phenomena' as well as ‘purity' of said in his later work (Hacking 1992 , 37; here Hacking uses the photoelectric effect as an exemplar). I think therefore it is not unreasonable to take Hacking at his own (repeated) word. If one does do that, then it appears from our discussion that Hacking's stipulation of experiment as ‘creation of phenomena', and his emphasis on ‘pure state', gives rise to anomalies in a range of cases of scientific practices as shown.

Hans Radder uses ‘scientific experimentation' in a way which reflects the importance of the processual nature of scientific work and practices, (Radder 2003b , 15).

Hacking too has used the term ‘experimentation', ‘Experimentation has many lives of its own' ( 1983 , 165). However, Hacking uses ‘experimentation' in contrast to theory, saying “…, let us not pretend that the various phenomenological laws of solid state physics required a theory—any theory—before they were known. Experimentation has many lives of its own” (ibid.). In contrast, David Gooding uses ‘experimentation' as a process qua process.

Also see Rouse (1996).

This case has been analyzed in detail by Shapere ( 1982 ) and Pinch ( 1985 ) as well as dealt with in summary by Bogen and Woodward ( 1988 , 316).

Others too have noted the ambiguities arising out of the very particular way Hacking stipulates his category of experiment, See Feest ( 2011 , 63–64).

See earlier references to each of these authors.

Rose-Mary Sargent too uses the term but for descriptive rather than analytical purposes (Steinle 1997 , S71).

See also Woodward ( 2013 ) for a review of the topic where he deals with the different positions on the subject matter, including his own.

In so doing, they avoid linguistic oddities such as, ‘What has been shown as well is that, in actual practice, making scientific observations often includes doing genuine experiments' (Radder 2003b , 15).

Since then it has been re-stated by Woodward on a number of occasions (Woodward 1989 , 2000 , 2011 ). In these revised versions, however, Woodward has been more concerned with dealing with the relationship between this account and its relationship with scientific theory. The data phenomena account has been contested on various grounds. These contestations have tended to focus on two areas. First, whether it is reasonable to draw a distinction between data and phenomena at all whereas both should be viewed as patterns within data sets (Glymour 2000 ). Further, even if one were to draw a distinction between them, how one does this—in particular, the role of assumptions in this process (McAllister 1997 ). Woodward ( 2011 , 175–176) amongst others (Apel 2011 , 27–31), have responded to these points in recent years. The other area of focus has been the relationship of data and phenomena within Bogen and Woodward's account to theory (Schindler 2007 , 2011 ); in particular, as it relates to the influence of theory on observation and its implications for reliability. Woodward counters this view in detail (Woodward 2011 , 172–174) by suggesting that the charge that data in the data phenomena account is assumed to be independent of ‘additional assumptions' or ‘theory free' is unfounded. However, where used to delineate scientific practice qua practice, it appears reasonably robust—as even its detractors concede (Schindler 2011 , 54).

For details of how the melting point of lead is determined under laboratory conditions, see Bogen and Woodward ( 1988 , 309–310).

See Feest ( 2011 , 63–64).

See Rheinberger ( 1997 , 28).

Ainsworth, M. D. S. (1991). Attachments and other affectional bonds across the life-cycle. In B. Caldwell & H. Riccuiti (Eds.), Review of child development research (pp. 1–94). Chicago: The University of Chicago Press.

Google Scholar  

Anstey, P. (2014). Philosophy of experiment in early modern England: The case of Bacon, Boyle and Hooke. Early Science and Medicine, 19 , 103–132.

Article   Google Scholar  

Apel, J. (2011). On the meaning and the epistemological relevance of the notion of scientific phenomena. Synthese, 182 , 23–38.

Aristotle. (1962). Meteorologica . Introduction and Translation by H. D. P. Lee. London: Heinemann, Loeb Classical Library Series.

Barnes, B., Bloor, D., & Henry, J. (1996). Scientific knowledge: A sociological analysis . Chicago: The University of Chicago Press.

Beaumont, W. (1833). Experiments and observations on the gastric juice and the physiology of digestion . Boston: F. P. Allen.

Bernard, C. (1957). An introduction to the study of experimental medicine . New York: Dover Publications Inc.

Blum, D. (2002). Love at Goon Park: Harry Harlow and the science of affection . Cambridge, MA: Perseus.

Bogen, J., & Woodward, J. (1988). Saving the phenomena. The Philosophical Review, 97 , 303–352.

Brandon, R. N. (1999). Introduction. Biology and Philosophy, 14 , 1–7.

Burian, R. (1997). Exploratory experimentation and the role of histochemical techniques in the work of Jean Brachet, 1938–1952. History and Philosophy of the Life Sciences, 19 , 27–45.

Burian, R. (2007). On micro RNA and the need for exploratory experimentation in post-genomic molecular biology. History and Philosophy of the Life Sciences, 29 , 285–312.

Chang, H. (2011). The philosophical grammar of scientific practice. International Studies in the Philosophy of Science, 25 , 205–221.

Collins, H. (1985). Changing order . Chicago: The University of Chicago Press.

Crary, J. (1992). Techniques of the observer: On vision and modernity in the nineteenth century . Cambridge, MA: MIT Press.

Crary, J. (2001). Suspensions of perception: Attention spectacle and modern culture . Cambridge, MA: MIT Press.

Danson, E. (2009). Weighing the world: The quest to measure the Earth . Oxford: Oxford University Press.

Daston, L. (2011). The empire of observation. In L. Daston & E. Lunbeck (Eds.), Histories of scientific observation (pp. 81–113). Chicago: The University of Chicago Press.

Daston, L., & Galison, P. (2007). Objectivity . New York: Zone Books.

Daston, L., & Lunbeck, E. (2011). Introduction. In L. Daston & E. Lunbeck (Eds.), Histories of scientific observation (pp. 1–9). Chicago: The University of Chicago Press.

Dean, D. R. (1973). James Hutton and his public, 1785–1802. Annals of Science, 30 , 89–105.

Dean, D. R. (1992). James Hutton and the history of geology . Ithaca: Cornell University Press.

Dear, P. (1995). Discipline and experience: The mathematical way in the scientific revolution . Chicago: The University of Chicago Press.

Book   Google Scholar  

Falkenburg, B. (2000). How to observe quarks. In E. Agazzi & M. Pauri (Eds.), The reality of the unobservable (pp. 329–341). Dordrecht: Kluwer.

Chapter   Google Scholar  

Feest, U. (2011). What exactly is stabilized when phenomena are stabilized? Synthese, 182 , 57–71.

Fodor, J. (1983). Observation reconsidered. Philosophy of Science, 51 , 23–43.

Franklin, A. D. (1981). Millikan’s published and unpublished data on oil drops. Historical Studies in the Physical Sciences, 11 , 185–201.

Franklin, A. (1986). The neglect of experiment . Cambridge: Cambridge University Press.

Franklin, A. (2012). Experiment in physics. In Stanford encyclopedia of philosophy . http://plato.stanford.edu . accessed on 14 May 2014.

Galison, P. (1982). Theoretical predispositions in experimental physics: Einstein and the gyromagnetic experiments, 1915–1925. Historical Studies in the Physical Sciences, 12 , 285–323.

Galison, P. (1983). How the first neutral current experiments ended. Reviews of Modern Physics, 55 , 477–509.

Galison, P. (1985). Bubble chambers and the experimental workplace. In P. Achinstein & O. Hannaway (Eds.), Observation, experiment and physical science (pp. 309–373). Cambridge, MA: MIT Press.

Galison, P. (1987). How experiments end . Chicago: The University of Chicago Press.

Glymour, B. (2000). Data and phenomena: A distinction reconsidered. Erkenntis, 52 , 29–37.

Gooding, D. (1982). Empiricism in practice: Teleology, economy, and observation in Faraday’s physics. Isis, 73 , 46–67.

Gooding, D. (1990). Experiment and the making of meaning . Boston: Kluwer Academic.

Gooding, D. (1992). Putting agency back into experiment. In A. Pickering (Ed.), Science as practice and culture (pp. 65–112). Chicago: The University of Chicago Press.

Gooding, D. (2000). Experiment. In W. H. Newton-Smith (Ed.), A companion to the philosophy of science (pp. 117–126). Oxford: Oxford University Press.

Goodstein, D. (2001). In defence of Robert Andrews Millikan. American Scientist, 89 , 54–60.

Hacking, I. (1983). Representing and intervening . Cambridge: Cambridge University Press.

Hacking, I. (1989). Extragalactic reality: The case of gravitational lensing. Philosophy of Science, 56 , 555–581.

Hacking, I. (1992). The self-vindication of the laboratory sciences. In A. Pickering (Ed.), Science as practice and culture (pp. 29–64). Chicago: The University of Chicago Press.

Hagen, J. B. (1999). Retelling experiments: H. B. D. Kettlewell’s studies of industrial melanism in peppered moths. Biology and Philosophy, 14 , 39–54.

Hanson, N. R. (1958). Patterns of discovery . Cambridge: Cambridge University Press.

Hoffmann, C. (2006). Unter Beobachtung: Naturforschung in der Zeit der Sinnesapparate . Gottingen: Wallstein.

Holton, G. (1978). Subelectrons, presuppositions, and the Millikan–Ehrenhaft dispute. Historical Studies in the Physical Sciences, 9 , 161–224.

Kettlewell, H. (1955). Selection experiments on industrial melanism in the Lepidoptera. Heredity, 9 , 323–342.

Kettlewell, H. (1956). Further selection experiments on industrial melanism in the Lepidoptera. Heredity, 10 , 287–301.

Kettlewell, H. (1958). A survey of the frequencies of Biston betularia (L.) and its melanic forms in Great Britain. Heredity, 12 , 51–72.

Kohler, R. (1991). Systems of production: Drosophila, neurospora and biochemical genetics. Historical Studies in the Physical and Biological Sciences, 22 , 87–130.

Kuhn, T. S. (1976). Mathematical vs. experimental traditions in the development of physical sciences. Journal of Interdisciplinary History, VII (I), 1–31.

Latour, B., & Woolgar, S. (1986). Laboratory life: The construction of scientific facts . Princeton: Princeton University Press.

Lindberg, D. C. (2007). The beginnings of western science . Chicago: The University of Chicago Press.

Lloyd, G. E. R. (2004). Ancient worlds, modern reflections: Philosophical perspectives on Greek and Chinese science and culture . Oxford: Oxford University Press.

Lloyd, G. E. R. (2006). Principles and practices in ancient Greek and Chinese science . Aldershot: Variorum.

Maxwell, G. (1962). The ontological status of theoretical entities. In H. Feigi & G. Maxwell (Eds.), Scientific explanation, space and time: Minnesota studies in the philosophy of science (pp. 181–192). Minnesota: University of Minnesota Press.

McAllister, J. (1997). Phenomena and patterns in data sets. Erkenntis, 47 , 217–228.

Millikan, R. A. (1913). On the elementary charge and the Avogadro constant. The Physical Review, II , 109–143.

Newton, I. (1846). Newton’s principia. The mathematical principles of natural philosophy . Trans. A. Motte. New York: Daniel Adee.

Niaz, M. (2005). An appraisal of the controversial nature of the oil drop experiment: Is closure possible? British Journal for the Philosophy of Science, 56 , 681–702.

O’Malley, M. A. (2007). Exploratory experimentation and scientific practice: metagenomics and the proteorhodopsin case. History and Philosophy of the Life Sciences, 29 , 337–360.

Park, K. (2011). Observation in the margins, 500–1500. In L. Daston & E. Lunbeck (Eds.), Histories of scientific observation (pp. 15–44). Chicago: The University of Chicago Press.

Pickering, A. (1981). The hunting of the quark. Isis, 72 , 216–236.

Pinch, T. (1985). Towards an analysis of scientific observation: The externality and evidential significance of observational reports in physics. Social Studies of Science, 15 , 3–36.

Pomata, G. (2011). Observation rising: Birth of an epistemic genre, 1500–1650. In L. Daston & E. Lunbeck (Eds.), Histories of scientific observation (pp. 45–80). Chicago: The University of Chicago Press.

Pormann, P., & Savage-Smith, E. (2007). Medieval Islamic medicine . Washington D.C.: Georgetown University Press.

Prior, V., & Glaser, D. (2006). Understanding attachment and attachment disorders: Theory, evidence and practice . London: Jessica Kingsley.

Radder, H. (Ed.). (2003a). The philosophy of scientific experimentation . Pittsburgh: University of Pittsburgh Press.

Radder, H. (2003b). Towards a more developed philosophy of scientific experimentation. In H. Radder (Ed.), The philosophy of scientific experimentation (pp. 1–18). Pittsburgh: University of Pittsburgh Press.

Repcheck, J. (2004). The man who found time . London: Perseus.

Rheinberger, H.-J. (1997). Towards a history of epistemic things: Synthesizing proteins in the test tube . Stanford: Stanford University Press.

Rudge, D. W. (1999). Taking the peppered moth with a grain of salt. Biology and Philosophy, 14 , 9–37.

Rudge, D. W. (2001). Kettlewell from an Error Statistician’s Point of View. Perspectives on Science, 9 , 59–77.

Rudge, D. W. (2005a). Did Kettlewell commit Fraud? Re-examining the evidence. Public Understanding of Science, 14 , 249–268.

Rudge, D. W. (2005b). The beauty of Kettlewell’s classic experimental demonstration of natural selection. Bio-Science, 55 , 369–375.

Rudge, D. W. (2006). H. D. P. Kettlewell’s research 1937–1953: The influence of E. B. Ford, E. A. Cockayne and P. M. Sheppard. History and Philosophy of the Life Sciences, 28 , 359–388.

Rudge, D. W. (2009). H. B. D. Kettlewell’s research 1934–1961: The influence of J. W. Heslop Harrison. Transactions of the American Philosophical Society, 99 , 243–270.

Rudge, D. W. (2010). Tut-tut tutt, not so fast: Did Kettlewell really test Tutt’s explanation of industrial melanism? History and Philosophy of the Life Sciences, 28 , 359–388.

Rudwick, M. J. S. (1985). The Great Devonian controversy: The shaping of scientific knowledge among gentlemanly specialists . Chicago: The University of Chicago Press.

Rudwick, M. J. S. (2004). The new science of geology: Studies in the earth sciences in the age of revolution . London: Ashgate.

Rudwick, M. J. S. (2005a). Lyell and Darwin, geologists: Studies in the earth sciences in the age of reform . London: Ashgate.

Rudwick, M. J. S. (2005b). Bursting the limits of time: the reconstruction of geohistory in the age of revolution . Chicago: The University of Chicago Press.

Sabra, A. I. (1996). Situating Arabic science: Locality versus essence. Isis, 87 , 654–670.

Sabra, A. I. (2003). Ibn al-Haytham’s revolutionary project in optics: The achievement and the obstacle. In J. P. Hogendijk & A. I. Sabra (Eds.), The enterprise of science in Islam (pp. 85–118). Boston, MA: The MIT Press.

Schickore, J. (2007). The microscope and the eye: A history of reflections, 1740–1870 . Chicago: The University of Chicago Press.

Schindler, S. (2007). Rehabilitating theory: Refusal of the ‘bottom-up’ construction of scientific phenomena. Studies in the History and Philosophy of Science, 38 , 160–184.

Schindler, S. (2011). Bogen and Wooodward’s data-phenomena distinction, forms of theory-ladenness, and the reliability of data. Synthese, 182 , 39–55.

Shapere, D. (1982). The concept of observation in science and philosophy. Philosophy of Science, 49 , 485–525.

Shapere, D. (1993). Astronomy and antirealism. Philosophy of Science, 60 , 134–150.

Smallwood, J. R. (2009). John Playfair on Schiehallion, 1801–1811. In C. L. E. Lewis & S. J. Knell (Eds.), The making of the Geological Society of London (pp. 279–298). London: The Geological Society.

Smith, A. M. (1996). Ptolemy’s theory of visual perception . Philadelphia: The American Philosophical Society.

Steinle, F. (1997). Entering new fields: Exploratory uses of experimentation. Philosophy of Science (Proceedings), 64 , S65–S74.

Steinle, F. (2002). Experiments in history and philosophy of science. Perspectives on Science, 10 , 408–432.

Stuewer, R. (1975). The Compton effect . New York: Science History Publications.

Tiles, J. E. (1993). Experiment as intervention. British Journal for the Philosophy of Science, 44 , 463–475.

Turnball, D., & Stokes, T. (1990). Manipulable systems and laboratory strategies in a biomedical institute. In H. E. Le Grand (Ed.), Experimental enquiries (pp. 167–192). Dordrecht: Kluwer.

van Fraassen, B. C. (1980). The scientific image . Oxford: Oxford University Press.

Waters, C. K. (2007). The nature and context of exploratory experimentation: An introduction to three case studies of exploratory research. History and Philosophy of the Life Sciences, 29 , 275–284.

Wheaton, B. (1983). The tiger and the shark . Cambridge: Cambridge University Press.

Woodward, J. (1989). Data and phenomena. Synthese, 79 , 393–472.

Woodward, J. (2000). Data, phenomena, and reliability. Philosophy of Science, 67 , S163–S179.

Woodward, J. (2003). Experimentation, causal inference, and instrumental realism. In H. Radder (Ed.), The philosophy of scientific experimentation (pp. 87–118). Pittsburg: University of Pittsburg Press.

Woodward, J. (2011). Data and phenomena: A restatement and defense. Synthese, 182 , 165–179.

Woodward, J. (2013). Causation and manipulability. In E. N. Zalta (Ed) The Stanford encyclopedia of philosophy . http://plato.stanford.edu/archives/win2013/entries/causation-mani/ .

Worral, J. (1982). The pressure of light: The strange case of the vacillating crucial experiment. Studies in History and Philosophy of Science, 13 , 133–171.

Download references

Acknowledgments

I would like to thank Emilie Savage-Smith and Nick Jardine for their comments during the early gestation of this work. I am also very grateful to the anonymous referees, as well as the Editors, for their very helpful comments during review.

Author information

Authors and affiliations.

Cardiff University, Cardiff, CF10 3EU, UK

Saira Malik

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Saira Malik .

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Malik, S. Observation Versus Experiment: An Adequate Framework for Analysing Scientific Experimentation?. J Gen Philos Sci 48 , 71–95 (2017). https://doi.org/10.1007/s10838-016-9335-y

Download citation

Published : 07 May 2016

Issue Date : March 2017

DOI : https://doi.org/10.1007/s10838-016-9335-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Philosophy of experiment
• Ian Hacking
• Observation
• Scientific experimentation
• Scientific practice
• Find a journal
• Publish with us
• Track your research