experimental philosophy and qualitative research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Taylor & Francis Open Select

Qualitative tools and experimental philosophy

James andow.

a Department of Philosophy , University of Reading , Reading, UK

Experimental philosophy brings empirical methods to philosophy. These methods are used to probe how people think about philosophically interesting things such as knowledge, morality, and freedom. This paper explores the contribution that qualitative methods have to make in this enterprise. I argue that qualitative methods have the potential to make a much greater contribution than they have so far. Along the way, I acknowledge a few types of resistance that proponents of qualitative methods in experimental philosophy might encounter, and provide reasons to think they are ill-founded.

Section 1 introduces experimental philosophy and outlines experimental philosophers’ current methods. Section 2 provides a basic introduction to qualitative methods, data gathering, and analysis and articulates how such methods might contribute to experimental philosophy. Section 3 articulates a major potential line of resistance to the incorporation of qualitative methods which focuses on the idea that experimental philosophers are interested in investigating particular types of mental processing— intuitive processing—for which a qualitative methodology would be a poor resource. Section 4 responds to this objection by examining the basic motivations for experimental philosophy and the various ways in which it has been claimed that empirical data can make an important philosophical contribution, finding no justification for a near exclusive focus on intuitive processing. Section 5 then deals with a number of other objections and clarifies my recommendations before I conclude in section 6.

1.  Experimental Philosophy

Experimental philosophy is a new sub-discipline of philosophy. Experimental philosophers aim to make philosophical contributions by using empirical tools to probe how people (typically ordinary folks, although sometimes philosophers themselves or other populations) think about philosophically interesting phenomena.

True to their name, experimental philosophers run experiments. In these experiments, participants consider cases and their judgments about the cases are recorded. For example, a participant might consider a Gettier ( 1963 ) case and be asked to what extent they agree with the statement ‘John knows that …’. Experimental philosophers aim not just to find out what judgments people make about particular cases, they “run systematic experiments aimed at understanding how people ordinarily think about the issues at the foundations of philosophical discussions” (The Experimental Philosophy Page, n.d. ). Philosophy experiments investigate the factors that influence participants’ judgments. These factors include interpersonal factors such as ethnicity, but more typically these factors are intrapersonal. An intrapersonal factor might concern the content of the case, such as how much rides on John’s being right. Or it might concern the conditions in which the participant views the case. Is the participant suffering from ego depletion? Is the participant in smelly or non-smelly environs (see Schnall et al., 2008 )? 1

While experimental philosophy is interested in how participants think about relevant issues, participants in these experiments are almost never asked what factors they think are relevant in their decision making. They are not generally even asked, for instance:

Indicate your level of agreement with the following statement:

Considerations of luck were important when coming to my decision

(disagree) 1 – 2 – 3 – 4 – 5 – 6 – 7 (agree)

In other words, experimental philosophers do not typically try to ascertain what is consciously going through participants’ heads when they think about philosophically interesting things.

We’ll consider why this is the case in section 3. First, however, we should note that there exists a host of empirical tools designed to get at how participants think in precisely this sense. The scale above would be a pretty crude way to do this. A better way would involve gathering qualitative data. Let’s quickly look at what this would entail and how philosophers might make use of qualitative data.

2.  Qualitative Methods

This section is not intended as a how-to guide for using qualitative methods in philosophy nor as a comprehensive survey of qualitative methodology. It is a brief and incomplete introduction to qualitative methods for those not familiar with them. I concentrate on those aspects of qualitative methodology which I envisage being of most use to philosophers.

2.1. Qualitative data gathering

The distinctive aspect of qualitative data is that it involves open response answers. 2 Participants are not asked to respond “yes” or “no,” nor are they asked to indicate a level of agreement with a particular statement. Rather, questions are asked which give participants the opportunity to provide an open response in their own terms.

The most straightforward way of gathering qualitative data would be via surveys which incorporate open response questions. This form of qualitative data gathering can be thought of as an alternative way of delivering a structured, qualitative interview. Verbal interviews are historically the meat and bread of qualitative research. Interviews can be more or less structured. A more structured interview closely resembles a research questionnaire, albeit one in which the interviewer reads the questions aloud and the interviewee responds aloud (Brinkmann, 2014 ). Highly structured interviews can also be administered by email. Telephone interviews can be used (experimental philosophers have made some use of telephone surveys, but not to gather qualitative data, see Ahlenius & Tännsjö, 2012 ).

Conducting interviews face-to-face is more typical in qualitative research, as it allows for more freedom in the structure of the interview. At the more unstructured end of the scale, “after the opening request for a narrative, the main role of the interviewer is to remain a listener … sporadically asking questions that may clarify” (Brinkmann, 2014 ). In all cases, structured or unstructured, the conversation is recorded and, when necessary, transcribed. It is worth noting that in addition to questions, an interviewer can make use of other prompts to elicit a response from participants, such as images, thought experiments, or philosophically interesting cases.

Focus groups are another way to gather qualitative data. A focus group is an informal discussion among a small group of participants, usually numbering between six and eight and rarely over twelve (Wilkinson, 1998 ). This discussion is on a topic provided by the researcher. Again, this might include questions, images, or a philosophically interesting case or thought experiment. Discussion can be more or less structured. The discussion may be directed by a moderator, frequently the researcher, but the aim is to capture discussion between participants. Again, the discussion is recorded and transcribed.

The final data gathering method which I will consider is what I shall call a ‘think aloud study’ (see Van Someren et al., 1994 for the background to such methods). In the context I have in mind, such a study might involve a participant being asked to complete a survey of the type used in standard philosophy experiments and, while completing it, being asked to think aloud by vocalizing their thought processes as they decide how to answer the questions on the survey. Again, this can be recorded and transcribed. (Such methods are sometimes called ‘concurrent protocol analysis’, see Baldacchino et al., 2014 .) An alternative method involves having participants fill out a survey and afterwards asking them to articulate why they answered the questions as they did.

2.2. Qualitative data analysis

The analysis of qualitative data can take a number of forms. There are some set qualitative research paradigms which stipulate certain approaches (for examples, see Bryant & Charmaz, 2007 on Grounded Theory or Smith et al., 2009 on Interpretative Phenomenological Analysis). However, I foresee a rather more straightforward approach being of most use to philosophers. Here I will describe a basic approach to the analysis of qualitative data (for more on the basics, see Saldaña, 2013 ).

Qualitative data gathering typically results in a large body of text. This text is then broken down into segments and coded. The appropriate length of the segments depends on the purpose of the research, but a fairly typical unit might be a sentence or small group of sentences. The researcher assigns segments to a code or set of codes which identify particular themes in the data. Codes can represent particular ideas expressed by the text. They can also represent other features of the text, such as confusion, apparent contradiction, or belief revision. Codes can also be clustered under higher-order codes in order to better represent the themes that emerge.

The researcher develops a coding manual containing a clear and precise description of each code. This helps to ensure consistency in coding. It facilitates the establishment of inter-coder reliability, so that when multiple coders work with the same manual one can check the degree to which the coders agree. A coding manual can be decided in advance (this is sometimes described as “a priori”). More typically, however, the construction of the final coding manual in the analysis of qualitative data is an iterative process. This means that the coding is done in several stages as the coding manual develops.

A completely preset coding manual might be appropriate for some research aims. In many philosophical applications, it may be appropriate to set certain codes in advance. For instance, in a think aloud study, one would likely be looking out for certain typical problems that participants encounter. At the other end of the scale, researchers might initially be completely open in the first pass of process of iterative coding. This means that a researcher approaches the text with no specific hypotheses in mind, trying to distance their analysis from any expectations they have about important themes. A completely open initial coding would likely be inappropriate for philosophical use as the research questions are likely to be more targeted. However, allowing a coding manual to develop via an iterative process of coding does not mean that the initial stages of coding must be open. The idea of such an iterative process is that as one codes the data the codes themselves develop as one’s understanding of the date develops. The descriptions of codes may be refined during this process. One may see a need to collapse codes or create new ones. Having developed a better understanding of the themes that emerge, one will likely have to go back and recode the data using the refined coding manual. The process of qualitative analysis is often iterative with researchers coding and recoding until they are happy that the codes capture the themes apparent in the data.

It is important to note that quantitative information can help one’s analysis of qualitative data. For instance, information about the frequencies of certain codes or words in the data, collocation data relating to words, or the overlap of particular codes can help one identify important themes in the data.

2.3. Philosophical uses of qualitative methods

Experimental philosophers do not tend to use qualitative methods. A brief quantitative survey illustrates this fact. A list was made of articles from 2014 that were indexed by a popular resource with experimental philosophers (The Experimental Philosophy Page, n.d. ). 3 The complete list contained 57 articles. These were categorized as non-empirical (i.e., presenting no novel data), quantitative, and qualitative. Papers were only classified as qualitative if qualitative data was collected, analyzed, and featured in the presented results (as something other than a manipulation check). Of the 36 empirical articles in the sample, only one was classified as qualitative. 4 My main aim in this paper is to argue that qualitative methods have the potential to make a much greater contribution to experimental philosphy than they do at present.

How might experimental philosophers use qualitative methods? I think that the most important contribution they have to make is in supplementing the methods already used by experimental philosophers. Qualitative tools are typically used to supplement other types of methods. For instance, Wilkinson ( 1998 ) notes that focus groups are often used either “in an initial exploratory or hypothesis-generation phase, prior to developing a…questionnaire” (p. 184) or “in a final follow-up phase, to pursue an interesting finding from a large-scale survey, or simply to add richness and depth to a project” (pp. 184–185).

What are the advantages of employing more qualitative tools? Qualitative methods can be used to gain insight into how participants think and talk about philosophically interesting phenomena for themselves. Of course, quantitative surveys can do this. However, qualitative methods can give researchers much deeper insights. The reason is that they remove certain barriers. For example, it might be otherwise very unclear how participants understand the question they are asked. Or there might be very important aspects of how a given participant thinks about an issue which a fixed response format might provide no ability to express, and which will consequently remain unknown to philosophy. Another barrier which is removed is that participants are able to respond in their own words, so one can gain an insight into the way in which the “philosophical” usage of a term, as employed in a philosopher’s experimental materials, may not match the ways ordinary folks use the term.

For example, Strohminger and Nichols (2014) probes folk ideas about personal identity in five experiments. 5 The headline finding is that “moral traits [such as honesty and racism] are considered more important to personal identity than any other part of the mind” (p. 168). One might wonder exactly how these results should feed into philosophical debate about personal identity. One might wonder whether participants should be interpreted as expressing any thoughts about numerical personal identify when they select one end of the scale (“Jack is completely different now”) as opposed to the other (“Jack is the same person as before”). These sorts of questions can be asked about many philosophical experiments. Sometimes they can be raised in a dismissive context, implying that participants simply fail to understand the relevant philosophical concepts, undermining the results. But that is not the context in which I raise them in here. Results such as Strohminger and Nichols’s are not valuable only insofar as they concern participants’ thoughts about personal identity as it is typically understood by philosophers . If it turns out that ordinary folks have little traction on a philosophical notion of personal identity and that the sense of a person “being the same person as before” which matters to them is something quite different, that is a philosophically important finding. The reason that I raise these questions here is that without further data, it is difficult to know which philosophically interesting lesson to take from such studies—the one concerning philosophers’ understanding of personal identity or the other. What further data is needed? The most effective method would involve gathering qualitative data, perhaps via an interview format which allowed researchers the flexibility to probe how participants think about personal identity. Such an interview could start by asking participants simply to explain in their own words why they give the answers they do, or what they mean when they say that “Jack is the same person as before,” and then follow up on that answer in order to gain deeper insight. 6

For other philosophical purposes, something like a focus group might be more helpful. Advantages to focus groups include the fact that they afford insight into the ways that participants speak about and use philosophical concepts in the real world. In a one-on-one interview, participants are engaged in conversation with a researcher who guides the discussion using particular language and according to the researcher's understanding of the issues. Of course, a focus group can’t guarantee complete ecological validity, as it were, as it is still a pretty artificial setting. 7 However, they do represent a marked improvement on survey responses or other interview types. The direction and language of the conversations is directed primarily by participants themselves rather than a researcher. A philosophical question which I can imagine being explored in this way is the extent to which ordinary thinking about distinctions between beliefs, thoughts, and knowledge map onto philosophical discussions in epistemology. Why would this be a valuable project? Well, many epistemologists acknowledge that terms such as ‘believe’ and ‘know’ as used by philosophers disconnect in important ways from the ordinary use of those terms. Whether that is a problem depends on your general metaphilosophical stance. Perhaps you think that the terms as used by philosophers are simply terms of art and the resemblance to ordinary English words is a red herring, in which case it would be no surprise if you were not interested in the kind of empirical work I am suggesting. Perhaps, however, you think that the importance and interest of many epistemological debates is rooted in puzzles that arise in our ordinary epistemic lives, our ordinary ways of thinking and talking about beliefs, and our ordinary practices of belief formation. And, if that description fits, you should recognize value in a project that attempts to articulate how people consciously think about beliefs and belief formation.

Hofmann and colleagues ( 2014 ) use a different technique for a similar purpose. They motivate their study thus:

Despite considerable scientific and practical interest in issues of morality, virtually no research has taken morality science out of these artificial settings and directly asked people about whether and how they think about morality and immorality in the course of their everyday lived experience.

They prompted each participant in their sample (N = 1252) to respond to a survey via their smart phone five times a day for three days. First, participants were asked to indicate whether they had committed, been the target of, or witnessed a moral or an immoral act. The follow up questions then included open response questions as to what the event was about, its location, and so on. These responses were coded according to a manual, drawing on research on Moral Foundations Theory (see, e.g., Graham et al., 2012 ). Hofmann and colleagues conclude that their “everyday-life approach” suggests new categories such as honesty and self-discipline which are not part of Moral Foundations Theory. The study also reveals that the differences in the ways that liberals and conservatives think about moral issues are more nuanced than controlled experiments would suggest.

One area in which qualitative data gathering could be particularly helpful for experimental philosophers is in scale development. A scale is a device used in a quantitative study to measure some construct or set of constructs. In experimental philosophy, the Free Will and Moral Responsibility Scale has been developed to gauge the extent to which participants think particular cases involve an agent who is free and morally responsible (see Andow & Cova, 2016 , for an example of this scale in use). Scales comprise a number of “items,” or statements. Participants are asked to indicate their level of agreement with each item or to indicate how closely they think the item describes them. 8

Scale development involves a number of steps. The first step typically involves generating a number of items and then refining them to find a subset or subsets with high internal validity. One way for researchers to do this is to come up with a number of items which they think might be relevant. However, this is potentially quite limiting if one really wants to understand how participants think about these issues.

The use of qualitative data can help avoid these limitations in at least two ways. First, open response data from a preliminary survey can be used in the initial stages as a source of items for later scale development. For instance, exact quotes from participants could be used as items. This provides some assurance that the scale will measure the most significant aspects of the ways participants think about the relevant issues. Second, qualitative data can be helpful in assessing the face validity of items or sets of items. For example, they can be applied to the end product, helping to assess the external validity of the refined scale. A researcher can subject a set of items to a think aloud study. Analysis of the gathered data can then allow one to identify problems with items which might cast the validity of any scale using them into doubt (such as items that participants systematically understand in a way which doesn’t match the researchers’ expectations). 9

3.  Qualitative methods and intuitive processing

In this section, I consider a principled objection to my proposal that qualitative methods have much to contribute to experimental philosophy. This is an objection I take seriously, as it is a plausible argument that experimental philosophers should avoid qualitative methods. Nonetheless, I think this objection is mistaken. This section is not concerned with why in fact experimental philosophers have made no great use of qualitative methods. There needn’t be an interesting answer to that question. It might be simply that it has never occurred to them, that they didn’t get around to it yet, or that they are leaving it to someone else.

Why might one think that qualitative methods are out of place in experimental philosophy? One plausible reason appeals to the idea that experimental philosophy is concerned with intuitions . Not all characterizations of experimental philosophy place as much weight on this. I haven’t, for reasons which should become clear. However, it is very common to characterize experimental philosophy as the empirical investigation of intuitions (see, e.g., Alexander, 2012 ). Indeed, it is common to premise the philosophical relevance of the empirical work done by experimental philosophers on the idea that they investigate intuitions. The idea is that intuitions play a role in philosophy, and so experimental philosophy can contribute to philosophy by speaking to that use of intuitions (more on this later).

If this is right, then there might be a reason to think qualitative methods have little to contribute. There is much debate about how to characterize intuitions, but a very common characterization takes intuitions to be non-inferential judgements that are not a product of conscious reasoning, are fairly immediate, and not slowly or carefully reasoned. There might seem to be good reason to think that qualitative methods will be next to useless for probing such intuitive thinking. The data which qualitative methods capture concern more “reflective” thinking. They capture participants’ thoughts about their deliberation process and their process of reasoning to arrive at an answer. Of course, there is no obvious reason to think that qualitative data could not in principle tell you anything about subpersonal, “intuitive” processing. Others have advocated methods such as think aloud studies as ways to do precisely that, in the context of exploring intuitive expertise (Baldacchino et al., 2014 ). However, there may be reason to think that qualitative methods are unsuitable for measuring the ordinary, subpersonal mechanisms underlying processes like moral judgment.

One way of elaborating this line of objection draws on psychological research about participants’ self-knowledge. Generally, one of the lessons we have learned from modern psychology is that participants often do not have good access to the reasons for the judgments they make or to how they arrive at decisions—they seem to have much less access than we might previously have assumed (for a survey of the relevant literature, see Schwitzgebel, 2014 ). Psychologists have, in recent decades, found evidence that much of our decision-making appears to be guided by processes other than conscious reasoning.

One sign of this is that it is now very common to use a dual-process framework when theorizing about cognition (see, e.g., Evans & Stanovich, 2013 ). Here’s a rather simplistic summary. This popular picture is that there are broadly two types of judgment delivery system installed in our heads: one is fast (system 1) and the other slow (system 2). The former is unconscious, fast, effortless, heuristic, associative, and emotional. The latter is conscious, slow, effortful, and logical. These two types of processes are often supposed to work in tandem, the former delivering quick answers via a process to which we have no introspective access while the latter then trundles away, consciously deliberating to a reasoned answer. For present purposes, it doesn’t really matter whether these two types of processing are best thought of as the result of two domain-general systems of processing. (For a useful summary of all the features associated with the two types of processing posited by dual-process and dual-system accounts, see Frankish, 2010 , p. 922.)

One reason dual-process accounts have garnered so much interest is that a rather larger proportion of our decision-making than one might have originally expected seems to fall into the system 1 category. Also, when our judgments are the result of such fast, intuitive processing, evidence suggests that we are very bad sources of information about the factors which led to those decisions. When asked why they made a decision, the answers people produce seem often to be pure confabulation (for an overview, see Carruthers, 2010, 2013 ).

Putting this together, we can ask the same questions again. Why might one think that qualitative tools have no significant contribution to make to experimental philosophy? It might seem that a rationale has been established. Intuitions play a role in philosophy. Experimental philosophy should thus focus on peoples’ intuitions about philosophically interesting things and in the intuitive processing underlying those intuitions. So there is little value in exploring what people say when you ask about their decision making.

4.  The relevance of reflective processing

Let’s grant a rough distinction between intuitive thinking in the narrow, system 1 sense and more reflective thinking. The objection I’m considering goes as follows. Philosophers use the products of intuitive thinking, and experimental philosophy is therefore philosophically relevant because it investigates intuitive thinking. Qualitative methods are no good are investigating intuitive thinking, and therefore they have no contribution to make to experimental philosophy.

Here’s the essence of my response. Philosophers don’t just use intuitive thinking in this narrow sense. Philosophers draw on more reflective aspects of ordinary thought about philosophically interesting things as well, so qualitative methods have a philosophical contribution to make. The motivations behind experimental philosophy and the typical reasons for thinking that the data it provides can make a philosophical contribution would in no way justify limiting experimental philosophy to the investigation of intuitive thought. So the philosophical contribution to be made by qualitative data seems to fit into the experimental philosophy project.

To make this response, I’ll first consider the more general explanations of the aims of experimental philosophy which have been given in the literature, finding in them no justification for an exclusive focus on intuitive thinking. Then, I move on to examine some specific ways in which the empirical data about people’s thinking about philosophically interesting things gathered by experimental philosophers can contribute to philosophical debate, and again I’ll find among them no justification for an exclusive focus on intuitive thinking. In fact, I will emphasize that all these various ways of motivating experimental philosophy in fact make it clear that qualitative methods have an important role to play and an important philosophical contribution to make.

4.1. General motivations

We can get a good picture by getting back to basics. In their Experimental Philosophy Manifesto , Knobe and Nichols ( 2008 ) open as follows:

It used to be a commonplace that the discipline of philosophy was deeply concerned with questions about the human condition. Philosophers thought about human beings and how their minds worked. They took an interest in reason and passion, culture and innate ideas, the origins of people’s moral and religious beliefs. On this traditional conception, it wasn’t particularly important to keep philosophy clearly distinct from psychology, history, or political science. Philosophers were concerned, in a very general way, with questions about how everything fit together. (p. 3) The new movement of experimental philosophy seeks a return to this traditional vision. Like philosophers of centuries past, we are concerned with questions about how human beings actually happen to be . We recognize that such an inquiry will involve us in the study of phenomena that are messy, contingent, and highly variable across times and places, but we do not see how that fact is supposed to make the inquiry any less genuinely philosophical. On the contrary, we think that many of the deepest questions of philosophy can only be properly addressed by immersing oneself in the messy, contingent, highly variable truths about how human beings really are (emphasis added). (p. 12)

Hopefully, it is clear in such statements that there is nothing that would suggest that experimental philosophers should be interested only in intuitions. However, I should note that this interest in “how human beings really are” and about “how their minds work” does, even in Knobe and Nichols's manifesto, end up being parsed in terms of intuitions.

More and more, philosophers are coming to feel that questions about how people ordinarily think have great philosophical significance in their own right. So, for example, it seems to us that there are important philosophical lessons to be gleaned from the study of people’s intuitions about causation, but we do not think that the significance of these intuitions is exhausted by the evidence they might provide for one or another metaphysical theory. On the contrary, we think that the patterns to be found in people’s intuitions point to important truths about how the mind works , and these truths—truths about people’s minds, not about metaphysics—have great significance for traditional philosophical questions (emphasis added). (p. 12)

Given the prevalence of intuition-talk in experimental philosophy, it is easy to read such statements as suggesting that what experimental philosophers are really interested in is intuitions . That interpretation is fine if ‘intuitions’ merely means ‘judgments about philosophically interesting things’. But if that’s the interpretation, such statements do not suggest that the real interest is in intuitions in the very specific sense discussed above . In fact, quite to the contrary, this interpretation would include reflective thinking and thus is no barrier to the idea that qualitative methods can contribute to experimental philosophy. On the other hand, if you do try to read ‘intuitions’ in the more specific sense, then the position seems rather strange. It is far from clear what part of the general motivations stated in the above passage would mandate this slide from a general interest in how people think to a more specific interest in their system 1 intuitions. It seems that one should take the more permissive reading.

I foresee a potential objection here. Why should we take the more general motivations to accurately capture the nature and philosophical relevance of experimental philosophy? Maybe the opening of this manifesto, and other more general statements of the motivation for doing experimental philosophy, are simply too loosely stated. Maybe the real interest is not in how people think about philosophically interesting stuff in general, but rather in their intuitive thinking in the more specific sense. That isn’t right. To see why, let’s look at some more specific ways in which the empirical data gathered by experimental philosophers about how people think has been claimed to be of potential philosophical value.

4.2. Specific motivations

There are five main ways in which experimental philosophers’ data about how people think about philosophically interesting things has been considered philosophically valuable. I’ll examine them in turn. 10

4.2.1. Enriching philosophers’ evidence base

Experimental philosophy has been claimed to contribute to philosophy by providing more evidence of the same type as that already used, albeit perhaps from a more diverse sample and collected more systematically. Philosophers have always given weight to ordinary ways of thinking about things. Philosophers of time, for example, pay attention to our ordinary understanding of time. They want an account which ultimately is not only coherent (and perhaps empirically adequate), but which makes sense of our ordinary ways of thinking about time, passage, the past, and so on. All else equal, accordance with our ordinary ways of thinking about time is a sign that a theory is true, while conflicting with those ways of thinking is a sign that a theory is false. Experimental philosophy can help by revealing all sorts of features of our ordinary ways of thinking about the world which were not apparent through introspection or consultation with those around us, and so can serve to provide us with better evidence. (This is the hope of the “positive program” in experimental philosophy, although not all experimental philosophers share this hope. See Alexander, 2012 .)

Note that nothing here suggests that an exclusive focus on our fast, intuitive, judgments about cases would be appropriate. For example, when considering whether a theory of time fits with the ordinary ways we think, the focus isn’t solely on intuitive judgments in some narrow sense. Many things are relevant—the way we talk about past events, the way we think about what is possible for people to achieve in the past, present, and future, and the fact that it simply feels odd to us to claim that dinosaurs are as real as our children. Consider what we would make of the following situation. Suppose that our intuitive, quick, unconsidered response to a case is P, but upon a moment’s reflection, every ordinary person would immediately think that P is incorrect. Would or should philosophers only be interested in the immediate reaction? The answer is clearly no. Perhaps philosophers should give it some attention. It is an interesting feature of the kind of beings that we are. But we are also the kind of beings that have and value considered responses to philosophically interesting cases. So it seems clear that qualitative methods have something to contribute by providing more evidence for philosophy of the same type that they already use .

4.2.2. Challenging philosophers’ supposed evidence base

Experimental philosophy has been claimed to contribute to philosophy by leading us to reconsider what we take to be evidence. As I have argued, philosophers give weight to ordinary ways of thinking. Sometimes experiments can contribute by revealing features of our ordinary ways of thinking about the world which might lead us to reconsider the amount of weight we give to such considerations in our theorizing. (This is the aim of the “negative program” of experimental philosophy. See Alexander, 2012 ). The influence of certain factors on our ways of thinking might, on reflection, be taken as a sign that our ordinary ways of thinking are erroneous, for example, if they are sensitive to irrelevant factors.

Nothing here suggests that an exclusive focus on our fast, intuitive judgments about such cases would be appropriate. It is true that philosophers’ thinking is influenced by fast, immediate responses. Sometimes, perhaps often, these immediate judgments are simply taken up by more explicit reasoning processes and have a significant effect on theory. So it makes sense to be interested in the provenance of our judgments about philosophically interesting phenomena. And, given our comparative ignorance about fast, intuitive judgments, we should perhaps be more worried about such judgments than those which are the product of more explicit processing. However, there seems to be no reason not to be interested in the way that people consciously reason about things. The provenance of our immediate responses is interesting, but is only one facet of a fully developed empirical program that investigates how people think about time, morality, art, knowledge, and other phenomena. Qualitative research could reveal, for example, common assumptions or associations that play a role in our thinking about philosophical issues, but are problematic for some reason. So it seems that qualitative methods have a significant part to play here too. Of course, causal links are difficult or impossible to establish with purely qualitative methods. One might need to do this in order to show that, for example, intuitions are affected by a certain irrelevant factor. But I am not suggesting that qualitative methods be used in isolation. They can help an attempt to challenge philosophers’ evidence base by providing a source of well-informed hypotheses.

4.2.3. Catalyzing reflection

Experimental philosophy has been claimed to contribute to philosophy by catalyzing reflection on ways of thinking that philosophers have taken for granted. All disciplines have to take some things for granted. However, it is healthy for such ways of thinking to be questioned and subjected to reflective scrutiny. The thought is that experimental findings, such as those showing that different types of people think in different ways about morality, can serve as an effective catalyst for philosophical reflection about morality (Knobe & Nichols, 2008 , p. 11).

But, again, there is no motivation here for an exclusive focus on intuitions. Different cultures have different religions, world-views, legal systems, cultural practices, gender roles, and so on. Indeed, these are precisely the types of difference that Knobe and Nichols have in mind. All of these things go far beyond the immediate responses people have to cases. Finding out that members of a culture reason about X and conceptualize X in a very different way than members of another culture can surely serve as an important catalyst for reflection. But, it is not only facts about intuitions that can play this role. Indeed, it seems that the fact that people consciously construct and reflectively endorse completely different moral systems is likely to serve as a much more powerful catalyst than their having divergent immediate responses (although I accept that this an empirical claim, susceptible to empirical disconfirmation). So, clearly qualitative methods have a role to play in providing these catalysts.

4.2.4. Otherwise informing our understanding of ordinary ways of thinking

Experimental philosophy’s contribution doesn’t have to be linked to evidence or to catalyzing reflection. Philosophers sometimes treat ordinary ways of thinking as a starting point. Sometimes it is thought to be part of the job of a philosopher to make sense of our ordinary ways of thinking. Experimental philosophy can therefore contribute by discovering surprising aspects of our ways of thinking—aspects which were not apparent from the armchair. Once philosophers become aware of surprising aspects of our ways of thinking, they can then think carefully about those aspects. Experimental philosophy might simply provide a better idea of the ordinary ways of thinking that it is the philosopher’s job to make sense of. Or it might help us examine which ordinary ways of thinking we should treat as a starting point. Experiments simply provide additional things to philosophize about.

My point will now be familiar. Data about intuitions, in the restricted sense, has an important contribution to make in this way. However, so does qualitative data relating to more reflective ways of thinking. When philosophers treat ordinary ways of thinking as a starting point for inquiry, or as something it is the philosopher’s job to make sense of, they do not restrict themselves to intuitive ways of thinking. We want to make sense of our ways of thinking about moral responsibility. That means that we are interested in aspects of thinking which qualitative methods provide a way to explore. We are interested in practices of praise and blame. We are interested in the sorts of principles people formulate for themselves. We ask about the role that responsibility plays in our wider moral framework. We are interested in the ways that people consciously reason about these things.

4.2.5. Conceptual analysis

Finally, experimental philosophy has been claimed to contribute to philosophy by contributing to the project of conceptual analysis. Whether as an end in itself or as a stepping-stone to inquiry about the world, philosophers have always been interested in the structure of our concepts and in the relations between our concepts. Views about what concepts are vary, as do views about what conceptual analysis is interested in analyzing. Be that as it may, it is common to treat data concerning how we apply concepts, or about how acceptable we find certain applications of concepts, as evidence about the concepts themselves. Philosophical experiments can provide a rich source of such data for the project of conceptual analysis (see, e.g., Knobe, 2007 ). Experiments can detect subtle differences between cases. They can detect subtle differences in how willing we are to attribute knowledge, intentional action, and other philosophically interesting predicates in such cases. They can detect subtle differences which are not readily apparent through introspection. Therefore, they enable us to paint a better picture of how we think about such concepts and about the various factors relevant to our application of those concepts.

Can qualitative data contribute as well? This isn’t a straightforward question. The idea of conceptual analysis is a little slippery. It is not always clear what the target of conceptual analysis is. There is disagreement among philosophers as to what they are doing when they do conceptual analysis. This is worth noting because there is an understanding of conceptual analysis on which data about intuitive thinking would be relevant but data about more reflective thinking would not. Suppose that concepts are such that the most useful information available to us in their analysis is information about people’s fast, immediate, non-reflective judgments about whether particular cases fall within the extension of the concept. Such concepts would be something like subpersonal psychological structures with application conditions (of some form) that represent features of the world.

However, I think it should be pretty clear that this sense of conceptual analysis is pretty restricted, and that most philosophers who analyze concepts are interested in more than simply having a clear understanding of the ways in which humans quickly divide up the world at the subpersonal level. That is not to say that philosophers are not interested in concepts in this more restricted sense, just that they are interested in something else too. For example, suppose that we were interested in analyzing the concept of action. Data about people’s more reflective ways of thinking is typically taken to be relevant to the philosophical project of analyzing such a concept. When people reflect on the relevant cases again, what type of considerations leap out at them? Which types of cases are they most confident that they have classified correctly? What relations do they consciously make with other concepts? What priority do they give to their first impressions about concept application, once they reflect on issues of moral responsibility or knowledge? Qualitative data could be invaluable in providing this richer understanding of people’s concepts.

5.  Clarifications

I am not raising an objection to experimental philosophy as currently practiced. Saying that X could be augmented by the incorporation of Y is not an objection to X. However, I think that there has sometimes been too much focus on intuitive thinking, in the restricted sense of fast, non-inferential processing.

I am not saying there is a problem with experimental methods for investigating the sort of thing which experimental philosophy investigates. I am recommending that experimental philosophers supplement their current toolkit. Using qualitative research to get some traction on how people think can provide an excellent basis upon which to design the necessary quantitative methods of experimental philosophy (including surveys, scales, and the like) to track participants’ thinking (including more reflective thinking) in quantifiable terms for use within experimental studies.

I am not saying that philosophers shouldn’t be interested in subpersonal, fast, intuitive classification of philosophically interesting cases. They should. But they are and should be interested in more reflective thinking too.

I am not claiming experimental philosophers have never collected qualitative data. They have, although typically in the form of open response survey questions, generally to perform manipulation checks and the like within an experimental project. 11 The claim is that qualitative methods have much more to contribute than they have so far. I am also not claiming to be the only researcher to have called for a greater use of qualitative tools. 12

I am not claiming that empirical work which draws purely on qualitative data would be of any value. Perhaps it might. Perhaps it mightn’t. There’s no obvious reason to think that it could never be. But I don’t aim to argue either way. For all I have said, it might be the case that any project which used qualitative data to investigate how people think about philosophically interesting things and which did not supplement this with more quantitative methods would be of only limited or even no philosophical value.

I am not claiming that there is nothing to be gleaned from quantitative data about reflective reasoning. As I have made clear throughout, I think that quantitative methods are simply a far from perfect tool for the job of tapping participants’ conscious thought processes and that it therefore makes sense to supplement them with qualitative methods.

I am not endorsing any kind of “qualitative philosophy.” Qualitative researchers sometimes buy into dubious, or at least controversial, “philosophies” such as the idea that there is no objective truth, that all knowledge is constructed, that such notions are morally problematic, and so on. I am inclined to think that endorsing any such thing would be a gross mistake. Fortunately, one need not buy any of that stuff in order to make use of qualitative tools.

I am not claiming that incorporating qualitative tools is easy nor that they don’t have shortcomings. Generalizing from qualitative data is problematic both because of the nature of the data and the size of the samples one typically has to work with. Qualitative techniques are laborious. This means that that the number of participants one can feasibly run is much lower than in more typical experimental philosophy. Additionally, for certain types of analysis you need to have members of the research team who are not aware of the precise hypotheses of the project. Drawing themes out of qualitative data can require a good deal of interpretation on the part of the researcher and so does make it particularly difficult to ensure objectivity. Allowing flexible forms of interview means that many variables may change between participants. Qualitative tools can really be used only to explore themes and construct theories rather than test concrete hypotheses or establish causal relations. As a result, they can sometimes, at least in the short term, lead to less clarity about how people consciously think about the relevant issues. So I am definitely not saying that there are no shortcomings. However, take note! The fact that the correct toolkit for the job of “immersing oneself in the messy, contingent, highly variable truths about how human beings really are” (Knobe and Nichols, 2008 , p. 3) includes tools which have these shortcomings ought not be too much of a surprise. And, remember, there is no suggestion here that experimental philosophers should up-sticks and adopt a purely qualitative methodology.

With these clarifications out the way, and before I conclude, I want to see off one last objection. I’m going to consider rather an extreme version of this objection. It should be clear how I would respond to less extreme versions.

The objection goes as follows. All conscious thinking (the only stuff qualitative methods have access to) is post hoc confabulation. Our conscious thoughts tell us nothing about our real decision making, classification of cases, and so on. They tell us only a complete fiction invented by some sort of modular commentary box, the only part of the mind that exists at a personal level, which has no access to our subpersonal thinking other than via the senses. We have no introspective access and we make sense of our own behavior and decisions by observing ourselves in precisely the same way as we do the behavior and actions of others. So, qualitative methods do not give us access to anything philosophically deep or interesting about ourselves . If the world is as the objection paints it, when we claim that something is X, all of our reasoning about why it is X bears absolutely no relation to the reasons we declared it to be X in the first place (unless by coincidence), as we have no introspective access to those reasons.

Now a model this extreme is probably not correct (but I don’t aim to argue for that here). 13 My response to this objection is to note that, even if this extreme model of a post hoc, disconnected mind were correct, the conclusion that qualitative methods do not give us access to anything philosophically deep or interesting about ourselves wouldn’t follow. Why wouldn’t the conclusion follow? Quite contrary to the idea that in such a scenario qualitative methods wouldn’t give one access to anything philosophically interesting, I think that they would give one access to perhaps the most philosophically interesting part of the whole setup. In such a scenario we probably should be interested in the causal story as to how certain decisions get made and how certain intuitions come to be had, however, much more interesting would be the conscious reasoning, justifications and theorizing that we do after the fact, precisely because that is where most of our ordinary thinking about philosophically interesting things takes place.

6.  Wrapping Up

The point of experimental philosophy is to bring empirical methods to philosophy in order to probe how people think about philosophically interesting things, such as knowledge, morality, and freedom. The point of this paper has been to argue that qualitative methods, which play very little part in experimental philosophy at the moment, have a significant contribution to make to this project. I have tried to stave off one particular line of resistance—that experimental philosophy is and should be focused (almost) exclusively on intuitive processing rather than the more reflective thinking which qualitative methods can access. I have argued that, given the general motivations behind experimental philosophy, and given the particular types of philosophical contribution that experimental philosophy is taken to be able to make, there is no reason to think that experimental philosophy should be focused exclusively on subpersonal intuitive processing, nor would it be justified to claim that qualitative methods have no contribution to make.

Disclosure statement

No potential conflict of interest was reported by the author.

1. For detailed introductions to the methods of experimental philosophy, see Alexander ( 2012 ), Knobe ( 2007 ), and Knobe et al. ( 2011 )., 2011b).

2. This is an imperfect characterization, as many methods of qualitative research involve, for example, naturalistic observation rather than the collection of responses from participants in any form. Here, I will simply restrict discussion to the methods involving open responses in some form, but this is not because I think that naturalistic observation has no role to play in empirically-minded philosophy.

3. All articles from “X-Phi of…” categories. Papers in languages other than English were not included. Citations for edited collections were not included in this count (the volumes, not the papers therein), nor were book reviews or conference summaries.

4. Thanks to a referee for suggesting this quick survey. Interested readers should also see Knobe ( 2015 ), whose more systematic survey tracks the extent to which quantitative data is used in philosophy.

5. Let me be clear. This work is valuable and important. By using this example, I don’t suggest that this paper has any problems or any peculiar features. It is simply useful to have an example.

6. For some philosophical purposes, there might even be a call for a more Socratic form of interview, in which the interviewer poses simple challenges to the interviewee’s position.

7. Ecological validity is defined as “the confidence with which the conclusions of an empirical investigation can be generalized to naturally occurring situations in which the phenomenon under investigation occurs” (Colman, 2015 ).

8. A score on the scale is then calculated for the participant. On more simple scales this simply amounts to the mean of their responses to the scale items, but less simple scales might measure multiple dimensions or weight different items by different amounts.

9. Feltz and Millan ( 2015 ) have recently made a similar point, calling for a turn to greater use of “protocol analysis” in experimental philosophy studies relating to freewill and moral responsibility.

10. A referee points out that one might think that there are important connections between the more specific motivations considered in this section and that one might even think that many of them are essentially the same motivation. By separating them here, I don’t mean to suggest they are completely distinct. The aim is to show that the standard ways of conceptualizing the potential philosophical import of experimental philosophy in no way imply that only evidence concerning intuitive thinking, in the more specific sense, is philosophically valuable.

11. There are some exceptions which I haven’t discussed in the above, for example, Berniūnas and Dranseika ( 2016 ), De Cruz (2016), Monroe and Malle ( 2010 ), and Skulmowski et al. ( 2015 ).

12. Some similar comments are made, albeit briefly, by a number of others (e.g., Devitt, 2015 ; Feltz & Millan, 2015 ; O’Brien, 2015 ).

13. For example, as a reviewer points out, it seems unlikely that reflective reasoning can tell us nothing important about intuitions.

• Ahlenius H., & Tännsjö T. (2012). Chinese and westerners respond differently to the trolley dilemmas . Journal of Cognition and Culture , , 195–201. [ Google Scholar ]
• Alexander J. (2012). Experimental philosophy: An introduction . Cambridge: Polity. [ Google Scholar ]
• Andow J., & Cova F. (2016). Why compatibilist intuitions are not mistaken: A reply to Feltz and Millan . Philosophical Psychology , , 550–566. [ Google Scholar ]
• Baldacchino L., Ucbasaran D., Lockett A., & Cabantous L. (2014). Capturing intuition through concurrent protocol analysis In Sinclair M. (Ed.), Handbook of research methods on intuition (pp. 160–175). Cheltenham: Elgar. [ Google Scholar ]
• Berniūnas R., & Dranseika V. (2016). Folk concepts of person and identity: A response to Nichols and Bruno . Philosophical Psychology , , 96–122. [ Google Scholar ]
• Brinkmann S. (2014). Unstructured and semi-structured interviewing In Leavy P. (Ed.), The Oxford handbook of qualitative research (pp. 277–297). Oxford: Oxford University Press. [ Google Scholar ]
• Bryant A., & Charmaz K. (Eds.). (2007). The SAGE handbook of grounded theory . London: Sage. [ Google Scholar ]
• Carruthers P. (2010). Introspection: Divided and partly eliminated . Philosophy and Phenomenological Research , , 76–111. [ Google Scholar ]
• Carruthers P. (2013). Understanding our minds . Oxford: Oxford University Press. [ Google Scholar ]
• Colman A. (2015). A dictionary of psychology (4th ed.). Oxford: Oxford University Press. [ Google Scholar ]
• De Cruz H. (2016). Irrelevant influences and philosophical practice: A qualitative study . (Manuscript under review).
• Devitt M. (2015). Testing theories of reference In Haukioja J. (Ed.), Advances in experimental philosophy of language (pp. 31–64). London: Bloomsbury. [ Google Scholar ]
• Evans J. S. B., & Stanovich K. E. (2013). Dual-process theories of higher cognition: Advancing the debate . Perspectives on Psychological Science , , 223–241. [ PubMed ] [ Google Scholar ]
• Feltz A., & Millan M. (2015). An error theory for compatibilist intuitions . Philosophical Psychology , , 529–555. [ Google Scholar ]
• Frankish K. (2010). Dual-process and dual-system theories of reasoning . Philosophy Compass , , 914–926. [ Google Scholar ]
• Gettier E. (1963). Is justified true belief knowledge? Analysis , , 121–123. [ Google Scholar ]
• Graham J., Haidt J., Koleva S., Motyl M., Iyer R., Wojcik S. P., & Ditto P. H. (2012). Moral foundations theory: The pragmatic validity of moral pluralism . Advances in Experimental Social Psychology , , 55–130. [ Google Scholar ]
• Hofmann W., Wisneski D. C., Brandt M. J., & Skitka L. J. (2014). Morality in everyday life . Science , , 1340–1343. [ PubMed ] [ Google Scholar ]
• Knobe J. (2007). Experimental philosophy . Philosophy Compass , , 81–92. [ Google Scholar ]
• Knobe J. (2015). Philosophers are doing something different now: Quantitative data . Cognition , , 36–38. [ PubMed ] [ Google Scholar ]
• Knobe J., Buckwalter W., Robbins P., Sarkissian H., Sommers T., & Nichols S. (2011). Experimental philosophy . Annual Review of Psychology , , 81–99. [ PubMed ] [ Google Scholar ]
• Knobe J., & Nichols S. (2008). An experimental philosophy manifesto In Knobe J. & Nichols S. (Eds.), Experimental philosophy (pp. 3–14). Oxford: Oxford University Press. [ Google Scholar ]
• Monroe A., & Malle B. (2010). From uncaused will to conscious choice: The need to study, not speculate about people’s folk concept of free will . Review of Philosophy and Psychology , , 211–224. [ Google Scholar ]
• O’Brien L. (2015). Side effects and asymmetry in act-type attribution . Philosophical Psychology , , 1012–1025. [ Google Scholar ]
• Saldaña J. M. (2013). The coding manual for qualitative researchers . London: Sage. [ Google Scholar ]
• Schnall S., Haidt J., Clore G., & Jordan A. (2008). Disgust as embodied moral judgment . Personality and Social Psychology Bulletin , , 1096–1109. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Schwitzgebel E. (2014). Introspection In Zalta E. N. (Ed.), The Stanford encyclopedia of philosophy (Summer 2014 ed.). Retrieved from http://plato.stanford.edu/entries/introspection/
• Skulmowski A., Bunge A., Cohen B. R., Kreilkamp B., & Troxler N. (2015). Investigating conceptions of intentional action by analyzing participant generated scenarios . Frontiers in Psychology , , 1630. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Smith J. A., Flowers P., & Larkin M. (2009). Interpretive phenomenological analysis: Theory, method and research . London: Sage. [ Google Scholar ]
• Strohminger N., & Nichols S. (2014). The essential moral self . Cognition , , 159–171. [ PubMed ] [ Google Scholar ]
• The Experimental Philosophy Page (n.d.). Retrieved August 19, 2016, from http://experimental-philosophy.yale.edu/ExperimentalPhilosophy.html
• Van Someren M. W., Barnard Y. F., & Sandberg J. A. (1994). The think aloud method: A practical guide to modelling cognitive processes (Vol. 2 ). London: Academic Press. [ Google Scholar ]
• Wilkinson S. (1998). Focus group methodology: A review . International Journal of Social Research Methodology , , 181–203. [ Google Scholar ]

• Table of Contents
• Random Entry
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Advanced Tools
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Entry Contents

Bibliography

Academic tools.

• Friends PDF Preview
• Author and Citation Info
• Back to Top

Experimental Philosophy

Experimental philosophy is an interdisciplinary approach that brings together ideas from what had previously been regarded as distinct fields. Specifically, research in experimental philosophy brings together two key elements:

• the kinds of questions and theoretical frameworks traditionally associated with philosophy;
• the kinds of experimental methods traditionally associated with psychology and cognitive science.

Though experimental philosophy is united by this broad approach, there is a diverse range of projects in experimental philosophy. Some use experimental evidence to support a “negative program” that challenges more traditional methods in analytic philosophy, others use experimental data to support positive claims about traditional questions, and still others explore questions about how people ordinarily think and feel insofar as these questions are important in themselves.

This entry provides a brief introduction to the core aims of contemporary experimental philosophy. It then reviews recent experimental work on the negative program, free will, moral judgment and epistemology. We conclude with a discussion of major objections to the field of experimental philosophy as a whole.

1. Overview

2.1.1 the argument from diversity, 2.1.2 the argument from sensitivity, 2.2 free will and moral responsibility, 2.3 impact of moral judgment, 2.4 epistemology, 2.5 other topics, 3.1.1 philosophers don’t rely on intuitions, 3.1.2 philosophers shouldn’t rely on intuitions, 3.2 defending privileged intuitions rather than those of ordinary experimental participants, 3.3 but is it philosophy, other internet resources, related entries.

Experimental philosophy is a relatively new approach, usually understood as beginning only in the early years of the 21st century. At the heart of this new approach is the idea of pursuing philosophical questions using methods more typically associated with the social sciences.

Within the broad banner of experimental philosophy research, one finds work using an enormous variety of methods and aims (see, e.g., Schwitzgebel & Rust 2014; Meskin et al. 2013; Bartels & Urminsky 2011). Nonetheless, most research in experimental philosophy makes use of a collection of closely connected methods that in some way involve the study of intuitions. The remainder of this section aims to characterize the different projects experimental philosophers have pursued using these methods and their relevance for broader questions in philosophy.

The practice of exploring intuitions has its origins in a more traditional philosophical approach that long predates the birth of experimental philosophy (see the entry on intuition ). Research within this more traditional approach often relies on the idea that we can make progress on one or another topic by looking at intuitions about that topic. For example, within epistemology, it has been suggested that we can make progress on questions about the nature of knowledge by looking at intuitions about whether certain states count as knowledge. Similarly, within moral philosophy, it has been suggested that we can make progress on questions about moral obligation by looking at intuitions about what actions certain agents are obligated to perform. Similar approaches have been advocated in numerous other areas of philosophy.

There is a complex literature within the analytic tradition about how to understand this traditional method. Some argue that the study of intuitions gives us insight into concepts (Jackson 1998), others argue that the study of intuitions gives us a more direct sort of insight into the actual properties or relations those concepts pick out (Sosa 2007), and still others argue that this whole way of conceiving of the project is a mistaken one (e.g., Cappelen 2012).

It is commonplace to divide existing research in experimental philosophy into distinct projects in accordance with their different relationships to this prior tradition. Dividing things up in this way, one arrives at three basic kinds of research in experimental philosophy.

First, some experimental philosophy research has a purely ‘negative’ relationship to this more traditional use of intuitions. Such research aims to provide evidence that the method used in the more traditional work is in some way flawed or unreliable. For example, it has been argued that intuitions differ across demographic factors such as gender or ethnicity, or that they are subject to order effects, or that they can be influenced by incidental emotion (e.g., Weinberg et al. 2001; Buckwalter & Stich 2014; Swain et al. 2008; Cameron et al. 2013). To the extent that intuitions show these effects, it is argued, we should not be relying uncritically on intuition as a method for addressing substantive philosophical questions. This first project is called ‘negative’ in that it is not intended to make progress on the original philosophical question (e.g., about the nature of knowledge) but only to argue against a specific method for addressing that question (appeal to intuition).

This project has triggered a large and multi-faceted literature among philosophers interested in its metaphilosophical implications (Brown 2013b; Cappelen 2012; Deutsch 2015; Weinberg 2007; Weinberg et al. 2010; Williamson 2007). This literature has explored the question as to whether empirical facts about the patterns in people’s intuitions could give us reason to change our philosophical practices. Much of this work is quite closely tied to prior philosophical work about the role of intuition in philosophy more generally.

Second, some research in experimental philosophy aims to make further progress on precisely the sorts of questions that motivated prior work within analytic philosophy. Thus, this research looks at epistemic intuitions as a way of making progress in epistemology, moral intuitions as a way of making progress in moral philosophy, and so forth. Experimental philosophers pursuing this second project have offered various different accounts of the way in which facts about intuitions could yield progress on these philosophical issues, but the most common approach proceeds by advancing some specific hypothesis about the underlying cognitive processes that generate intuitions in a particular domain. The suggestion is then that this hypothesis can help us assess which intuitions in this domain are worthy of our trust and which should simply be dismissed or ignored (Gerken 2017; Leslie 2013; Greene 2008; Nagel 2010).

Work within this second project has inspired a certain amount of metaphilosophical debate, but its main impact on the philosophical literature has been not at the level of metaphilosophy but rather in discussions of individual philosophical questions. Thus, philosophers interested in epistemic contextualism discuss experiments on people’s intuitions about knowledge (DeRose 2011), philosophers interested in incompatibilism discuss experiments on people’s intuitions about free will (Björnsson & Pereboom 2014; Vargas 2013), and philosophers interested in interventionist accounts of causation discuss experiments on people’s intuitions about causation (Woodward 2014). Work in this vein typically does not focus primarily on more abstract theories about the role of intuitions in philosophy. Instead, it draws more on theories about the particular topic under study (theories of knowledge, free will, causation).

The third type of research being conducted in experimental philosophy is not concerned either way with the kind of project pursued in more traditional analytic philosophy; it is just doing something else entirely. Specifically, in many cases, experimental philosophers are not looking at people’s thoughts and feelings about some topic as a way of making progress on questions about that topic; they are instead trying to make progress on questions that are directly about people’s thoughts and feelings themselves . For example, much of the experimental philosophy research in moral psychology is concerned with questions that truly are about moral psychology itself.

Research in this third vein tends to be highly interdisciplinary. Thus, work on any particular topic within this third vein tends to be at least relatively continuous with work on that same topic in other disciplines (psychology, neuroscience, linguistics, etc.), and the impact of such work is often felt just as much in those other disciplines as in philosophy specifically.

The distinction between these projects has proven helpful within metaphilosophical work on the significance of experimental philosophy, but it should be noted that the metaphilosophical distinction between these three projects does not correspond in any straightforward way to the distinctions between the different concrete research programs experimental philosophers pursue (on free will intuitions, on moral intuitions, on epistemic intuitions, etc.). Each of these concrete research programs can be relevant to a number of different projects, and indeed, it often happens that a single paper reports a result that seems relevant to more than one of these projects. Thus, as we review the actual experimental research coming out of experimental philosophy, we will need to turn away from the metaphilosophical distinction between projects and turn instead to distinctions between concrete research topics.

2. Research in Experimental Philosophy

The best way to get a sense of what experimental philosophy is all about is not just to consider it in the abstract but to look in detail at a few ongoing research programs in the field. Accordingly, we proceed in this section by reviewing existing research in four specific areas: the negative program, free will, the impact of moral judgment, and epistemology.

We focus on these four areas because they have received an especially large amount of attention within the existing experimental philosophy of literature. We should note, however, that experimental philosophers have explored an enormous range of different questions, and work in these four specific areas comprises only a relatively small percentage of the experimental philosophy literature as a whole.

2.1 The Negative Program

In the Theaetetus, Socrates asks, “Herein lies the difficulty which I can never solve to my satisfaction—What is knowledge?” (146a). The subsequent philosophical discussion often proceeds by setting out various hypotheses, e.g., that knowledge is true belief, and considering possible counterexamples to the hypothesis. So, for instance, Socrates argues that knowledge isn’t simply true belief because a skilled lawyer can persuade a person to have a belief that is true, but that belief wouldn’t actually be knowledge [see the entry on the Theaetetus ]. Socrates typically expects, and receives, agreement from his interlocutor. Nor does Socrates ask his interlocutor, “What is your conception of knowledge” or “What counts as knowledge for Athenians?” Rather, he seems to expect a global answer about what knowledge is. In addition, he seems to expect that knowledge has a single nature, as suggested by his telling Theaetetus, “I want you… to give one single account of the many branches of knowledge” (148d).

Work in the negative program of experimental philosophy uses empirical work to challenge this traditional philosophical project. Two somewhat different challenges have been developed.

One challenge arises from the prospect of systematic diversity in how different populations of people think about philosophical questions. The possibility of such diversity had been raised before (e.g., Stich 1990), but experimental philosophers have sought to provide evidence of such diversity. For instance, an early study reported differences between East Asian students and Western students on famous cases from epistemology (Weinberg et al. 2001). Another early study provided evidence for cultural differences in judgments about reference. East Asians were more likely than Westerners to have descriptivist judgments about the reference of proper names (Machery et al. 2004). Some studies have also found gender differences in intuitions about philosophical cases (see, e.g., Buckwalter & Stich 2014; Friesdorf et al. 2015). In addition, there are systematic individual differences in philosophical intuition; for example, people who are more extraverted are more inclined to compatibilism about responsibility (Feltz & Cokely 2009).

This apparent diversity in intuitions about philosophical matters has been used to challenge the use of intuitions in philosophy to tell us about the nature of things like knowledge and reference. If intuitions about knowledge turn out to exhibit diversity between populations, then this looks to put pressure on a traditional philosophical project. In rough form, the worry arises from the following claims:

• D1. The philosophical tradition uses intuitions regarding philosophically important categories or kinds like knowledge in an effort to determine the nature of those categories
• D2. Knowledge (like many other philosophical categories) has a single nature. It’s not the case that knowledge is one thing in Athens and another thing in Sparta.
• D3. Intuitions about philosophical categories systematically vary between populations (by culture, for instance)
• D4. The diversity in intuition cannot be dismissed by privileging the intuitions of one population.

Each of these claims has been challenged. Some argue that philosophers do not—or should not—rely on intuitions (thus rejecting D1) (see section 3.1 ); others hold, contra D4, that certain populations (e.g., professional philosophers) are specially positioned to have reliable intuitions (see section 3.2 ).

Another way to defuse the challenge is to argue (contra D2) that we needn’t suppose that knowledge has a single nature, but instead allow for a kind of pluralism. For instance, “knowledge” might pick out different epistemic notions in different communities. A pluralist might allow, or even celebrate, this diversity. Even if other communities have different epistemic values than we do, this need not undermine our valuing knowledge, as it is construed in our community (e.g., Sosa 2009: 109; also Lycan 2006). For a pluralist, empirical demonstrations of diversity needn’t undermine traditional philosophical methods, but might instead reveal important epistemic features that we have missed.

A more conservative response to the challenge, which leaves traditional philosophy largely untouched, is to question whether there really is diversity between populations in intuitions about philosophical categories. One way to develop this response is to claim that participants in different populations might simply interpret the scenarios in different ways; in that case, we could explain their different answers by saying that they are responding to different questions (e.g., Sosa 2009).

More importantly, a growing body of empirical evidence has called into question the claim that there really are large differences in philosophical intuitions across populations. Some of the original findings of culture differences have not replicated (e.g., Nagel et al. 2013; Kim & Yuan 2015); similarly, many of the original findings of gender differences haven’t replicated (e.g., Seyedsayamdost 2015; Adleberg et al. 2015). These findings provide strong reason to believe that some of the effects suggested by early experimental philosophy studies do not, in fact, exist at all. Moreover, experimental philosophers have also uncovered robust cross-cultural uniformity. For instance, one recent cross-cultural study examined intuitions about Gettier cases across four very different cultures (Brazil, India, Japan, and the USA), with participants in all groups tending to deny knowledge to the protagonist in Gettier cases (Machery et al. 2015). This suggests that there might be a universal “core folk epistemology” (Machery et al. 2015). In any case, these kinds of results suggest that there is less diversity than had been suggested.

The foregoing argument is based on diversity between populations. But experimental philosophers in the negative program have also used intra-individual diversity to undermine traditional philosophical methods (Swain et al. 2008; Sinnott-Armstrong 2008; Weinberg 2016). Experimental philosophers have found that people’s judgments about philosophical cases are sensitive to various kinds of contextual factors that seem to be philosophically irrelevant. The same person will give different responses depending on apparently irrelevant factors of presentation. People’s judgments about cases are affected by the induction of irrelevant emotions (Cameron et al. 2013), the order in which cases are presented (Petrinovich and O’Neill 1996; Swain et al. 2008; Wright 2010), and the way an outcome is described (e.g., Petrinovich and O’Neill 1996; Schwitzgebel & Cushman 2015).

Sensitivity to contextual factors has been used to challenge the philosophical use of intuition in a way that is somewhat distinct from the diversity argument. The challenge begins with the same assumption about the role of intuitions in philosophy, but then draws on somewhat different considerations:

• S1. The philosophical tradition uses intuitions regarding philosophically important categories or kinds like knowledge in an effort to determine the nature of those categories
• S2. A person’s judgments about philosophical cases are sensitive to contextual factors like the order of presentation.
• S3. Sensitivity to these factors is epistemically inappropriate
• S4. This inappropriate sensitivity cannot be dismissed by privileging the intuitions of one population (e.g., philosophers)
• S5. We can’t tell, from the armchair, which of our judgments are inappropriately sensitive in this way.

This set of claims presents a challenge because it seems that even philosophers are susceptible to these epistemically inapt influences, and we can’t tell which of our intuitions are to be trusted. Thus, philosophers are on shaky epistemic ground when they rely on their intuitions to try to glean philosophical truths.

Obviously, the argument from sensitivity is developed in different ways depending on the category in question and the evidence of sensitivity, but it’s useful to see how the general claims (S1–S5) might be questioned. (See section 3.1 for the rejection of intuition in philosophy (S1) and section 3.2 for a defense of privileged populations [contra S4]).

Although there are replicable effects on the influence of contextual factors, pace S2 many of these effects seem too small to threaten the practice of relying on intuitions (see, e.g., Demaree-Cotton 2016; May 2014). The effect might amount to the difference between 2.2 and 2.5 on a 7 point scale. It’s hard to see how such a difference threatens the practice of relying on the operative intuitions.

In some cases, contextual factors have more pronounced effects, and do lead to changes in participants’ verdict about a case. For instance, judgments about certain moral dilemmas and judgments about certain epistemic cases are changed depending on previously seen cases (e.g., Petrinovich & O’Neill 1996; Swain et al. 2008). However, it’s possible that participants respond differently to a case because the contextual differences actually provide an epistemically appropriate basis for changing one’s judgment. For instance, in the order effect studies, seeing one case can provide evidence about the appropriate response on another case (Horne & Livengood 2016). On this view, we can grant that participants change their judgment, but deny that they are doing so in a way that is epistemically inappropriate.

Finally, even if people’s judgments do change in epistemically inappropriate ways, people might be able to recognize which judgments are especially trustworthy. For instance, only some thought experiments are susceptible to order effects, and it turns out that for these thought experiments, people have lower confidence in their responses (e.g., Wright 2010; Zamzow & Nichols 2009). This suggests (contra D5) that there might be an internal resource—confidence—that can be used to discern which judgments are epistemically unstable.

Research in experimental philosophy has explored many aspects of lay beliefs regarding free will. Experimental philosophers have designed improved scales for measuring belief in free will (Nadelhoffer et al. 2014; Deery et al. 2015), they have investigated the role of the desire to punish in attributing free will (Clark et al. 2014), and they have examined the impact of the belief in free will on moral behavior (Baumeister et al. 2009). But the most intensively studied issue concerns intuitions about whether free will is compatible with determinism.

Experimental philosophers have argued that the philosophical defense of incompatibilism depends on intuitions (e.g., Nahmias et al., 2006). The question about whether incompatibilism is true depends on a wide variety of factors, but experimental philosophers have argued that one factor that plausibly matters is the alleged intuitiveness of the thought that determinism is incompatible with free will (Murray & Nahmias 2014, but see Sommers 2010). This then generates a question that invites an empirical inquiry: is incompatibilism intuitive? (Nahmias et al. 2006).

One of the first experimental studies on free will found that people seemed to have compatibilist intuitions. Participants were presented with a scenario describing a deterministic universe, and then asked whether a person in the scenario was free and morally responsible (Nahmias et al. 2006). In one case, participants were asked to imagine a future scenario in which there is a supercomputer that is capable of predicting all future human behavior when provided with a complete description of the universe along with the laws of nature. In this scenario, a man robs a bank, and participants are asked whether the man is morally responsible for his action. Somewhat surprisingly, most participants gave compatibilist answers, saying that the person was morally blameworthy. This basic finding held across a number of scenarios.

In these early studies on intuitions about free will and moral responsibility, the description of determinism focused on the fact that in a deterministic universe, every event is in principle predictable from the past and the laws. In addition, the scenarios involved particular agents in our world doing bad things. Later studies emphasized the causal nature of determinism—that what happens at a given point is completely caused by what happened previously—and stressed that what happens in a deterministic universe is inevitable given the past. Even with this description of determinism, participants still tend to say that a specific concrete individual in such a universe who commits a heinous crime is free and responsible (Nichols & Knobe 2007; Roskies & Nichols 2008). However, when asked a more abstract question about whether it is possible in general for people in such a deterministic universe to be free and responsible, participants tend to say that morally responsibility is not possible in a deterministic universe. This incompatibilist response was also found in a cross cultural sample with participants from India, Hong Kong, Colombia, and the United States (Sarkissian et al. 2010). In addition to the abstract nature of the question, another important element seems to be whether one is considering an alternate deterministic universe or contemplating the possibility that our own universe is deterministic. When led to consider our own universe as deterministic, participants were more likely to say that people would still be morally responsible (Roskies & Nichols 2008).

Thus, it seems like people give compatibilist responses under some conditions and incompatibilist responses under others. One reaction to this apparent inconsistency is to treat one set of responses as defective. Some experimental philosophers maintain that it’s the incompatibilist responses that don’t reflect people’s true judgments. The best developed version of this view maintains people aren’t affirming incompatibilist responses at all (Nahmias & Murray 2011; Murray & Nahmias 2014). Instead, when people deny free will and responsibility it’s because they misunderstand the description of determinism. In particular, people mistakenly interpret the description of determinism to mean that our mental states lack causal efficacy, that the production of our behavior “bypasses” our mental states. That is, on this view, people wrongly think that determinism means that a person will behave as she does regardless of what she thinks, wants, or intends (Murray & Nahmias 2014).

Of course, if people’s mental states have no impact on their behavior, that is an excellent reason to think that people aren’t morally responsible for their behaviors. So, if people interpret determinism to mean bypassing , it is perfectly rational for them to infer the lack of free will and responsibility from bypassing. However, it seems to be a flat-out confusion to interpret determinism as bypassing. Even if determinism is true, our behavior might be caused (not bypassed) by our mental states. Thus, if people give incompatibilist responses because they confuse determinism with bypassing, then people’s responses don’t reflect a real commitment to incompatibilism.

Surprisingly, people do make bypassing judgments when given a description of causal determinism. For instance, when presented with a description of a determinist universe, many participants agreed that in that universe, “what a person wants has no effect on what they end up doing” (Murray & Nahmias 2014). This suggests that people go through the following confused process: determinism means bypassing, and bypassing means no free will. If that’s right, then the incompatibilist response really is a confusion. However, another explanation is that people think that determinism means no free will, and it’s the denial of free will that leads to the bypassing judgments. The idea would be roughly that if we don’t have free will, then in some way our mental states don’t lead to our behavior in the way we had thought. Some experimental philosophers have used statistical causal modeling to try to tease these two possibilities apart, arguing that it’s the latter explanation that is the right one (Björnsson 2014; Rose & Nichols 2013). That is, people take determinism to entail that there is no free will, and it is this judgment that there is no free will that leads to the bypassing judgment.

Thus, there is some reason to think that incompatibilist responses do reflect many people’s intuitions. What about the compatibilist responses? Some experimental philosophers maintain that it is these judgments that are distorted. On one view, the distortion is caused by emotional reactions (e.g., Nichols & Knobe 2007). However, a meta-analysis indicates that there is very little evidence that emotions play a critical role in generating compatibilist judgments (Feltz & Cova 2014). A different argument for demoting compatibilist judgments holds that many people who affirm free will in deterministic scenarios lack any sensitivity to compatibilist considerations, but instead will affirm free will even under fatalistic conditions in which it is explicitly stipulated that John’s behavior is inevitable “regardless of the past events in John’s life and the laws of nature”. (This view is dubbed “free will no matter what”; Feltz & Millan 2015.) One line of argument based on these results is that if people’s attributions of free will are so insensitive, it can hardly be said that people appreciate the consistency of free will and determinism. However, subsequent studies found that in these fatalistic scenarios, subjects who affirmed free will still tended to think that the source of the action was in the agent, in harmony with “source compatibilism” (Andow & Cova 2016).

Thus, the state of the evidence currently suggests that people do have both incompatibilist and compatibilist intuitions. Future empirical work might uncover more clearly what factors and processes draws people in one direction or the other. There are also open questions about whether the role of different psychological mechanisms in intuitions about free will has implications for philosophical questions for whether we are truly free and responsible.

It is common to distinguish between two kinds of judgments that people make about morally significant situations. On one hand, people can make straightforwardly moral judgments (e.g., judgments about moral wrongness, about obligation, about blameworthiness). On the other, they can make judgments that might be morally relevant but that still appear to be in some important sense non-moral judgments (about whether the agent acted intentionally, whether she caused certain outcomes, whether she knew what she was doing). A question now arises as to how to understand the relationship between these two different kinds of judgments.

One possible view would be that the relationship is entirely unidirectional. Thus, it might be thought that (a) people’s moral judgments depend on prior non-moral judgments, but (b) people’s non-moral judgments do not depend on prior moral judgments. We can illustrate this view with the example of the relationship between people’s moral judgments and their intentional action judgments. It seems clear that people’s moral judgments about whether an agent is deserving of blame might depend on prior non-moral judgments about whether this agent acted intentionally. However, one might think that things do not go in the opposite direction. It is not as though your non-moral judgment that the agent acted intentionally could depend on a prior moral judgment that her action was wrong.

Although this view might seem intuitively compelling, a series of studies in experimental philosophy have called it into question. These studies suggest that people’s moral judgments can impact their judgments even about what might appear to be entirely non-moral questions. Such results have been obtained for a wide variety of different apparently non-moral judgments.

• When an agent knows that she will bring about an outcome but is not specifically trying to bring it about, people are more inclined to say that she brought it about intentionally when it is morally bad than when it is morally good (Knobe 2003).
• When an agent correctly believes that an outcome will arise but is only correct in this belief as the result of a coincidence, people are more inclined to say that she has knowledge when the outcome is morally bad than when it is morally good (Beebe & Shea 2013; Buckwalter 2014).
• When an agent has a lot of positive emotion and a high opinion of her life, people are less inclined to say that she is truly happy when her life is morally bad than when it is morally good (Phillips, Nyholm & Liao 2014).
• When a number of different factors are each individually necessary for an outcome to arise, people are more inclined to regard one of the factors as a cause when it is morally bad than when it is morally good (Alicke 1992; Hitchcock & Knobe 2009).

Effects of moral judgment have also been observed on numerous other judgments, including everything from action individuation (Ulatowski, 2012) to attributions of weakness of will (May & Holton 2012) to the semantics of gradable adjectives (Egré & Cova 2015).

These findings might be philosophically relevant at two different levels. On one hand, each individual effect might be relevant to philosophical work that aims to understand the corresponding concept or property. Thus, the findings about intentional action judgments might be relevant to philosophical work about intentional action, those about happiness judgments might be relevant to philosophical work about happiness, and so forth. At the same time, the general finding that moral judgment has this pervasive influence might be relevant to philosophical work that focuses on the human mind and the way people make sense of the world. For example, these findings could help us to understand the nature of folk psychology or the relationship between our ordinary folk theories and more systematic scientific theories.

To make progress on these two issues, research has focused on trying to understand why these effects arise. That is, researchers have aimed to provide hypotheses about the precise cognitive processes that give rise to the patterns observed in people’s judgments. These hypotheses then, in turn, have implications for philosophical questions both about specific concepts and properties and about the human mind.

Existing research has led to a proliferation of hypotheses, drawing on theoretical frameworks from a variety of fields (see Cova 2016 for a review of seventeen hypotheses about the intentional action effect). Still, although there are numerous distinct specific hypotheses, it seems that the basic approaches can be grouped into four broad families.

First, it might be that the effect is not truly driven by moral judgment . Existing studies show that people make different judgments depending on whether the agent is doing something helpful or harmful, but of course, there are many differences between helpful and harmful actions other than their moral status. For example, a number of researchers have argued that the effect is in fact driven by people’s beliefs about the mental states of the agents in the vignettes (Sloman, Fernbach & Ewing 2012; Sripada & Konrath 2011). Agents will tend to have different sorts of mental states when they are doing something helpful than when they are doing something harmful, and it might be that this difference in mental states is driving all of the observed effects.

Second, it might be that the effect is indeed driven by moral judgment but that it is the result of an error . On this view, moral considerations do not play any real role in the concepts at work here (people’s concepts of intentional action, of happiness, etc.). Rather, people’s judgments are being biased or distorted by some further process which gets in the way of their ability to correctly apply their own concepts. For example, some researchers have argued that the effect is due to a process of motivated cognition (Alicke, Rose & Bloom 2011). People believe the agent to be blameworthy and want to justify that belief. This desire to justify blame then distorts their judgments about what might seem to be purely factual matters.

Third, it might be that the effect is driven by moral judgment and doesn’t involve an error but nonetheless simply reflects a fact about how people use words , rather than a fact about their application of the corresponding concepts. Researchers often make inferences from facts about how people use certain words (‘intentionally,’ ‘happy,’ ‘knows’) about how people apply the corresponding concepts (the concept of intentional action, of happiness, of knowledge). However, it is also possible for factors to influence the use of our words without influencing the use of these concepts, and some researchers have suggested that this is the process at work in the present effects. For example, it has been suggested that these effects arise as a result of conversational pragmatics, with people trying to avoid the pragmatic implicatures that would be generated by making certain claims that are in fact literally true (Adams & Steadman 2004). Alternatively, it has been suggested that the relevant words (e.g., ‘intentionally’) are actually associated with more than one different concept and that the impact of morality arises not because morality plays a role in any of these concepts but rather because it plays a role in the way people resolve the ambiguity of the word itself (Nichols & Ulatowski 2007). On these sorts of views, people are not necessarily making a mistake when their use of language is impacted by moral judgment, but all the same, moral judgment is not playing a role in their more basic capacities to make sense of the world.

Fourth, it might be that moral judgment actually plays a role in people’s basic capacities to apply the relevant concepts . For example, it has been argued that the concept of happiness is itself a value-laden concept (Phillips et al. 2014). Similarly, it has been suggested the concepts of intentional action and causation make use of a form of counterfactual thinking in which moral judgments play a key role (Icard, Kominsky & Knobe 2017; Phillips, Luguri & Knobe 2015). On this last view, the effects observed in these experiments point to a genuine role for moral judgment in the most basic capacities underlying people’s application of the relevant concepts.

Debates between these rival views remain ongoing. Within the more recent literature, discussion of these questions has become increasingly interdisciplinary, with many of the key contributions turning to methods from cognitive neuroscience, developmental psychology, or computational cognitive science.

Within experimental work in epistemology, the primary focus of research has been on the patterns of people’s ordinary attributions of knowledge . As we’ve seen ( section 2.1 ), evidence on epistemic intuitions plays a prominent role in the negative program. But work in experimental epistemology has not been dominated by any one single issue or question. Rather, it has been divided among a number of different strands of research, which have each been pursued separately.

One important topic has been the role of stakes in people’s knowledge attributions. Suppose that Keith considers some available evidence and then concludes (correctly) that the bank will be open on Saturday. Now consider two cases. In the low-stakes case, it is not especially important whether the bank actually is open. By contrast, in the high-stakes case, Keith’s whole financial future depends on whether the bank is open or not. The key question now is whether this difference in stakes has any impact on whether it is correct to say: “Keith knows that the bank will be open”.

Within the non-experimental literature, philosophers have appealed to a wide variety of arguments to help resolve this question. Although many of these arguments do not directly involve people’s intuitions about cases (Brown 2013a; see also Fantl & McGrath 2009; Hawthorne 2004), some specifically rely on the empirical claim that people would be more willing to attribute knowledge when the stakes are low than when the stakes are high (DeRose 1992). Among philosophers who accept this empirical claim, there has been considerable debate about precisely how to explain the purported impact of stakes (DeRose 1992; Hawthorne 2004; Rysiew 2001; Stanley 2005).

Surprisingly, a number of early findings from the experimental epistemology literature suggested that people’s ordinary knowledge attributions actually don’t depend on stakes. For example, people seem to say that Keith knows the bank will be open on Saturday not only in the low-stakes case but also in the high-stakes case (Buckwalter 2010; Feltz & Zarpentine 2010; May et al. 2010). This experimental finding threatens to undermine the entire debate within the non-experimental epistemology literature. After all, if there is no effect of stakes, then there is no question as to how to understand this effect.

Subsequent experimental work in this area has therefore focused on the question as to whether the stakes effect even exists at all. Some have criticized the early experiments that did not find an effect (DeRose 2011). Others have shown that although the effect does not emerge in the experimental paradigms used by those early experiments, it does emerge in other paradigms (Pinillos 2012; Sripada & Stanley 2012; but see Buckwalter & Schaffer 2015, for a critique). Regardless of how these debates are resolved, recent experimental work seems to have established, at a very minimum, that the pattern of people’s epistemic intuitions is not quite the way it was assumed to be within the previous non-experimental literature.

A second question concerns the relationship between knowledge and belief. Clearly, a mental state can only count as knowledge if it satisfies certain conditions that go beyond anything that would be required for the state to count as belief. Thus, there can be cases in which a person believes that p but does not know that p . A question arises, however, as to whether the converse also holds. That is, a question arises as to whether a mental state must satisfy certain conditions to count as a belief that go beyond what would be required for it to count as knowledge. Can there be cases in which a person knows that p but does not believe that p ?

Strikingly, a series of studies suggest that people do attribute knowledge in certain cases in which they would not be willing to attribute belief (Myers-Schulz & Schwitzgebel 2013; see also Murray et al. 2013; Rose & Schaffer 2013; Buckwalter et al. 2015; Shields 2016). In one study, participants were given a vignette about a student taking a history test who faces the question: “What year did Queen Elizabeth die?” She has reviewed this date many times, but at that one moment, she is flustered by the pressure and can’t recall the answer. She therefore decides just to guess, and she writes down ‘1603.’ In fact, this is the correct answer. When given this vignette, experimental participants tended to say that (a) the student knows that Queen Elizabeth died in 1603 but to deny that (b) she believes that Queen Elizabeth died in 1603 (Myers-Schulz & Schwitzgebel 2013, drawing on a vignette from Radford). Similar effects have been obtained for numerous other cases (Murray et al. 2013; Rose & Schaffer 2013; Buckwalter et al. 2015; Shields 2016).

Research in this area aims to understand why this effect arises and what implications it has for epistemology. One view is that people’s concept of belief truly does involve certain conditions that are not required by their concept of knowledge (Myers-Schulz & Schwitzgebel 2013). An alternative view is that there is more than one sense of ‘belief,’ such that knowledge requires the mental state picked out by one of the senses but not the other. Within work that adopts this latter approach, there have been a number of more specific suggestions about how to spell out the difference between the two senses and what relation each has to the ordinary concept of knowledge (Rose & Schaffer 2013; Buckwalter et al. 2015).

Experimental epistemology has also explored numerous other issues. A series of studies indicate that people actually do attribute knowledge in ‘fake barn’ cases (Colaço et al. 2014; Turri 2017). Others show that judgments about whether a person’s mental state counts as knowledge depend on whether that person’s evidence comes from facts about an object itself or from statistical base rates (Friedman & Turri 2015). Still others have explored issues at the intersection of formal semantics and epistemology, exploring the impact of specific linguistic factors on knowledge attributions (Schaffer & Szabó 2014).

We have been focusing in on four specific areas in which there have been especially prominent contributions from experimental philosophy, but we should emphasize that it is not as though the majority of experimental philosophy research falls into one or another of these areas. On the contrary, research in experimental philosophy is highly diverse, and it has actually been getting steadily more heterogeneous in recent years.

First, experimental philosophers have been pursuing an ever more diverse array of topics. On one hand, there has been a surge of experimental research using more formal, mathematical tools, including work on causation using Bayes nets (e.g., Livengood & Rose 2016). and work in formal semantics on everything from gradable adjectives to conditionals to epistemic modals (Liao & Meskin 2017; Cariani & Rips 2017; Khoo 2015). On the other, there has been a proliferation of work addressing core topics in the humanities, including art, religion and even questions at the intersection of experimental philosophy and the history of philosophy (De Cruz & De Smedt 2016; Liao et al. 2014; Nichols 2015).

Secondly one finds an ever-growing diversity of experimental methods. There are still plenty of studies that proceed by giving participants vignettes and asking for their intuitions, but in contemporary experimental philosophy, one also finds studies using corpora (Reuter 2011), reaction times (Philips & Cushman 2017), neuroimaging (Greene et al. 2001), even studies that look at whether ethics professors actually behave ethically (Schwitzgebel & Rust 2014).

Finally, and perhaps most noticeably, there is an increasingly close connection between research in experimental philosophy and research in psychology. For example, the experimental research program on intuitions about trolley problems has been dominated by contributions from psychology (e.g., Cushman et al. 2006; Wiegmann et al. 2012), but there have also been important contributions from philosophers (e.g., Mikhail 2011; Kahane & Shackel 2008). Conversely, there have been numerous recent papers in psychology that aim to contribute to research programs that originated in experimental philosophy (Samland & Waldmann 2016; Feldman & Chandrashekar forthcoming; Starmans & Friedman 2012).

3. Challenges to Experimental Philosophy

As is the case with any healthy research area, there is lots of dispute about issues within experimental philosophy. There are disagreements about particular studies, the implications of different kinds of results, and so on. But there are also broad challenges to the very idea that experimental philosophy research could prove helpful in addressing the philosophical questions. We focus here on three of the most prominent of these challenges.

3.1 Disputing the Role of Intuitions in Philosophy

As we’ve seen, much work in experimental philosophy presupposes that intuitions play an important role in philosophical inquiry. Work in the negative program characteristically starts with the assumption that intuitions play a central role in the philosophical tradition. Outside of the negative program, experimental philosophers want to understand what people’s intuitions are about philosophical matters and why they have these intuitions. Several philosophers, however, challenge the role of intuitions in philosophy in ways that also pose a challenge to the philosophical significance of much experimental philosophy.

One way to reject the role of intuitions is simply to deny that philosophers use intuitions as justification for their views (Williamson 2007; Cappelen 2012; Deutsch 2009, 2010, 2015). According to such “intuition deniers”, the experimental investigation of intuitions is thoroughly irrelevant to philosophy (e.g., Cappelen 2012: 1; for discussion, Nado 2016). Obviously if this is right, then the negative program is arguing against a thoroughly mistaken conception of philosophy.

Although work in metaphilosophy often assumes that philosophers use intuitions as evidence, this is exactly what is challenged by intuition deniers. It is granted on all sides that philosophers sometimes mention intuitions, but according to the intuition deniers, intuitions are not integral to the philosophical work. In particular, intuition deniers maintain that a careful inspection of philosophical practice reveals that philosophers don’t rely on intuitions to justify philosophical views; rather, philosophers rely on arguments (see, e.g., Cappelen 2012: 170; Deutsch 2009: 451).

There have been several responses to the intuition deniers, but perhaps the most prominent response to is that the arguments of intuition deniers depend on an implausibly strong conception of the notion of intuition (e.g., Chalmers 2014; Devitt 2015; Weinberg 2014). Once we focus on a less demanding notion of intuition, it’s plausible that philosophers often rely on intuitions as evidence for philosophical theses (Devitt 2015). Indeed, some have argued that for classic examples like Gettier cases, it’s hard to see how the argument works if it doesn’t rely on intuitions (see, e.g., Brown 2017; Sytsma & Livengood 2015: 92–93) Experimental philosophers have also argued against intuition deniers on experimental grounds, noting that a recent study found that over 50% of philosophers agree with the statement “intuitions are useful for justifying philosophical claims” (Kuntz & Kuntz 2011; see Sytsma & Livengood 2015: 91).

A rather different way to challenge the study of intuitions in experimental philosophy is to deny that the study of intuitions is an apt subject matter for philosophical inquiry. On this view, we can grant that it’s a fact that philosophers rely on intuitions, but it’s a lamentable fact. The use of intuitions in philosophy is misguided for reasons that have nothing in particular to do with experimental philosophy—the appeal to intuitions is a relic, which should be rejected because it doesn’t actually answer the philosophical questions. This conclusion threatens positive applications of experimental philosophy (see, e.g., sections 2.2–2.4 ), but is of course, perfectly consistent with the conclusion urged by the negative program in experimental philosophy ( section 2.1 ).

One influential argument against the use of intuitions builds on the rejection of descriptivist theories of reference, according to which concepts refer to kinds via a set of associated descriptions. In place of descriptivism, some maintain that concepts refer in virtue of the function of the concept (e.g., Millikan 2000). Other views maintain that concepts refer in virtue of a causal chain connecting the concept to the kind (Putnam 1973). On these anti-descriptivist views, people can have wildly mistaken intuitions regarding the application of their concepts. As a result, probing lay intuitions might be an ineffective way to investigate the kinds of things to which our concepts refer (e.g., Fischer 2015; Kornblith 2002).

Anti-descriptivism itself doesn’t entail that appeal to intuitions is philosophically irrelevant. Indeed, some of the most influential arguments against descriptivist theories of reference seem to depend on intuitions (Devitt 2015). However, some argue that rather than relying on intuitions about kinds, we should investigate the kinds themselves. So, if the concept knowledge picks out a natural kind, we can consult the distribution and characteristics of knowledge as it is instantiated in the world. Using intuitions to understand knowledge would be like using intuitions to understand gold. The way we come to understand the nature of gold is to examine samples of gold rather than people’s intuitions about gold. Similarly, the way to understand knowledge is to examine samples of knowledge as it presents in animals, rather than people’s intuitions about knowledge (Kornblith 2002). To examine knowledge by intuitions is at best inefficient, and at worst a complete distraction from the task of understanding what knowledge is. This objection is primarily directed at traditional forms of conceptual analysis, but insofar as experimental philosophy focuses on intuitions, it is in the same leaky boat (Kornblith 2013: 197).

The claim that philosophers shouldn’t rely on intuitions constitutes a broad attack on conceptual analysis, in both its traditional and experimental guises. Not surprisingly, there have been several defenses of the importance of intuitions for doing philosophy. For instance, some philosophers argue that in order even to pick out the kind of interest, we need to rely on our intuitive sense of what belongs in the category (e.g., Goldman 2015). To determine the characteristics of knowledge, we need to have a way of picking out which items are genuine members of the kind, and for this we must rely on our intuitive understanding of knowledge. In addition, if we reject outright the appeal to what intuitively belongs to a category, it’s hard to make sense of the intelligibility of eliminativism (e.g., Bermúdez 2006: 305), since eliminativists typically argue that there is a mismatch between intuitive notions of, e.g., free will, and the kinds of things in the world. To give up on the significance of characterizing our intuitive commitments is to preemptively exclude eliminativist views, which have long been regarded as of central philosophical interest.

A second objection would be that even if intuitions do matter, we should not be concerned with just any old kind of intuition. Rather, our concern should be with a distinctive class of intuitions. For example, research in philosophy has traditionally been conducted by trained philosophers who spent years thinking about difficult problems. There is good reason to suspect that the intuitions generated by this type of process will have a special sort of epistemic status, and perhaps these sorts of intuitions can play a legitimate role in philosophy. By contrast, the intuitions explored within experimental philosophy research tend to be those of ordinary folks, with no prior background in philosophy, and one might think that intuitions of this latter type have no real philosophical significance.

One way of spelling out this concern is in terms of what has come to be known as the expertise objection . The key contention here is that trained philosophers have a distinctive type of expertise. Thus, if we want to understand the process at the core of traditional philosophical practice, we need to study people who have this type of expertise. It is no good just looking at the judgments of people who have never taken a single philosophy course. A number of philosophers have developed objections along more or less these lines though with important differences (Williamson 2007; Ludwig 2007).

This is an important objection, and to address it, experimental philosophers launched a major effort to study the intuitions of trained philosophers. The results show that trained philosophers still show order effects (Schwitzgebel & Cushman 2012), actor/observer effects (Tobia et al. 2013), and effects of temperament (Schulz, Cokely, & Feltz 2011). Thus, existing work provides at least some evidence against the claim that trained philosophers have a distinctive expertise that allows them to escape the sorts of biases that plague the judgments of ordinary folks.

Of course, there are numerous ways of defending the objection against this type of response. It could be argued that although philosophers do not have an ability to avoid biases of the type studied within experimental philosophy, their judgments do differ from those of ordinary folks in some other important respect. Similarly, it could be argued that what gives certain intuitions their privileged epistemic state is not the fact that they come from a particular type of person (trained philosophers) but rather the fact that they are the product of a particular way of approaching the question (sustained reflection) (see, e.g., Kauppinen 2007).

Finally, it might be objected that experimental philosophy simply isn’t philosophy at all. On this view, there are certain properties that differentiate work in philosophy from work in other disciplines. Research in experimental philosophy lacks these properties and is therefore best understood as falling outside the philosophical tradition entirely. Note that this last objection is not concerned with the question as to whether experimental philosophy has any value but rather with the question as to whether it should be considered part of a particular discipline. As one recent paper puts it,

… what is at issue is not whether there is room for such empirical study, but whether there is room for it now as a branch of philosophy . (Sorell forthcoming: 6)

In actual practice, debate over this objection has tended to focus on questions in the history of philosophy. Clearly, numerous philosophers from Aristotle through Nietzsche were deeply concerned with empirical questions about human nature, so it might seem that the default view, at least in the absence of any counterarguments, should be that work on these issues can indeed count as philosophy. The key question, then, is whether there are any legitimate counterarguments.

One possible argument would be that although the people we now regard as philosophers did work on these issues, this aspect of their work should not be regarded as falling within the discipline of philosophy. Anthony Appiah questions this gambit:

You would have a difficult time explaining to most of the canonical philosophers that this part of the work was echt philosophy and that part of their work was not. Trying to separate out the “metaphysical” from the “psychological” elements in this corpus is like trying to peel a raspberry. (Appiah 2008: 13)

According to this response, there is a well-established practice within the history of philosophy of exploring empirical and psychological questions, and it is actually the idea of carefully separating the psychological from the philosophical that should be regarded as a departure from philosophical tradition.

More recent work on these issues has been concerned especially with the early modern period. It has been noted that some of the most prominent philosophers in this period actually conducted experimental studies (Sytsma & Livengood 2015), and some explicitly referred to themselves as ‘experimental philosophers’ (Anstey & Vanzo 2016). Though contemporary experimental philosophy obviously differs in certain respects from these historical antecedents, one might argue that the work of contemporary experimental philosophers is best understood as a continuation of this broad historical tradition.

On the other side, it has been argued that this historical continuity picture fails to take account of a change in the use of the word ‘philosophy’ (Sorell forthcoming). In the Renaissance, physics was referred to as ‘philosophy,’ but we would not say that all research in contemporary physics belongs in the discipline of philosophy. Similarly, even if work on the psychology of moral judgment was historically classified as philosophy, one might think that it should not be regarded today as falling into the discipline of philosophy but rather into a distinct discipline.

Certainly, partisans on both sides of this debate should agree that the boundaries of a discipline can change over time, but this point cuts both ways. Just as the boundaries of a discipline may have changed in the past, they can change in the future. It will therefore be interesting to see how the boundaries of the discipline of philosophy evolve over the course of the next few decades and how this evolution impacts the status of experimental philosophy.

• Adams, Fred & Annie Steadman, 2004, “Intentional Action in Ordinary Language: Core Concept or Pragmatic Understanding?”, Analysis , 64(282): 173–181. doi:10.1111/j.1467-8284.2004.00480.x
• Adleberg, Toni, Morgan Thompson, & Eddy Nahmias, 2015, “Do men and women have different philosophical intuitions? Further data”, Philosophical Psychology , 28(5): 615–641. doi:10.1080/09515089.2013.878834
• Alicke, Mark D., 1992, “Culpable Causation”, Journal of Personality and Social Psychology , 63(3): 368–378. doi:10.1037/0022-3514.63.3.368
• Alicke, Mark D., David Rose, & Dori Bloom, 2011, “Causation, Norm Violation, and Culpable Control”, The Journal of Philosophy , 108(12): 670–696. doi:10.5840/jphil20111081238
• Andow, James & Florian Cova, 2016, “Why Compatibilist Intuitions Are Not Mistaken: A reply to Feltz and Millan” Philosophical Psychology , 29(4): 550–566. doi:10.1080/09515089.2015.1082542
• Anstey, Peter R. & Alberto Vanzo, 2016, “Early Modern Experimental Philosophy”, Sytsma & Buckwalter 2016: 87–102. doi:10.1002/9781118661666.ch6
• Appiah, Kwame Anthony, 2008, Experiments in Ethics , (The Mary Flexner Lectures of Bryn Mawr College), Cambridge, MA: Harvard University Press.
• Bartels, Daniel M. & Oleg Urminsky, 2011, “On Intertemporal Selfishness: How the Perceived Instability of Identity Underlies Impatient Consumption”, Journal of Consumer Research , 38(1): 182–198. doi:10.1086/658339
• Baumeister, Roy F., E.J. Masicampo, & C. Nathan DeWall, 2009, “Prosocial Benefits of Feeling Free: Disbelief in Free Will Increases Aggression and Reduces Helpfulness”, Personality and Social Psychology Bulletin , 35(2): 260–268. doi:10.1177/0146167208327217
• Beebe, James R., & Joseph Shea, 2013, “Gettierized Knobe Effects”, Episteme , 10(3): 219–240. doi:10.1017/epi.2013.23
• Bermúdez, José Luis, 2006, “Knowledge, Naturalism, and Cognitive Ethology: Kornblith’s Knowledge and Its Place in Nature”, Philosophical Studies , 127(2): 299–316. doi:10.1007/s11098-005-4960-z
• Björnsson, Gunnar, 2014, “Incompatibilism and ‘Bypassed’ Agency”, in Surrounding Free Will , edited by Alfred R. Mele, New York: Oxford University Press, pp. 95–122. doi:10.1093/acprof:oso/9780199333950.003.0006
• Björnsson, Gunnar & Derk Pereboom, 2014, “Free Will Skepticism and Bypassing”, in Walter Sinnott-Armstrong (ed.), Moral Psychology, Volume 4: Free Will and Moral Responsibility , Cambridge, MA: MIT Press, pp. 27–35.
• Brown, Jessica, 2013a, “Experimental-Philosophy, Contextualism and Subject-Sensitive Invariantism”, Philosophy and Phenomenological Research , 86(2): 233–494. doi:10.1111/j.1933-1592.2010.00461.x
• –––, 2013b, “Intuitions, Evidence and Hopefulness”. Synthese 190(12): 2021–2046. doi:10.1007/s11229-011-9952-2
• –––, 2017, “Gettier and Philosophical Methodology”, in Rodrigo Borges, Claudio de Almeida, & Peter D. Klein (eds.), Explaining Knowledge: New Essays on the Gettier Problem , Oxford: Oxford University Press, pp. 191–212.
• Buckwalter, Wesley, 2010, “Knowledge isn’t Closed on Saturday: A Study in Ordinary Language”, Review of Philosophy and Psychology , 1(3): 395–406. doi:10.1007/s13164-010-0030-3
• –––, 2014, “Gettier Made ESEE”, Philosophical Psychology , 27(3): 368–383. doi:10.1080/09515089.2012.730965
• Buckwalter, Wesley & Jonathan Schaffer, 2015, “Knowledge, Stakes, and Mistakes”, Noûs , 49(2): 201–234. doi:10.1111/nous.12017
• Buckwalter, Wesley, David Rose, & John Turri, 2015, “Belief Through Thick and Thin”, Noûs , 49(4): 748–775. doi:10.1111/nous.12048
• Buckwalter, Wesley, & Stephen Stich, 2014, “Gender and Philosophical Intuition”, in Experimental Philosophy , volume 2, edited by Joshua Knobe and Shaun Nichols, New York: Oxford University Press, pp. 307–346. doi:10.1093/acprof:osobl/9780199927418.003.0013
• Cameron, C. Daryl, B. Keith Payne, & John M. Doris, 2013, “Morality in High Definition: Emotion Differentiation Calibrates the Influence of Incidental Disgust on Moral Judgments”, Journal of Experimental Social Psychology , 49(4): 719–725. doi:10.1016/j.jesp.2013.02.014
• Cappelen, Herman, 2012, Philosophy Without Intuitions , Oxford: Oxford University Press. doi:10.1093/acprof:oso/9780199644865.001.0001
• Cariani, Fabrizio & Lance J. Rips, 2017, “Conditionals, Context, and the Suppression Effect”, Cognitive Science , 41(3): 540–589. doi:10.1111/cogs.12336
• Chalmers, David J., 2014, “Intuitions in Philosophy: a Minimal Defense”, Philosophical Studies , 171(3): 535–544. doi:10.1007/s11098-014-0288-x
• Clark, Cory J., Jamie B. Luguri, Peter H. Ditto, Joshua Knobe, Azim F. Shariff, & Roy F. Baumeister, 2014, “Free to Punish: a Motivated Account of Free Will Belief”, Journal of Personality and Social Psychology , 106(4): 501–513. doi:10.1037/a0035880
• Colaço, David, Wesley Buckwalter, Stephen Stich, & Edouard Machery, 2014, “Epistemic Intuitions in Fake-Barn Thought Experiments”, Episteme , 11(2): 199–212. doi:10.1017/epi.2014.7
• Cova, Florian, 2016, “The Folk Concept of Intentional Action: Empirical Approaches”, Sytsma & Buckwalter 2016: 117–141. doi:10.1002/9781118661666.ch8
• Cushman, Fiery, Liane Young, & Marc Hauser, 2006, “The Role of Conscious Reasoning and Intuition in Moral Judgment: Testing Three Principles of Harm”, Psychological Science , 17(12): 1082–1089. doi:10.1111/j.1467-9280.2006.01834.x
• De Cruz, Helen & Johan De Smedt, 2016, “How Do Philosophers Evaluate Natural Theological Arguments? An Experimental Philosophical Investigation”, in Advances in Religion, Cognitive Science, and Experimental Philosophy , Helen De Cruz and Ryan Nichols (eds), London: Bloomsbury Publishing, pp. 119–142.
• Demaree-Cotton, Joanna, 2016, “Do Framing Effects Make Moral Intuitions Unreliable?”, Philosophical Psychology , 29(1): 1–22. doi:10.1080/09515089.2014.989967
• Deery, Oisin, Taylor Davis, & Jasmine Carey, 2015, “The Free-Will Intuitions Scale and the Question of Natural Compatibilism”, Philosophical Psychology , 28(6): 776–801. doi:10.1080/09515089.2014.893868
• DeRose, Keith, 1992, “Contextualism and Knowledge Attributions”, Philosophy and Phenomenological Research , 52(4): 913–929. doi:10.2307/2107917
• –––, 2011, “Contextualism, Contrastivism, and X-Phi Surveys”, Philosophical Studies , 156(1): 81–110. doi:10.1007/s11098-011-9799-x
• Deutsch, Max Emil, 2009, “Experimental Philosophy and the Theory of Reference”, Mind & Language , 24(4): 445–466. doi:10.1111/j.1468-0017.2009.01370.x
• –––, 2010, “Intuitions, Counter-Examples, and Experimental Philosophy”, Review of Philosophy and Psychology , 1(3): 447–460. doi:10.1007/s13164-010-0033-0
• –––, 2015, The Myth of the Intuitive: Experimental Philosophy and Philosophical Method , Cambridge, MA: MIT Press. doi:10.7551/mitpress/9780262028950.001.0001
• Devitt, Michael, 2015, “Relying on Intuitions: Where Cappelen and Deutsch Go Wrong”, Inquiry , 58(7–8): 669–699. doi:10.1080/0020174X.2015.1084824
• Egré, Paul & Florian Cova, 2015, “Moral Asymmetries and the Semantics of Many”, Semantics and Pragmatics , 8: 13–1. doi:10.3765/sp.8.13
• Fantl, Jeremy & Matthew McGrath, 2009, Knowledge in an Uncertain World , Oxford: Oxford University Press. doi:10.1093/acprof:oso/9780199550623.001.0001
• Feldman, Gilad & Subramanya Prasad Chandrashekar, forthcoming, “Laypersons’ Beliefs and Intuitions About Free Will and Determinism: New Insights Linking the Social Psychology and Experimental Philosophy Paradigms”, Social Psychological and Personality Science , first published 25 July 2017. doi:10.1177/1948550617713254
• Feltz, Adam & Edward T. Cokely, 2009. “Do Judgments About Freedom and Responsibility Depend on Who You Are? Personality Differences in Intuitions About Compatibilism and Incompatibilism”, Consciousness and Cognition , 18(1): 342–350. doi:10.1016/j.concog.2008.08.001
• Feltz, Adam & Florian Cova, 2014, “Moral Responsibility and Free Will: A Meta-Analysis”, Consciousness and Cognition , 30: 234–246. doi:10.1016/j.concog.2014.08.012
• Feltz, Adam & Melissa Millan, 2015, “An Error Theory for Compatibilist Intuitions”, Philosophical Psychology , 28(4): 529–555. doi:10.1080/09515089.2013.865513
• Feltz, Adam & Chris Zarpentine, 2010, “Do You Know More When it Matters Less?”, Philosophical Psychology , 23(5): 683–706. doi:10.1080/09515089.2010.514572
• Fisher, Justin C., 2015, “Pragmatic Experimental Philosophy”, Philosophical Psychology , 28(3): 412–433. doi:10.1080/09515089.2013.870546
• Friedman, Ori & John Turri, 2015, “Is Probabilistic Evidence a Source of Knowledge?”, Cognitive Science , 39(5): 1062–1080. doi:10.1111/cogs.12182
• Friesdorf, Rebecca, Paul Conway, & Bertram Gawronski, 2015, “Gender Differences in Responses to Moral Dilemmas: a Process Dissociation Analysis”, Personality and Social Psychology Bulletin , 41(5): 696–713. doi:10.1177/0146167215575731
• Gerken, Mikkel, 2017, On Folk Epistemology: How we Think and Talk about Knowledge , Oxford: Oxford University Press.
• Goldman, Alvin I., 2015. “Review of Hilary Kornblith, A Naturalistic Epistemology: Selected Papers ”, Notre Dame Philosophical Reviews , 2015.06.11. [ Goldman 2015 available online ]
• Greene, Joshua D., 2008, “The Secret Joke of Kant’s Soul”, Moral Psychology, Volume 3 , Walter Sinnott-Armstrong (ed.), Cambridge, MA: MIT Press, pp. 35–79.
• Greene, Joshua D., R. Brian Sommerville, Leigh E. Nystrom, John M. Darley, & John D. Cohen, 2001, “An fMRI Investigation of Emotional Engagement in Moral Judgment”, Science , 293(5537): 2105–2108. doi:10.1126/science.1062872
• Hawthorne, John, 2004, Knowledge and Lotteries , Oxford: Oxford University Press. doi:10.1093/0199269556.001.0001
• Hitchcock, Christopher & Joshua Knobe, 2009, “Cause and Norm”, The Journal of Philosophy , 106(11): 587–612. doi:10.5840/jphil20091061128
• Horne, Zachary & Jonathan Livengood, 2017, “Ordering Effects, Updating Effects, and the Specter of Global Skepticism”, Synthese , 194(4): 1189–1218. doi:10.1007/s11229-015-0985-9
• Icard, Thomas F., Jonathan F. Kominsky, & Joshua Knobe, 2017, “Normality and Actual Causal Strength”, Cognition , 161: 80–93. doi:10.1016/j.cognition.2017.01.010
• Ichikawa, Jonathan Jenkins, 2012, “Experimentalist Pressure Against Traditional Methodology”, Philosophical Psychology , 25(5): 743–765. doi:10.1080/09515089.2011.625118
• Jackson, Frank, 1998, From Metaphysics to Ethics: A Defence of Conceptual Analysis , Oxford: Clarendon Press. doi:10.1093/0198250614.001.0001
• Kahane, Guy & Nicholas Shackel, 2008, “Do Abnormal Responses Show Utilitarian Bias?”, Nature , 452(7185): 908–911. doi:10.1038/nature06785
• Kauppinen, Antti, 2007, “The Rise and Fall of Experimental Philosophy”, Philosophical Explorations , 10(2): 95–118. doi:10.1080/13869790701305871
• Khoo, Justin, 2015, “Modal Disagreements”, Inquiry , 58(5): 511–534. doi:10.1080/0020174X.2015.1033005
• Kim, Minsun, & Yuan Yuan, 2015, “No Cross-Cultural Differences in the Gettier Car Case Intuition: a Replication Study of Weinberg et al. 2001”, Episteme , 12(03): 355–361. doi:10.1017/epi.2015.17
• Knobe, Joshua, 2003, “Intentional Action and Side Effects in Ordinary Language”, Analysis , 63(279): 190–194. doi:10.1111/1467-8284.00419
• Knobe, Joshua, Tania Lombrozo, & Shaun Nichols (eds), 2014, Oxford Studies in Experimental Philosophy , volume 1, Oxford: Oxford University Press.
• Kornblith, Hilary, 2002, Knowledge and Its Place in Nature , Oxford: Clarendon. doi:10.1093/0199246319.001.0001
• –––, 2013, “Is There Room for Armchair Theorizing in Epistemology?”, in Matthew C. Haug (ed.), Philosophical Methodology: The Armchair or the Laboratory? , New York: Routledge, pp. 195–216.
• Kuntz, J.R. & J.R.C. Kuntz, 2011, “Surveying Philosophers about Philosophical Intuition”, Review of Philosophy and Psychology 2(4): 643–65. doi:10.1007/s13164-011-0047-2
• Leslie, Sarah-Jane, 2013, “Essence and Natural Kinds: When Science Meets Preschooler Intuition”, in Tamar Szabó & John Hawthorne, Oxford Studies in Epistemology, volume 4 , Oxford: Oxford University Press. pp. 108–165. doi:10.1093/acprof:oso/9780199672707.003.0005
• Liao, Shen-yi & Aaron Meskin, 2017, “Aesthetic Adjectives: Experimental Semantics and Context-Sensitivity”, Philosophy and Phenomenological Research , 94(2): 371–398. doi:10.1111/phpr.12217
• Liao, Shen-yi, Nina Strohminger, & Chandra Sekhar Sripada, 2014, “Empirically Investigating Imaginative Resistance”, British Journal of Aesthetics , 54(3): 339–355. doi:10.1093/aesthj/ayu027
• Livengood, Jonathan & David Rose, 2016, “Experimental Philosophy and Causal Attribution”, Sytsma & Buckwalter 2016: 434–449. doi:10.1002/9781118661666.ch30
• Ludwig, Kirk, 2007, “The Epistemology of Thought Experiments: First Person Versus Third Person Approaches”, Midwest Studies in Philosophy , 31(1): 128–159. doi:10.1111/j.1475-4975.2007.00160.x
• Lycan, William G., 2006, “On the Gettier Problem Problem”, in Stephen Hetherington (ed.), Epistemology Futures , Oxford: Clarendon Press, pp. 148–168, .
• Machery, Edouard, Ron Mallon, Shaun Nichols, & Stephen P. Stich, 2004, “Semantics, Cross-Cultural Style”, Cognition , 92(3): B1–B12. doi:10.1016/j.cognition.2003.10.003
• Machery, Edouard, Stephen Stich, David Rose, Amita Chatterjee, Kaori Karasawa, Noel Struchiner, Smita Sirker, Naoki Usui, & Takaaki Hashimoto, 2015, “Gettier Across Cultures”, Noûs , 51(3): 645–664. doi:10.1111/nous.12110
• May, Joshua, 2014, “Does Disgust Influence Moral Judgment?”, Australasian Journal of Philosophy , 92(1): 125–141. doi:10.1080/00048402.2013.797476
• May, Joshua & Richard Holton, 2012, “What in the World is Weakness of Will?”, Philosophical Studies , 157(3): 341–360. doi:10.1007/s11098-010-9651-8
• May, Joshua, Walter Sinnott-Armstrong, Jay G. Hull, & Aaron Zimmerman, 2010, “Practical Interests, Relevant Alternatives, and Knowledge Attributions: An Empirical Study”, Review of Philosophy and Psychology , 1(2): 265–273. doi:10.1007/s13164-009-0014-3
• Meskin, Aaron, Mark Phelan, Margaret Moore, & Matthew Kieran, 2013, “Mere Exposure to Bad Art”, The British Journal of Aesthetics , 53(2): 139–164. doi:10.1093/aesthj/ays060
• Mikhail, John, 2011, Elements of Moral Cognition: Rawls’ Linguistic Analogy and the Cognitive Science of Moral and Legal Judgment , Cambridge: Cambridge University Press. doi:10.1017/CBO9780511780578
• Millikan, Ruth Garrett, 2000, On Clear and Confused Ideas: An Essay about Substance Concepts , Cambridge: Cambridge University Press. doi:10.1017/CBO9780511613296
• Murray, Dylan & Eddy Nahmias, 2014, “Explaining Away Incompatibilist Intuitions”, Philosophy and Phenomenological Research , 88(2): 434–467. doi:10.1111/j.1933-1592.2012.00609.x
• Murray, Dylan, Justin Sytsma, & Jonathan Livengood, 2013, “God Knows (But Does God Believe?)”, Philosophical Studies , 166(1): 83–107. doi:10.1007/s11098-012-0022-5
• Myers-Schulz, Blake & Eric Schwitzgebel, 2013, “Knowing that P Without Believing that P”, Noûs , 47(2): 371–384. doi:10.1111/nous.12022
• Nadelhoffer, Thomas, Jason Shepard, Eddy Nahmias, Chandra Sripada, & Lisa Thomson Ross, 2014, “The Free Will Inventory: Measuring Beliefs About Agency and Responsibility”, Consciousness and Cognition , 25: 27–41. doi:10.1016/j.concog.2014.01.006
• Nado, Jennifer, 2016, “The Intuition Deniers”, Philosophical Studies , 173(3): 781–800. doi:10.1007/s11098-015-0519-9
• Nagel, Jennifer, 2010, “Knowledge Ascriptions and the Psychological Consequences of Thinking About Error”, The Philosophical Quarterly , 60(239): 286–306. doi:10.1111/j.1467-9213.2009.624.x
• Nagel, Jennifer, Valerie San Juan, , & Raymond A. Mar, 2013, “Lay Denial of Knowledge for Justified True Beliefs”, Cognition , 129(3): 652–661. doi:10.1016/j.cognition.2013.02.008
• Nahmias, Eddy, Stephen G. Morris, Thomas Nadelhoffer, & Jason Turner, 2006, “Is Incompatibilism Intuitive?”, Philosophy and Phenomenological Research , 73(1): 28–53. doi:10.1111/j.1933-1592.2006.tb00603.x
• Nahmias, Eddy & Dylan Murray, 2011, “Experimental Philosophy on Free Will: An Error Theory for Incompatibilist Intuitions”, in New Waves in Philosophy of Action , Jesús H. Aguilar, Andrei A. Bckaref, & Keith Frankish (eds), London: Palgrave Macmillan UK, pp. 189–216.
• Nichols, Ryan, 2015, “Hypothesis-Testing of the Humanities: the Hard and Soft Humanities as Two Emerging Cultures”, Southwest Philosophy Review , 31(1): 1–19. doi:10.5840/swphilreview20153111
• Nichols, Shaun & Joshua Knobe, 2007, “Moral Responsibility and Determinism: the Cognitive Science of Folk Intuitions”, Nous , 41(4): 663–685. doi:10.1111/j.1468-0068.2007.00666.x
• Nichols, Shaun & Joseph Ulatowski, 2007, “Intuitions and Individual Differences: the Knobe Effect Revisited”, Mind & Language , 22(4): 346–365. doi:10.1111/j.1468-0017.2007.00312.x
• Petrinovich, Lewis & Patricia O’Neill, 1996, “Influence of Wording and Framing Effects on Moral Intuitions”, Ethology and Sociobiology , 17(3): 145–171. doi:10.1016/0162-3095(96)00041-6
• Phillips, Jonathan & Fiery Cushman, 2017, “Morality Constrains the Default Representation of What is Possible”, Proceedings of the National Academy of Sciences , 114(18): 4649–4654. doi:10.1073/pnas.1619717114
• Phillips, Jonathan, James B. Luguri, & Joshua Knobe, 2015, “Unifying Morality’s Influence on Non-Moral Judgments: the Relevance of Alternative Possibilities”, Cognition , 145: 30–42. doi:10.1016/j.cognition.2015.08.001
• Phillips, Jonathan, Sven Nyholm, & Shen-yi Liao, 2014, “The Good in Happiness”, in Knobe, Lombrozo, & Nichols 2014: 253–293. doi:10.1093/acprof:oso/9780198718765.003.0011
• Pinillos, Ángel, 2012, “Knowledge, Experiments and Practical Interests”, Knowledge Ascriptions , Jessica Brown & Mikkel Gerken (eds), Oxford: Oxford University Press, pp. 192–219. doi:10.1093/acprof:oso/9780199693702.003.0009
• Plato, Theaetetus , translated by B. Jowett, in B. Jowett, (ed.), The Dialogues of Plato , Oxford: Oxford University Press, 1892.
• Putnam, Hilary, 1973, “Meaning and Reference”, The Journal of Philosophy , 70(19): 699–711. doi:10.2307/2025079
• Radford, Colin, 1966, “Knowledge: By Examples”, Analysis , 27(1): 1–11. doi:10.2307/3326979
• Reuter, Kevin, 2011, “Distinguishing the Appearance from the Reality of Pain”, Journal of Consciousness Studies , 18(9–10): 94–109
• Rose, David & Shaun Nichols, 2013, “The Lesson of Bypassing”, Review of Philosophy and Psychology , 4(4): 599–619. doi:10.1007/s13164-013-0154-3
• Rose, David & Jonathan Schaffer, 2013, “Knowledge Entails Dispositional Belief”, Philosophical Studies , 166(1): 19–50. doi:10.1007/s11098-012-0052-z
• Roskies, Adina L. & Shaun Nichols, 2008, “Bringing Moral Responsibility Down to Earth”, The Journal of Philosophy , 105(7): 371–388. doi:10.5840/jphil2008105737
• Rysiew, Patrick, 2001, “The Context-Sensitivity of Knowledge Attributions”, Noûs , 35(4): 477–514. doi:10.1111/0029-4624.00349
• Samland, Jana, & Michael R. Waldmann, 2016, “How Prescriptive Norms Influence Causal Inferences”, Cognition , 156: 164–176. doi:10.1016/j.cognition.2016.07.007
• Sarkissian, Hagop, Amita Chatterjee, Felipe De Brigard, Joshua Knobe, Shaun Nichols, & Smita Sirker, 2010, “Is Belief in Free Will a Cultural Universal?”, Mind & Language , 25(3): 346–358. doi:10.1111/j.1468-0017.2010.01393.x
• Schaffer, Jonathan & Zoltán Gendler Szabó, 2014, “Epistemic Comparativism: a Contextualist Semantics for Knowledge Ascriptions”, Philosophical Studies , 168(2): 491–543. doi:10.1007/s11098-013-0141-7
• Schulz, Eric, Edward T. Cokely, & Adam Feltz, 2011, “Persistent Bias in Expert Judgments About Free Will and Moral Responsibility: a Test of the Expertise Defense”, Consciousness and Cognition , 20(4): 1722–1731. doi:10.1016/j.concog.2011.04.007
• Schwitzgebel, Eric & Fiery Cushman, 2015, “Philosophers’ Biased Judgments Persist Despite Training, Expertise and Reflection”, Cognition , 141: 127–137. doi:10.1016/j.cognition.2015.04.015
• Schwitzgebel, Eric & Joshua Rust, 2014, “The Moral Behavior of Ethics Professors: Relationships Among Self-Reported Behavior, Expressed Normative Attitude, and Directly Observed Behavior”, Philosophical Psychology , 27(3): 293–327. doi:10.1080/09515089.2012.727135
• Seyedsayamdost, Hamid, 2015, “On Gender and Philosophical Intuition: Failure of Replication and Other Negative Results”, Philosophical Psychology , 28(5): 642–673. doi:10.1080/09515089.2014.893288
• Shields, Kenneth, 2016, “Moral Internalism, Amoralist Skepticism and the Factivity Effect”, Philosophical Psychology , 29(8): 1095–1111. doi:10.1080/09515089.2016.1234596
• Sinnott-Armstrong, Walter, 2008, “Framing Moral Intuitions”, in Walter Sinnott-Armstrong (ed.), Moral Psychology, Volume 2: The Cognitive Science of Morality , Cambridge, MA: MIT Press, pp. 47–76.
• Sloman, Steven A., Philip M. Fernbach, & Scott Ewing, 2012, “A Causal Model of Intentionality Judgment”, Mind & Language , 27(2): 154–180. doi:10.1111/j.1468-0017.2012.01439.x
• Sommers, Tamler, 2010, “Experimental Philosophy and Free Will”, Philosophy Compass , 5(2): 199–212. doi:10.1111/j.1747-9991.2009.00273.x
• Sorell, Tom, forthcoming, “Experimental Philosophy and the History of Philosophy”, British Journal for the History of Philosophy , published online 18 May 2017, pp. 1–21. doi:10.1080/09608788.2017.1320971
• Sosa, Ernest, 2007, “Experimental Philosophy and Philosophical Intuition”, Philosophical Studies , 132(1): 99–107. doi:10.1007/s11098-006-9050-3
• –––, 2009, “A Defense of the Use of Intuitions in Philosophy”, in Dominic Murphy and Michael Bishop (eds), Stich and his Critics , Malden, MA: Wiley-Blackwell, pp. 101–112.
• Sripada, Chandra Sekhar, & Sara Konrath, 2011, “Telling More Than We Can Know About Intentional Action”, Mind & Language , 26(3): 353–380. doi:10.1111/j.1468-0017.2011.01421.x
• Sripada, Chandra Sekhar, & Jason Stanley, 2012, “Empirical Tests of Interest-Relative Invariantism”, Episteme , 9(01): 3–26. doi:10.1017/epi.2011.2
• Stanley, Jason, 2005, Knowledge and Practical Interests , Oxford: Clarendon Press. doi:10.1093/0199288038.001.0001
• Starmans, Christina & Ori Friedman, 2012, “The Folk Conception of Knowledge”, Cognition , 124(3): 272–283. doi:10.1016/j.cognition.2012.05.017
• Stich, Stephen, 1990, The Fragmentation of Reason: Preface to a Pragmatic Theory of Cognitive Evaluation , Cambridge, MA: MIT Press.
• Swain, Stacey, Joshua Alexander, & Jonathan M. Weinberg, 2008, “The Instability of Philosophical Intuitions: Running Hot and Cold on Truetemp”, Philosophy and Phenomenological Research , 76(1): 138–155. doi:10.1111/j.1933-1592.2007.00118.x
• Sytsma, Justin & Wesley Buckwalter (eds), 2016, A Companion to Experimental Philosophy , Malden, MA: Wiley Blackwell. doi:10.1002/9781118661666
• Sytsma, Justin & Jonathan Livengood, 2015, The Theory and Practice of Experimental Philosophy , Peterborough, Ontario: Broadview Press.
• Tobia, Kevin, Wesley Buckwalter, & Stephen Stich, 2013, “Moral Intuitions: Are Philosophers Experts?”, Philosophical Psychology , 26(5): 629–638. doi:10.1080/09515089.2012.696327
• Turri, John, 2017, “Knowledge Attributions in Iterated Fake Barn Cases”, Analysis , 77(1): 104–115. doi:10.1093/analys/anx036
• Ulatowski, Joseph, 2012, “Act Individuation: An Experimental Approach”, Review of Philosophy and Psychology , 3(2): 249–262. doi:10.1007/s13164-012-0096-1
• Vargas, Manuel, 2013, Building Better Beings: A Theory of Moral Responsibility , Oxford: Oxford University Press. doi:10.1093/acprof:oso/9780199697540.001.0001
• Weinberg, Jonathan M., 2007, “How to Challenge Intuitions Empirically Without Risking Scepticism”, Midwest Studies in Philosophy , 31(1): 318–343. doi:10.1111/j.1475-4975.2007.00157.x
• –––, 2014, “Cappelen between a Rock and a Hard Place”, Philosophical Studies , 171(3): 545–553. doi:10.1007/s11098-014-0286-z
• –––, 2016, “Going Positive by Going Negative”, in Justin Sytsma & Wesley Buckwalter (eds.), A Companion to Experimental Philosophy , Malden, MA and Oxford: John Wiley & Sons, pp. 72–86.
• –––, 2017, “What is Negative Experimental Philosophy Good For?”, in Giuseppina D’Oro & Søren Overgaard (eds.) The Cambridge Companion to Philosophical Methodology , Cambridge: Cambridge University Press, pp. 161–184. doi:10.1017/9781316344118.010
• Weinberg, Jonathan M., Chad Gonnerman, Cameron Buckner, & Joshua Alexander, 2010, “Are Philosophers Expert Intuiters?”, Philosophical Psychology , 23(3): 331–355. doi:10.1080/09515089.2010.490944
• Weinberg, Jonathan M., Shaun Nichols, & Stephen Stich, 2001, “Normativity and Epistemic Intuitions”, Philosophical Topics , 29(1/2): 429–460. doi:10.5840/philtopics2001291/217
• Wiegmann, Alex, Yasmina Okan, & Jonas Nagel, 2012, “Order Effects in Moral Judgment”, Philosophical Psychology , 25(6): 813–836. doi:10.1080/09515089.2011.631995
• Williamson, Timothy, 2007, The Philosophy of Philosophy , Oxford: Blackwell Publishing.
• Woodward, James, 2014, “Causal Reasoning: Philosophy and Experiment”, in Knobe, Lombrozo, & Nichols 2014: 294–324. doi:10.1093/acprof:oso/9780198718765.003.0012
• Wright, Jennifer C., 2010, “On Intuitional Stability: the Clear, the Strong, and the Paradigmatic”, Cognition , 115(3): 491–503. doi:10.1016/j.cognition.2010.02.003
• Zamzow, Jennifer L. & Shaun Nichols, 2009, “Variations in Ethical Intuitions”, Philosophical Issues , 19(1): 368–388. doi:10.1111/j.1533-6077.2009.00164.x

How to cite this entry . Preview the PDF version of this entry at the Friends of the SEP Society . Look up topics and thinkers related to this entry at the Internet Philosophy Ontology Project (InPhO). Enhanced bibliography for this entry at PhilPapers , with links to its database.

• PhilPapers Experimental Philosophy , edited by Wesely Buckwalter.

experimental moral philosophy | intuition | Plato: on knowledge in the Theaetetus

Acknowledgments

We’d like to thank Jonathan Weinberg, the editors, and an anonymous referee for helpful comments on some of this material.

Copyright © 2017 by Joshua Knobe Shaun Nichols

• Accessibility

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2023 by The Metaphysics Research Lab , Department of Philosophy, Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Qualitative tools and experimental philosophy

Affiliation.

• 1 Department of Philosophy, University of Reading , Reading , UK.
• PMID: 28392629
• PMCID: PMC5359736
• DOI: 10.1080/09515089.2016.1224826

Experimental philosophy brings empirical methods to philosophy. These methods are used to probe how people think about philosophically interesting things such as knowledge, morality, and freedom. This paper explores the contribution that qualitative methods have to make in this enterprise. I argue that qualitative methods have the potential to make a much greater contribution than they have so far. Along the way, I acknowledge a few types of resistance that proponents of qualitative methods in experimental philosophy might encounter, and provide reasons to think they are ill-founded.

Keywords: Conceptual analysis; experimental philosophy; intuitions; intuitive processing; philosophical methodology; qualitative methods; quantitative methods; reflective processing.

PubMed Disclaimer

Similar articles

• On the pursuitworthiness of qualitative methods in empirical philosophy of science. Hangel N, ChoGlueck C. Hangel N, et al. Stud Hist Philos Sci. 2023 Apr;98:29-39. doi: 10.1016/j.shpsa.2022.12.009. Epub 2023 Feb 25. Stud Hist Philos Sci. 2023. PMID: 36842291
• Philosophical producers, philosophical consumers, and the metaphilosophical value of original texts. Landes E. Landes E. Philos Stud. 2023;180(1):207-225. doi: 10.1007/s11098-022-01900-8. Epub 2022 Oct 29. Philos Stud. 2023. PMID: 36691421 Free PMC article.
• Experimental philosophy of medicine and the concepts of health and disease. Veit W. Veit W. Theor Med Bioeth. 2021 Aug;42(3-4):169-186. doi: 10.1007/s11017-021-09550-3. Epub 2021 Dec 1. Theor Med Bioeth. 2021. PMID: 34853990 Free PMC article.
• Experimental philosophy. Knobe J, Buckwalter W, Nichols S, Robbins P, Sarkissian H, Sommers T. Knobe J, et al. Annu Rev Psychol. 2012;63:81-99. doi: 10.1146/annurev-psych-120710-100350. Epub 2011 Jul 29. Annu Rev Psychol. 2012. PMID: 21801019 Review.
• The future of philosophy. Searle JR. Searle JR. Philos Trans R Soc Lond B Biol Sci. 1999 Dec 29;354(1392):2069-80. doi: 10.1098/rstb.1999.0544. Philos Trans R Soc Lond B Biol Sci. 1999. PMID: 10670025 Free PMC article. Review.
• Broad, subjective, relative: the surprising folk concept of basic needs. Pölzler T, Tomabechi T, Hannikainen IR. Pölzler T, et al. Philos Stud. 2024;181(1):319-347. doi: 10.1007/s11098-023-02080-9. Epub 2023 Dec 21. Philos Stud. 2024. PMID: 38268665 Free PMC article.
• Does suffering suffice? An experimental assessment of desert retributivism. Bauer PC, Poama A. Bauer PC, et al. PLoS One. 2020 Apr 20;15(4):e0230304. doi: 10.1371/journal.pone.0230304. eCollection 2020. PLoS One. 2020. PMID: 32310957 Free PMC article. Clinical Trial.
• New Issues for New Methods: Ethical and Editorial Challenges for an Experimental Philosophy. Polonioli A. Polonioli A. Sci Eng Ethics. 2017 Aug;23(4):1009-1034. doi: 10.1007/s11948-016-9838-2. Epub 2016 Nov 28. Sci Eng Ethics. 2017. PMID: 27896613 Free PMC article.
• Ahlenius H., & Tännsjö T. (2012). Chinese and westerners respond differently to the trolley dilemmas. Journal of Cognition and Culture , , 195–201.
• Alexander J. (2012). Experimental philosophy: An introduction. Cambridge: Polity.
• Andow J., & Cova F. (2016). Why compatibilist intuitions are not mistaken: A reply to Feltz and Millan. Philosophical Psychology , , 550–566.
• Baldacchino L., Ucbasaran D., Lockett A., & Cabantous L. (2014). Capturing intuition through concurrent protocol analysis In Sinclair M. (Ed.), Handbook of research methods on intuition (pp. 160–175). Cheltenham: Elgar.
• Berniūnas R., & Dranseika V. (2016). Folk concepts of person and identity: A response to Nichols and Bruno. Philosophical Psychology , , 96–122.

LinkOut - more resources

Full text sources.

• Europe PubMed Central
• PubMed Central

Other Literature Sources

• scite Smart Citations
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

The Four Types of Research Paradigms: A Comprehensive Guide

• 5-minute read
• 22nd January 2023

In this guide, you’ll learn all about the four research paradigms and how to choose the right one for your research.

Introduction to Research Paradigms

A paradigm is a system of beliefs, ideas, values, or habits that form the basis for a way of thinking about the world. Therefore, a research paradigm is an approach, model, or framework from which to conduct research. The research paradigm helps you to form a research philosophy, which in turn informs your research methodology.

Your research methodology is essentially the “how” of your research – how you design your study to not only accomplish your research’s aims and objectives but also to ensure your results are reliable and valid. Choosing the correct research paradigm is crucial because it provides a logical structure for conducting your research and improves the quality of your work, assuming it’s followed correctly.

Three Pillars: Ontology, Epistemology, and Methodology

Before we jump into the four types of research paradigms, we need to consider the three pillars of a research paradigm.

Ontology addresses the question, “What is reality?” It’s the study of being. This pillar is about finding out what you seek to research. What do you aim to examine?

Epistemology is the study of knowledge. It asks, “How is knowledge gathered and from what sources?”

Methodology involves the system in which you choose to investigate, measure, and analyze your research’s aims and objectives. It answers the “how” questions.

Let’s now take a look at the different research paradigms.

1.   Positivist Research Paradigm

The positivist research paradigm assumes that there is one objective reality, and people can know this reality and accurately describe and explain it. Positivists rely on their observations through their senses to gain knowledge of their surroundings.

In this singular objective reality, researchers can compare their claims and ascertain the truth. This means researchers are limited to data collection and interpretations from an objective viewpoint. As a result, positivists usually use quantitative methodologies in their research (e.g., statistics, social surveys, and structured questionnaires).

This research paradigm is mostly used in natural sciences, physical sciences, or whenever large sample sizes are being used.

2.   Interpretivist Research Paradigm

Interpretivists believe that different people in society experience and understand reality in different ways – while there may be only “one” reality, everyone interprets it according to their own view. They also believe that all research is influenced and shaped by researchers’ worldviews and theories.

As a result, interpretivists use qualitative methods and techniques to conduct their research. This includes interviews, focus groups, observations of a phenomenon, or collecting documentation on a phenomenon (e.g., newspaper articles, reports, or information from websites).

3.   Critical Theory Research Paradigm

The critical theory paradigm asserts that social science can never be 100% objective or value-free. This paradigm is focused on enacting social change through scientific investigation. Critical theorists question knowledge and procedures and acknowledge how power is used (or abused) in the phenomena or systems they’re investigating.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

Researchers using this paradigm are more often than not aiming to create a more just, egalitarian society in which individual and collective freedoms are secure. Both quantitative and qualitative methods can be used with this paradigm.

4.   Constructivist Research Paradigm

Constructivism asserts that reality is a construct of our minds ; therefore, reality is subjective. Constructivists believe that all knowledge comes from our experiences and reflections on those experiences and oppose the idea that there is a single methodology to generate knowledge.

This paradigm is mostly associated with qualitative research approaches due to its focus on experiences and subjectivity. The researcher focuses on participants’ experiences as well as their own.

Choosing the Right Research Paradigm for Your Study

Once you have a comprehensive understanding of each paradigm, you’re faced with a big question: which paradigm should you choose? The answer to this will set the course of your research and determine its success, findings, and results.

To start, you need to identify your research problem, research objectives , and hypothesis . This will help you to establish what you want to accomplish or understand from your research and the path you need to take to achieve this.

You can begin this process by asking yourself some questions:

• What is the nature of your research problem (i.e., quantitative or qualitative)?
• How can you acquire the knowledge you need and communicate it to others? For example, is this knowledge already available in other forms (e.g., documents) and do you need to gain it by gathering or observing other people’s experiences or by experiencing it personally?
• What is the nature of the reality that you want to study? Is it objective or subjective?

Depending on the problem and objective, other questions may arise during this process that lead you to a suitable paradigm. Ultimately, you must be able to state, explain, and justify the research paradigm you select for your research and be prepared to include this in your dissertation’s methodology and design section.

Using Two Paradigms

If the nature of your research problem and objectives involves both quantitative and qualitative aspects, then you might consider using two paradigms or a mixed methods approach . In this, one paradigm is used to frame the qualitative aspects of the study and another for the quantitative aspects. This is acceptable, although you will be tasked with explaining your rationale for using both of these paradigms in your research.

Choosing the right research paradigm for your research can seem like an insurmountable task. It requires you to:

●  Have a comprehensive understanding of the paradigms,

●  Identify your research problem, objectives, and hypothesis, and

●  Be able to state, explain, and justify the paradigm you select in your methodology and design section.

Although conducting your research and putting your dissertation together is no easy task, proofreading it can be! Our experts are here to make your writing shine. Your first 500 words are free !

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

9-minute read

How to Use Infographics to Boost Your Presentation

Is your content getting noticed? Capturing and maintaining an audience’s attention is a challenge when...

8-minute read

Why Interactive PDFs Are Better for Engagement

Are you looking to enhance engagement and captivate your audience through your professional documents? Interactive...

7-minute read

Seven Key Strategies for Voice Search Optimization

Voice search optimization is rapidly shaping the digital landscape, requiring content professionals to adapt their...

4-minute read

Five Creative Ways to Showcase Your Digital Portfolio

Are you a creative freelancer looking to make a lasting impression on potential clients or...

How to Ace Slack Messaging for Contractors and Freelancers

Effective professional communication is an important skill for contractors and freelancers navigating remote work environments....

3-minute read

How to Insert a Text Box in a Google Doc

Google Docs is a powerful collaborative tool, and mastering its features can significantly enhance your...

Make sure your writing is the best it can be with our expert English proofreading and editing.

Estimating the Reproducibility of Experimental Philosophy

• Published: 14 June 2018
• Volume 12 , pages 9–44, ( 2021 )

Cite this article

• Florian Cova   ORCID: orcid.org/0000-0001-9360-8290 1 , 2 ,
• Brent Strickland 3 , 4 ,
• Angela Abatista 5 ,
• Aurélien Allard 6 ,
• James Andow 7 ,
• Mario Attie 8 ,
• James Beebe 9 ,
• Renatas Berniūnas 10 ,
• Jordane Boudesseul 11 ,
• Matteo Colombo 12 ,
• Fiery Cushman 13 ,
• Rodrigo Diaz 14 ,
• Noah N’Djaye Nikolai van Dongen 15 ,
• Vilius Dranseika 16 ,
• Brian D. Earp 17 ,
• Antonio Gaitán Torres 18 ,
• Ivar Hannikainen 19 ,
• José V. Hernández-Conde 20 ,
• Wenjia Hu 21 ,
• François Jaquet 1 ,
• Kareem Khalifa 22 ,
• Hanna Kim 23 ,
• Markus Kneer 24 ,
• Joshua Knobe 25 ,
• Miklos Kurthy 26 ,
• Anthony Lantian 27 ,
• Shen-yi Liao 28 ,
• Edouard Machery 29 ,
• Tania Moerenhout 30 ,
• Christian Mott 25 ,
• Mark Phelan 21 ,
• Jonathan Phillips 13 ,
• Navin Rambharose 21 ,
• Kevin Reuter 31 ,
• Felipe Romero 15 ,
• Paulo Sousa 32 ,
• Jan Sprenger 33 ,
• Emile Thalabard 34 ,
• Kevin Tobia 25 ,
• Hugo Viciana 35 ,
• Daniel Wilkenfeld 29 &
• Xiang Zhou 36  

5443 Accesses

103 Citations

43 Altmetric

Explore all metrics

A Correction to this article was published on 22 June 2021

A Correction to this article was published on 10 August 2018

This article has been updated

Responding to recent concerns about the reliability of the published literature in psychology and other disciplines, we formed the X-Phi Replicability Project (XRP) to estimate the reproducibility of experimental philosophy ( osf.io/dvkpr ). Drawing on a representative sample of 40 x-phi studies published between 2003 and 2015, we enlisted 20 research teams across 8 countries to conduct a high-quality replication of each study in order to compare the results to the original published findings. We found that x-phi studies – as represented in our sample – successfully replicated about 70% of the time. We discuss possible reasons for this relatively high replication rate in the field of experimental philosophy and offer suggestions for best research practices going forward.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

What is Qualitative in Qualitative Research

Reporting reliability, convergent and discriminant validity with structural equation modeling: a review and best-practice recommendations.

Literature reviews as independent studies: guidelines for academic practice

Change history, 10 august 2018.

A Correction to this paper has been published: https://doi.org/10.1007/s13164-018-0407-2

22 June 2021

A Correction to this paper has been published: https://doi.org/10.1007/s13164-021-00559-0

Meaning, the ratio of published studies that would replicate versus not replicate if a high-quality replication study were carried out.

http://experimental-philosophy.yale.edu/xphipage/Experimental%20Philosophy-Replications.html .

In practice, it can be hard to determine whether the ‘sufficiently similar’ criterion has actually been fulfilled by the replication attempt, whether in its methods or in its results (Nakagawa and Parker 2015 ). It can therefore be challenging to interpret the results of replication studies, no matter which way these results turn out (Collins 1975 ; Earp and Trafimow 2015 ; Maxwell et al. 2015 ). Thus, our findings should be interpreted with care: they should be seen as a starting point for further research, not as a final statement about the existence or non-existence of any individual effect. For instance, we were not able to replicate Machery et al. ( 2004 ), but this study has been replicated on several other occasions, including in children (Li et al. 2018 ; for a review, see Machery, 2017a , chapter 2).

Note that this page is basically a mirror of the “Experimental philosophy” category of the Philpapers database.

Despite two important studies published in 2001 (Greene et al. 2001 ; Weinberg et al. 2001 ), no experimental philosophy paper is to be found for 2002.

There was some initial debate about whether to include papers reporting negative results, that is, results that failed to reject the null hypothesis using NHST. We decided to do so when such results were used as the basis for a substantial claim. The reason for this was that negative results are sometimes treated as findings within experimental philosophy. For example, in experimental epistemology, the observation of negative results has led some to reach the substantive conclusion that practical stakes do not impact knowledge ascriptions (see for example Buckwalter 2010 ; Feltz and Zarpentine 2010 ; Rose et al. in press ). Accordingly, papers reporting ‘substantive’ negative results were not excluded.

Note, however, that the more ‘demanding’ paper that was originally selected was not discarded from our list, but remained there in case research teams with the required resources agreed to replicate these studies.

It should be noted that two other papers were replaced during the replication process. For the year 2006, Malle ( 2006 ) was replaced with Nichols ( 2006 ), given that the data kindly provided by author for Study 1 did not completely match the results presented in the original paper. Note that this does not mean that the effect reported in Malle ( 2006 )'s Study 1 is not real, as subsequent studies in the same paper (as well as unpublished replications by the same author) found a similar effect. For the same year, Cushman et al. ( 2006 ) proved to be too resource-demanding after all and was replaced by Nahmias et al. ( 2006 ).

In this respect, our methodology differed from the OSC’s methodology, which instructed replication teams to focus on the papers’ last study.

N s were computed not from the total N recruited for the whole study but from the number of data points included in the relevant statistical analysis.

For this analysis, studies for which power > 0.99 were counted as power = 0.99.

For studies reporting statistically significant results, we counted studies for which the original effect size was smaller than the replication 95% CI as successful replications on the ground that, given the studies’ original hypotheses, a greater effect size than originally expected constituted even more evidence in favor of these hypotheses. Of course, theoretically, this need not always be the case, for example if a given hypothesis makes precise predictions about the size of an effect. But for the studies we attempted to replicate, a greater effect size did indeed signal greater support for the hypothesis.

As pointed out by a reviewer on this paper, this criterion might even be considered too stringent. This is because, in certain circumstances in which no prediction is made about the size of an effect, a replication for which the 95% CI falls below the original effect size might still be considered as a successful replication, given that there is a significant effect in the predicted direction. Other ways of assessing replication success using effect sizes might include computing whether there is a statistical difference between the original and replication effect size (which would present the disadvantage of rewarding underpowered studies), or considering whether the replication effect size fell beyond the lower bound of the 95% CI of the original effect size (which returns a rate of 28 successful replications out of 34 original studies, i.e. 82.4%). Nevertheless, we decided to err on the side of stringency.

This analysis was done on the basis of Google Scholar’s citation count (as of March 23rd, 2018).

In a previous version of this manuscript, we reported 30 content-based studies and 5 demographic effects . However, helpful commentaries from readers, including Wesley Buckwalter, led us to revise our classification for Nichols ( 2004 ).

A low replication rate for demographic-based effects should not be taken as direct evidence for the nonexistence of variations between demographic groups. Indeed, out of 3 demographic-based effects that failed to replicate, one was a null effect, meaning that the failed replication found an effect where there was none in the original study.

Possible reasons for such transparency might be that (i) experimental philosophy is still a smaller academic community where individual researchers are likelier to be well known to each other and thus able and willing to hold each other accountable, and (ii) research resources (such as online survey accounts) used to be shared among researchers in the early days of the field, thus making questionable research practices more difficult to obscure (see Liao 2015 ).

One more cynical explanation would simply be that experimental philosophers are less well versed in into statistics, and that certain questionable research practices are only available to those who have sufficient skills in this area (i.e., the ability to take advantage of highly complex statistical models or approaches to produce ‘findings’ that are of questionable value).

For example, as of November 2017, the Wikipedia page for “Experimental Philosophy” dedicates a large part of its “Criticisms” section to the “Problem of Reproducibility,” arguing that “a parallel with experimental psychology is likely.”

Alfano, M. & Loeb, D. 2014. Experimental moral philosophy. In The Stanford Encyclopedia of Philosophy (Fall 2017 Edition) , ed. E. N. Zalta. Retrieved from https://plato.stanford.edu/archives/fall2017/entries/experimental-moral/

American Statistical Association. 2016. American Statistical Association statement on statistical significance and p-values. American Statistical Association . Retrieved from http://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf

Amrhein, V., and S. Greenland. 2017. Remove, rather than redefine, statistical significance. Nature Human Behaviour. https://doi.org/10.1038/s41562-017-0224-0 .

Anderson, S.F., K. Kelley, and S.E. Maxwell. 2017. Sample-size planning for more accurate statistical power: A method adjusting sample effect sizes for publication bias and uncertainty. Psychological Science 28 (11): 1547–1562. https://doi.org/10.1177/0956797617723724 .

Article   Google Scholar  

Baker, M. 2016. Is there a reproducibility crisis? Nature 533 (1): 452–454.

Benjamin, D.J., J.O. Berger, M. Johannesson, B.A. Nosek, E.-J. Wagenmakers, R. Berk, et al. in press. Redefine statistical significance. Nature Human Behaviour. https://doi.org/10.1038/s41562-017-0189-z .

Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E. J., Berk, R., ... & Cesarini, D. 2018. Redefine statistical significance. Nature Human Behaviour 2 (1): 6

Boyle, G. J. (in press). Proving a negative? Methodological, statistical, and psychometric flaws in Ullmann et al. (2017) PTSD study. Journal of Clinical and Translational Research .

Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R., … van ’t Veer, A. 2014. The replication recipe: What makes for a convincing replication? Journal of Experimental Social Psychology , 50 (supplement C), 217–224. https://doi.org/10.1016/j.jesp.2013.10.005 , The Replication Recipe: What makes for a convincing replication?

Buckwalter, W. 2010. Knowledge isn’t closed on Saturday: A study in ordinary language. Review of Philosophy and Psychology 1 (3): 395–406. https://doi.org/10.1007/s13164-010-0030-3 .

Button, K.S., J.P. Ioannidis, C. Mokrysz, B.A. Nosek, J. Flint, E.S. Robinson, and M.R. Munafò. 2013. Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14 (5): 365–376. https://doi.org/10.1038/nrn3475 .

Casler, K., L. Bickel, and E. Hackett. 2013. Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior 29 (6): 2156–2160. https://doi.org/10.1016/j.chb.2013.05.009 .

Cesario, J. 2014. Priming, replication, and the hardest science. Perspectives on Psychological Science 9 (1): 40–48. https://doi.org/10.1177/1745691613513470 .

Chambers, C., & Munafò, M. 2013. Trust in science would be improved by study pre-registration. The Guardian. Retrieved from http://www.theguardian.com/science/blog/2013/jun/05/trust-in-science-study-pre-registration

Champely, S. 2018. Package ‘pwr’. Retrieved from http://cran.r-project.org/package=pwr

Chang, A.C., and P. Li. 2015. Is economics research replicable? Sixty published papers from thirteen journals say “usually not”, Finance and Economics Discussion Series 2015–083 . Washington, DC: Board of Governors of the Federal Reserve System.

Google Scholar  

Clavien, C., C.J. Tanner, F. Clément, and M. Chapuisat. 2012. Choosy moral punishers. PLoS One 7 (6): e39002. https://doi.org/10.1371/journal.pone.0039002 .

Collins, H.M. 1975. The seven sexes: A study in the sociology of a phenomenon, or the replication of experiments in physics. Sociology 9 (2): 205–224. https://doi.org/10.1177/003803857500900202 .

Colombo, M., Duev, G., Nuijten, M. B., & Sprenger, J. 2017. Statistical reporting inconsistencies in experimental philosophy. Retrieved from https://osf.io/preprints/socarxiv/z65fv

Cova, F. 2012. Qu’est-ce que la philosophie expérimentale ? In La Philosophie Expérimentale , ed. F. Cova, J. Dutant, E. Machery, J. Knobe, S. Nichols, and E. Nahmias. Paris: Vuibert.

Cova, F. 2016. The folk concept of intentional action: Empirical approaches. In A Companion to Experimental Philosophy , ed. W. Buckwalter and J. Sytsma, 121–141 Wiley-Blackwell.

Cova, F. 2017. What happened to the trolley problem? Journal of Indian Council of Philosophical Research 34 (3): 543–564.

Crandall, C.S., and J.W. Sherman. 2016. On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology 66 (Supplement C): 93–99. https://doi.org/10.1016/j.jesp.2015.10.002 .

Cullen, S. 2010. Survey-driven romanticism. Review of Philosophy and Psychology 1 (2): 275–296. https://doi.org/10.1007/s13164-009-0016-1 .

Cumming, G. 2013. Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis . Routledge.

Cushman, F., Young, L., & Hauser, M. 2006. The role of conscious reasoning and intuition in moral judgment testing three principles of harm. Psychological Science 17 (12): 1082–1089.

De Villiers, J., R.J. Stainton, and P. Szatmari. 2007. Pragmatic abilities in autism spectrum disorder: A case study in philosophy and the empirical. Midwest Studies in Philosophy 31 (1): 292–317. https://doi.org/10.1111/j.1475-4975.2007.00151.x .

Del Re, A. C. 2015. Package “compute.es”. Available from https://cran.r-project.org/web/packages/compute.es/compute.es.pdf Accessed 08 Apr 2018.

Doyen, S., O. Klein, D.J. Simons, and A. Cleeremans. 2014. On the other side of the mirror: Priming in cognitive and social psychology. Social Cognition 32 (Supplement): 12–32. https://doi.org/10.1521/soco.2014.32.supp.12 .

Dunaway, B., A. Edmonds, and D. Manley. 2013. The folk probably do think what you think they think. Australasian Journal of Philosophy 91 (3): 421–441.

Earp, B.D. 2017. The need for reporting negative results – a 90 year update. Journal of Clinical and Translational Research 3 (S2): 1–4. https://doi.org/10.18053/jctres.03.2017S2.001 .

Earp, B.D. in press. Falsification: How does it relate to reproducibility? In Key concepts in research methods , ed. J.-F. Morin, C. Olsson, and E.O. Atikcan. Abingdon: Routledge.

Earp, B.D., and D. Trafimow. 2015. Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology 6 (621): 1–11. https://doi.org/10.3389/fpsyg.2015.00621 .

Earp, B.D., and D. Wilkinson. 2017. The publication symmetry test: a simple editorial heuristic to combat publication bias. Journal of Clinical and Translational Research 3 (S2): 5–7. https://doi.org/10.18053/jctres.03.2017S2.002 .

Feltz, A., and F. Cova. 2014. Moral responsibility and free will: A meta-analysis. Consciousness and Cognition 30: 234–246. https://doi.org/10.1016/j.concog.2014.08.012 .

Feltz, A., and C. Zarpentine. 2010. Do you know more when it matters less? Philosophical Psychology 23 (5): 683–706. https://doi.org/10.1080/09515089.2010.514572 .

Fiedler, K., and N. Schwarz. 2016. Questionable research practices revisited. Social Psychological and Personality Science 7 (1): 45–52. https://doi.org/10.1177/1948550615612150 .

Findley, M.G., N.M. Jensen, E.J. Malesky, and T.B. Pepinsky. 2016. Can results-free review reduce publication bias? The results and implications of a pilot study. Comparative Political Studies 49 (13): 1667–1703. https://doi.org/10.1177/0010414016655539 .

Fraley, R.C., and S. Vazire. 2014. The N-pact factor: Evaluating the quality of empirical journals with respect to sample size and statistical power. PLoS One 9 (10): e109019. https://doi.org/10.1371/journal.pone.0109019 .

Franco, A., N. Malhotra, and G. Simonovits. 2014. Publication bias in the social sciences: Unlocking the file drawer. Science 345 (6203): 1502–1505. https://doi.org/10.1126/science.1255484 .

Gilbert, D.T., G. King, S. Pettigrew, and T.D. Wilson. 2016. Comment on “estimating the reproducibility of psychological science”. Science 351 (6277): 1037–1037. https://doi.org/10.1126/science.aad7243 .

Greene, J.D., R.B. Sommerville, L.E. Nystrom, J.M. Darley, and J.D. Cohen. 2001. An fMRI investigation of emotional engagement in moral judgment. Science 293 (5537): 2105–2108. https://doi.org/10.1126/science.1062872 .

Greene, J. D., Morelli, S.A., Lowenberg, K., Nystrom, L.E., & Cohen, J.D. 2008. Cognitive load selectively interferes with utilitarian moral judgment. Cognition 107 (3): 1144–1154.

Grens, K. (2014). The rules of replication. Retrieved November 8, 2017, from http://www.the-scientist.com/?articles.view/articleNo/41265/title/The-Rules-of-Replication/

Heine, S.J., D.R. Lehman, K. Peng, and J. Greenholtz. 2002. What's wrong with cross-cultural comparisons of subjective Likert scales? The reference-group effect. Journal of Personality and Social Psychology 82 (6): 903–918. https://doi.org/10.1037//0022-3514.82.6.903 .

Hendrick, C. 1990. Replications, strict replications, and conceptual replications: Are they important? Journal of Social Behavior and Personality 5 (4): 41–49.

Hitchcock, C., & Knobe, J. (2009). Cause and norm. The Journal of Philosophy 106 (11): 587–612.

Ioannidis, J.P.A. 2005. Why most published research findings are false. PLoS Medicine 2 (8): e124. https://doi.org/10.1371/journal.pmed.0020124 .

John, L.K., G. Loewenstein, and D. Prelec. 2012. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science 23 (5): 524–532. https://doi.org/10.1177/0956797611430953 .

Knobe, J. 2016. Experimental philosophy is cognitive science. In A companion to experimental philosophy , ed. J. Sytsma and W. Buckwalter, 37–52. John Wiley & Sons, Ltd. https://doi.org/10.1002/9781118661666.ch3 .

Knobe, J. 2003a. Intentional action and side effects in ordinary language. Analysis 63 (279): 190–94.

Knobe, J. 2003b. Intentional action in folk psychology: An experimental investigation. Philosophical psychology 16 (2): 309–324.

Knobe, J., & Burra, A. 2006. The folk concepts of intention and intentional action: A cross-cultural study. Journal of Cognition and Culture 6 (1): 113–132.

Knobe, J. 2007. Experimental Philosophy. Philosophy Compass 2 (1): 81–92.

Knobe, J., and S. Nichols. 2008. Experimental philosophy . Oxford University Press.

Knobe, J., W. Buckwalter, S. Nichols, P. Robbins, H. Sarkissian, and T. Sommers. 2012. Experimental philosophy. Annual Review of Psychology 63 (1): 81–99. https://doi.org/10.1146/annurev-psych-120710-100350 .

Lakens, D. 2013. Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology 4: 863.

Lakens, D., F.G. Adolfi, C. Albers, F. Anvari, M.A.J. Apps, S.E. Argamon, et al. 2017. Justify your alpha: a response to “Redefine statistical significance”. PsyArXiv . https://doi.org/10.17605/OSF.IO/9S3Y6 .

Lam, B. 2010. Are Cantonese-speakers really descriptivists? Revisiting cross-cultural semantics. Cognition 115 (2), 320–329.

Lash, T.L., and J.P. Vandenbroucke. 2012. Should preregistration of epidemiologic study protocols become compulsory? Reflections and a counterproposal. Epidemiology 23 (2): 184–188. https://doi.org/10.1097/EDE.0b013e318245c05b .

Li, J., L. Liu, E. Chalmers, and J. Snedeker. 2018. What is in a name?: The development of cross-cultural differences in referential intuitions. Cognition 171: 108–111. https://doi.org/10.1016/j.cognition.2017.10.022 .

Liao, S. 2015. The state of reproducibility in experimental philosophy Retrieved from http://philosophycommons.typepad.com/xphi/2015/06/the-state-of-reproducibility-in-experimental-philosophy.html

Locascio, J. 2017. Results blind science publishing. Basic and Applied Social Psychology 39 (5): 239–246. https://doi.org/10.1080/01973533.2017.1336093 .

Machery, E., Mallon, R., Nichols, S., & Stich, S. P. 2004. Semantics, cross-cultural style. Cognition 92 (3): B1–B12.

Machery, E. 2017a. Philosophy within its proper bounds . Oxford: Oxford University Press.

Book   Google Scholar  

Machery, E. 2017b. What is a replication? Unpublished manuscript.

Makel, M.C., & Plucker, J.A. 2014. Facts are more important than novelty: Replication in the educationsciences. Educational Researcher 43 (6), 304–316.

Malle, B. F. 2006. Intentionality, morality, and their relationship in human judgment. Journal of Cognition and Culture 6 (1), 87–112.

Maxwell, S.E., M.Y. Lau, and G.S. Howard. 2015. Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? The American Psychologist 70 (6): 487–498. https://doi.org/10.1037/a0039400 .

McShane, B.B., Gal, D., Gelman, A., Robert, C., & Tackett, J L. (2017). Abandon Statistical Significance. arXiv preprint . arXiv:1709.07588.

Munafò, M.R., B.A. Nosek, D.V.M. Bishop, K.S. Button, C.D. Chambers, N.P. du Sert, et al. 2017. A manifesto for reproducible science. Nature Human Behaviour 1 (21): 1–9. https://doi.org/10.1038/s41562-016-0021 .

Murtaugh, P.A. 2014. In defense of p-values. Ecology 95 (3): 611–617. https://doi.org/10.1890/13-0590.1 .

Nadelhoffer, T., & Feltz, A. 2008. The actor–observer bias and moral intuitions: adding fuel to Sinnott-Armstrong’s fire. Neuroethics 1 (2): 133–144.

Nadelhoffer, T., Kvaran, T., & Nahmias, E. 2009. Temperament and intuition: A commentary on Feltz and Cokely. Consciousness and cognition, 18 (1): 351–355.

Nakagawa, S., and T.H. Parker. 2015. Replicating research in ecology and evolution: Feasibility, incentives, and the cost-benefit conundrum. BMC Biology 13 (88): 1–6. https://doi.org/10.1186/s12915-015-0196-3 .

Nahmias, E., Morris, S.G., Nadelhoffer, T., & Turner, J. (2006). Is incompatibilism intuitive? Philosophy and Phenomenological Research 73 (1): 28–53.

Nichols, S. 2004. After objectivity: An empirical study of moral judgment. Philosophical Psychology 17 (1): 3–26.

Nichols, S. 2006. Folk intuitions on free will. Journal of Cognition and Culture 6 (1): 57–86.

Nichols, S., & Knobe, J. 2007. Moral responsibility and determinism: The cognitive science of folk intuitions. Nous 41 (4): 663–685.

Nosek, B.A., and T.M. Errington. 2017. Reproducibility in cancer biology: Making sense of replications. eLife 6: e23383. https://doi.org/10.7554/eLife.23383 .

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (in press). The preregistration revolution. Proceedings of the National Academy of Sciences .

O’Neill, E., and E. Machery. 2014. Experimental philosophy: What is it good for? In Current controversies in experimental philosophy , ed. E. Machery and E. O’Neill. New York: Routledge.

Open Science Collaboration. 2015. Estimating the reproducibility of psychological science. Science 349 (6251): aac4716. https://doi.org/10.1126/science.aac4716 .

Reuter, K. 2011. Distinguishing the appearance from the reality of pain. Journal of Consciousness Studies 18 (9-10): 94–109.

Rose, D., and D. Danks. 2013. In defense of a broad conception of experimental philosophy. Metaphilosophy 44 (4): 512–532. https://doi.org/10.1111/meta.12045 .

Rose, D., Machery, E., Stich, S., Alai, M., Angelucci, A., Berniūnas, R., … & Cohnitz, D. (in press). Nothing at stake in knowledge. Noûs.

Rosenthal, R. 1979. The file drawer problem and tolerance for null results. Psychological Bulletin 86 (3): 638–641. https://doi.org/10.1037/0033-2909.86.3.638 .

Schmidt, S. 2009. Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology 13 (2): 90–100. https://doi.org/10.1037/a0015108 .

Scott, S. 2013. Pre-registration would put science in chains. Retrieved July 29, 2017, from https://www.timeshighereducation.com/comment/opinion/pre-registration-would-put-science-in-chains/2005954.article

Simmons, J.P., L.D. Nelson, and U. Simonsohn. 2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22 (11): 1359–1366. https://doi.org/10.1177/0956797611417632 .

Simonsohn, U., L.D. Nelson, and J.P. Simmons. 2014. P-curve: A key to the file-drawer. Journal of Experimental Psychology: General 143 (2): 534.

Sprouse, J., & Almeida, D. 2017. Setting the empirical record straight: Acceptability judgments appear to be reliable, robust, and replicable. Behavioral and Brain Sciences 40: e311.

Stroebe, W., and F. Strack. 2014. The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science 9 (1): 59–71.

Trafimow, D., and B.D. Earp. 2017. Null hypothesis significance testing and type I error: The domain problem. New Ideas in Psychology 45: 19–27.

Weinberg, J.M., S. Nichols, and S. Stich. 2001. Normativity and epistemic intuitions. Philosophical Topics 29 (1/2): 429–460.

Woolfolk, R.L. 2013. Experimental philosophy: A methodological critique. Metaphilosophy 44 (1–2): 79. https://doi.org/10.1111/meta.12016 .

Young, N.S., Ioannidis, J.P., & Al-Ubaydli, O. 2008. Why current publication practices may distort science. PLoS medicine 5 (10): e201.

Zalla, T., & Leboyer, M. 2011. Judgment of intentionality and moral evaluation in individuals with high functioning autism. Review of Philosophy and Psychology 2 (4), 681–698.

Download references

Acknowledgments

This project could not have been possible without the financial support of multiple organizations. Florian Cova’s work on this project was supported by a grant from the Cogito Foundation (Grant No. S-131/13, “Towards an Experimental Philosophy of Aesthetics”).

Brent Strickland’s work was supported by two grants from the Agence Nationale de la Recherche (Grants No. ANR-10-IDEX-0001-02 PSL*, ANR-10-LABX-0087 IEC).

Matteo Colombo, Noah van Dongen, Felipe Romero and Jan Sprenger’s work was supported by the European Research Council (ERC) through Starting Grant. No. 640638 (“Making Scientific Inferences More Objective”).

Rodrigo Diaz and Kevin Reuter would like to acknowledge funding from the Swiss National Science Foundation, Grant No. 100012_169484.

Antonio Gaitán Torres and Hugo Viciana benefited from funding from the Ministerio de Economía y Competitividad for the project “La constitución del sujeto en la interacción social” (Grant No. FFI2015-67569-C2-1-P & FFI2015-67569-C2-2-P).

José Hernández-Conde carried out his work as a Visiting Scholar at the University of Pittsburgh’s HPS Department. He was financially supported by a PhD scholarship and mobility grant from the University of the Basque Country, and by the Spanish Ministry of Economy and Competitiveness research project No. FFI2014-52196-P. His replication research was supported by the Pittsburgh Empirical Philosophy Lab.

Hanna Kim’s work was supported by the Pittsburgh Empirical Philosophy Lab.

Shen-yi Liao’s work was supported by the University of Puget Sound Start-up Funding.

Tania Moerenhout carried out her work as a Visiting Researcher at the Center for Bioethics and Health Law, University of Pittsburgh, PA (Aug 2016-July 2017).

Aurélien Allard, Miklos Kurthy, and Paulo Sousa are grateful to Rashmi Sharma for her help in the replication of Knobe & Burra ( 2006 ), in particular for her help in translating the demographic questions from English to Hindi.

Ivar Hannikainen and Florian Cova would like to thank Uri Simonsohn for his help in discussing the meaning and best interpretation of p-curves.

Finally, we would like to thank all the authors of original studies who accepted to take the time to answer our questions, share their original material and data, and discuss the results of our replication attempts with us.

Author information

Authors and affiliations.

Centre Interfacultaire en Sciences Affectives, Université de Genève, Geneva, Switzerland

Florian Cova & François Jaquet

Swiss Center for Affective Sciences, Campus Biotech, CISA – University of Geneva, Chemin des Mines, 9, 1202, Geneva, Switzerland

Florian Cova

Département d’Etudes Cognitives, Ecole Normale Supérieure, Paris, France

Brent Strickland

Institut Jean Nicod, CNRS, Paris, France

Faculté de Psychologie et des Sciences de l’Education, Université de Genève, Geneva, Switzerland

Angela Abatista

Laboratoire des Théories du Politique, Université Paris 8 Vincennes, Saint-Denis, France

Aurélien Allard

University of Reading, Reading, UK

James Andow

Department of Philosophy, Yale University, New Haven, CT, USA

Mario Attie

University at Buffalo, Buffalo, NY, USA

James Beebe

Department of General Psychology, Vilnius University, Vilnius, Lithuania

Renatas Berniūnas

Instituto de Investigación Científica, Universidad de Lima, Lima, Peru

Jordane Boudesseul

Tilburg Center for Logic, Ethics and Philosophy of Science, Tilburg University, Tilburg, Netherlands

Matteo Colombo

Department of Psychology, Harvard University, Cambridge, MA, USA

Fiery Cushman & Jonathan Phillips

University of Bern, Bern, Switzerland

Rodrigo Diaz

Tilburg University, Tilburg, Netherlands

Noah N’Djaye Nikolai van Dongen & Felipe Romero

Department of Logic and History of Philosophy, Faculty of Philosophy, Vilnius University, Vilnius, Lithuania

Vilius Dranseika

Departments of Philosophy and Psychology, Yale University, New Haven, CT, USA

Brian D. Earp

Departamento de Humanidades, Universidad Carlos III de Madrid, Getafe, Spain

Antonio Gaitán Torres

Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, Brazil

Ivar Hannikainen

Department of Linguistics and Basque Studies, University of the Basque Country, Leioa, Spain

José V. Hernández-Conde

Lawrence University, Appleton, WI, USA

Wenjia Hu, Mark Phelan & Navin Rambharose

Philosophy Department, Middlebury College, Middlebury, VT, USA

Kareem Khalifa

Washington and Jefferson College, Washington, PA, USA

University of Zurich, Zurich, Switzerland

Markus Kneer

Yale University, New Haven, CT, USA

Joshua Knobe, Christian Mott & Kevin Tobia

University of Sheffield, Sheffield, UK

Miklos Kurthy

Laboratoire Parisien de Psychologie Sociale, UPL, Université Paris Nanterre, Nanterre, France

Anthony Lantian

Department of Philosophy, University of Puget Sound, Tacoma, WA, 98416, USA

Shen-yi Liao

Department of History and Philosophy of Science, Center for Philosophy of Science, University of Pittsburgh, Pittsburgh, PA, USA

Edouard Machery & Daniel Wilkenfeld

Department of Philosophy and Moral Sciences and Department of Family Medicine and Primary Health Care, University of Ghent, Ghent, Belgium

Tania Moerenhout

Institute of Philosophy, University of Bern, Bern, Switzerland

Kevin Reuter

Queen’s University Belfast, Belfast, UK

Paulo Sousa

Center for Logic, Language and Cognition, Department of Philosophy and Educational Sciences, University of Turin, Turin, Italy

Jan Sprenger

Sciences, Normes, Décision (FRE 3593), Université Paris-Sorbonne, Paris, France

Emile Thalabard

Juan de la Cierva Research Fellow, Instituto de Estudios Sociales Avanzados (IESA-CSIC), Córdoba, Spain

Hugo Viciana

University of Pittsburgh, Pittsburgh, PA, USA

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Florian Cova .

Additional information

Osf repository.

Details, methods and results for all replications can be found online at https://osf.io/dvkpr/

Most of the analyses reported in this manuscript were conducted using the R {compute.es} and {pwr} packages (Champely 2018 ; Del Re 2015 ). We are also indebted to Lakens’ R2D2 sheet (Lakens 2013 ).

The original article has been revised: Appendix 1 has been corrected.

The original version of this article was revised: Footnote 8 was incomplete in the original version of this article. The original article has been corrected.

Appendix 1. List of Studies Selected for Replication

(Crossed-out studies are studies who were planned for replications but did not get replicated.)

Most cited: Knobe, J. ( 2003a ). Intentional action and side effects in ordinary language.

Analysis , 63 (279), 190–194. [Study 1] (Content-based, successful, osf.io/hdz5x/ ).

Random: Knobe, J. ( 2003b ). Intentional action in folk psychology: An experimental investigation. Philosophical Psychology , 16 (2), 309–324. [Study 1] (Content-based, successful, osf.io/78sqa/ ).

Most cited: Machery, E., Mallon, R., Nichols, S., & Stich, S. P. ( 2004 ). Semantics, cross-cultural style. Cognition , 92 (3), B1-B12. (Demographic effect, successful, osf.io/qdekc/ )

Replacement: Knobe, J. (2004). Intention, intentional action and moral considerations. Analysis , 64 (282), 181–187. [Study 1] (Content-based, successful, osf.io/ka5wv/ )

Random 1: Nadelhoffer, T. (2004). Blame, Badness, and Intentional Action: A Reply to Knobe and Mendlow. Journal of Theoretical and Philosophical Psychology , 24 (2), 259–269. (Content-based, unsuccessful, osf.io/w9bza/ ).

Random 2: Nichols, S. ( 2004 ). After objectivity: An empirical study of moral judgment. Philosophical Psychology , 17 (1), 3–26. [Study 3] (Content-based, successful, osf.io/bv4ep/ ).

Most cited: Nahmias, E., Morris, S., Nadelhoffer, T., & Turner, J. (2005). Surveying freedom: Folk intuitions about free will and moral responsibility. Philosophical Psychology , 18 (5), 561–584. [Study 1] (Content-based, successful, osf.io/4gvd5/ ).

Random 1: McCann, H. J. (2005). Intentional action and intending: Recent empirical studies. Philosophical Psychology , 18 (6), 737–748. [Study 1] (Context-based, null effect, successful, osf.io/jtsnn/ ).

Random 2: Nadelhoffer, T. (2005). Skill, luck, control, and intentional action. Philosophical Psychology , 18 (3), 341–352. [Study 1] (Content-based, successful, osf.io/6ds5e/ ).

Most cited: Cushman, F., Young, L., & Hauser, M. (2006). The role of conscious reasoning and intuition in moral judgment testing three principles of harm. Psychological Science, 17(12), 1082–1089.

Replacement: Nahmias, E., Morris, S. G., Nadelhoffer, T., & Turner, J. ( 2006 ). Is incompatibilism intuitive? Philosophy and Phenomenological Research , 73 (1), 28–53. [Study 2] (Content-based, unsuccessful, osf.io/m8t3k/ )

Random 1: Knobe, J., & Burra, A. ( 2006 ). The folk concepts of intention and intentional action: A cross-cultural study. Journal of Cognition and Culture , 6 (1), 113–132. (Content-based, successful, osf.io/p48sa/ )

Replacement: Malle, B. F. (2006). Intentionality, morality, and their relationship in human judgment. Journal of Cognition and Culture, 6(1), 87–112.

Replacement: Nichols, S. ( 2006 ). Folk intuitions on free will. Journal of Cognition and Culture , 6 (1), 57–86. [Study 2] (Content-based, successful, osf.io/8kf3p/ )

Random 2: Nadelhoffer, T. (2006). Bad acts, blameworthy agents, and intentional actions: Some problems for juror impartiality. Philosophical Explorations , 9 (2), 203–219. (Content-based, successful, osf.io/bv42c/ ).

Most cited: Nichols, S., & Knobe, J. ( 2007 ). Moral responsibility and determinism: The cognitive science of folk intuitions. Nous , 41 (4), 663–685. [Study 1] (Content-based, successful, osf.io/stjwg/ ).

Random 1: Nahmias, E., Coates, D. J., & Kvaran, T. (2007). Free will, moral responsibility, and mechanism: Experiments on folk intuitions. Midwest studies in Philosophy , 31 (1), 214–242. (Content-based, successful, osf.io/pjdkg/ ).

Random 2: Livengood, J., & Machery, E. (2007). The folk probably don’t think what you think they think: Experiments on causation by absence. Midwest Studies in Philosophy , 31 (1), 107–127. [Study 1] (Content-based, successful, osf.io/7er6r/ ).

Most cited: Greene, J. D., Morelli, S. A., Lowenberg, K., Nystrom, L. E., & Cohen, J. D. ( 2008 ). Cognitive load selectively interferes with utilitarian moral judgment. Cognition , 107 (3), 1144–1154. (Context-based, unsuccessful, but with deviations from the original procedure, see osf.io/yb38c ).

Random 1: Gonnerman, C. (2008). Reading conflicted minds: An empirical follow-up to Knobe and Roedder. Philosophical Psychology , 21 (2), 193–205. (Content-based, successful, osf.io/wy8ab/ ).

Random 2: Nadelhoffer, T., & Feltz, A. ( 2008 ). The actor–observer bias and moral intuitions: adding fuel to Sinnott-Armstrong’s fire. Neuroethics , 1 (2), 133–144. (Context-based, unsuccessful, osf.io/jb8yp/ ).

Most cited: Hitchcock, C., & Knobe, J. ( 2009 ). Cause and norm. The Journal of Philosophy , 106 (11), 587–612. (Content-based, successful, osf.io/ykt7z/ ).

Random 1: Roxborough, C., & Cumby, J. (2009). Folk psychological concepts: Causation. Philosophical Psychology , 22 (2), 205–213. (Content-based, unsuccessful, osf.io/5eanz/ ).

Random 2: Nadelhoffer, T., Kvaran, T., & Nahmias, E. ( 2009 ). Temperament and intuition: A commentary on Feltz and Cokely. Consciousness and Cognition , 18 (1), 351–355. (Demographic effect, null effect, unsuccessful, osf.io/txs86/ ).

Most cited: Beebe, J. R., & Buckwalter, W. (2010). The epistemic side-effect effect. Mind & Language , 25 (4), 474–498. (Content-based, successful, osf.io/n6r3b/ )

Random 1: Lam, B. (2010). Are Cantonese-speakers really descriptivists? Revisiting crosscultural semantics. Cognition, 115(2), 320–329.

Replacement: Sytsma, J., & Machery, E. (2010). Two conceptions of subjective experience. Philosophical Studies , 151 (2), 299–327. [Study 1] (Demographic effect, successful, osf.io/z2fj8/ )

Random 2: De Brigard, F. (2010). If you like it, does it matter if it’s real? Philosophical Psychology , 23 (1), 43–57. (Content-based, successful, osf.io/cvuwy/ ).

Most cited: Alicke, M. D., Rose, D., & Bloom, D. (2011). Causation, norm violation, and culpable control. The Journal of Philosophy , 108 (12), 670–696. [Study 1] (Content-based, unsuccessful, osf.io/4yuym/ )

Random 1: Zalla, T., & Leboyer, M. (2011). Judgment of intentionality and moral evaluation in individuals with high functioning autism. Review of Philosophy and Psychology, 2(4), 681–698.

Replacement: Reuter, K. (2011). Distinguishing the Appearance from the Reality of Pain. Journal of Consciousness Studies , 18 (9–10), 94–109. (Observational data, successful, osf.io/3sn6j/ )

Random 2: Sarkissian, H., Park, J., Tien, D., Wright, J. C., & Knobe, J. (2011). Folk moral relativism. Mind & Language , 26 (4), 482–505. [Study 1] (Content-based, successful, osf.io/cy4b6/ ).

Most cited: Paxton, J. M., Ungar, L., & Greene, J. D. (2012). Reflection and reasoning in moral judgment. Cognitive Science , 36 (1), 163–177. [Study 1] (Context-based, unsuccessful, osf.io/ejmyw/ ).

Random 1: Schaffer, J., & Knobe, J. (2012). Contrastive knowledge surveyed. Noûs , 46 (4), 675–708. [Study 1] (Content-based, successful, osf.io/z4e45/ ).

Random 2: May, J., & Holton, R. (2012). What in the world is weakness of will? Philosophical Studies , 157 (3), 341–360. [Study 3] (Content-based, successful, osf.io/s37h6/ ).

Most cited: Nagel, J., San Juan, V., & Mar, R. A. (2013). Lay denial of knowledge for justified true beliefs. Cognition , 129(3), 652–661. (Content-based, successful, osf.io/6yfxz/ ).

Random 1: Beebe, J. R., & Shea, J. (2013). Gettierized Knobe effects. Episteme , 10 (3), 219. (Content-based, successful, osf.io/k89fc/ ).

Random 2: Rose, D., & Nichols, S. (2013). The lesson of bypassing. Review of Philosophy and Psychology , 4 (4), 599–619. [Study 1] (Content-based, null effect, successful, osf.io/ggw7c/ ).

Most cited: Murray, D., & Nahmias, E. (2014). Explaining away incompatibilist intuitions. Philosophy and Phenomenological Research , 88 (2), 434–467. [Study 1] (Content-based, successful, osf.io/rpkjk/ ).

Random 1: Grau, C., & Pury, C. L. (2014). Attitudes towards reference and replaceability. Review of Philosophy and Psychology , 5 (2), 155–168. (Demographic effect, unsuccessful, osf.io/xrhqe/ ).

Random 2: Liao, S., Strohminger, N., & Sripada, C. S. (2014). Empirically investigating imaginative resistance. The British Journal of Aesthetics , 54 (3), 339–355. [Study 2] (Content-based, successful, osf.io/7e8hz/ ).

Most cited: Buckwalter, W., & Schaffer, J. (2015). Knowledge, stakes, and mistakes. Noûs , 49 (2), 201–234. [Study 1] (Content-based, successful, osf.io/2ukpq/ ).

Random 1: Björnsson, G., Eriksson, J., Strandberg, C., Olinder, R. F., & Björklund, F. (2015). Motivational internalism and folk intuitions. Philosophical Psychology , 28 (5), 715–734. [Study 2] (Content-based, successful, osf.io/d8uvg/ ).

Random 2: Kominsky, J. F., Phillips, J., Gerstenberg, T., Lagnado, D., & Knobe, J. (2015). Causal superseding. Cognition , 137 , 196–209. [Study 1] (Content-based, successful, osf.io/f5svw/ ).

Appendix 2. Pre-replication form

Reference of the paper: ….

Replication team: ….

*Which study in the paper do you replicate? ….

*If it is not the first study, please explain your choice: ….

*In this study, what is the main result you will focus on during replication? Please give all relevant statistical details present in the paper: ….

*What is the corresponding hypothesis? ….

*What is the corresponding effect size? ….

*Was the original effect size:

Explicitly reported in the original paper

Not explicitly reported in the original paper, but inferable from other information present in the original paper

Not inferable from information present in the original paper.

*What is the corresponding confidence interval (if applicable)?

*Was the original confidence interval:

*From which population was the sample used in the original study drawn? (Which country, language, students/non-students, etc.)

*Was the nature of the original population:

*What was the original sample size (N): ….

*Was the original sample size:

*Does the study involve a selection procedure (e.g. comprehension checks)? (YES/NO).

*If YES, describe it briefly: ….

*Were all the steps of the selection procedure (including, e.g., comprehension checks):

*Overall, would you say that the original paper contained all the information necessary to properly conduct the replication (YES/NO).

*If NO, explain what information was lacking: ….

Power analysis and required sample size:

(Please, describe briefly the power analysis you conducted to determine the minimum required sample size. If the original effect is a null effect, just describe the required sample size you obtained by doubling the original sample size.)

Projected sample size:

(Please, describe the actual sample size you plan to use in the replication.)

Appendix 3. Post-replication form

Please, describe briefly the power analysis you conducted to determine the minimum required sample size. If the original effect is a null effect, just describe the required sample size you obtained by doubling the original sample size.)

Actual sample size and population:

(Describe the number of participants you actually recruited, and the nature of the population they are drawn from. Indicate whether the number of participants you actually recruited matched the one you planned on the OSF pre-registration. Describe briefly any difference between the population you drew your sample from and the population the original study drew its sample from.)

Materials and Procedure:

(Describe the procedure you employed for the replication, like you would in the Methods section of a paper. At the end, indicate all important differences between the original study and replication, e.g. language,)

Data analysis - Target effect:

(Focusing on the effect you singled out as the target effect for replication, describe the results you obtained. Then describe the statistical analyses you performed, detailing the effect size, the significance of the effect and, when applicable, the confidence interval.)

Data analysis - Other effects:

(If the original study included other effects and you performed the corresponding analyses, please, describe them in this section.)

Data analysis - Exploratory Analysis:

(If you conducted additional analyses that were absent from the original study, feel free to report them here. Just indicate whether they were planned in the OSF pre-registration, or exploratory.)

Success assessment:

(Did you succeed in replicating the original result? If applicable, does the original team agree with you?)

Rights and permissions

Reprints and permissions

About this article

Cova, F., Strickland, B., Abatista, A. et al. Estimating the Reproducibility of Experimental Philosophy. Rev.Phil.Psych. 12 , 9–44 (2021). https://doi.org/10.1007/s13164-018-0400-9

Download citation

Published : 14 June 2018

Issue Date : March 2021

DOI : https://doi.org/10.1007/s13164-018-0400-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Qualitative Research - Philosophies

2009, International Journal of Therapy and Rehabilitation

Background: Qualitative research has made great strides in recent years and it now makes an important contribution to our understanding of health and illness. But there are still many practitioners, academics and researchers who are totally bemused by its principles and practices. Content: In the first of a series of three articles exploring qualitative research philosophies, methodologies and methods, I attempt to unravel some of its complexities and peculiarities, in the hope that those readers new to qualitative research will study it further and consider using it in the future. This article deals with broad questions of philosophy; most especially the fundamental difference between a quantitative and qualitative worldview. Conclusions: This article explores the difference between a belief in a single objective reality and multiple realities, and relates these to quantitative and qualitative research methods. It also considers the role of theory, focusing on the difference between inductive and deductive reasoning. These concepts are the basis for the second article in the series on qualitative methodologies, to appear in the next issue.

Related Papers

Catherine Pope

Australian and New Zealand Journal of Public Health

Objective: To provide an overview of qualitative methodologies for health researchers in order to inform better research practices.Approach: Different possible goals in health research are outlined: quantifying relationships between variables, identifying associations, exploring experience, understanding process, distinguishing representations, comprehending social practices and achieving change. Three important issues in understanding qualitative approaches to research are discussed: the partiality of our view of the world, deductive and inductive approaches to research, and the role of the researcher in the research process. The methodologies of phenomenology, grounded theory, discourse analysis, ethnography, ethnomethodology and action research are illustrated.Conclusion: In order to undertake high-quality qualitative research, it is important for researchers to consider their analytic focus and methodological position.

Douglas Ezzy

Online Journal of …

Kalaiselvan Ganapathy

Jordan Tovera Salvador

Various researchers employing either qualitative or quantitative researches should better understand the different competing research paradigms and its philosophical underpinnings in relation to ontology, epistemology, methodology and even the methods in collecting and analyzing data to produce an excellent research. Moreover, this paper uncovers the distinctive philosophical keystones of each competing paradigms of qualitative research: scientific, interpretive, and critical theory. In qualitative research, realities are multiple in nature and subjective; dissimilar to quantitative research there is only one objective reality.The qualitative researcher should be able to breach the objective separateness or distance between him and the participants to enter their subjective world for only then the researcher can fully well understand the experience of the phenomenon from the perspectives of the participants (emic perspective) and not according to his viewpoint (etic perspective). Thus, this article would help researchers to fully understand the nature, components, and uniqueness of each competing paradigms of qualitative research.

International Journal of Therapy and Rehabilitation

David Nicholls

Background: There are still many practitioners, academics and researchers who are bemused by the principles and practices of qualitative research. The second paper in this three part series on qualitative research explores the important question of research methodologies. Content: Focusing on four of the more common methodologies – phenomenology, grounded theory, ethnography and discourse analysis – the article shows how each represents a distinctively different view of reality (a feature of qualitative research that was unpacked in the first article in the series). Conclusions: These methodologies are then used to highlight some of the fundamental methodological differences between quantitative and qualitative research. Having set down these principles, I move on, in the third article, to discuss qualitative methods of data collection and analysis.

John W Hogan (Senior Research Fellow) , Brendan K O'Rourke , Marian Crowley-Henry , Olivia Freeman

Abstract This chapter presents a discussion on the nature of qualitative research. In it, a number of contributors to the book sit down for a general roundtable discussion on qualitative methodologies. Here they express their thinking in relation to a range of questions on qualitative methodologies put to them by the moderator, one of the editors of the volume. The objective of the chapter is to provide readers with an insight into a free flowing discussion amongst academics on the nature of qualitative research.

Dr. Awais H. Gillani

UNICAF University - Zambia

Ivan Steenkamp

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Qualitative Health Research

Miles Little

Journal of Evaluation in Clinical Practice

Mary Boulton

Tanu Tandon

Colum Cronin

Nawa Mukumbuta

Qualitative research methods: A practice-oriented introduction

Arthur Cropley

International Journal of Recent Innovations in Medicine and Clinical Research

Qualitative Research: Practices and Challenges

Elizabeth M Pope

Martin J Packer

Journal of Nursing and Health

Mario Cardano

Nurse Education Today

Peter Ashworth

Jurij Fikfak

Horsub Uciha

The Psychiatrist

Steven Agius

Ciência & Saúde Coletiva

Vera L P Alves

Sandra Mathison

Michael Murray , Kerry Chamberlain

Nick Schuermans

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Saunders’ Research Onion: Explained Simply

Peeling the onion, layer by layer (with examples).

By: David Phair (PhD) and Kerryn Warren (PhD) | January 2021

If you’re learning about research skills and methodologies, you may have heard the term “ research onion ”. Specifically, the research onion developed by Saunders et al in 2007 . But what exactly is this elusive onion? In this post, we’ll break Saunders’ research onion down into bite-sized chunks to make it a little more digestible.

Saunders’ (2007) Research Onion – What is it?

At the simplest level, Saunders’ research onion describes the different decisions you’ll need to make when developing a  research methodology   – whether that’s for your dissertation, thesis or any other formal research project. As you work from the outside of the onion inwards , you’ll face a range of choices that progress from high-level and philosophical to tactical and practical in nature. This also mimics the general structure for the methodology chapter .

While Saunders’ research onion is certainly not perfect, it’s a useful tool for thinking holistically about methodology. At a minimum, it helps you understand what decisions you need to make in terms of your research design and methodology.

The layers of Saunders’ research onion

The onion is made up of 6 layers, which you’ll need to peel back one at a time as you develop your research methodology:

• Research philosophy
• Research approach
• Research strategy
• Time horizon
• Techniques & procedures

Onion Layer 1: Research Philosophy

The very first layer of the onion is the research philosophy . But what does that mean? Well, the research philosophy is the foundation of any study as it describes the set of beliefs the research is built upon . Research philosophy can be described from either an  ontological  or  epistemological  point of view. “A what?!”, you ask?

In simple terms,  ontology  is the “what” and “how” of what we know – in other words, what is the nature of reality and what are we really able to know and understand. For example, does reality exist as a single objective thing, or is it different for each person? Think about the simulated reality in the film The Matrix.

Epistemology , on the other hand, is about “how” we can obtain knowledge and come to understand things – in other words, how can we figure out what reality is, and what the limits of this knowledge are. This is a gross oversimplification, but it’s a useful starting point (we’ll cover ontology and epistemology another post).

With that fluffy stuff out the way, let’s look at three of the main research philosophies that operate on different ontological and epistemological assumptions:

• Interpretivism

These certainly aren’t the only research philosophies, but they are very common and provide a good starting point for understanding the spectrum of philosophies.

Research Philosophy 1:  Positivism

Positivist research takes the view that knowledge exists outside of what’s being studied . In other words, what is being studied can only be done so objectively , and it cannot include opinions or personal viewpoints – the researcher doesn’t interpret, they only observe. Positivism states that there is only one reality  and that all meaning is consistent between subjects.

In the positivist’s view, knowledge can only be acquired through empirical research , which is based on measurement and observation. In other words, all knowledge is viewed as a posteriori knowledge – knowledge that is not reliant on human reasoning but instead is gained from research.

For the positivist, knowledge can only be true, false, or meaningless . Basically, if something is not found to be true or false, it no longer holds any ground and is thus dismissed.

Let’s look at an example, based on the question of whether God exists or not. Since positivism takes the stance that knowledge has to be empirically vigorous, the knowledge of whether God exists or not is irrelevant. This topic cannot be proven to be true or false, and thus this knowledge is seen as meaningless.

Kinda harsh, right? Well, that’s the one end of the spectrum – let’s look at the other end.

Research Philosophy 2: Interpretivism

On the other side of the spectrum, interpretivism emphasises the influence that social and cultural factors can have on an individual. This view focuses on  people’s thoughts and ideas , in light of the socio-cultural backdrop. With the interpretivist philosophy, the researcher plays an active role in the study, as it’s necessary to draw a holistic view of the participant and their actions, thoughts and meanings.

Let’s look at an example. If you were studying psychology, you may make use of a case study in your research which investigates an individual with a proposed diagnosis of schizophrenia. The interpretivist view would come into play here as social and cultural factors may influence the outcome of this diagnosis.

Through your research, you may find that the individual originates from India, where schizophrenic symptoms like hallucinations are viewed positively, as they are thought to indicate that the person is a spirit medium. This example illustrates an interpretivist approach since you, as a researcher, would make use of the patient’s point of view, as well as your own interpretation when assessing the case study.

Research Philosophy 3: Pragmatism

Pragmatism highlights the importance of using the best tools possible to investigate phenomena. The main aim of pragmatism is to approach research from a practical point of view , where knowledge is not fixed, but instead is constantly questioned and interpreted. For this reason, pragmatism consists of an element of researcher involvement and subjectivity, specifically when drawing conclusions based on participants’ responses and decisions. In other words, pragmatism is not committed to (or limited by) one specific philosophy.

Let’s look at an example in the form of the trolley problem, which is a set of ethical and psychological thought experiments. In these, participants have to decide on either killing one person to save multiple people or allowing multiple people to die to avoid killing one person. 

This experiment can be altered, including details such as the one person or the group of people being family members or loved ones. The fact that the experiment can be altered to suit the researcher’s needs is an example of pragmatism – in other words, the outcome of the person doing the thought experiment is more important than the philosophical ideas behind the experiment.

To recap, research philosophy is the foundation of any research project and reflects the ontological and epistemological assumptions of the researcher. So, when you’re designing your research methodology , the first thing you need to think about is which philosophy you’ll adopt, given the nature of your research.

Onion Layer 2: Research Approach

Let’s peel off another layer and take a look at the research approach . Your research approach is the broader method you’ll use for your research –  inductive  or  deductive . It’s important to clearly identify your research approach as it will inform the decisions you take in terms of data collection and analysis in your study (we’ll get to that layer soon).

Inductive approaches entail generating theories from research , rather than starting a project with a theory as a foundation.  Deductive approaches, on the other hand, begin with a theory and aim to build on it (or test it) through research.

Sounds a bit fluffy? Let’s look at two examples:

An  inductive approach  could be used in the study of an otherwise unknown isolated community. There is very little knowledge about this community, and therefore, research would have to be conducted to gain information on the community, thus leading to the formation of theories.

On the other hand, a  deductive approach  would be taken when investigating changes in the physical properties of animals over time, as this would likely be rooted in the theory of evolution. In other words, the starting point is a well-established pre-existing body of research.

Closely linked to research approaches are  qualitative and  quantitative  research. Simply put, qualitative research focuses on textual , visual or audio-based data, while quantitative research focuses on numerical data. To learn more about qualitative and quantitative research, check out our dedicated post here .

What’s the relevance of qualitative and quantitative data to research approaches? Well, inductive approaches are usually used within qualitative research, while quantitative research tends to reflect a deductive approach, usually informed by positivist philosophy. The reason for using a deductive approach here is that quantitative research typically begins with theory as a foundation, where progress is made through hypothesis testing. In other words, a wider theory is applied to a particular context, event, or observation to see whether these fit in with the theory, as with our example of evolution above.

So, to recap, the two research approaches are  inductive  and  deductive . To decide on the right approach for your study, you need to assess the type of research you aim to conduct. Ask yourself whether your research will build on something that exists, or whether you’ll be investigating something that cannot necessarily be rooted in previous research. The former suggests a deductive approach while the latter suggests an inductive approach.

Need a helping hand?

Onion Layer 3: Research Strategy

So far, we’ve looked at pretty conceptual and intangible aspects of the onion. Now, it’s time to peel another layer off that onion and get a little more practical – introducing research strategy . This layer of the research onion details how, based on the aims of the study, research can be conducted. Note that outside of the onion, these strategies are referred to as research designs.

There are several strategies  you can take, so let’s have a look at some of them.

• Experimental research
• Action research
• Case study research
• Grounded theory
• Ethnography
• Archival research

Strategy 1: Experimental research

Experimental research involves manipulating one variable (the independent variable ) to observe a change in another variable (the dependent variable ) – in other words, to assess the relationship between variables. The purpose of experimental research is to support, refute or validate a  research hypothesis . This research strategy follows the principles of the  scientific method  and is conducted within a controlled environment or setting (for example, a laboratory).

Experimental research aims to test existing theories rather than create new ones, and as such, is deductive in nature. Experimental research aligns with the positivist research philosophy, as it assumes that knowledge can only be studied objectively and in isolation from external factors such as context or culture.

Let’s look at an example of experimental research. If you had a hypothesis that a certain brand of dog food can raise a dogs’ protein levels, you could make use of experimental research to compare the effects of the specific brand to a “regular” diet. In other words, you could test your hypothesis.

In this example, you would have two groups, where one group consists of dogs with no changes to their diet (this is called  the control group) and the other group consists of dogs being fed the specific brand that you aim to investigate (this is called the experimental/treatment group). You would then test your hypothesis by comparing the protein levels in both groups.

Strategy 2: Action research

Next, we have action research . The simplest way of describing action research is by saying that it involves learning through… wait for it… action. Action research is conducted in practical settings such as a classroom, a hospital, a workspace, etc – as opposed to controlled environments like a lab. Action research helps to inform researchers of problems or weaknesses related to interactions within the real-world . With action research, there’s a strong focus on the participants (the people involved in the issue being studied, which is why it’s sometimes referred to as “participant action research” or PAR.

An example of PAR is a community intervention (for therapy, farming, education, whatever). The researcher comes with an idea and it is implemented with the help of the community (i.e. the participants). The findings are then discussed with the community to see how to better the intervention. The process is repeated until the intervention works just right for the community. In this way, a practical solution is given to a problem and it is generated by the combination of researcher and community (participant) feedback.

This kind of research is generally applied in the social sciences , specifically in professions where individuals aim to improve on themselves and the work that they are doing. Action research is most commonly adopted in qualitative studies and is rarely seen in quantitative studies. This is because, as you can see in the above examples, action research makes use of language and interactions rather than statistics and numbers.

Strategy 3: Case study research

A case study is a detailed, in-depth study of a single subject – for example, a person, a group or an institution, or an event, phenomenon or issue. In this type of research, the subject is analysed to gain an in-depth understanding of issues in a real-life setting. The objective here is to gain an in-depth understanding within the context of the study – not (necessarily) to generalise the findings.

It is vital that, when conducting case study research, you take the social context and culture into account, which means that this type of research is (more often than not) qualitative in nature and tends to be inductive. Also, since the researcher’s assumptions and understanding play a role in case study research, it is typically informed by an interpretivist philosophy.

For example, a study on political views of a specific group of people needs to take into account the current political situation within a country and factors that could contribute towards participants taking a certain view.

Strategy 4: Grounded theory

Next up, grounded theory. Grounded theory is all about “letting the data speak for itself”. In other words, in grounded theory, you let the data inform the development of a new theory, model or framework. True to the name, the theory you develop is “ grounded ” in the data. Ground theory is therefore very useful for research into issues that are completely new or under-researched.

Grounded theory research is typically qualitative (although it can also use quantitative data) and takes an inductive approach. Typically, this form of research involves identifying commonalities between sets of data, and results are then drawn from completed research without the aim of fitting the findings in with a pre-existing theory or framework.

For example, if you were to study the mythology of an unknown culture through artefacts, you’d enter your research without any hypotheses or theories, and rather work from the knowledge you gain from your study to develop these.

Strategy 5: Ethnography

Ethnography involves observing people in their natural environments and drawing meaning from their cultural interactions. The objective with ethnography is to capture the subjective experiences of participants, to see the world through their eyes. Creswell (2013) says it best: “Ethnographers study the meaning of the behaviour, the language, and the interaction among members of the culture-sharing group.”

For example, if you were interested in studying interactions on a mental health discussion board, you could use ethnography to analyse interactions and draw an understanding of the participants’ subjective experiences.

For example, if you wanted to explore the behaviour, language, and beliefs of an isolated Amazonian tribe, ethnography could allow you to develop a complex, complete description of the social behaviours of the group by immersing yourself into the community, rather than just observing from the outside.  

Given the nature of ethnography, it generally reflects an interpretivist research philosophy and involves an inductive , qualitative research approach. However, there are exceptions to this – for example, quantitative ethnography as proposed by David Shafer.

Strategy 6: Archival research

Last but not least is archival research. An archival research strategy draws from materials that already exist, and meaning is then established through a review of this existing data. This method is particularly well-suited to historical research and can make use of materials such as manuscripts and records.

For example, if you were interested in people’s beliefs about so-called supernatural phenomena in the medieval period, you could consult manuscripts and records from the time, and use those as your core data set.

Onion Layer 4: Choices

The next layer of the research onion is simply called “choices” – they could have been a little more specific, right? In any case, this layer is simply about deciding how many data types (qualitative or quantitative) you’ll use in your research. There are three options – mono , mixed , and multi-method .

Let’s take a look at them.

Choosing to use a  mono method  means that you’ll only make use of one data type – either qualitative or quantitative. For example, if you were to conduct a study investigating a community’s opinions on a specific pizza restaurant, you could make use of a qualitative approach only, so that you can analyse participants’ views and opinions of the restaurant.

If you were to make use of both quantitative and qualitative data, you’d be taking a  mixed-methods approach. Keeping with the previous example, you may also want to assess how many people in a community eat specific types of pizza. For this, you could make use of a survey to collect quantitative data and then analyse the results statistically, producing quantitative results in addition to your qualitative ones.

Lastly, there’s  multi-method . With a multi-method approach, you’d make use of a wider range of approaches, with more than just a one quantitative and one qualitative approach. For example, if you conduct a study looking at archives from a specific culture, you could make use of two qualitative methods (such as thematic analysis and content analysis ), and then additionally make use of quantitative methods to analyse numerical data.

As with all the layers of the research onion, the right choice here depends on the nature of your research, as well as your research aims and objectives . There’s also the practical consideration of viability – in other words, what kind of data will you be able to access, given your constraints.

Onion Layer 5: Time horizon

What’s that far in the distance? It’s the time horizon. But what exactly is it? Thankfully, this one’s pretty straightforward. The time horizon simply describes how many points in time you plan to collect your data at . Two options exist – the  cross-sectional  and  longitudinal  time horizon.

Imagine that you’re wasting time on social media and think, “Ooh! I want to study the language of memes and how this language evolves over time”. For this study, you’d need to collect data over multiple points in time – perhaps over a few weeks, months, or even years. Therefore, you’d make use of a  longitudinal time horizon. This option is highly beneficial when studying changes and progressions over time.

If instead, you wanted to study the language used in memes at a certain point in time (for example, in 2020), you’d make use of a  cross-sectional  time horizon. This is where data is collected at one point in time, so you wouldn’t be gathering data to see how language changes, but rather what language exists at a snapshot point in time. The type of data collected could be qualitative, quantitative or a mix of both, as the focus is on the time of collection, not the data type.

As with all the other choices, the nature of your research and your research aims and objectives are the key determining factors when deciding on the time horizon. You’ll also need to consider practical constraints , such as the amount of time you have available to complete your research (especially in the case of a dissertation or thesis).

Onion Layer 6: Techniques and Procedures

Finally, we reach the centre of the onion – this is where you get down to the real practicalities of your research to make choices regarding specific techniques and procedures .

Specifically, this is where you’ll:

• Decide on what data you’ll collect and what data collection methods you’ll use (for example, will you use a survey? Or perhaps one-on-one interviews?)
• Decide how you’ll go about sampling the population (for example, snowball sampling, random sampling, convenience sampling, etc).
• Determine the type of data analysis you’ll use to answer your research questions (such as content analysis or a statistical analysis like correlation).
• Set up the materials you’ll be using for your study (such as writing up questions for a survey or interview)

What’s important to note here is that these techniques and procedures need to align with all the other layers of the research onion – i.e., research philosophy, research approaches, research strategy, choices, and time horizon.

For example, you if you’re adopting a deductive, quantitative research approach, it’s unlikely that you’ll use interviews to collect your data, as you’ll want high-volume, numerical data (which surveys are far better suited to). So, you need to ensure that the decisions at each layer of your onion align with the rest, and most importantly, that they align with your research aims and objectives.

Let’s Recap: Research Onion 101

The research onion details the many interrelated choices you’ll need to make when you’re crafting your research methodology. These include:

• Research philosophy – the set of beliefs your research is based on (positivism, interpretivism, pragmatism)
• Research approaches – the broader method you’ll use (inductive, deductive, qualitative and quantitative)
• Research strategies – how you’ll conduct the research (e.g., experimental, action, case study, etc.)
• Choices – how many methods you’ll use (mono method, mixed-method or multi-method)
• Time horizons – the number of points in time at which you’ll collect your data (cross-sectional or longitudinal)
• Techniques and procedures (data collection methods, data analysis techniques, sampling strategies, etc.)

Psst... there’s more!

This post was based on one of our popular Research Bootcamps . If you're working on a research project, you'll definitely want to check this out ...

You Might Also Like:

61 Comments

This is good

Wow this was sooo helpful. I don’t feel so blank about my research anymore. With this information I can conquer my research. Going ‘write’ into it. Get it write not right hahahaha

I am doing research with Bolton University so i would like to empower myself.

Really thoughtful presentation and preparation. I learnt too much to teach my students in a very simple and understandable way

Very useful, thank you.

You’re most welcome. Good luck with your research!

thanks alot for your brief and brilliant notes

I am a Student at Malawi Institute of Management, pursuing a Masters’ degree in Business Administration. I find this to be very helpful

Extremely useful, well explained. Thank you so much

I would like to download this file… I can’t find the attachment file. Thanks

Thank you so much for explaining it in the most simple and precise manner!

Very thoughtful and well expained, thanks.

This is good for upgrade my research knowledge

I have enjoying your videos on YouTube, they are very educative and useful. I have learned a lot. Thanks

Thank you this has really helped me with writing my dissertation methodology !

Thanks so much for this piece. Just to be clear, which layer do interviews fit in?

well explained i found it to be very engaging. now i’m going to pass my research methods course. thank you.

Thank you so much this has really helped as I can’t get this insight from uni due to covid

well explained with more clarity!

this is an excellent piece i find it super helpful

Beautiful, thank you!

Beautiful and helpful. Thank you!

This is well done!

A complex but useful approach to research simplified! I would like to learn more from the team.

A very simplified version of a complex topic. I found it really helpful. I would like to know if this publication can be cited for academic research. Thank you

You’re welcome to cite this page, but it would be better to cite the original work of Saunders.

Thirteen odd years since my MSc in HRM & HRD at UoL. I’d like to say thank you for the effort to produce such an insightful discussion of a rather complex topic.

I am a PhD in Media Studies student. I found this enormously helpful when stringing together the methodology chapter, especially the research philosophy section.

Hello there. Thank you for summarising the work on the onion. A more recent version of the onion (Saunders et al., 2019) refers to ‘methodological choices’ rather than choices. This can be downloaded, along with the chapter dealing with research philosophies at: https://www.researchgate.net/publication/330760964_Research_Methods_for_Business_Students_Chapter_4_Understanding_research_philosophy_and_approaches_to_theory_development or https://www.academia.edu/42304065/Research_Methods_for_Business_Students_Chapter_4_Understanding_research_philosophy_and_approaches_to_theory_development_8th_edition

Thank you Mark Saunders. Your work is very insightful

Thank you for the update and additional reading Mark, very helpful indeed.

THROUGHLY AND SIMPLY BRIEFED TO MAKE SENSE AND A CLEAR INSIGHT. THANK YOU, VERY MUCH.

Thank you for the sharing the recent version of the Onion!

I want to keep it in my reference of my assignment. May I??

Great summary, thank you taking the time to put this together. I’m sure it’s been a big help to lots of people. It definitely was to me.

I love the analysis… some people do not recognize qualitative or quantitative as an approach but rather have inductive, abductive, and deductive.

This has been helpful in the understanding of research . Thank you for this valuable information.

Great summary. Well explained. Thank you, guys.

This makes my fears on methodology go away. I confidently look forward to working on my methodology now. Thank you so much I ma doing a PhD with UNIMA, School of Education

simple and clear

Simple guide to crafting a research methodology. Quite impactful. Thank you

Thank you for this, this makes things very clear. Now I’m off to conquer my research proposal. Thanks again.

Thank you for this very informative and valuable information. What would the best approach be to take if you are using secondary data to form a qualitative study and relying on industry reports and peer journals to distinguish what factors influence the use of say cryptocurrency ?

Thanks for providing the whole idea/knowledge in the simplest way with essential factors which made my entire research process more efficient as well as valuable.

what is about research design such as descriptive, causal-comparative, correlation, developmental where these fall in the research onion?

This is very helpful. Thank you for this wonderful piece. However, it would be nicer to have References to the knowledge provided here. My suggestion

This material is very important for researchers, particularly for PhD scholars to conduct further study.

This was insightful. Thank you for the knowledge.

Thank you for the wonderful knowledge !Easy to understand and grasp.

thanks very much very simple. will need a coach

Hi this is a great article giving much help to my research. I just wanted to mention here that the example where you mentioned that ” schizophrenic symptoms like hallucinations are viewed positively, as they are thought to indicate the person is a spirit medium” is completely false as those are different cases and a bit out of context here. We are medically and psychologically well versed and obviously understand the difference between the two. As much as I am grateful to this article I would like to suggest you to give proper examples.

Thank you very much, sincerely I appreciate your efforts, it is insightful information. Once again I’m grateful .

In short, a complete insight of and for writing research methodology.

This information was very helpful, I was having difficulties in writing my methodology now I can say I have the full knowledge to write a more informative research methodology.

Thank you so much for this amazing explanation. As a person who hasn’t ever done a research project, this video helped me to clear my doubts and approach my research in a clear and concise manner. Great work

very well explained , after going through this there is no need any material to study . a very concise and to the point.

I have one small query. If I choose mixed -methods (quantitative and qualitative techniques), Then, my research Philosophy will also change to both Positivists and Interpretivist. Isn’t?

well explained and thank you

Thanks for this presentation. Quite simple and easy to understand, and to teach others.

Hello! Having made a decision to use a particular research philosophy, how then do we go about justifying that choice with references? Thank you.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• Corpus ID: 149059399

Experimental Philosophy, Folk Metaethics and Qualitative Methods

• Published 1 October 2017

9 Citations

Misunderstanding metaethics: difficulties measuring folk objectivism and relativism, empirical research on folk moral objectivism, measuring metaaesthetics: challenges and ways forward, the irrationality of folk metaethics.

• Highly Influenced

How to Measure Moral Realism

Beyond objectivism: new methods for studying metaethical intuitions, should morality be abolished an empirical challenge to the argument from intolerance, does intentionality decision-making depend on who you are the role of individual differences., experiments in moral and political philosophy, 40 references, the empirical study of folk metaethics.

• Highly Influential

Moral Objectivism in Cross-Cultural Perspective

The meta-ethical grounding of our moral beliefs: evidence for meta-ethical pluralism, experimental philosophy., the rise and fall of experimental philosophy, different voices, perfect storms, and asking grandma what she thinks: situating experimental philosophy in relation to feminist philosophy, thought experiments and experimental philosophy, experimental philosophy and the philosophical tradition, generic qualitative research in psychology, folk moral relativism, related papers.

Showing 1 through 3 of 0 Related Papers