how is procedural rigour demonstrated in a research report

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 26: Rigour

Darshini Ayton

Learning outcomes

Upon completion of this chapter, you should be able to:

Understand the concepts of rigour and trustworthiness in qualitative research.
Describe strategies for dependability, credibility, confirmability and transferability in qualitative research.
Define reflexivity and describe types of reflexivity

What is rigour?

In qualitative research, rigour, or trustworthiness, refers to how researchers demonstrate the quality of their research. 1, 2 Rigour is an umbrella term for several strategies and approaches that recognise the influence on qualitative research by multiple realities; for example, of the researcher during data collection and analysis, and of the participant. The research process is shaped by multiple elements, including research skills, the social and research environment and the community setting. 2

Research is considered rigorous or trustworthy when members of the research community are confident in the study’s methods, the data and its interpretation. 3 As mentioned in Chapters 1 and 2, quantitative and qualitative research are founded on different research paradigms and, hence, quality in research cannot be addressed in the same way for both types of research studies. Table 26.1 provides a comparison overview of the approaches of quantitative and qualitative research in ensuring quality in research.

Table 26.1: Comparison of quantitative and qualitative approaches to ensuring quality in research

Qualitative research - Concept	Qualitative research - Definition	Quantitative research - concept	Quantitative research - Definition
Dependability	Consistency in the research and the ability for another researcher to achieve the same results with the same research process. Dependability is demonstrated through detailing the changes and context of the research setting. This includes any changes that may occur in the setting, and description and explanation of how these changes may have affected the research process.	Reliability	The extent to which results are consistent over time and an accurate representation of the study population and an assessment of whether the results of a study can be reproduced under a similar methodology.
Credibility	Confidence in the truth of the findings.	Validity	An assessment of whether the research measures what it was meant to measure, or how truthful the results are.
Confirmability	The extent by which the findings of a study are shaped by the respondents and not research bias, motivation or interest.	Objectivity	Strategies to reduce bias in research.
Transferability	Provides sufficient information about the context and process of the research to enable another person to determine if their context is similar and therefore the findings can be applied to the setting.	Generalisability	The extent to which the findings from the research sample can be applied to the broader population.
Authenticity	Demonstrates the range of participant realities, and provides rich and detailed descriptions of these realities, using quotes and narratives.

Below is an overview of the main approaches to rigour in qualitative research. For each of the approaches, examples of how rigour was demonstrated are provided from the author’s PhD thesis.

Approaches to dependability

Dependability requires the researcher to provide an account of changes to the research process and setting. 3 The main approach to dependability is an audit trail.

Audit trail – the researcher records or takes notes on the conduct of the research and the process of reaching conclusions from the data. The audit trail includes information on the data collection and data analysis, including decision-making and interpretations of the data that influence the study’s results. 8 , 9

The interview questions for this study evolved as the study progressed, and accordingly, the process was iterative. I spent 12 months collecting data, and as my understanding and responsiveness to my participants and to the culture and ethos of the various churches developed, so did my line of questioning. For example, in the early interviews for phase 2, I included questions regarding the qualifications a church leader might look for in hiring someone to undertake health promotion activities. This question was dropped after the first couple of interviews, as it was clear that church leaders did not necessarily view their activities as health promoting and therefore did not perceive the relevance of this question. By ‘being church’, they were health promoting, and therefore activities that were health promoting were not easily separated from other activities that were part of the core mission of the church 10 ( pp93–4)

Approaches to credibility

Credibility requires the researcher to demonstrate the truth or confidence in the findings. The main approaches to credibility include triangulation, prolonged engagement, persistent observation, negative case analysis and member checking. 3

Triangulation – the assembly of data and interpretations from multiple methods (methods triangulation), researchers (research triangulation), theory (theory triangulation) and data sources (different participant groups). 9 Refer to Chapter 28 for a detailed discussion of this process.
Prolonged engagement – the requirement for researchers to spend sufficient time with participants and/or within the research context to familiarise them with the research setting, to build trust and rapport with participants and to recognise and correct any misinformation. 9

Prolonged engagement with churches was also achieved through the case study phase as the ten case study churches were involved in more than one phase of data collection. These ten churches were the case studies in which significant time was spent conducting interviews and focus groups, and attending activities and programs. Subsequently, there were many instances where I interacted with the same people on more than one occasion, thereby facilitating the development of interactive and deeper relationships with participants 10 (pp.94–5)

Persistent observation – the identification of characteristics and elements that are most relevant to the problem or issue under study, and upon which the research will focus in detail. 9

In the following chapters, I present my analysis of the world of churches in which I was immersed as I conducted fieldwork. I describe the processes of church practice and action, and explore how this can be conceptualised into health promotion action 10 (p97)

Negative case analysis – the process of finding and discussing data that contradicts the study’s main findings. Negative case analysis demonstrates that nuance and granularity in perspectives of both shared and divergent opinions have been examined, enhancing the quality of the interpretation of the data.

Although I did not use negative case selection, the Catholic churches in this study acted as examples of the ‘low engagement’ 10 (p97 )

Member checking – the presentation of data analysis, interpretations and conclusions of the research to members of the participant groups. This enables participants or people with shared identity with the participants to provide their perspectives on the research. 9

Throughout my candidature – during data collection and analysis, and in the construction of my results chapters – I engaged with a number of Christians, both paid church staff members and volunteers, to test my thoughts and concepts. These people were not participants in the study, but they were embedded in the cultural and social context of churches in Victoria. They were able to challenge and also affirm my thinking and so contributed to a process of member checking 10 (p96)

Approaches to confirmability

Confirmability is demonstrated by grounding the results in the data from participants. 3 This can be achieved through the use of quotes, specifying the number of participants and data sources and providing details of the data collection.

Quotes from participants are used to demonstrate that the themes are generated from the data. The results section of the thesis chapters commences with a story based on the field notes or recordings, with extensive quotes from participants presented throughout. 10
The number of participants in the study provides the context for where the data is ‘sourced’ from for the results and interpretation. Table 26.2 is reproduced with permission from the Author’s thesis and details the data sources for the project. This also contributes to establishing how triangulation across data sources and methods was achieved.
Details of data collection – Table 26.2 provides detailed information about the processes of data collection, including dates and locations but the duration of each research encounter was not specified.

Table 26.2 Data sources for the PhD research project of the Author.

Study phase	Date	Data source	Data collection	Participant numbers
Phase 1 - Exploration	April - Oct 2009 Jan- Mar 2010	Documents Qualitative interviews	Annual reports of funding agencies, local government councils and church affiliated organisations, strategic plans of primary care partnerships. In-depth interviews with local church leaders and individuals from church affiliated organisations in Victoria	5 participants from local churches 5 participants from church affiliated organisations
Phase 2 - Description	April - June 2010	Qualitative telephone interviews	Qualitative semi-structured telephone interviews with church leaders of 25 Victorian churches	25 church ministers
Phase 3 - Case studies	July - Dec 2010	Qualitative Interviews Focus groups Observation Document analysis	Face-to-face qualitative in-depth interviews with the church staff and/or key volunteers of 10 case study churches. Focus groups with church volunteers. Direct observation of case study churches in their conduct of health promotion activities. Annual reports and/or church newsletters	37 participants 10 focus groups 17 direct observations 12 document analyses

Approaches to transferability

To enable the transferability of qualitative research, researchers need to provide information about the context and the setting. A key approach for transferability is thick description. 6

Thick description – detailed explanations and descriptions of the research questions are provided, including about the research setting, contextual factors and changes to the research setting. 9

I chose to include the Catholic Church because it is the largest Christian group in Australia and is an example of a traditional church. The Protestant group were represented through the Uniting, Anglican Baptist and Church of Christ denominations. The Uniting Church denomination is unique to Australia and was formed in 1977 through the merging of the Methodist, Presbyterian and Congregationalist denominations. The Church of Christ denomination was chosen to represent a contemporary less hierarchical denomination in comparison to the other protestant denominations. The last group, the Salvation Army, was chosen because of its high profile in social justice and social welfare, therefore offering different perspectives on the role and activities of the church in health promotion 10 (pp82–3)

What is reflexivity?

Reflexivity is the process in which researchers engage to explore and explain how their subjectivity (or bias) has influenced the research. 12 Researchers engage in reflexive practices to ensure and demonstrate rigour, quality and, ultimately, trustworthiness in their research. 13 The researcher is the instrument of data collection and data analysis, and hence awareness of what has influenced their approach and conduct of the research – and being able to articulate them – is vital in the creation of knowledge. One important element is researcher positionality (see Chapter 27), which acknowledges the characteristics, interests, beliefs and personal experiences of the researcher and how this influences the research process. Table 26.3 outlines different types of reflexivity, with examples from the author’s thesis.

Table 26.3: Types of reflexivity

Reflexivity type	Examples from the author’s thesis
Personal – reflections on the researcher's personal expectations, assumptions, biases and reactions to the research contexts, participants and data.	‘It was with hesitant steps that I entered the field for my research. I was known in some of these church communities, and my background and experience in churches was what drove me to do this research. As mentioned above, I identified as an insider to this research as I shared experiences, religious affiliation and language with the research participants. In undertaking this research, I was required to be true in what I captured and interpreted, and reflexive in acknowledging my own biases that may have coloured my approach and interpretations.’
Interpersonal – reflections on how relationships influence the research process.	‘My time in the field was peppered with statements such as “Oh, you know this person?” or “I don’t need to explain this church terminology to you.” I identified myself as an insider and by positioning myself in this way my participants treated me as someone who shared their beliefs.’
Methodological – reflections on how decisions were made regarding the study’s methods and methodological approach, and the implications of these.	‘I sought to understand what it meant to “be church” and how this played out in health promoting practices in their local community...The church is the social context. The aim of the inquiry is to understand and re-examine the constructions that the participants and I, as the researcher, hold in relation to the local church as a setting and partner for health promotion.’
Contextual – reflections on how the research context shapes and influences the research process.	‘This experience from the field involved a shared experience of a church service, however during this process the participants and I became acutely aware of our differences in social position. I was attending my own church service afterwards and had dressed according to the middle class norms of this service. The attendees at Redgum Church of Christ were experiencing poverty and health issues and therefore their dress and manner reflected their circumstances in life. Despite being an insider in some aspects (religious background, familiarity with church culture and practices), there were social facets that were not shared with my participants including generational, socio-economic and ethnic differences.’

The quality of qualitative research is measured through the rigour or trustworthiness of the research, demonstrated through a range of strategies in the processes of data collection, analysis, reporting and reflexivity.

Chowdhury IA. Issue of quality in qualitative research: an overview. Innovative Issues and Approaches in Social Sciences . 2015;8(1):142-162. doi:10.12959/issn.1855-0541.IIASS-2015-no1-art09
Cypress BS. Rigor or reliability and validity in qualitative research: perspectives, strategies, reconceptualization, and recommendations. Dimens Crit Care Nurs . 2017;36(4):253-263. doi:10.1097/DCC.0000000000000253
Connelly LM. Trustworthiness in qualitative research. Medsurg Nurs . 2016;25(6):435-6.
Golafshani N. Understanding reliability and validity in qualitative research. Qual Rep . 2003;8(4):597-607. Accessed September 18, 2023. https://nsuworks.nova.edu/tqr/vol8/iss4/6/
Yilmaz K. Comparison of quantitative and qualitative research traditions: epistemological, theoretical, and methodological differences. Eur J Educ . 2013;48(2):311-325. doi:10.1111/ejed.12014
Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. Education for Information 2004;22:63-75. Accessed September 18, 2023. https://content.iospress.com/articles/education-for-information/efi00778
Varpio L, O’Brien B, Rees CE, Monrouxe L, Ajjawi R, Paradis E. The applicability of generalisability and bias to health professions education’s research. Med Educ . Feb 2021;55(2):167-173. doi:10.1111/medu.14348
Carcary M. The Research Audit Trail: Methodological guidance for application in practice. Electronic Journal of Business Research Methods . 2020;18(2):166-177. doi:10.34190/JBRM.18.2.008
Korstjens I, Moser A. Series: Practical guidance to qualitative research. Part 4: Trustworthiness and publishing. Eur J Gen Pract . Dec 2018;24(1):120-124. doi:10.1080/13814788.2017.1375092
Ayton D. ‘From places of despair to spaces of hope’ – the local church and health promotion in Victoria . PhD. Monash University; 2013. https://figshare.com/articles/thesis/_From_places_of_despair_to_spaces_of_hope_-_the_local_church_and_health_promotion_in_Victoria/4628308/1
Hanson A. Negative case analysis. In: Matthes J, ed. The International Encyclopedia of Communication Research Methods . John Wiley & Sons, Inc.; 2017. doi: 10.1002/9781118901731.iecrm0165
Olmos-Vega FM. A practical guide to reflexivity in qualitative research: AMEE Guide No. 149. Med Teach . 2023;45(3):241-251. doi: 10.1080/0142159X.2022.2057287
Dodgson JE. Reflexivity in qualitative research. J Hum Lact . 2019;35(2):220-222. doi:10.1177/08903344198309

Qualitative Research – a practical guide for health and social care researchers and practitioners Copyright © 2023 by Darshini Ayton is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

Rigour in qualitative case-study research

Affiliation.

1 School of Nursing and Midwifery Studies, National University of Ireland, Galway. [email protected]
PMID: 23520707
DOI: 10.7748/nr2013.03.20.4.12.e326

Aim: To provide examples of a qualitative multiple case study to illustrate the specific strategies that can be used to ensure the credibility, dependability, confirmability and transferability of a study.

Background: There is increasing recognition of the valuable contribution qualitative research can make to nursing knowledge. However, it is important that the research is conducted in a rigorous manner and that this is demonstrated in the final research report.

Data sources: A multiple case study that explored the role of the clinical skills laboratory in preparing students for the real world of practice. Multiple sources of evidence were collected: semi-structured interviews (n=58), non-participant observations at five sites and documentary sources.

Discussion: Strategies to ensure the rigour of this research were prolonged engagement and persistent observation, triangulation, peer debriefing, member checking, audit trail, reflexivity, and thick descriptions. Practical examples of how these strategies can be implemented are provided to guide researchers interested in conducting rigorous case study research.

Conclusion: While the flexible nature of qualitative research should be embraced, strategies to ensure rigour must be in place.

PubMed Disclaimer

Related information

Cited in Books

LinkOut - more resources

Full text sources.

Ovid Technologies, Inc.

Other Literature Sources

scite Smart Citations
Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

- Google Chrome

Intended for healthcare professionals

My email alerts
BMA member login
Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

Advanced search
Search responses
Search blogs
Qualitative Research:...

Qualitative Research: Rigour and qualitative research

Related content
Peer review
Nicholas Mays , director of health services research a ,
Catherine Pope , lecturer in social and behavioural medicine b
a King's Fund Institute, London W2 4HT
b Department of Epidemiology and Public Health, University of Leicester, Leicester LE1 6TP
Correspondence to: Mr Mays.

Various strategies are available within qualitative research to protect against bias and enhance the reliability of findings. This paper gives examples of the principal approaches and summarises them into a methodological checklist to help readers of reports of qualitative projects to assess the quality of the research.

Criticisms of qualitative research

In the health field--with its strong tradition of biomedical research using conventional, quantitative, and often experimental methods--qualitative research is often criticised for lacking scientific rigour. To label an approach “unscientific” is peculiarly damning in an era when scientific knowledge is generally regarded as the highest form of knowing. The most commonly heard criticisms are, firstly, that qualitative research is merely an assembly of anecdote and personal impressions, strongly subject to researcher bias; secondly, it is argued that qualiative research lacks reproducibility--the research is so personal to the researcher that there is no guarantee that a different researcher would not come to radically different conclusions; and, finally, qualitative research is criticised for lacking generalisability. It is said that qualitative methods tend to generate large amounts of detailed information about a small number of settings.

Is qualitative research different?

The pervasive assumption underlying all these criticisms is that quantitative and qualitative approaches are fundamentally different in their ability to ensure the validity and reliability of their findings. This distinction, however, is more one of degree than of type. The problem of the relation of a piece of research to some presumed underlying “truth” applies to the conduct of any form of social research. “One of the greatest methodological fallacies of the last half century in social research is the belief that science is a particular set of techniques; it is, rather, a state of mind, or attitude, and the organisational conditions which allow that attitude to be expressed.” 1 In quantitative data analysis it is possible to generate statistical …

Log in using your username and password

BMA Member Log In

If you have a subscription to The BMJ, log in:

Need to activate
Log in via institution
Log in via OpenAthens

Log in through your institution

Subscribe from £184 *.

Subscribe and get access to all BMJ articles, and much more.

* For online subscription

Access this article for 1 day for: £50 / $60/ €56 ( excludes VAT )

You can download a PDF version for your personal record.

Buy this article

Cancer Nursing Practice
Emergency Nurse
Evidence-Based Nursing
Learning Disability Practice
Mental Health Practice
Nurse Researcher
Nursing Children and Young People
Nursing Management
Nursing Older People
Nursing Standard
Primary Health Care
RCN Nursing Awards
Nursing Live
Nursing Careers and Job Fairs
CPD webinars on-demand
--> Advanced -->

how is procedural rigour demonstrated in a research report

Clinical articles
Expert advice
Career advice
Revalidation
CPD Quizzes

Issues in research Previous Next

Rigour in qualitative research: mechanisms for control, kimberley ryan-nicholls associate professor, university of calgary, calgary, canada, department of psychiatric nursing, school of health studies, brandon university, brandon, canada, constance will associate professor, department of nursing, school of health studies, brandon university, brandon, canada.

Qualitative researchers have been criticised for a perceived failure to demonstrate methodological rigour. Kimberley D Ryan-Nicholls and Constance I Will offer cautionary recommendations related to the mechanisms for control of methodological rigour in qualitative inquiry

Qualitative research methods are accepted as congruent with and relevant to the perspective and goals of nursing. There exists, however, continued criticism of the methodological rigour of qualitative work ( Sandelowski 1986 , 2004 , Farmer et al 2006 , Morse 2006a , 2006b ). This now informs how qualitative work is considered with the current emphasis on evidence-based practice ( Grypdonck 2006 , Morse 2006a , 2006b , Sandelowski et al 2006 ). What is the basis of this criticism? How can methodological rigour be demonstrated? Furthermore, what is rigour and what are the key issues for qualitative researchers? This paper attempts to answer these questions.

Nurse Researcher . 16, 3, 70-85. doi: 10.7748/nr2009.04.16.3.70.c6947

qualitative research - methodology - rigour - nursing research

User not found

Want to read more?

Already have access log in, 3-month trial offer for £5.25/month.

Unlimited access to all 10 RCNi Journals
RCNi Learning featuring over 175 modules to easily earn CPD time
NMC-compliant RCNi Revalidation Portfolio to stay on track with your progress
Personalised newsletters tailored to your interests
A customisable dashboard with over 200 topics

Alternatively, you can purchase access to this article for the next seven days. Buy now

Are you a student? Our student subscription has content especially for you. Find out more

01 April 2009 / Vol 16 issue 3

TABLE OF CONTENTS

DIGITAL EDITION

LATEST ISSUE
SIGN UP FOR E-ALERT
WRITE FOR US
PERMISSIONS

Share article: Rigour in qualitative research: mechanisms for control

We use cookies on this site to enhance your user experience.

By clicking any link on this page you are giving your consent for us to set cookies.

The Ultimate Guide to Qualitative Research - Part 3: Presenting Qualitative Data

Presenting qualitative data
Data visualization
Research paper writing
Introduction

What is rigor in qualitative research?

Why is transparent research important, how do you achieve transparency and rigor in research.

How to publish a research paper

Transparency and rigor in research

Qualitative researchers face particular challenges in convincing their target audience of the value and credibility of their subsequent analysis . Numbers and quantifiable concepts in quantitative studies are relatively easier to understand than their counterparts associated with qualitative methods . Think about how easy it is to make conclusions about the value of items at a store based on their prices, then imagine trying to compare those items based on their design, function, and effectiveness.

The goal of qualitative data analysis is to allow a qualitative researcher and their audience to make determinations about the value and impact of the research. Still, before the audience can reach these determinations, the process of conducting research that produces the qualitative analysis must first be perceived as credible. It is the responsibility of the researcher to persuade their audience that their data collection process and subsequent analysis are rigorous.

Qualitative rigor refers to the meticulousness, consistency, and transparency of the research. It is the application of systematic, disciplined, and stringent methods to ensure the credibility, dependability, confirmability, and transferability of research findings. In qualitative inquiry, these attributes ensure the research accurately reflects the phenomenon it is intended to represent, that its findings can be used by others, and that its processes and results are open to scrutiny and validation.

Credibility

Credibility refers to the extent to which the results accurately represent the participants' experiences. To achieve credibility, qualitative researchers, especially those conducting research on human research participants , employ a number of strategies to bolster the credibility of the data and the subsequent analysis. Prolonged engagement and persistent observation , for example, involve spending significant time in the field to gain a deep understanding of the research context and to continuously observe the phenomenon under study. Peer debriefing involves discussing the research and findings with knowledgeable peers to assess their validity . Member checking involves sharing the findings with the research participants to confirm that they accurately reflect their experiences. These and other methods ensure an abundantly rich data set from which the researcher describes in vivid detail the phenomenon under study, and which other scholars can audit to challenge the strength of the findings if necessary.

Dependability

Dependability refers to the consistency of the research process such that it is logical and clearly documented. It addresses the potential for others to build on the research through subsequent studies. To achieve dependability, researchers should provide a 'decision trail' detailing all decisions made during the course of the study. This allows others to understand how conclusions were reached and to replicate the study if necessary. Ultimately, the documentation of a researcher's process while collecting and analyzing data provides a clear record not only for other scholars to consider but also for those conducting the study and refining their methods for future research.

Confirmability

Confirmability requires the research findings to be directly linked to the data. While it is important to acknowledge researcher positionality (e.g., through reflexive memos) in social science research, researchers still have a responsibility to make assertions and identify insights rooted in the data for the resulting knowledge to be considered confirmable. By transparently communicating how the data was analyzed and conclusions were reached, researchers can allow their audience to perform a sort of audit of the study. This practice helps remind researchers about the importance of ensuring that there are sufficient connections to the raw data collected from the field and the findings that are presented as consequential developments of theory.

Transferability

Transferability refers to the applicability of the research findings in other contexts or with other participants. While dependability is more relevant to the application of research within its own situated context, transferability is determined by how findings generated in one set of circumstances (e.g., a geographic location or a culture) apply to another set of circumstances. This is essentially a significant challenge since, given the unique focus on context in qualitative research , researchers can't usually claim that their findings are universally applicable. Instead, they provide a rich, detailed description of the context and participants, enabling others to determine if the findings may apply to their own context. As a result, such detail necessitates discussion of transparency in research, which will be discussed later in this section.

Reflexivity

The concept of reflexivity also contributes to rigor in qualitative research. Reflexivity involves the researcher critically reflecting on the research and their own role in it, including how their biases , values, experiences, and presence may influence the research. Any discussion of reflexivity necessitates a recognition that knowledge about the social world is never objective, but always from a particular perspective. Reflexivity begins with an acknowledgment that those who conduct qualitative research do so while perceiving the social world through an analytical lens that is unique and distinct from that of others. As subjectivity is an inevitable circumstance in any research involving humans as sources or instruments of data collection , the researcher is responsible for providing a thick description of the environment in which they are collecting data as well as a detailed description of their own place in the research. Subjectivity can be considered as an asset, whereby researchers acknowledge and indicate how their positionality informed the analysis in ways that were insightful and productive.

Triangulation

Triangulation is another key aspect of rigor, referring to the use of multiple data sources, researchers, or methods to cross-check and validate findings. This can increase the depth and breadth of the research, improve its quality, and decrease the likelihood of researcher bias influencing the findings. Particularly given the complexity and dynamic nature of the social world, one method or one analytical approach will seldom be sufficient in holistically understanding the phenomenon or concept under study. Instead, a researcher benefits from examining the world through multiple methods and multiple analytical approaches, not to garner perfectly consistent results but to gather as much rich detail as possible to strengthen the analysis and subsequent findings.

In qualitative research , rigor is not about seeking a single truth or reality, but rather about being thorough, transparent, and critical in the research to ensure the integrity and value of the study. Rigor can be seen as a commitment to best practices in research, with researchers consistently questioning their methods and findings, checking for alternative interpretations , and remaining open to critique and revision. This commitment to rigor helps ensure that qualitative research provides valid, reliable , and meaningful contributions to our understanding of the complex social world.

Transparently document your research with ATLAS.ti

Choose ATLAS.ti for getting the most out of your data. Download a free trial of our software today.

When you read a story in a newspaper or watch a news report on television, do you ever get the feeling that you may not be receiving all the information or context necessary to understand the overarching messages being conveyed? Perhaps a salesperson is trying to convince you to buy something from them by explaining all the benefits of a product but doesn't tell you how they know these benefits are real. When you're choosing a movie to watch, you might look at a critic's review or a score in an online movie database without actually knowing how that review or score was actually determined.

In all of these situations, it is easier to trust the information presented to you if there is a rigorous analysis process behind that information and if that process is explicitly detailed. The same is true for qualitative research results, making transparency a key element in qualitative research methodologies . Transparency is a fundamental aspect of rigor in qualitative research. It involves the clear, detailed, and explicit documentation of all stages of the research process. This allows other researchers to understand, evaluate, transfer, and build upon the study. The key aspects of transparency in qualitative research include methodological transparency, analytical transparency, and reflexive transparency.

Methodological transparency involves providing a comprehensive description of the research methods and procedures used in the study. This includes detailing the research design, sampling strategy, data collection methods , and ethical considerations . For example, researchers should thoroughly describe how participants were selected, how and where data were collected (e.g., interviews , focus groups , observations ), and how ethical issues such as consent, confidentiality , and potential harm were addressed. They should also clearly articulate the theoretical and conceptual frameworks that guided the study. Methodological transparency allows other researchers to understand how the study was conducted and assess its trustworthiness.

Analytical transparency refers to the clear and detailed documentation of the data analysis process. This involves explaining how the raw data were transformed into findings, including the coding process , theme/category development, and interpretation of results . Researchers should describe the specific analytical strategies they used, such as thematic analysis , grounded theory , or discourse analysis . They should provide evidence to support their findings, such as direct quotes from participants. They may also describe any software they used to assist with analyzing data . Analytical transparency allows other researchers to understand how the findings were derived and assess their credibility and confirmability.

Reflexive transparency involves the researcher reflecting on and disclosing their own role in the research, including their potential biases , assumptions, and influences. This includes recognizing and discussing how the researcher's background, beliefs, and interactions with participants may have shaped the data collection and analysis . Reflexive transparency may be achieved through the use of a reflexivity journal, where the researcher regularly records their thoughts, feelings, and reactions during research. This aspect of transparency ensures that the researcher is open about their subjectivity and allows others to assess the potential impact of the researcher's positionality on the findings.

Transparency in qualitative research is essential for maintaining rigor, trustworthiness, and ethical integrity . By being transparent, researchers allow their work to be scrutinized, critiqued, and improved upon, contributing to the ongoing development and refinement of knowledge in their field.

Rigorous, trustworthy research is research that applies the appropriate research tools to meet the stated objectives of the investigation. For example, to determine if an exploratory investigation was rigorous, the investigator would need to answer a series of methodological questions: Do the data collection tools produce appropriate information for the level of precision required in the analysis ? Do the tools maximize the chance of identifying the full range of what there is to know about the phenomenon? To what degree are the collection techniques likely to generate the appropriate level of detail needed for addressing the research question(s) ? To what degree do the tools maximize the chance of producing data with discernible patterns?

Once the data are collected, to what degree are the analytic techniques likely to ensure the discovery of the full range of relevant and salient themes and topics? To what degree do the analytic strategies maximize the potential for finding relationships among themes and topics? What checks are in place to ensure that the discovery of patterns and models are relevant to the research question? Finally, what standards of evidence are required to ensure readers that results are supported by the data?

The clear challenge is to identify what questions are most important for establishing research rigor (trustworthiness) and to provide examples of how such questions could be answered for those using qualitative data . Clearly, rigorous research must be both transparent and explicit; in other words, researchers need to be able to describe to their colleagues and their audiences what they did (or plan to do) in clear, simple language. Much of the confusion that surrounds qualitative data collection and analysis techniques comes from practitioners who shroud their behaviors in mystery and jargon. For example, clearly describing how themes are identified, how codebooks are built and applied, and how models were induced helps to bring more rigor to qualitative research .

Researchers also must become more familiar with the broad range of methodological techniques available, such as content analysis , grounded theory , and discourse analysis . Cross-fertilization across methodological traditions can also be extremely valuable to generate meaningful understanding rather than attacking all problems with the same type of methodological tool.

The introduction of methodologically neutral and highly flexible qualitative analysis software like ATLAS.ti can be considered as extremely helpful indeed. It is highly apt to both support interdisciplinary cross-pollination and to bring about a great deal of trust in the presented results. By allowing the researcher to combine both the source material and his/her findings in a structured, interactive platform while producing both quantifiable reports and intuitive visual renderings of their results, ATLAS.ti adds new levels of trustworthiness to qualitative research . Moreover, it permits the researcher to apply multiple approaches to their research, to collaborate across philosophical boundaries, and thus significantly enhance the level of rigor in qualitative research. Dedicated research software like ATLAS.ti helps the researcher to catalog, explore and competently analyze the data generated in a given research project.

Ultimately, transparency and rigor are indispensable elements of any robust research study. Achieving transparency requires a systematic, deliberate, and thoughtful approach. It revolves around clarity in the formulation of research objectives, comprehensiveness in methods, and conscientious reporting of the results. Here are several key strategies for achieving transparency and rigor in research:

Clear research objectives and methods

Transparency begins with the clear and explicit statement of research objectives and questions. Researchers should explain why they are conducting the study, what they hope to learn, and how they plan to achieve their objectives. This involves identifying and articulating the study's theoretical or conceptual framework and delineating the key research questions . Ensuring clarity at this stage sets the groundwork for transparency throughout the rest of the study.

Transparent research includes a comprehensive and detailed account of the research design and methodology. Researchers should describe all stages of their research process, including the selection and recruitment of participants, the data collection methods , the setting of the research, and the timeline. Each step should be explained in enough detail that another researcher could replicate the study. Furthermore, any modifications to the research design or methodology over the course of the study should be clearly documented and justified.

Thorough data documentation and analysis

In the data collection phase, researchers should provide thorough documentation, including original data records such as transcripts , field notes , or images . The specifics of how data was gathered, who was involved, and when and where it took place should be meticulously recorded.

During the data analysis phase , researchers should clearly describe the steps taken to analyze the data, including coding processes , theme identification , and how conclusions were drawn. Researchers should provide evidence to support their findings and interpretations , such as verbatim quotes or detailed examples from the data. They should also describe any analytic software or tools used, including how they were used and why they were chosen.

Reflexivity and acknowledgment of bias

Transparent research involves a process of reflexivity , where researchers critically reflect on their own role in the research process. This includes considering how their own beliefs, values, experiences, and relationships with participants may have influenced the data collection and analysis . Researchers should maintain reflexivity journals to document these reflections, which can then be incorporated into the final research report. Researchers should also explicitly acknowledge potential biases and conflicts of interest that could influence the research. This includes personal, financial, or institutional interests that could affect the conduct or reporting of the research.

Transparent reporting and publishing

Transparency also involves the open sharing of research materials and data, where ethical and legal guidelines permit. This may include providing access to interview guides , survey instruments , data analysis scripts, raw data , and other research materials. Open sharing allows others to scrutinize, transfer, or extend the research, thereby enhancing its transparency and trustworthiness.

Finally, the reporting and publishing phase should adhere to the principles of transparency. Researchers should follow the relevant reporting guidelines for their field. Such guidelines provide a framework for reporting research in a comprehensive, systematic, and transparent manner.

Furthermore, researchers should choose to publish in open-access journals or other accessible formats whenever possible, to ensure the research is publicly accessible. They should also be open to critique and engage in post-publication discussion and debate about their findings.

By adhering to these strategies, researchers can ensure the transparency of their research, enhancing its credibility, trustworthiness, and contribution to their field. Transparency is more than just a good research practice—it's a fundamental ethical obligation to the research community, participants, and wider society.

Rigorous research starts with ATLAS.ti

Click here for a free trial of our powerful and intuitive data analysis software.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

20.1 Introduction to qualitative rigor

We hear a lot about fake news these days. Fake news has to do with the quality of journalism that we are consuming. It begs questions like: does it contain misinformation, is it skewed or biased in its portrayal of stories, does it leave out certain facts while inflating others. If we take this news at face value, our opinions and actions may be intentionally manipulated by poor quality information. So, how do we avoid or challenge this? The oversimplified answer is, we find ways to check for quality. While this isn’t a chapter dedicated to fake news, it does offer an important comparison for the focus of this chapter, rigor in qualitative research. Rigor is concerned with the quality of research that we are designing and consuming. While I devote a considerable amount of time in my clinical class talking about the importance of adopting a non-judgmental stance in practice, that is not the case here; I want you to be judgmental, critical thinkers about research! As a social worker who will hopefully be producing research (we need you!) and definitely consuming research, you need to be able to differentiate good science from rubbish science. Rigor will help you to do this.

This chapter will introduce you to the concept of rigor and specifically, what it looks like in qualitative research. We will begin by considering how rigor relates to issues of ethics and how thoughtfully involving community partners in our research can add additional dimensions in planning for rigor. Next, we will look at rigor in how we capture and manage qualitative data, essentially helping to ensure that we have quality raw data to work with for our study. Finally, we will devote time to discussing how researchers, as human instruments, need to maintain accountability throughout the research process. Finally, we will examine tools that encourage this accountability and how they can be integrated into your research design. Our hope is that by the end of this chapter, you will begin to be able to identify some of the hallmarks of quality in qualitative research, and if you are designing a qualitative research proposal, that you consider how to build these into your design.

19.1 Introduction to qualitative rigor

Learning objectives.

Learners will be able to…

Identify the role of rigor in qualitative research and important concepts related to qualitative rigor
Discuss why rigor is an important consideration when conducting, critiquing and consuming qualitative research
Differentiate between quality in quantitative and qualitative research studies

In Chapter 11 we talked about quality in quantitative studies, but we built our discussion around concepts like reliability and validity . With qualitative studies, we generally think about quality in terms of the concept of rigor . The difference between quality in quantitative research and qualitative research extends beyond the type of data (numbers vs. words/sounds/images). If you sneak a peek all the way back to Chapter 7 , we discussed the idea of different paradigms or fundamental frameworks for how we can think about the world. These frameworks value different kinds of knowledge, arrive at knowledge in different ways, and evaluate the quality of knowledge with different criteria. These differences are essential in differentiating qualitative and quantitative work.

Quantitative research generally falls under a positivist paradigm, seeking to uncover knowledge that holds true across larger groups of people. To accomplish this, we need to have tools like reliability and validity to help produce internally consistent and externally generalizable findings (i.e. was our study design dependable and do our findings hold true across our population).

In contrast, qualitative research is generally considered to fall into an alternative paradigm (other than positivist), such as the interpretive paradigm which is focused on the subjective experiences of individuals and their unique perspectives. To accomplish this, we are often asking participants to expand on their ideas and interpretations. A positivist tradition requires the information collected to be very focused and discretely defined (i.e. closed questions with prescribed categories). With qualitative studies, we need to look across unique experiences reflected in the data and determine how these experiences develop a richer understanding of the phenomenon we are studying, often across numerous perspectives.

Rigor is a concept that reflects the quality of the process used in capturing, managing, and analyzing our data as we develop this rich understanding. Rigor helps to establish standards through which qualitative research is critiqued and judged, both by the scientific community and by the practitioner community.

For the scientific community, people who review qualitative research studies submitted for publication in scientific journals or for presentations at conferences will specifically look for indications of rigor, such as the tools we will discuss in this chapter. This confirms for them that the researcher(s) put safeguards in place to ensure that the research took place systematically and that consumers can be relatively confident that the findings are not fabricated and can be directly connected back to the primary sources of data that was gathered or the secondary data that was analyzed.

As a note here, as we are critiquing the research of others or developing our own studies, we also need to recognize the limitations of rigor. No research design is flawless and every researcher faces limitations and constraints. We aren’t looking for a researcher to adopt every tool we discuss below in their design. In fact, one of my mentors, speaks explicitly about “misplaced rigor”, that is, using techniques to support rigor that don’t really fit what you are trying to accomplish with your research design. Suffice it to say that we can go overboard in the area of rigor and it might not serve our study’s best interest. As a consumer or evaluator of research, you want to look for steps being taken to reflect quality and transparency throughout the research process, but they should fit within the overall framework of the study and what it is trying to accomplish.

From the perspective of a practitioner, we also need to be acutely concerned with the quality of research. Social work has made a commitment, outlined in our Code of Ethics (NASW,2017) , to competent practice in service to our clients based on “empirically based knowledge” (subsection 4.01). When I think about my own care providers, I want them to be using “good” research—research that we can be confident was conducted in a credible way and whose findings are honestly and clearly represented. Don’t our clients deserve the same from us?

As providers, we will be looking to qualitative research studies to provide us with information that helps us better understand our clients, their experiences, and the problems they encounter. As such, we need to look for research that accurately represents:

Who is participating in the study
What circumstances is the study being conducted under
What is the research attempting to determine

Further, we want to ensure that:

Findings are presented accurately and reflect what was shared by participants ( raw data )
A reasonably good explanation of how the researcher got from the raw data to their findings is presented
The researcher adequately considered and accounted for their potential influence on the research process

As we talk about different tools we can use to help establish qualitative rigor, I will try to point out tips for what to look for as you are reading qualitative studies that can reflect these. While rigor can’t “prove” quality, it can demonstrate steps that are taken that reflect thoughtfulness and attention on the part of the researcher(s). This is a link from the American Psychological Association on the topic of reviewing qualitative research manuscripts. It’s a bit beyond the level of critiquing that I would expect from a beginning qualitative research student, however, it does provide a really nice overview of this process. Even if you aren’t familiar with all the terms, I think it can be helpful in giving an overview of the general thought process that should be taking place.

To begin breaking down how to think about rigor, I find it helpful to have a framework to help understand different concepts that support or are associated with rigor. Lincoln and Guba (1985) have suggested such a framework for thinking about qualitative rigor that has widely contributed to standards that are often employed for qualitative projects. The overarching concept around which this framework is centered is trustworthiness . Trustworthiness is reflective of how much stock we should put in a given qualitative study—is it really worth our time, headspace, and intellectual curiosity? A study that isn’t trustworthy suggests poor quality resulting from inadequate forethought, planning, and attention to detail in how the study was carried out. This suggests that we should have little confidence in the findings of a study that is not trustworthy.

According to Lincoln and Guba (1985) [1] trustworthiness is grounded in responding to four key ideas and related questions to help you conceptualize how they relate to your study. Each of these concepts is discussed below with some considerations to help you to compare and contrast these ideas with more positivist or quantitative constructs of research quality.

Truth value

You have already been introduced to the concept of internal validity . As a reminder, establishing internal validity is a way to ensure that the change we observe in the dependent variable is the result of the variation in our independent variable—did we actually design a study that is truly testing our hypothesis. In much/most qualitative studies we don’t have hypotheses, independent or dependent variables, but we do still want to design a study where our audience (and ourselves) can be relatively sure that we as the researcher arrived at our findings through a systematic and scientific process, and that those findings can be clearly linked back to the data we used and not some fabrication or falsification of that data; in other words, the truth value of the research process and its findings. We want to give our readers confidence that we didn’t just make up our findings or “see what we wanted to see”.

Applicability

who we were studying
how we went about studying them
what we found

Consistency

These concepts reflect a set of standards that help to determine the integrity of qualitative studies. At the end of this chapter you will be introduced to a range of tools to help support or reflect these various standards in qualitative research. Because different qualitative designs (e.g. phenomenology, narrative, ethnographic), that you will learn more about in Chapter 22 emphasize or prioritize different aspects of quality, certain tools will be more appropriate for these designs. Since this chapter is intended to give you a general overview of rigor in qualitative studies, exploring additional resources will be necessary to best understand which of these concepts are prioritized in each type of design and which tools best support them.

Key Takeaways

Qualitative research is generally conducted within an interpretativist paradigm. This differs from the post-positivist paradigm in which most quantitative research originates. This fundamental difference means that the overarching aim of these different approaches to knowledge building differ, and consequently, our standards for judging the quality of research within these paradigms differ.
Assessing the quality of qualitative research is important, both from a researcher and a practitioner perspective. On behalf of our clients and our profession, we are called to be critical consumers of research. To accomplish this, we need strategies for assessing the scientific rigor with which research is conducted.
Trustworthiness and associated concepts, including credibility, transferablity, dependability and confirmability, provide a framework for assessing rigor or quality in qualitative research.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry . Newberry Park, CA: Sage ↵

Rigor is the process through which we demonstrate, to the best of our ability, that our research is empirically sound and reflects a scientific approach to knowledge building.

The degree to which an instrument reflects the true score rather than error. In statistical terms, reliability is the portion of observed variability in the sample that is accounted for by the true variability, not by error. Note : Reliability is necessary, but not sufficient, for measurement validity.

The extent to which the scores from a measure represent the variable they are intended to.

a paradigm guided by the principles of objectivity, knowability, and deductive logic

Findings form a research study that apply to larger group of people (beyond the sample). Producing generalizable findings requires starting with a representative sample.

a paradigm based on the idea that social context and interaction frame our realities

in a literature review, a source that describes primary data collected and analyzed by the author, rather than only reviewing what other researchers have found

Data someone else has collected that you have permission to use in your research.

unprocessed data that researchers can analyze using quantitative and qualitative methods (e.g., responses to a survey or interview transcripts)

Trustworthiness is a quality reflected by qualitative research that is conducted in a credible way; a way that should produce confidence in its findings.

Ability to say that one variable "causes" something to happen to another variable. Very important to assess when thinking about studies that examine causation such as experimental or quasi-experimental designs.

The level of confidence that research is obtained through a systematic and scientific process and that findings can be clearly connected to the data they are based on (and not some fabrication or falsification of that data).

The ability to apply research findings beyond the study sample to some broader population,

This is a synonymous term for generalizability - the ability to apply the findings of a study beyond the sample to a broader population.

The potential for qualitative research findings to be applicable to other situations or with other people outside of the research study itself.

Consistency is the idea that we use a systematic (and potentially repeatable) process when conducting our research.

a single truth, observed without bias, that is universally applicable

one truth among many, bound within a social and cultural context

The idea that qualitative researchers attempt to limit or at the very least account for their own biases, motivations, interests and opinions during the research process.

Share This Book

Basic Search
Advanced search
Individual Login
Create an account
Purchase options
Subscribe to the MJA
Publish with us
Current issue
Online first
Anatomy and physiology
Anesthesia, analgesia and pain
Cardiovascular diseases
Complementary therapies
Diagnostic techniques and procedures
Digestive system diseases
Emergency medicine
Endocrine system diseases
Environment and public health
Ethics and law
Eye diseases
General medicine
Gerontology
Global health
Health occupations
Health services administration
Hematologic diseases
History and humanities
Immune system diseases
Indigenous health
Infectious diseases
Information science
Medical education
Medical genetics
Men's health
Mental disorders
Musculoskeletal diseases
Nervous system diseases
Nutritional and metabolic diseases
Occupational diseases
Otorhinolaryngologic diseases
Palliative care
Pediatric medicine
Pharmaceutical preparations
Public and environmental health
Rehabilitation
Respiratory tract diseases
Rheumatology
Sexual health
Skin and connective tissue diseases
Social determinants of health
Sports medicine
Statistics, epidemiology and research design
Substance-related disorders
Substance‐related disorders
Surgical procedures, operative
Urologic diseases
Vascular diseases
Women's health
Wounds and injuries
Research letters
Guidelines and statements
Narrative reviews
Perspectives
Reflections
Competitions
The Medical Journal of Australia

Archive keyboard_arrow_down
search SEARCH -->

Issues by year

Supplements

Article types

Quality in qualitative research

picture_as_pdf Download

Qualitative research most commonly involves the systematic collection, ordering, description and interpretation of textual data generated from talk, observation or documentation.

A report of qualitative research should address the following criteria:

Clarification and justification;

Procedural rigour;

Representativeness;

Interpretative rigour;

Reflexivity and evaluative rigour; and

Transferability.

Because of the limitations on article length for the Medical Journal of Australia , authors should focus on only a couple of aspects of the research, rather than trying to present a simplified description of multiple aspects.

R esearch in health care, especially clinical medicine, is an increasingly complex field that ranges from small-scale, cutting-edge benchtop science to large-scale population studies. Recently, there has been an increasing recognition by evidence-based clinical researchers that it is important to look towards qualitative research. In addition, an expanding interest in examining the attitudes, beliefs and experiences of those involved or affected by the delivery of health care have brought qualitative research into the spotlight. 1 The increased interest in this form of research has led to concerns by readers and reviewers about the assessment of quality. 2 - 10 In this article, we provide definitions and standards for qualitative research that can be used for assessment. We expect that authors who submit work for publication that adheres to the suggestions in this article will increase the likelihood of acceptance for publication and, more importantly, will enhance the transferability of findings into policy and practice in the future.

Most commonly, qualitative research is concerned with the systematic collection, ordering, description and interpretation of textual data generated from talk, observation or documentation. Qualitative research methods include the techniques of interviewing, observation, and document analysis. Its goal is to explore the behaviour, processes of interaction, and the meanings, values and experiences of purposefully sampled individuals and groups in their “natural” context. 11 - 14 The capacity to make conceptual generalisations from the local context of a qualitative study to other settings is the desired outcome.

In contrast, most quantitative research is concerned with measuring the magnitude, size or extent of a phenomenon. Data collection is by formal rules of procedure and verification, analysis is through the use of standardised statistical formats, and prediction and empirical generalisation are the desired outcomes. 15 , 16 The conventional methodological criteria of quantitative research — validity, reliability and empirical generalisability — are generally not directly applied to qualitative research because of the different frameworks, sampling approaches, size of sample and goals of qualitative research. Instead, terms such as rigour (thoroughness and appropriateness of the use of research methods), credibility (meaningful, well presented findings) and relevance (utility of findings) are used to judge the quality or “trustworthiness” of a study (see Box 1 for definitions of some common terms in qualitative research). 17

There has been a strong move towards standardising research reporting in quantitative research, as can be seen through the development of standard reporting mechanisms like the CONSORT (Consolidated Standards of Reporting Trials) statement. 18 However, some question the appropriateness of using checklists to assess qualitative research because of the diversity of approaches in collecting, analysing and interpreting qualitative data. 4 , 19 Furthermore, the sheer number of checklists can prove overwhelming and confusing for new researchers. A recent review of criteria for assessing interview and focus group studies revealed more than 22 checklists actively being used, and subsequently resulted in the formulation of a new 32-item guideline. 20

Nevertheless, it is possible to develop clear and useful generic guidelines for assessing and presenting qualitative research. 2 , 7 , 17 , 21 Box 2 outlines some criteria that have been constructed with the MJA readership in mind that can be used to assess and enhance the rigour of qualitative studies. In themselves, these criteria do not ensure rigour. However, they can strengthen rigour if they are used in concordance with a broader understanding of qualitative research design, data collection and analysis. 19

As in all forms of research, clarity of research question reflected in the aims of the study is essential for evaluating results and their interpretation. The demonstration of theoretical rigour (referring to the soundness of fit of the research question, aims and the choice of methods appropriate to the research problem 11 ) is extremely important.

There is a wide variety of named qualitative approaches that are underpinned by particular theoretical perspectives. In addition, the researcher may use basic field research (question, investigation, interpretation). Regardless of the theoretical approach used, the choice requires justification in reference to the research question of the study.

Procedural, or methodological, rigour concerns the transparency or “explicitness” of the description of the way the research was conducted. It involves detailing issues of accessing subjects; development of rapport and trust; how data are collected, recorded, coded and analysed; and accounts of the manner in which errors or subject refusals are dealt with. 4 , 11 , 22 In this regard, readers and reviewers may ask the following questions while examining descriptions of qualitative methods: How were participants/settings accessed? Who was interviewed/observed? How often? For how long? What interview questions were asked? What was the purpose of any observation? Which policy documents/case notes were accessed? How were they assessed? How was collected data managed?

There are a number of commonly available, non-probability sampling approaches. Maximum variation sampling seeks representativeness of all aspects of the topic in terms of participants. Homogenous sampling consists of the selection of a group fitting specified criteria. Snowball sampling involves networking from one difficult-to-access type of participant to a wider range of participants. Finally, convenience sampling involves studying easily accessed individuals or groups. This last technique obviously presents its own ethical dilemmas of the “insider” type and is possibly the weakest form of sampling in terms of allowing conceptual generalisability. 4 , 15 , 16 , 22 , 23 Maximum variation is the ideal when a holistic overview of the phenomenon is sought; for instance, the question of how a particular hospital department operates may involve sampling in the wider organisation as well as within the individual department and among recipients of services.

Simply mentioning the sampling strategy in the methods section of a qualitative research paper is not sufficient. The key findings of the research need to be evaluated in reference to the diverse characteristics of the research subjects. Through constantly comparing the experiences and responses of the participants against each other, subtle but significant differences can be uncovered that can generate profound insights into the phenomena under study. 19

Interpretative rigour relates to as full as possible a demonstration of the data/evidence. In qualitative research, a commonly used concept is inter-rater reliability . This refers to using a type of researcher triangulation by which multiple researchers are involved in the analytical process. This is an attempt to increase the validity and reliability of the study 19 through the provision of a more complex and nuanced understanding of the possible interpretations of the objects of the research. 11 In contrast to the quantitative research paradigm, what is important in this process is not the level of consensus, but the opportunity for discussion among analysts to provide opportunities for developing further coding. 19

A related technique is that of respondent validation , or member checking. This entails offering subjects interviewed the opportunity to view and amend their transcripts as a type of validity. 12 However, this approach does have limitations due to the evolution over time of the positions and purposes of the researchers and participants, thereby potentially affecting interpretations and accounts. Respondent validation should be thought of as part of a process of reducing error, which involves the generation of further original data, which then requires interpretation. 8

Other techniques that enhance interpretative rigour are the differing forms of triangulation: data (multiple evidentiary sources; ie, documents, interviews, survey data, observation), methods (multiple methods), and theory (multiple theoretical and conceptual frames applied to the research to enhance insights into phenomena). Using these forms of triangulation allows the development of a comprehensive understanding of the phenomena and can ameliorate the potential bias of simply using one method. 4 , 5 , 8 , 11 , 16 , 22

In the interpretive process, accounts of “negative” or “deviant” cases are especially important. These are explanations pertaining to data or evidence that contradicts the researchers overall explanatory account of the phenomena. 5

In sum, a clear description of what forms of analysis were used, the process and what were the major outcomes of the analytical process in terms of findings is needed to ensure quality for the author, and to enable an assessment to be made in terms of the analytical quality of the research by the reader.

Reflexivity is where researchers openly acknowledge and address the influence that the relationship among the researchers, the research topic and subjects may have on the results. 4 , 11 , 13 Fundamentally, reflexivity requires a demonstration by the researchers that they are aware of the sociocultural position they inhabit and how their value systems might affect the selection of the research problem, research design, collection and analysis of data. 15 It also refers to an awareness by the researchers of the social setting of the research and of the wider social context in which it is placed. 4

Evaluative rigour refers to ensuring that the ethical and political aspects of research are addressed. Typically, this refers to proper ethics approval from appropriate committees covering confidentiality, informed consent and steps to avoid possible adverse effects on the subjects. Importantly, where appropriate, relevant community leaders should be consulted in the design and conduct of the research. 11 Researchers should revisit their actions and interactions within the research process to ensure as “accurate” as possible portrayal of the production of their findings.

Conceptual generalisability and transferability refer to how well the study’s findings inform health care contexts that differ from that in which the original study was undertaken. 4 For example, a review of data from qualitative studies was conducted on a wide variety of doctor–patient interactions about medication compliance. 24 The authors examined barriers to patients taking prescribed medication as directed by their doctors and found that patients were often inclined to resist taking medicines, not because of problems with the patients, doctors or systems, but because patients were concerned about the medicines. This type of study allows for the construction and transfer of general policy on medicine-taking (through, for example, less emphasis on patient behaviour modification and more emphasis on production of safer medicines) and practice (suggesting, for instance, that doctors should assist lay evaluations through provision of more information, support, feedback and safe prescribing practices).

The formats of biomedical journals such as the Medical Journal of Australia and BMJ provide particular challenges for comprehensive analysis and reporting of qualitative studies. 4 , 11 , 22 The house style of biomedical journals involves tight restrictions on word length and particular forms of presentation. For instance, lengthy ethnographic studies of health issues can require 6000–8000 words to successfully elucidate the phenomena. 25 Obviously, this is not tenable within most biomedical journals, where article lengths of 2000–3000 words are more common. These raise significant limitations to an exhaustive analysis of the volume of data that is commonly produced within qualitative studies. 5

When writing for the MJA , we suggest that qualitative authors focus on only a couple of aspects of their research findings and use visual displays strategically (ie, charts, quotes and tables). 22 We suggest that authors avoid an overabundance of simplified descriptions of multiple aspects of the phenomena under study. This detracts from a focused, transparent and considered analysis of the core issues relevant to the objectives of the research.

We have set out some useful general rules that, if followed, will allow for concise and informative assessment of qualitative findings. These assessment criteria will be particularly relevant to articles that use the most common qualitative data collection techniques: interviews, focus groups, document analysis, or observation techniques (alone or in combination). We suggest that potential MJA authors also consider using the article format and distribution of word count presented in Box 3 to maximise the rigour and clarity of their research articles. Using this outline should allow the authors to provide the reviewers and readers with enough information on each aspect of the research to engender a sense of research rigour and the trustworthiness of research findings.

1 Glossary of terms

Credibility: refers to whether the findings are well presented and meaningful

Evaluative rigour: the transparent description of ethical and political aspects of the conduct of research

Procedural rigour: the transparent description of the conduct of research

Reflexivity: open acknowledgement of the complex influences among the researchers, the research topic and subjects on the research results

Transferability or relevance: refers to how useful the findings are to the context and phenomenon under study

Triangulation: a comprehensive approach to the conduct of research using multiple theories, data and methods

2 Criteria for assessing qualitative research

Clarification

What are the aims of the research?

What is the research question?

Justification

Why is a qualitative approach the best option to answer this question?

Why was the particular qualitative research design chosen?

Procedural rigour

Have the techniques of data collection been clearly documented?

Are the forms of data analysis completely transparent?

Representativeness

What sampling techniques have been used to answer the research question?

Do the sampling techniques support conceptual generalisability?

Interpretation

Has a more conceptual discussion of the results and linkage to existing theory or new theory been developed to explain the relevance of findings to a targeted audience or discipline?

Have any negative cases been included and discussed?

Reflexivity and evaluative rigour

Has a clear statement of the effect on the data of the researcher’s views and the methods chosen been included?

Has an explicit evaluation of the relationship between the researcher and those under research, addressing any ethical issues, been discussed?

Has ethics approval been obtained from an appropriate institution?

Transferability

Has a critical evaluation of the application of findings to other similar contexts been made?

Has the relevance of these findings to current knowledge, policy, and practice or to current research been discussed?

3 Format of qualitative research articles

Simon C Kitto 1
Janice Chesters 1
Carol Grbich 2
1 School of Rural Health, Monash University, Moe, VIC.
2 Flinders University, Adelaide, SA.

None identified.

1. Green J, Britten N. Qualitative research and evidence based medicine. BMJ 1998; 316: 1230-1232.
2. Boulton M, Fitzpatrick R, Swinburn C. Qualitative research in health care: II. A structured review and evaluation of studies. J Eval Clin Pract 1996; 2: 171-179.
3. Daly J, Willis K, Small R, et al. A hierarchy of evidence for assessing qualitative health research. J Clin Epidemiol 2007; 60: 43-49.
4. Green J, Thorogood N. Qualitative methods for health research. London: Sage, 2004.
5. Mays N, Pope C. Qualitative research: rigour and qualitative research. BMJ 1995; 311: 109-112.
6. Mays N, Pope C. Qualitative research: observational methods in health care settings. BMJ 1995; 311: 182-184.
7. Mays N, Pope C. Quality in qualitative health research. In: Pope C, Mays N, editors. Qualitative research in health care. London: BMJ Books, 1999: 89-101.
8. Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative research. BMJ 2000; 320: 50-52.
9. Pope C, Mays N. Qualitative research: reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ 1995; 311: 42-45.
10. Pope C, Ziebland S, Mays N. Qualitative research in health care: analysing qualitative data. BMJ 2000; 320: 114-116.
11. Liamputtong P, Ezzy D. Qualitative research methods. Melbourne: Oxford University Press, 2005.
12. Lincoln YS, Guba EG. Naturalistic inquiry. Newbury Park, Calif: Sage, 1985.
13. Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet 2001; 358: 483-488.
14. Miles MB, Huberman AM. Qualitative data analysis: an expanded sourcebook. 2nd ed. Thousand Oaks, Calif: Sage, 1994.
15. Grbich C. Qualitative research in health: an introduction. London: Sage, 1999.
16. Grbich C. Qualitative data analysis: an introduction. London: Sage, 2007.
17. Critical Appraisal Skills Programme (CASP). 10 questions to help you make sense of qualitative research. Oxford: Public Health Resource Unit, 2006. http://www.phru.nhs.uk/Doc_Links/Qualitative%20Appraisal%20 Tool.pdf (accessed Oct 2007).
18. Moher D, Schulz K, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001; 285: 1987-1991.
19. Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ 2001; 322: 1115-1117.
20. Tong A, Sainsbury P, Craig JC. Critical appraisal criteria for interviews and focus groups. In: Qualitative Research in Evidence-based Healthcare — an Exploration of Scope and Methods. 2006; Jul 10–11; Adelaide.
21. Blaxter M. Criteria for qualitative research. Med Sociol News 2000; 26: 34-37.
22. Hansen EC. Successful qualitative health research: a practical introduction. Sydney: Allen and Unwin, 2006.
23. Rice PL, Ezzy D. Qualitative research methods: a health focus. Melbourne: Oxford University Press, 1999.
24. Pound P, Britten N, Morgan M, et al. Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med 2005; 61: 133-155.
25. Moore D. Governing street-based injecting drug users: a critique of heroin overdose prevention in Australia. Soc Sci Med 2004; 59: 1547-1557.

Publication of your online response is subject to the Medical Journal of Australia 's editorial discretion. You will be notified by email within five working days should your response be accepted.

Do you have any competing interests to declare? *

Ensuring rigour and trustworthiness of qualitative research in clinical pharmacy

Research Article
Published: 14 December 2015
Volume 38 , pages 641–646, ( 2016 )

Cite this article

Muhammad Abdul Hadi 1 &
S. José Closs 2

18k Accesses

134 Citations

Explore all metrics

The use of qualitative research methodology is well established for data generation within healthcare research generally and clinical pharmacy research specifically. In the past, qualitative research methodology has been criticized for lacking rigour, transparency, justification of data collection and analysis methods being used, and hence the integrity of findings. Demonstrating rigour in qualitative studies is essential so that the research findings have the “integrity” to make an impact on practice, policy or both. Unlike other healthcare disciplines, the issue of “quality” of qualitative research has not been discussed much in the clinical pharmacy discipline. The aim of this paper is to highlight the importance of rigour in qualitative research, present different philosophical standpoints on the issue of quality in qualitative research and to discuss briefly strategies to ensure rigour in qualitative research. Finally, a mini review of recent research is presented to illustrate the strategies reported by clinical pharmacy researchers to ensure rigour in their qualitative research studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

Get 10 units per month
Download Article/Chapter or eBook
1 Unit = 1 Article or 1 Chapter
Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Evidence Produced While Using Qualitative Methodologies Including Research Trustworthiness

Evaluating Qualitative Health Research from Inside and Outside

Reconsidering ‘ethics’ and ‘quality’ in healthcare research: the case for an iterative ethical paradigm

Denzin NK, Lincoln YS, editors. The SAGE handbook of qualitative research. Thousand Oaks: Sage; 2005.

Google Scholar

Pope C, Mays N. Reaching the parts other methods cannot: an introduction to qualitative methods in health and health services research. BMJ. 1995;311:42–5.

Article CAS PubMed PubMed Central Google Scholar

Anderson C. Presenting and evaluating qualitative research. Am J Pharm Educ. 2010;74(8):6 Article 141 .

Article Google Scholar

Rolfe G. Validity, trustworthiness and rigour: quality and the idea of qualitative research. J Appl Nurs. 2006;53:304–10.

Long T, Johnson M. Rigour, reliability and validity in qualitative research. Clin Eeffect Nurs. 2000;4:30–7.

Cohen DJ, Crabtree BF. Evaluative criteria for qualitative research in health care: controversies and recommendations. Ann Fam Med. 2008;6(4):331–9.

Article PubMed PubMed Central Google Scholar

Lincoln YS, Guba EG. Naturalistic inquiry. Beverly Hills: Sage; 1985.

Sandelowski M. Rigor or rigor mortis: the problem of rigour in qualitative research revisited. ANS Adv Nurs Sci. 1993;16:1–8.

Article CAS PubMed Google Scholar

Morse JM, Barrett M, Mayan M, Olson K, Spiers J. Verification strategies for establishing reliability and validity in qualitative research. Int J Qual Methods. 2002;1:13–22.

Noble H, Smith J. Issues of validity and reliability in qualitative research. Evid Based Nurs. 2015;18:34–5.

Article PubMed Google Scholar

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19:349–57.

O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89:1245–51.

Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12:181.

Creswell JW. Qualitative inquiry and research design: choosing among five approaches. 2nd ed. Thousand Oaks: Sage; 2006.

Hammersley M, Atkinson P. Ethnography: principles in practice. 2nd ed. London: Routledge; 1995.

Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. Edu Inform. 2004;22:63–75.

Sandelowski M. The problem of rigor in feminist research. ANS Adv Nurs Sci. 1986;8:27–37.

Nguyen TST. Peer debriefing. The Sage encyclopedia of qualitative research methods. 2008. http://srmo.sagepub.com/view/sage-encyc-qualitative-research-methods/n312.xml . Accessed 5 June 2014.

Ogunbayo OJ, Schafheutle EI, Cutts C, Noyce PR. A qualitative study exploring community pharmacists’ awareness of, and contribution to, self-care support in the management of long-term conditions in the United Kingdom. Res Soc Adm Pharm 2015. doi: 10.1016/j.sapharm.2014 [Published online Feb, 2015].

Shiyanbola OO, Mort JR. Exploring consumer understanding and preferences for pharmacy quality information. Pharm Pract. 2014;12:1–11.

Murphy A, Szumilas M, Rowe D, et al. Pharmacy students’ experiences in provision of community pharmacy mental health services. Can Pharm J. 2014;147:55–65.

Ryder H, Aspden T, Sheridan J. The Hawke’s Bay Condom Card Scheme: a qualitative study of the views of service providers on increased, discreet access for youth to free condoms. Int J Pharm Pract. 2015. doi: 10.1111/ijpp.12178 .

Ziaei Z, Hassell K, Schafheutle EI. Internationally trained pharmacists’ perception of their communication proficiency and their views on the impact on patient safety. Res Soc Adm Pharm. 2015;11:428–41.

Hasan S, Stewart K, Chapman CB, Hasan MY, Kong DCM. Physicians’ attitudes towards provision of primary care services in community pharmacy in the United Arab Emirates. Int J Pharm Pract. 2014. doi: 10.1111/ijpp.12157 .

PubMed Google Scholar

Marquis J, Schneider MP, Spencer B, Bugnon O, Pasquier SD. Exploring the implementation of a medication adherence programme by community pharmacists: a qualitative study Int J. Clin Pharm. 2014;36:1014–22.

Swain L, Griffits C, Lisa Pont L, Barclay L. Attitudes of pharmacists to provision of home medicines review for indigenous Australians. Int. J Clin Pharm. 2014;36:1260–7.

Brazinha I, Fernandez-Llimos F. Barriers to the implementation of advanced clinical pharmacy services at Portuguese hospitals Int J. Clin Pharm. 2014;36:1031–8.

Odukoya OK, Stone JA, Chui MA. Barriers and facilitators to recovering from e- prescribing errors in community pharmacies. J Am Pharm Assoc. 2015;55:52–8.

Download references

No funding from any governmental or non-governmental agency was obtained for this paper.

Author information

Authors and affiliations.

Department of Clinical Pharmacy, Faculty of Pharmacy, Umm-Al-Qura University, Al-Abdia, Makkah, 13578, Saudi Arabia

Muhammad Abdul Hadi

School of Healthcare, University of Leeds, LS2 9UT, Leeds, UK

S. José Closs

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Abdul Hadi .

Ethics declarations

Conflicts of interest.

None declared.

Rights and permissions

Reprints and permissions

About this article

Hadi, M.A., José Closs, S. Ensuring rigour and trustworthiness of qualitative research in clinical pharmacy. Int J Clin Pharm 38 , 641–646 (2016). https://doi.org/10.1007/s11096-015-0237-6

Download citation

Received : 16 August 2015

Accepted : 08 December 2015

Published : 14 December 2015

Issue Date : June 2016

DOI : https://doi.org/10.1007/s11096-015-0237-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Clinical pharmacy
Qualitative research
Trustworthiness
Find a journal
Publish with us
Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

Advanced Search
Journal List

Framework for advancing rigorous research

Walter j koroshetz.

1 National Institute of Neurological Disorders and Stroke, Bethesda, United States

Shannon Behrman

2 iBiology, San Francisco, United States

Cynthia J Brame

3 Center for Teaching and Department of Biological Sciences, Vanderbilt University, Nashville, United States

Janet L Branchaw

4 Department of Kinesiology and Wisconsin Institute for Science Education and Community Engagement, University of Wisconsin - Madison, Madison, United States

Emery N Brown

5 Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, United States

6 Department of Brain and Cognitive Science, Institute of Medical Engineering and Sciences, the Picower Institute for Learning and Memory, and the Institute for Data Systems and Society, Massachusetts Institute of Technology, Boston, United States

Erin A Clark

7 Department of Biology and Program in Neuroscience, Brandeis University, Waltham, United States

David Dockterman

8 Harvard Graduate School of Education, Harvard University, Cambridge, United States

Jordan J Elm

9 Department of Public Health Sciences, Medical University of South Carolina, Charleston, United States

Pamela L Gay

10 Planetary Science Institute, Tucson, United States

Katelyn M Green

11 Cellular and Molecular Biology Graduate Program, University of Michigan, Ann Arbor, United States

12 The Concord Consortium, Emeryville, United States

Michael G Kaplitt

13 Department of Neurological Surgery, Weill Cornell Medical College, New York, United States

Benedict J Kolber

14 Department of Biological Sciences, Duquesne University, Pittsburgh, United States

Alex L Kolodkin

15 Solomon H. Snyder Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, United States

Diane Lipscombe

16 Carney Institute for Brain Science, Department of Neuroscience, Brown University, Providence, United States

Malcolm R MacLeod

17 Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom

Caleb C McKinney

18 Biomedical Graduate Education, Georgetown University Medical Center, Washington, United States

Marcus R Munafò

19 MRC Integrative Epidemiology Unit, School of Psychological Science, University of Bristol, Bristol, United Kingdom

Barbara Oakley

20 Oakland University, Rochester, United States

Jeffrey T Olimpo

21 Department of Biological Sciences, The University of Texas at El Paso, El Paso, United States

Nathalie Percie du Sert

22 National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs), London, United Kingdom

Indira M Raman

23 Department of Neurobiology, Northwestern University, Evanston, United States

24 Complexly, Missoula, United States

Amy L Shelton

25 Center for Talented Youth and School of Education, Johns Hopkins University, Baltimore, United States

Stephen Miles Uzzo

26 New York Hall of Science, Flushing Meadows Corona Park, United States

Devon C Crawford

Shai d silberberg.

There is a pressing need to increase the rigor of research in the life and biomedical sciences. To address this issue, we propose that communities of 'rigor champions' be established to campaign for reforms of the research culture that has led to shortcomings in rigor. These communities of rigor champions would also assist in the development and adoption of a comprehensive educational platform that would teach the principles of rigorous science to researchers at all career stages.

The scientific enterprise relies on mentors teaching their students and trainees how to design and conduct studies that produce reliable scientific knowledge. A crucial part of this is teaching students and trainees how to minimize the risks that chance observations, subconscious biases, or other factors might lead to incorrect or inflated claims. However, as the demands on mentors increase, some of them unintentionally overlook this crucial aspect of scientific investigation, meaning that students and trainees are not taught how to distinguish between high- and low-quality evidence when working on their own studies and when reading about other studies ( Ioannidis et al., 2014 ; Bosch and Casadevall, 2017 ; Landis et al., 2012 ).

Additional complications stem from the welcome rise in team-based science and a greater sophistication and range of experimental techniques ( National Research Council, 2015 ), which may, in part, be driven by a feeling that only exciting and complete stories will appeal to journals and funders ( Nosek et al., 2012 ; Casadevall et al., 2016 ). Increasingly, an individual scientist cannot be an expert in all the techniques used in a research project.

Taken together, these developments suggest that enhanced training in the fundamental principles of rigorous research common to most, if not all, experimental practices is needed to ensure that the outputs of scientific research remain reliable and robust. Such principles include strong reasoning and inference based on valid assertions, which requires the proper interpretation of uncertainty and a motivation to identify inconsistencies ( Bosch and Casadevall, 2017 ; Casadevall and Fang, 2016 ; Munafò and Davey Smith, 2018 ; Wasserstein et al., 2019 ). For studies that test hypotheses, researchers should: clearly define interventions; identify and disclose possible confounding factors; transparently report project workflows, experimental plans, methods, data analyses, and any divergence from pre-planned procedures; and fully report their competing interests (see https://www.equator-network.org/ for reporting guidelines). The requirements for studies intended to generate hypotheses will be different but should be equally described ( Dirnagl, 2019 ).

Before formulating solutions to these issues, we assessed current training practices at the graduate and postdoctoral levels by surveying all 41 institutions in the United States that held at least one training grant from the National Institute of Neurological Disorders and Stroke (NINDS) in May 2018. Only 5 of the 37 institutions that responded to the survey reported providing a course predominantly dedicated to principles of rigorous research, with others using a range of approaches – such as seminars, lectures within other coursework, workshops, and informal mentoring – to teach good research practices. However, few if any of the institutions covered the full range of principles that need to be learned and understood. Although the sample in our survey was small, the responses reinforced the common belief that formal training in rigorous research needs to be enhanced ( Ioannidis et al., 2014 ; Munafò et al., 2017 ).

While numerous training materials related to rigorous research are available online, finding suitable materials and assembling them into a cohesive course is challenging. Having access to a free, organized suite of educational resources could greatly reduce the energy barrier for institutions and scientists to implement enhanced training at all levels, from undergraduate education to faculty professional development.

Towards this end NINDS convened a workshop attended by a range of stakeholders: basic, translational, and clinical neuroscientists; scholars of education and science communication; educational platform developers; and trainees. Although neuroscience served as a focal point, the four outcomes of the discussions apply widely across the biomedical sciences: i) there is a clear need for a platform that teaches the principles of rigorous research and covers the needs of scientists at all career stages; ii) effective educational interventions should lead to measurable behavioral change; iii) academic institutions need to play a proactive role in promoting rigorous research practices; iv) progress in this area will require cultural change at academic institutions, funders, and publishers ( Casadevall et al., 2016 ; Munafò et al., 2017 ; Collins and Tabak, 2014 ; Begley et al., 2015 ; Casadevall and Fang, 2012 ).

Building communities of rigor champions

To unleash the motivation for a cultural change evident in discussions between the authors and early-career researchers and others, and to provide momentum for change across different sectors, we propose the establishment of inter- and intra-institutional communities of 'rigor champions' who are committed to promoting rigor and transparency in research. We know there are many such individuals working at different levels of seniority in different types of organizations (such as universities, funders, publishers, and scientific societies), but they often feel isolated and under-resourced. To seed this effort and to help like-minded individuals in different organizations to find each other and join forces, NINDS has created a website for researchers, educators, trainees, organizational leaders and others who are passionate about the issues discussed here. This website includes currently available resources for making science more rigorous and transparently reporting results, as well as instructions for identifying yourself as a rigor champion.

More information about the different activities that these communities could undertake are given in Table 1 . Researchers, educators and trainees are best placed to collaborate on new tools, share best practices, and promote rigorous research in their local scientific communities. Societies are in a position to advocate for widespread policy changes, while funders and journals have important gatekeeping roles ( Collins and Tabak, 2014 ; McNutt, 2014 ; Cressey, 2015 ; PLOS Biology, 2018 ). The recently established UK Reproducibility Network ( Munafò et al., 2020 ) and the PREMIER project ( Dirnagl et al., 2018 ), both of which aim to improve scientific practices, may serve as models for these communities.

Community	Intra-organizational activities	Inter-organizational activities
	• Promote transparency and other rigorous practices among colleagues and mentors • Advocate for resources to facilitate rigorous research practices	• Share institutional resources and practices in education and training • Call for changes in institutional culture and policies
	• Transparently report all experiments, including neutral outcomes • Promote rigorous practices among colleagues and trainees • Call for changes to institutional culture, policies, and infrastructure	• Share effective training practices and useful laboratory resources • Coordinate with the broader scientific community to promote better incentive structures
	• Suggest improvements to available resources that address rigor • Integrate rigorous research principles into all coursework	• Share resources and educational best practices • Share effective learning evaluation methods
	• Enact policies and support infrastructure to incentivize transparency and other rigorous research practices • Explicitly incorporate mentoring, collaboration, and rigorous research practices into promotion procedures • Initiate and share outcomes from piloted educational resources	• Support and promote communities of rigor champions • Disseminate policy changes, new initiatives, educational successes, and implementation strategies • Develop tangible outcome measures to evaluate impact
	• Promote thorough review of research practices in publications • Explicitly support research transparency and neutral outcomes • Educate reviewers on which scientific practices are valued by the journal	• Collaborate to implement best practices consistently across different publishers
	• Support the founding of communities of rigor champions • Compile and encourage best practices used by the scientific community • Host workshops and educational materials for members	• Promote and maintain communities of rigor champions • Encourage institutional policies that promote research quality and effective education
	• Emphasize attention to rigor in peer review • Reward rigorous research practices and outstanding mentorship • Support infrastructure for transparent and rigorous science • Support educational resources and initiatives	• Support and promote communities of rigor champions • Share best practices for incentivizing rigorous research and educating scientists • Develop partnerships to support better training and facilitate cultural changes

NINDS, for example, has proactively sought effective approaches to support greater transparency in reporting . An NINDS meeting with publishers led to changes in journal policies regarding transparency of reporting at various journals ( Nature, 2013 ; Kelner, 2013 ). Recommendations for greater transparency at scientific meetings stemmed from an NINDS roundtable with conference organizing bodies ( Silberberg et al., 2017 ) and are being piloted by the Federation of American Societies for Experimental Biology (FASEB ) . To recognize outstanding mentors, NINDS established the Landis Mentoring Award , and by providing greater stability to meritorious scientists though the NINDS R35 Program , it is anticipated that the pressures to rush studies to publication will be mitigated.

In particular we hope that leaders at academic institutions – such as department chairs, deans, and vice-presidents of research – will become involved because they are uniquely placed to shape the culture and social norms of institutions ( Begley et al., 2015 ). For example, faculty evaluation criteria should be modified to place greater emphasis on data sharing, methods transparency, demonstrated rigor, collaboration, and mentoring, with less emphasis on the number of publications and journal impact factors ( Casadevall and Fang, 2012 ; Moher et al., 2018 ; Bertuzzi and Jamaleddine, 2016 ; Lundwall, 2019 ; Strech et al., 2020 ; Casci and Adams, 2020 ; see also https://sfdora.org/read ). When publications are being evaluated, rigorously obtained null results should be valued as highly as positive findings. Institutional leaders are also uniquely placed to ensure that scientific rigor is properly taught to trainees and incorporated into day-to-day lab work ( Casadevall et al., 2016 ; Begley et al., 2015 ; Bosch, 2018 ; Button et al., 2020 ). Moreover, evaluations of trainees should emphasize experimental and analytic skills rather than where papers are published.

Building an educational resource for rigorous research

The establishment of communities of rigor champions will set the stage for the creation of an educational platform designed by the scientific community to communicate the principles of rigorous research. Given the rapid evolution of technologies and learning practices, it is difficult to predict what resource formats will be most effective in the future, so the platform will need to be open and freely available, easily discoverable, engaging, modular, adaptable, and upgradable. It will also need to be available during coursework and beyond so that scientists can use it to answer questions when they are doing research or as part of life-long learning ( Figure 1 ). This means that the platform will have to embody a number of principles of effective teaching and mentoring (see Table 2 ).

An external file that holds a picture, illustration, etc.
Object name is elife-55915-fig1.jpg

We envision a comprehensive resource that can be used by scientists at all stages of their career to explore the principles of rigorous research at various levels of detail. We envision modules on a range of topics (such as reducing cognitive biases), each of which contains a number of topics (such as blinding), each of which contains a number of lessons (such as practical examples).

Key element	Teaching and learning principle
	Define the learning objectives upfront, identify ways to measure achievement of these objectives, and then design activities to support learning ( ).
	Encourage students to pose their own questions, apply commonly used tools and methods to actively explore their questions, and provide evidence when explaining phenomena ( ; ; ; ).
	Provide feedback on real-world experiments, whether in the classroom or the laboratory, as a way to demonstrate relevance and stimulate interest. Opportunities for personalized application and discussion in the local setting with the help of a facilitator’s guide are particularly critical, as adults typically learn most effectively when given the opportunity for immediate personal utility and value ( ). Emphasize the ability to contribute to a larger purpose or gain social standing ( ).
	Include a range of approaches to teaching and learning to accommodate different levels of knowledge and skills, motivations, and senses of self-efficacy ( ; ).
	Allow individuals to gain self-efficacy by experiencing a feeling of progress, being challenged in low-stakes environments, and working through confusing concepts successfully ( ). This is more effective when the person feels psychologically safe to take risks and fail in front of their local scientific community.
	Facilitate learning, foster collaboration, and recognize diverse perspectives in order to encourage learners to gain agency and forge a connection with the intellectual community ( ; ).
	Include complexity and inconsistencies in training examples rather than simplification for the sake of a persuasive story ( ; ). This counteracts the drive to smooth over inconvenient but potentially important details and highlights the importance of confounding variables, potential artefactual influences, reproducibility, and robustness of the findings.
	Nurture positive behaviors, like acknowledging and learning from mistakes, rather than penalize imperfect practices ( ). Mentors at all career stages are encouraged to model these positive behaviors and to share their own failures, the drudgery and frustrations of science, and their approaches to coping emotionally and growing intellectually while maintaining rigorous research practices.
	Measure success via gains in learner competency and changes to their real-world approaches to research. Changes in laboratory practice could be assessed by user self-reports, by analysis of research presented at meetings ( ) and in publications ( ), or by querying scientists on whether discussions with their mentors and colleagues led to changes in laboratory and institutional culture. Collaborate from the beginning with individuals who specialize in assessment design in higher education settings ( ).

We envision the platform being developed via a hub-and-spoke approach as discussed at a recent National Advisory Neurological Disorders and Stroke Council meeting. A centralized mechanism (the 'hub') will provide financial and infrastructural support and guidance (possibly via a steering committee) and facilitate sharing and coordination between groups, while rigor champions will come together to design specific modules (spokes) for the platform by using existing resources or designing new ones from scratch as needed. We envision worldwide teams of experts collaborating on building and testing the resource. Rigor champions with experience in defining clear learning objectives, building curricula, and evaluating success, for example, will collaborate with content experts to design topics needed in the resource. Importantly, potential users will be involved from the beginning of the development stage, and onwards through the design and implementation stages, to provide feedback about effectiveness and usability.

Given the importance of being able to measure the effectiveness (or otherwise) of the platform ( Table 2 ), individual components should be released publicly as they are completed to allow educators and users to iteratively test and improve the resource as it unfolds. As with science itself, the developers will need to experiment with content and delivery. If the resource does not improve the comprehension and research practice of individuals, or add value to the research community, rigorous approaches should be applied to improve it.

Once a functioning and effective resource has been built, it will be essential to promote its use and adoption. One approach would be to host 'train-the-trainer' programs ( Spencer et al., 2018 ; Pfund et al., 2006 ): those involved in building the resource share it with small groups of mentors, who are then better equipped to use the resource with their own mentees and to encourage their colleagues to use it. This form of dissemination also creates buy-in from mentors who need to model the behaviors they are teaching. Rigor champions, meanwhile, can encourage their institutions and colleagues to adopt and use the resource.

Setting up and supporting communities of rigor champions and developing educational resources on rigorous research will be complex and likely require multiple sources of support. However, with the participation of all sectors of the scientific enterprise, the actions proposed herein should, within a decade, lead to improvements in the culture of science as well as improvements in the design, conduct, analysis, and reporting of biomedical research. The result will be a healthier and more effective scientific community.

The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US Government.

Biographies

Walter J Koroshetz is at the National Institute of Neurological Disorders and Stroke, Rockville, MD, United States

Shannon Behrman is at iBiology, San Francisco, CA, United States

Cynthia J Brame is at the Center for Teaching and Department of Biological Sciences, Vanderbilt University, Nashville, TN, United States

Janet L Branchaw is in the Department of Kinesiology and Wisconsin Institute for Science Education and Community Engagement, University of Wisconsin - Madison, Madison, WI, United States

Emery N Brown is in the Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, and the Department of Brain and Cognitive Science, Institute of Medical Engineering and Sciences, the Picower Institute for Learning and Memory, and the Institute for Data Systems and Society, Massachusetts Institute of Technology, Cambridge, MA, United States

Erin A Clark is in the Department of Biology and Program in Neuroscience, Brandeis University, Waltham, MA, United States

David Dockterman is at the Harvard Graduate School of Education, Harvard University, Cambridge, MA, United States

Jordan J Elm is in the Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States

Pamela L Gay is at the Planetary Science Institute, Tucson, AZ, United States

Katelyn M Green is in the Cellular and Molecular Biology Graduate Program, University of Michigan, Ann Arbor, MI, United States

Sherry Hsi is with The Concord Consortium, Emeryville, CA, United States

Michael G Kaplitt is in the Department of Neurological Surgery, Weill Cornell Medical College, New York, NY, United States

Benedict J Kolber is in the Department of Biological Sciences, Duquesne University, Pittsburgh, PA, United States

Alex L Kolodkin is in the Solomon H. Snyder Department of Neuroscience, Johns Hopkins School of Medicine, Baltimore, MD, United States

Diane Lipscombe is in the Carney Institute for Brain Science, Department of Neuroscience, Brown University, Providence, RI, United States

Malcolm R MacLeod is in the Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom

Caleb C McKinney is in Biomedical Graduate Education, Georgetown University Medical Center, Washington, DC, United States

Marcus R Munafò is in the MRC Integrative Epidemiology Unit, School of Psychological Science, University of Bristol, Bristol, United Kingdom

Barbara Oakley is at Oakland University, Rochester, MI, United States

Jeffrey T Olimpo is in the Department of Biological Sciences, The University of Texas at El Paso, El Paso, TX, United States

Nathalie Percie du Sert is in the National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs), London, United Kingdom

Indira M Raman is in the Department of Neurobiology, Northwestern University, Evanston, IL, United States

Ceri Riley is with Complexly, Missoula, MT, United States

Amy L Shelton is at the Center for Talented Youth and School of Education, Johns Hopkins University, Baltimore, MD, United States

Stephen Miles Uzzo is at the New York Hall of Science, Flushing Meadows Corona Park, NY, United States

Devon C Crawford is at the National Institute of Neurological Disorders and Stroke, Rockville, MD, United States

Shai D Silberberg is at the National Institute of Neurological Disorders and Stroke, Rockville, MD, United States

Funding Statement

Funded by the National Institute of Neurological Disorders and Stroke (NINDS).

Competing interests

No competing interests declared.

Author contributions

Conceptualization, Writing - review and editing.

Conceptualization, Writing - original draft, Writing - review and editing. DCC and SDS wrote the manuscript; all authors provided intellectual input and contributed to the editing of the manuscript.

Alberts B, Cicerone RJ, Fienberg SE, Kamb A, McNutt M, Nerem RM, Schekman R, Shiffrin R, Stodden V, Suresh S, Zuber MT, Pope BK, Jamieson KH. Self-correction in science at work. Science. 2015; 348 :1420–1422. doi: 10.1126/science.aab3847. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Begley CG, Buchan AM, Dirnagl U. Robust Research: Institutions must do their part for reproducibility. Nature. 2015; 525 :25–27. doi: 10.1038/525025a. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bertuzzi S, Jamaleddine Z. Capturing the value of biomedical research. Cell. 2016; 165 :9–12. doi: 10.1016/j.cell.2016.03.004. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bjork RA, Dunlosky J, Kornell N. Self-regulated learning: beliefs, techniques, and illusions. Annual Review of Psychology. 2013; 64 :417–444. doi: 10.1146/annurev-psych-113011-143823. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bosch G. Train PhD students to be thinkers not just specialists. Nature. 2018; 554 :277. doi: 10.1038/d41586-018-01853-1. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bosch G, Casadevall A. Graduate biomedical science education needs a new philosophy. mBio. 2017; 8 :17. doi: 10.1128/mBio.01539-17. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Bradforth SE, Miller ER, Dichtel WR, Leibovich AK, Feig AL, Martin JD, Bjorkman KS, Schultz ZD, Smith TL. University Learning: Improve undergraduate science education. Nature. 2015; 523 :282–284. doi: 10.1038/523282a. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Brown JS, Adler RP. Minds on fire: open education, the long tail, and learning 2.0. EDUCAUSE Review. 2008; 43 :16–32. [ Google Scholar ]
Button KS, Chambers CD, Lawrence N, Munafò MR. Grassroots training for reproducible science: a consortium-based approach to the empirical dissertation. Psychology Learning & Teaching. 2020; 19 :77–90. doi: 10.1177/1475725719857659. [ CrossRef ] [ Google Scholar ]
Casadevall A, Ellis LM, Davies EW, McFall-Ngai M, Fang FC. A framework for improving the quality of research in the biological sciences. mBio. 2016; 7 :e01256. doi: 10.1128/mBio.01256-16. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Casadevall A, Fang FC. Reforming science: methodological and cultural reforms. Infection and Immunity. 2012; 80 :891–896. doi: 10.1128/IAI.06183-11. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Casadevall A, Fang FC. Rigorous Science: A how-to guide. mBio. 2016; 7 :e01902. doi: 10.1128/mBio.01902-16. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Casci T, Adams E. Setting the right tone. eLife. 2020; 9 :e55543. doi: 10.7554/eLife.55543. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Coleman B. Science writing: Too good to be true? [February 29, 2020]; New York Times. 1987 https://www.nytimes.com/1987/09/27/books/sceince-writing-too-good-to-be-true.html
Collins FS, Tabak LA. NIH plans to enhance reproducibility. Nature. 2014; 505 :612–613. doi: 10.1038/505612a. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Corwin LA, Graham MJ, Dolan EL. Modeling course-based undergraduate research experiences: an agenda for future research and evaluation. CBE—Life Sciences Education. 2015; 14 :es1. doi: 10.1187/cbe.14-10-0167. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Cressey D. UK funders demand strong statistics for animal studies. Nature. 2015; 520 :271–272. doi: 10.1038/520271a. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Dirnagl U, Kurreck C, Castaños-Vélez E, Bernard R. Quality management for academic laboratories: burden or boon? EMBO Reports. 2018; 19 :e47143. doi: 10.15252/embr.201847143. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Dirnagl U. Handbook of Experimental Pharmacology. Berlin, Heidelberg: Springer; 2019. Resolving the ttension between exploration and confirmation in preclinical biomedical research. [ PubMed ] [ CrossRef ] [ Google Scholar ]
D’Mello S, Lehman B, Pekrun R, Graesser A. Confusion can be beneficial for learning. Learning and Instruction. 2014; 29 :153–170. doi: 10.1016/j.learninstruc.2012.05.003. [ CrossRef ] [ Google Scholar ]
Handelsman J, Ebert-May D, Beichner R, Bruns P, Chang A, DeHaan R, Gentile J, Lauffer S, Stewart J, Tilghman SM, Wood WB. Scientific teaching. Science. 2004; 304 :521–522. doi: 10.1126/science.1096022. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Howitt SM, Wilson AN. Revisiting “Is the scientific paper a fraud?” EMBO Reports. 2014; 15 :481–484. doi: 10.1002/embr.201338302. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Ioannidis JPA, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Increasing value and reducing waste in research design, conduct, and analysis. The Lancet. 2014; 383 :166–175. doi: 10.1016/S0140-6736(13)62227-8. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Kelner KL. Playing our part. Science Translational Medicine. 2013; 5 :190ed7. doi: 10.1126/scitranslmed.3006661. [ CrossRef ] [ Google Scholar ]
Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, Crystal RG, Darnell RB, Ferrante RJ, Fillit H, Finkelstein R, Fisher M, Gendelman HE, Golub RM, Goudreau JL, Gross RA, Gubitz AK, Hesterlee SE, Howells DW, Huguenard J, Kelner K, Koroshetz W, Krainc D, Lazic SE, Levine MS, Macleod MR, McCall JM, Moxley RT, Narasimhan K, Noble LJ, Perrin S, Porter JD, Steward O, Unger E, Utz U, Silberberg SD. A call for transparent reporting to optimize the predictive value of preclinical research. Nature. 2012; 490 :187–191. doi: 10.1038/nature11556. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Lundwall RA. Changing institutional incentives to foster sound scientific practices: one department. Infant Behavior and Development. 2019; 55 :69–76. doi: 10.1016/j.infbeh.2019.03.006. [ PubMed ] [ CrossRef ] [ Google Scholar ]
MacLeod MR, Lawson McLean A, Kyriakopoulou A, Serghiou S, de Wilde A, Sherratt N, Hirst T, Hemblade R, Bahor Z, Nunes-Fonseca C, Potluru A, Thomson A, Baginskaite J, Baginskitae J, Egan K, Vesterinen H, Currie GL, Churilov L, Howells DW, Sena ES. Risk of bias in reports of in vivo research: a focus for improvement. PLOS Biology. 2015; 13 :e1002273. doi: 10.1371/journal.pbio.1002273. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
McNutt M. Journals unite for reproducibility. Science. 2014; 346 :679. doi: 10.1126/science.aaa1724. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Minner DD, Levy AJ, Century J. Inquiry-based science instruction-what is it and does it matter? results from a research synthesis years 1984 to 2002. Journal of Research in Science Teaching. 2010; 47 :474–496. doi: 10.1002/tea.20347. [ CrossRef ] [ Google Scholar ]
Moher D, Naudet F, Cristea IA, Miedema F, Ioannidis JPA, Goodman SN. Assessing scientists for hiring, promotion, and tenure. PLOS Biology. 2018; 16 :e2004089. doi: 10.1371/journal.pbio.2004089. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, Percie du Sert N, Simonsohn U, Wagenmakers E-J, Ware JJ, Ioannidis JPA. A manifesto for reproducible science. Nature Human Behaviour. 2017; 1 :0021. doi: 10.1038/s41562-016-0021. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Munafò MR, Chambers CD, Collins AM, Fortunato L, Macleod MR. Research culture and reproducibility. Trends in Cognitive Sciences. 2020; 24 :91–93. doi: 10.1016/j.tics.2019.12.002. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Munafò MR, Davey Smith G. Robust research needs many lines of evidence. Nature. 2018; 553 :399–401. doi: 10.1038/d41586-018-01023-3. [ PubMed ] [ CrossRef ] [ Google Scholar ]
National Research Council . Enhancing the Effectiveness of Team Science. The National Academies Press; 2015. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Nature Reducing our irreproducibility. Nature. 2013; 496 :398. doi: 10.1038/496398a. [ CrossRef ] [ Google Scholar ]
Nosek BA, Spies JR, Motyl M. Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science. 2012; 7 :615–631. doi: 10.1177/1745691612459058. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Pfund C, Maidl Pribbenow C, Branchaw J, Miller Lauffer S, Handelsman J. The merits of training mentors. Science. 2006; 311 :473–474. doi: 10.1126/science.1123806. [ PubMed ] [ CrossRef ] [ Google Scholar ]
PLOS Biology Fifteen years in, what next for PLOS biology? PLOS Biology. 2018; 16 :e3000049. doi: 10.1371/journal.pbio.3000049. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Raman IM. How to be a graduate advisee. Neuron. 2014; 81 :9–11. doi: 10.1016/j.neuron.2013.12.030. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Silberberg SD, Crawford DC, Finkelstein R, Koroshetz WJ, Blank RD, Freeze HH, Garrison HH, Seger YR. Shake up conferences. Nature. 2017; 548 :153–154. doi: 10.1038/548153a. [ PubMed ] [ CrossRef ] [ Google Scholar ]
Spencer KC, McDaniels M, Utzerath E, Rogers JG, Sorkness CA, Asquith P, Pfund C. Building a sustainable national infrastructure to expand research mentor training. CBE—Life Sciences Education. 2018; 17 :ar48. doi: 10.1187/cbe.18-03-0034. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Strech D, Weissgerber T, Dirnagl U, QUEST Group Improving the trustworthiness, usefulness, and ethics of biomedical research through an innovative and comprehensive institutional initiative. PLOS Biology. 2020; 18 :e3000576. doi: 10.1371/journal.pbio.3000576. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
Walkington C, Bernacki ML. Personalization of instruction: design dimensions and implications for cognition. The Journal of Experimental Education. 2018; 86 :50–68. doi: 10.1080/00220973.2017.1380590. [ CrossRef ] [ Google Scholar ]
Wasserstein RL, Schirm AL, Lazar NA. Moving to a world beyond "p < 0.05". The American Statistician. 2019; 73 :1–19. doi: 10.1080/00031305.2019.1583913. [ CrossRef ] [ Google Scholar ]
Yeager DS, Henderson MD, Paunesku D, Walton GM, D'Mello S, Spitzer BJ, Duckworth AL. Boring but important: a self-transcendent purpose for learning fosters academic self-regulation. Journal of Personality and Social Psychology. 2014; 107 :559–580. doi: 10.1037/a0037637. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]

Login to your account

If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password

If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password

Property	Value
Status
Version
Ad File
Disable Ads Flag
Environment
Moat Init
Moat Ready
Contextual Ready
Contextual URL
Contextual Initial Segments
Contextual Used Segments
AdUnit
SubAdUnit
Custom Targeting
Ad Events
Invalid Ad Sizes

AACP Member Login Submit

A Review of the Quality Indicators of Rigor in Qualitative Research

Download started

Download PDF Download PDF
Add to Mendeley
qualitative research design
standards of rigor
best practices

INTRODUCTION

Qualitative research involves the studied use and collection of a variety of empirical materials – case study; personal experience; introspection; life story; interview; artifacts; cultural texts and productions; observational, historical, interactional, and visual texts – that describe the routine and problematic moments and meanings in individual lives. Accordingly, qualitative researchers deploy a wide range of interconnected interpretative practices, hoping always to get a better understanding of the subject matter at hand. It is understood, however, that each practice makes the world visible in a different way. Hence there is frequently a commitment to using more than one interpretative practice in any study. 1

BEST PRACTICES: STEP-WISE APPROACH

Step 1: identifying a research topic.

Step 2: Qualitative Study Design

Step 3: data analysis, step 4: drawing valid conclusions, step 5: reporting research results, article metrics, related articles.

Download Hi-res image
Download .PPT
Access for Developing Countries
Articles & Issues
Articles In Press
Current Issue
Past Issues
Journal Information
About Open Access
Aims & Scope
Editorial Board
Editorial Team
History of AJPE
Contact Information
For Authors
Guide for Authors
Researcher Academy
Rights & Permissions
Submission Process
Submit Article
For Reviewers
Reviewer Instructions
Reviewer Frequently Asked Questions

The content on this site is intended for healthcare professionals.

Privacy Policy
Terms and Conditions
Accessibility
Help & Contact

Information

Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

Active Journals
Find a Journal
Journal Proposal
Proceedings Series
For Authors
For Reviewers
For Editors
For Librarians
For Publishers
For Societies
For Conference Organizers
Open Access Policy
Institutional Open Access Program
Special Issues Guidelines
Editorial Process
Research and Publication Ethics
Article Processing Charges
Testimonials
Preprints.org
SciProfiles
Encyclopedia

Article Menu

Subscribe SciFeed
Recommended Articles
Google Scholar
on Google Scholar
Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Advancing phantom fabrication: exploring 3d-printed solutions for abdominal imaging research.

1. Introduction

2. materials and methods, 2.1. characterization of appendix simulation in an anthropomorphic phantom, 2.2. material selection and characterization, 2.3. printing of the abdominal mold, 2.4. phantom fabrication, 4. discussion, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

Mørup, S.D.; Stowe, J.; Precht, H.; Gervig, M.H.; Foley, S. Design of a 3D printed coronary artery model for CT optimization. Radiography 2022 , 28 , 426–432. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Tino, R.; Yeo, A.; Leary, M.; Brandt, M.; Kron, T. A systematic review on 3D-printed imaging and dosimetry phantoms in radiation therapy. Technol. Cancer Res. Treat. 2019 , 18 , 1533033819870208. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Hazelaar, C.; van Eijnatten, M.; Dahele, M.; Wolff, J.; Forouzanfar, T.; Slotman, B.; Verbakel, W.F. Using 3D printing techniques to create an anthropomorphic thorax phantom for medical imaging purposes. Med. Phys. 2018 , 45 , 92–100. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Amini, I.; Akhlaghi, P.; Sarbakhsh, P. Construction and verification of a physical chest phantom from suitable tissue equivalent materials for computed tomography examinations. Radiat. Phys. Chem. 2018 , 150 , 51–57. [ Google Scholar ] [ CrossRef ]
Tejo-Otero, A.; Fenollosa-Artés, F.; Achaerandio, I.; Rey-Vinolas, S.; Buj-Corral, I.; Mateos-Timoneda, M.Á.; Engel, E. Soft-tissue-mimicking using hydrogels for the development of phantoms. Gels 2022 , 8 , 40. [ Google Scholar ] [ CrossRef ]
Buchwald, R.; Breed, M.D.; Greenberg, A.R. The thermal properties of beeswaxes: Unexpected findings. J. Exp. Biol. 2008 , 211 , 121–127. [ Google Scholar ] [ CrossRef ]
Yan, Q.; Dong, H.; Su, J.; Han, J.; Song, B.; Wei, Q.; Shi, Y. A review of 3D printing technology for medical applications. Engineering 2018 , 4 , 729–742. [ Google Scholar ] [ CrossRef ]
Rajan, K.; Samykano, M.; Kadirgama, K.; Harun, W.S.; Rahman, M.M. Fused deposition modeling: Process, materials, parameters, properties, and applications. Int. J. Adv. Manuf. Technol. 2022 , 120 , 1531–1570. [ Google Scholar ] [ CrossRef ]
Vyavahare, S.; Teraiya, S.; Panghal, D.; Kumar, S. Fused deposition modelling: A review. Rapid Prototyp. J. 2020 , 26 , 176–201. [ Google Scholar ] [ CrossRef ]
Gear, J.I.; Cummings, C.; Craig, A.J.; Divoli, A.; Long, C.D.; Tapner, M.; Flux, G.D. Abdo-Man: A 3D-printed anthropomorphic phantom for validating quantitative SIRT. EJNMMI Phys. 2016 , 3 , 17. [ Google Scholar ] [ CrossRef ]
Qiu, J.; Hou, K.; Dyer, B.A.; Chen, J.C.; Shi, L.; Sun, Y.; Xu, L.; Zhao, H.; Li, Z.; Chen, T.; et al. Constructing customized multimodal phantoms through 3D printing: A preliminary evaluation. Front. Phys. 2021 , 9 , 605630. [ Google Scholar ] [ CrossRef ]
Filippou, V.; Tsoumpas, C. Recent advances on the development of phantoms using 3D printing for imaging with CT, MRI, PET, SPECT, and ultrasound. Med. Phys. 2018 , 45 , e740–e760. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Wang, K.; Ho, C.C.; Zhang, C.; Wang, B. A review on the 3D printing of functional structures for medical phantoms and regenerated tissue and organ applications. Engineering 2017 , 3 , 653–662. [ Google Scholar ] [ CrossRef ]
Coles-Black, J.; Bolton, D.; Robinson, D.; Chuen, J. Utility of 3D printed abdominal aortic aneurysm phantoms: A systematic review. ANZ J. Surg. 2021 , 91 , 1673–1681. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Higgins, M.; Leung, S.; Radacsi, N. 3D printing surgical phantoms and their role in the visualization of medical procedures. Ann. 3D Print. Med. 2022 , 6 , 100057. [ Google Scholar ] [ CrossRef ]
Anwari, V.; Lai, A.; Ursani, A.; Rego, K.; Karasfi, B.; Sajja, S.; Paul, N. 3D printed CT-based abdominal structure mannequin for enabling research. 3D Print. Med. 2020 , 6 , 3. [ Google Scholar ] [ CrossRef ]
Schenkl, S.; Muggenthaler, H.; Hubig, M.; Erdmann, B.; Weiser, M.; Zachow, S.; Heinrich, A.; Güttler, F.V.; Teichgräber, U.; Mall, G. Automatic CT-based finite element model generation for temperature-based death time estimation: Feasibility study and sensitivity analysis. Int. J. Leg. Med. 2017 , 131 , 699–712. [ Google Scholar ] [ CrossRef ]
Rengier, F.; Mehndiratta, A.; Von Tengg-Kobligk, H.; Zechmann, C.M.; Unterhinninghofen, R.; Kauczor, H.U.; Giesel, F.L. 3D printing based on imaging data: Review of medical applications. Int. J. Comput. Assist. Radiol. Surg. 2010 , 5 , 335–341. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Okkalidis, N. 3D printing methods for radiological anthropomorphic phantoms. Phys. Med. Biol. 2022 , 67 , 15TR04. [ Google Scholar ] [ CrossRef ]
Kairn, T.; Crowe, S.B.; Markwell, T. Use of 3D printed materials as tissue-equivalent phantoms. In Proceedings of the World Congress on Medical Physics and Biomedical Engineering, Toronto, ON, Canada, 7–12 June 2015; Springer International Publishing: Cham, Switzerland, 2015; pp. 728–731. [ Google Scholar ]
Lamba, R.; McGahan, J.P.; Corwin, M.T.; Li, C.S.; Tran, T.; Seibert, J.A.; Boone, J.M. CT Hounsfield numbers of soft tissues on unenhanced abdominal CT scans: Variability between two different manufacturers’ MDCT scanners. Am. J. Roentgenol. 2014 , 203 , 1013–1020. [ Google Scholar ] [ CrossRef ]
D’Souza, W.D.; Madsen, E.L.; Unal, O.; Vigen, K.K.; Frank, G.R.; Thomadsen, B.R. Tissue mimicking materials for a multi-imaging modality prostate phantom. Med. Phys. 2001 , 28 , 688–700. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Holmes, R.B.; Negus, I.S.; Wiltshire, S.J.; Thorne, G.C.; Young, P.; Alzheimer’s Disease Neuroimaging Initiative. Creation of an anthropomorphic CT head phantom for verification of image segmentation. Med. Phys. 2020 , 47 , 2380–2391. [ Google Scholar ] [ CrossRef ]
Kim, S.Y.; Park, J.W.; Park, J.; Yea, J.W.; Oh, S.A. Fabrication of 3D printed head phantom using plaster mixed with polylactic acid powder for patient-specific QA in intensity-modulated radiotherapy. Sci. Rep. 2022 , 12 , 17500. [ Google Scholar ] [ CrossRef ]
Marro, A.; Bandukwala, T.; Mak, W. Three-dimensional printing and medical imaging: A review of the methods and applications. Curr. Probl. Diagn. Radiol. 2016 , 45 , 2–9. [ Google Scholar ] [ CrossRef ]
Haleem, A.; Javaid, M.; Suman, R.; Singh, R.P. 3D printing applications for radiology: An overview. Indian J. Radiol. Imaging 2021 , 31 , 10–17. [ Google Scholar ] [ PubMed ]
Giannopoulos, A.A.; Steigner, M.L.; George, E.; Barile, M.; Hunsaker, A.R.; Rybicki, F.J.; Mitsouras, D. Cardiothoracic applications of 3-dimensional printing. J. Thorac. Imaging 2016 , 31 , 253–272. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Rai, R.; Manton, D.; Jameson, M.G.; Josan, S.; Barton, M.B.; Holloway, L.C.; Liney, G.P. 3D printed phantoms mimicking cortical bone for the assessment of ultrashort echo time magnetic resonance imaging. Med. Phys. 2018 , 45 , 758–766. [ Google Scholar ] [ CrossRef ]
Cook, T.S.; Steingall, S.J.; Steingall, S.R.; Boonn, W.W. Establishing and running a three-dimensional and advanced imaging laboratory. Radiographics 2018 , 38 , 1799–1809. [ Google Scholar ] [ CrossRef ]
Linguraru, M.G.; Yao, J.; Gautam, R.; Peterson, J.; Li, Z.; Linehan, W.M.; Summers, R.M. Renal tumor quantification and classification in contrast-enhanced abdominal CT. Pattern Recognit. 2009 , 42 , 1149–1161. [ Google Scholar ] [ CrossRef ]
Cardenas, C.E.; Yang, J.; Anderson, B.M.; Court, L.E.; Brock, K.B. Advances in auto-segmentation. In Seminars in Radiation Oncology ; WB Saunders: Philadelphia, PA, USA, 2019; Volume 29, pp. 185–197. [ Google Scholar ]
Demehri, S.; Muhit, A.; Zbijewski, W.; Stayman, J.W.; Yorkston, J.; Packard, N.; Senn, R.; Yang, D.; Foos, D.; Thawait, G.K.; et al. Assessment of image quality in soft tissue and bone visualization tasks for a dedicated extremity cone-beam CT system. Eur. Radiol. 2015 , 25 , 1742–1751. [ Google Scholar ] [ CrossRef ]
Goldman, L.W. Principles of CT: Radiation dose and image quality. J. Nucl. Med. Technol. 2007 , 35 , 213–225. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Mitsouras, D.; Liacouras, P.; Imanzadeh, A.; Giannopoulos, A.A.; Cai, T.; Kumamaru, K.K.; George, E.; Wake, N.; Caterson, E.J.; Pomahac, B.; et al. Medical 3D printing for the radiologist. Radiographics 2015 , 35 , 1965–1988. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Jusufbegović, M.; Pandžić, A.; Šehić, A.; Jašić, R.; Julardžija, F.; Vegar-Zubović, S.; Beganović, A. Computed tomography tissue equivalence of 3D printing materials. Radiography 2022 , 28 , 788–792. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Hepburn, H.R.; Pirk, C.W.; Duangphakdee, O.; Hepburn, H.R.; Pirk, C.W.; Duangphakdee, O. The chemistry of beeswax. In Honeybee Nests: Composition, Structure, Function ; Springer: Berlin/Heidelberg, Germany, 2014; pp. 319–339. [ Google Scholar ]
European Food Safety Authority (EFSA). Risk Assessment of Beeswax Adulterated with Paraffin and/or Stearin/Stearic Acid When Used in Apiculture and as Food (Honeycomb). EFSA Support. Publ. 2020 , 17 , 1859E. [ Google Scholar ]

Click here to enlarge figure

Source/Comparison	Key Findings	Description
Filippou et al. [ ]	Three-dimensional printing offers precision and customization.	Three-dimensional printing enables the production of phantoms that closely mimic human tissue properties, which is crucial for accurate radiation dose measurements.
Wang et al. [ ]	Potential for dose reduction through personalized phantom design.	This study highlights the comparison of different materials and printing techniques used to achieve tissue-equivalent properties, emphasizing dose reduction.
Coles-Black et al. [ ]	Clinical applications in surgery and radiation therapy.	This article discusses the importance of phantoms in training and preoperative planning, in which accurate tissue simulation is critical for clinical applications.
Higgins et al. [ ]	Comparison of commercial and 3D-printed phantoms.	This research compares the pros and cons of commercial and 3D-printed phantoms in terms of cost, accuracy, and clinical utility, providing insights into their relative effectiveness.

		U (kV)	I (mA)	t (ms)	p	T (mm)	CTDI (mGy)	Kernel
Series	1	80	300	750	0.816	0.5	3.9	FC18
	2	100	153	750	0.816	0.5	4.9	FC18
	3	120	87	750	0.816	0.5	6.0	FC18
	4	135	80	750	0.816	0.5	7.6	FC18
	5	80	300	750	0.816	0.5	3.9	FC08
	6	80	300	750	0.816	1.0	3.9	FC18
	7	100	153	750	0.816	0.5	4.9	FC08
	8	100	153	750	0.816	1.0	4.9	FC18
	9	120	87	750	0.816	0.5	6.0	FC08
	10	120	87	750	0.816	1.0	6.0	FC18
	11	135	80	750	0.816	0.5	7.6	FC08
	12	135	80	750	0.816	1.0	7.6	FC18

		U	I	CTDI	T	σ	D	C
		(kV)	(mA)	(mGy)	(mm)	(HU)	(mm)	(HU)
Series	1	80	300	3.9	0.5	1.00	7.46	810
	2	100	153	4.9	0.5	1.01	7.55	877
	3	120	87	6.0	0.5	0.93	7.35	884
	4	135	80	7.6	0.5	0.80	7.50	865
	5	80	300	3.9	0.5	1.00	7.46	808
	6	80	300	3.9	1.0	1.09	7.65	825
	7	100	153	4.9	0.5	1.01	7.56	874
	8	100	153	4.9	1.0	1.02	7.68	879
	9	120	87	6.0	0.5	0.93	7.22	883
	10	120	87	6.0	1.0	1.00	7.58	876
	11	135	80	7.6	0.5	0.80	7.50	864
	12	135	80	7.6	1.0	0.84	7.75	869

Evaluated Object	Fat Tissue ± σ)	Muscle Tissue ± σ)	Bone ± σ)
Patient images	−113.6 ± 10.4	49.72 ± 14.7	376 ± 120.6
3D-printed phantom	−115.41 ± 20.29	65.61 ± 18.06	510 ± 131.2
Commercially available phantom	−74.78 ± 12.83	56.34 ± 12.6	541 ± 101.8
p-Values of Student’s t-test
	0.428	<0.001	<0.001
	<0.001	<0.001	<0.001
	<0.001	<0.001	0.063

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

Becircic, M.; Delibegovic, S.; Sehic, A.; Julardzija, F.; Beganovic, A.; Ljuca, K.; Pandzic, A.; Jusufbegovic, M. Advancing Phantom Fabrication: Exploring 3D-Printed Solutions for Abdominal Imaging Research. Appl. Sci. 2024 , 14 , 8384. https://doi.org/10.3390/app14188384

Becircic M, Delibegovic S, Sehic A, Julardzija F, Beganovic A, Ljuca K, Pandzic A, Jusufbegovic M. Advancing Phantom Fabrication: Exploring 3D-Printed Solutions for Abdominal Imaging Research. Applied Sciences . 2024; 14(18):8384. https://doi.org/10.3390/app14188384

Becircic, Muris, Samir Delibegovic, Adnan Sehic, Fuad Julardzija, Adnan Beganovic, Kenana Ljuca, Adi Pandzic, and Merim Jusufbegovic. 2024. "Advancing Phantom Fabrication: Exploring 3D-Printed Solutions for Abdominal Imaging Research" Applied Sciences 14, no. 18: 8384. https://doi.org/10.3390/app14188384

Article Metrics

Further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

IMAGES

Promoting and evaluating scientific rigour in qualitative research
Criteria for ensuring rigour in qualitative research
1: Criteria of rigour in a qualitative report
Promoting and evaluating scientific rigour in qualitative research
(PDF) Using the TACT Framework to Learn the Principles of Rigour in
Initiative to Improve Education in the Principles of Rigorous Research

VIDEO

SAMPLING PROCEDURE AND SAMPLE (QUALITATIVE RESEARCH)
Managing the Limitations of Qualitative Research
Real-time Energy Beam in EEVEE!
WHAT IS PROCEDURAL MEMORY #mind #facts #shorts
Prof Alan Nevill Limits of Human Performance
Business Research: Rigour in Qualitative Research

COMMENTS

A Review of the Quality Indicators of Rigor in Qualitative Research
Abstract. Attributes of rigor and quality and suggested best practices for qualitative research design as they relate to the steps of designing, conducting, and reporting qualitative research in health professions educational scholarship are presented. A research question must be clear and focused and supported by a strong conceptual framework ...
Ensuring Rigor in Qualitative Data Analysis: A Design Research Approach
Rigor is demonstrated by this depth of engagement that enables the designer "to reach through to the concealed plums" (Cross, 2001, p. 53). Demonstrating Rigor in Research. It is important to clarify that the requirements for demonstrating rigor in design research and in grounded theory qualitative analysis vary from those required in ...
Chapter 26: Rigour
What is rigour? In qualitative research, rigour, or trustworthiness, refers to how researchers demonstrate the quality of their research. 1, 2 Rigour is an umbrella term for several strategies and approaches that recognise the influence on qualitative research by multiple realities; for example, of the researcher during data collection and analysis, and of the participant.
Rigour in qualitative case-study research
However, it is important that the research is conducted in a rigorous manner and that this is demonstrated in the final research report. ... Strategies to ensure the rigour of this research were prolonged engagement and persistent observation, triangulation, peer debriefing, member checking, audit trail, reflexivity, and thick descriptions ...
PDF Achieving Rigor in Qualitative Analysis: The Role of Active
1967: 244). When conducting qualitative analysis, we generally identify categories in our data. These categories are generally described as codes or groupings of codes, such as the first- and. second-order codes and overarching categories often described in classical grounded theory. (Strauss & Corbin, 1990).
Criteria for Good Qualitative Research: A Comprehensive Review
This review aims to synthesize a published set of evaluative criteria for good qualitative research. The aim is to shed light on existing standards for assessing the rigor of qualitative research encompassing a range of epistemological and ontological standpoints. Using a systematic search strategy, published journal articles that deliberate criteria for rigorous research were identified. Then ...
A Review of the Quality Indicators of Rigor in Qualitative Research
Peer review, another common standard of rigor, is a process by which researchers invite an independent third-party researcher to analyze a detailed audit trail maintained by the study author. The audit trail methodically describes the step-by-step processes and decision-making through-out the study.
Qualitative Research: Rigour and qualitative research
Various strategies are available within qualitative research to protect against bias and enhance the reliability of findings. This paper gives examples of the principal approaches and summarises them into a methodological checklist to help readers of reports of qualitative projects to assess the quality of the research. In the health field--with its strong tradition of biomedical research ...
Rigour in qualitative research: mechanisms for control
What is the basis of this criticism? How can methodological rigour be demonstrated? Furthermore, what is rigour and what are the key issues for qualitative researchers? This paper attempts to answer these questions. Nurse Researcher. 16, 3, 70-85. doi: 10.7748/nr2009.04.16.3.70.c6947
Rigor, Transparency, Evidence, and Representation in Discourse Analysis
The challenge of representation is twofold. First, representation requires researchers to find ways to present the process of data analysis in textual and/or visual form in order to publicly disclose the research process and to demonstrate the rigor of the analysis (Anfara et al., 2002; Harry et al., 2005).
Rigor and Transparency in Research
Transparency is a fundamental aspect of rigor in qualitative research. It involves the clear, detailed, and explicit documentation of all stages of the research process. This allows other researchers to understand, evaluate, transfer, and build upon the study. The key aspects of transparency in qualitative research include methodological ...
20.1 Introduction to qualitative rigor
In Chapter 11 we talked about quality in quantitative studies, but we built our discussion around concepts like reliability and validity.With qualitative studies, we generally think about quality in terms of the concept of rigor.The difference between quality in quantitative research and qualitative research extends beyond the type of data (numbers vs. words/sounds/images).
Quality in qualitative research
Qualitative research most commonly involves the systematic collection, ordering, description and interpretation of textual data generated from talk, observation or documentation. A report of qualitative research should address the following criteria: Clarification and justification; Procedural rigour; Representativeness; Interpretative rigour;
PDF Methodological rigour within a qualitative framework
qualitative research is a scientiﬁc process that has a valued contribution to make to the advancement of knowledge. Rigour is the means by which we demonstrate integrity and competence (Aroni et al. 1999), a way of demonstrating the legitimacy of the research process. Without rigour, there is a danger that research may become ﬁctional ...
Ensuring rigour and trustworthiness of qualitative research ...
The use of qualitative research methodology is well established for data generation within healthcare research generally and clinical pharmacy research specifically. In the past, qualitative research methodology has been criticized for lacking rigour, transparency, justification of data collection and analysis methods being used, and hence the integrity of findings. Demonstrating rigour in ...
Indicators of rigor in dissertation research
Clarity, a mark of writing rigor, facilitates reader immersion in dissertation narratives based on the writer consistently writing for the reader-who-does-not-know the subject matter as well as the writer (Williams and Bizup, 2017). •. Logical Flow: Logical flow is paramount in developing a dissertation.
Framework for advancing rigorous research
Building communities of rigor champions. To unleash the motivation for a cultural change evident in discussions between the authors and early-career researchers and others, and to provide momentum for change across different sectors, we propose the establishment of inter- and intra-institutional communities of 'rigor champions' who are committed to promoting rigor and transparency in research.
Increasing rigor and reducing bias in qualitative research: A document
We present this example to extend the knowledge base regarding applied thematic analysis and to demonstrate how step-by-step implementation of a purposeful methodology using trustworthy documentary data can effectively increase rigor and transparency, thereby reducing potential bias, in a qualitative analysis.
PDF This document is an introduction to the concept of rigour as related to
proposal, but it can also provide models of how to address rigour in your own research process and the ways you communicate it. Rigour in Quantitative Research . Rigour can be assessed by addressing . reliability. and . validity. in your quantitative research proposal. Reliability. is the quality of consistency in your research. Liu's ...
PDF Rigor in the Research Approach
Rigor of the Prior Research A careful assessment of the rigor of the prior research that serves as the key support for a proposed project will help applicants identify any weaknesses or gaps in the line of research. Describe the strengths and weaknesses in the rigor of the prior research (both
PDF What is Research Rigor? Lessons for a Transdiscipline
Rigor definitions tend to fall into one of two categories: criteria-based and compliance-based. Which is appropriate depends on the research context. Even more variation was found with respect to relevance, which is often used as a catch-all for research characteristics that aren't associated with rigor.
A Review of the Quality Indicators of Rigor in Qualitative Research
Attributes of rigor and quality and suggested best practices for qualitative research design as they relate to the steps of designing, conducting, and reporting qualitative research in health professions educational scholarship are presented. A research question must be clear and focused and supported by a strong conceptual framework, both of which contribute to the selection of appropriate ...
Applied Sciences
Background: The development of novel medical imaging technologies and treatment procedures hinges on the availability of accurate and versatile phantoms. This paper presents a cost-effective approach for creating anthropomorphic abdominal phantoms. Methods: This study proposes a cost-effective method using 3D printing and readily available materials (beeswax, plaster, and epoxy resin) to ...
'Rigour', 'discipline' and the 'systematic' in educational research
This paper seeks to engage with three particular areas of discussion: the first is the connection (as I would argue) between research and truth; the second is the essentially rigorous and hence 'disciplined' nature of research and its location within academic disciplines; and the third is the challenge presented to both these points of view by the idea of 'alternative epistemologies'.

Chapter 26: Rigour

Learning outcomes

What is rigour?

Table 26.1: Comparison of quantitative and qualitative approaches to ensuring quality in research

Approaches to dependability

Approaches to credibility

Table 26.2 Data sources for the PhD research project of the Author.

Approaches to transferability

What is reflexivity?

Table 26.3: Types of reflexivity

Share This Book

Save citation to file

Add to My Bibliography

Rigour in qualitative case-study research

Similar articles

Related information

LinkOut - more resources

Other Literature Sources

Search form

Qualitative Research: Rigour and qualitative research

Criticisms of qualitative research

Is qualitative research different?

Log in using your username and password

Log in through your institution

Issues in research Previous Next

Want to read more?

Share article: Rigour in qualitative research: mechanisms for control

The Ultimate Guide to Qualitative Research - Part 3: Presenting Qualitative Data

What is rigor in qualitative research?

Transparency and rigor in research

Credibility

Dependability

Confirmability

Transferability

Reflexivity

Triangulation

Transparently document your research with ATLAS.ti

Clear research objectives and methods

Thorough data documentation and analysis

Reflexivity and acknowledgment of bias

Transparent reporting and publishing

Rigorous research starts with ATLAS.ti

20.1 Introduction to qualitative rigor

19.1 Introduction to qualitative rigor

Truth value

Applicability

Consistency

Key Takeaways

Share This Book

Quality in qualitative research

Do you have any competing interests to declare? *

Ensuring rigour and trustworthiness of qualitative research in clinical pharmacy

Cite this article

Access this article

Similar content being viewed by others

Evidence Produced While Using Qualitative Methodologies Including Research Trustworthiness

Evaluating Qualitative Health Research from Inside and Outside

Reconsidering ‘ethics’ and ‘quality’ in healthcare research: the case for an iterative ethical paradigm

Author information

Corresponding author

Ethics declarations

Rights and permissions

About this article

Share this article

Framework for advancing rigorous research

Shannon Behrman

Cynthia J Brame

Janet L Branchaw

Emery N Brown

Erin A Clark

David Dockterman

Jordan J Elm

Pamela L Gay

Katelyn M Green

Michael G Kaplitt

Benedict J Kolber

Alex L Kolodkin

Diane Lipscombe

Malcolm R MacLeod

Caleb C McKinney